opencti performance improvement for exact searches #1048
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Hello,
Please find a proposal for improving the observables exact search by using the
.read()
API call with a properfilters
set.A dictionary is used with main Cortex observable types to be translated into OpenCTI
key
value.For the moment it supports the following Cortex types:
ip
,url
,domain
,mail
,hash
andfilename
.As well the helper function
get_hash_type()
has been added to do a regex match on Cortex typehash
and determine if it's an MD5, a SHA1 or a SH256 in order to translate to the proper OpenCTI observable type.For any observable type that is not present in the
cortex2opencti_types
full text search (slow) will be used.Performance here without the patch for querying one observable is ~10sec, with the patch it is under the second.
Without this, doing batch analyzer runs is prone to failure as the connection between Cortex instance and OpenCTI will easily timeout.
Cheers,
-- Mathieu