HuntingAbuseAPI analyzer #2885
HuntingAbuseAPI analyzer #2885
Conversation
|
Hi @fgibertoni, hope you are doing well. The PR is ready to be reviewed. |
|
FP found Response:
FP not found Response: |
There was a problem hiding this comment.
Thanks for your work!
I think it would be useful to add a Data Model too.
You can check out the doc for guidelines on how to write one (evaluation, reliability and maybe additional details can be mapped) or just check other analyzers like GreyNoise Intel or Crowdsec.
Since it's a relatively new feature feel free to ask for additional guidance if you're having trouble 😄
| logger = logging.getLogger(__name__) | ||
|
|
||
|
|
||
| class HuntingAbuseAPI(ObservableAnalyzer): |
There was a problem hiding this comment.
For abuse.ch services we introduced a mixin called AbuseCHMixin that handles the required API key, you just need to add the authentication_header property to the other headers and add the Mixin in the superclasses list. You can find some examples about this in ThreatFox or URLHaus analyzers.
| logger.info(f"Fetching fp_status for {self.observable_name}") | ||
| if value_dict["entry_value"] == self.observable_name: | ||
| return {"fp_status": "true", "details": value_dict} | ||
| return {"fp_status": "False"} |
There was a problem hiding this comment.
| return {"fp_status": "False"} | |
| return {"fp_status": False} |
So it gets automatically converted in JSON compatible format
| for _key, value_dict in fp_list.items(): | ||
| logger.info(f"Fetching fp_status for {self.observable_name}") | ||
| if value_dict["entry_value"] == self.observable_name: | ||
| return {"fp_status": "true", "details": value_dict} |
There was a problem hiding this comment.
| return {"fp_status": "true", "details": value_dict} | |
| return {"fp_status": True, "details": value_dict} |
So it gets automatically converted in JSON compatible format
|
Hi @spoiicy I had previously explored this issue a bit and just wanted to share a quick suggestion. Would it make sense to use a TTL-based cache for the false positives list? That way, we can store it once and refresh it periodically, instead of fetching the full list on every search. This could help with performance and reduce API load. |
|
@fgibertoni As per suggestion from @AnshSinghal, it actually makes sense to store the results for some amount of time, update them after, let's say 1 day, so that the server doesn't have to query on every call. And I am aware that we used a file-based approach previously to perform such tasks and now @cristinaascari is working on solution, making it a DB-based approach and refactoring old code. So how should I proceed with this? Let me know what you think. |
|
Yes I do agree with you and @AnshSinghal. At the moment @cristinaascari is blocked by other work on that task so it may take longer than expected. |
spoiicy
force-pushed
the
gsoc_25_enhancements
branch
from
7cebbd4 to
08a3184
Compare
Closes #2778
Description
This PR aims to add an analyzer to identify if the provided observable is present in the false positive list or not via Hunting Abuse API.
Type of change
Please delete options that are not relevant.
Checklist
developdumpplugincommand and added it in the project as a data migration. ("How to share a plugin with the community")test_files.zipand you added the default tests for that mimetype in test_classes.py.FREE_TO_USE_ANALYZERSplaybook by following this guide.urlthat contains this information. This is required for Health Checks (HEAD HTTP requests)._monkeypatch()was used in its class to apply the necessary decorators.MockUpResponseof the_monkeypatch()method. This serves us to provide a valid sample for testing.DataModelfor the new analyzer following the documentation# This file is a part of IntelOwl https://github.com/intelowlproject/IntelOwl # See the file 'LICENSE' for copying permission.Black,Flake,Isort) gave 0 errors. If you have correctly installed pre-commit, it does these checks and adjustments on your behalf.testsfolder). All the tests (new and old ones) gave 0 errors.DeepSource,Django Doctorsor other third-party linters have triggered any alerts during the CI checks, I have solved those alerts.Important Rules