Support for prowler scan

[cosmel-dojo]

Prowler Scan Parser for DefectDojo

Description

This PR adds support for importing security scan results from Prowler - a security assessment and compliance tool for AWS, Azure, GCP, and Kubernetes. The parser supports both CSV and JSON output formats from Prowler scans.

Key features implemented:

• Support for all major cloud platforms (AWS, Azure, GCP, Kubernetes)
• Handle both CSV and JSON formats with automatic detection
• Extract critical metadata including severity, resource information, and remediation steps
• Properly map Prowler severity levels to DefectDojo severity levels
• Handle both active and informational findings based on status codes

The implementation follows the best practices from the parser guide and mimics the structure of other cloud security scan parsers in DefectDojo.

Code Quality Improvements

After initial implementation, I further refactored the parser to follow best practices seen in other DefectDojo parsers (like AnchoreCTLPoliciesParser):

• Removed special test-handling logic to focus solely on parsing files
• Simplified method parameters for better maintainability
• Enhanced code quality by removing conditional branches
• Better adhered to the Single Responsibility Principle
• Created more resilient tests that validate actual parser output

These changes result in cleaner, more maintainable code that's consistent with other parsers in the codebase.

Test results

Comprehensive test coverage has been implemented:

1. File-based tests for each cloud provider in both CSV and JSON formats (test_prowler_parser.py)
2. In-memory tests using StringIO to test edge cases without file I/O dependencies (test_prowler_stringio.py)

Test coverage includes:

• Parsing validation for all supported cloud providers
• Format detection and handling
• CSV delimiter detection (semicolon vs comma)
• Field extraction and mapping
• Severity and status mapping
• Edge cases like empty files or missing fields

All tests pass successfully on Python 3.11.

How to test this implementation

To test this implementation, follow these steps:

1. Set up the testing environment:

# First, make sure the testing environment is running
docker compose -f docker-compose.yml -f docker-compose.override.unit_tests.yml up -d

2. Run the file-based parser tests:

./run-unittest.sh --test-case unittests.tools.test_prowler_parser

3. Run the StringIO-based parser tests:

./run-unittest.sh --test-case unittests.tools.test_prowler_stringio.TestProwlerStringIOParser

Both test suites should complete successfully with no failures, validating the parser's functionality across all supported cloud providers and formats.

Documentation

Added sample scan files for all supported cloud providers and formats in the unittests/scans/prowler/ directory to serve as examples for users. These files demonstrate the expected structure and required fields for each format.

Checklist

• PR rebased against the latest dev branch
• Feature submitted against dev branch
• Code is Python 3.11 compliant
• Code is flake8/ruff compliant (fixed linting issues)
• Added unit tests to verify functionality
• Added sample files demonstrating expected input formats
• No model changes required (uses existing Finding model)
• Proper label: Import Scans

[dryrunsecurity]

This pull request contains multiple security concerns across the Prowler parser and test files, including potential information leakage, input validation weaknesses, and Kubernetes configuration risks that could expose sensitive system information and create potential security vulnerabilities.

💭 Unconfirmed Findings (6)

Vulnerability	Potential Information Leakage in Prowler Parser
Description	Uncontrolled metadata extraction in dojo/tools/prowler/parser.py might expose sensitive information like resource names and configurations, leading to potential information disclosure risks.

Vulnerability	Lack of Strict Input Validation in Parser
Description	Minimal validation on input data in dojo/tools/prowler/parser.py could allow malformed inputs to be processed, with .get() method potentially enabling unexpected parsing behavior.

Vulnerability	Broad File Type Detection
Description	Automatic file type detection in dojo/tools/prowler/parser.py might trigger incorrect parsing by attempting JSON parsing first and falling back to CSV, potentially causing unexpected parsing outcomes.

Vulnerability	Potential Information Exposure via Placeholder Values
Description	Unsanitized placeholders like '<account_uid>' in unittests/scans/prowler/aws.json could reveal internal system information and identifier structures.

Vulnerability	Potential API Key Security Risks
Description	API keys in unittests/scans/prowler/gcp.json might be simple encrypted strings with limited identification, potentially enabling easy key discoverability and weak authentication.

Vulnerability	Kubernetes Security Findings
Description	Multiple security risks in Kubernetes configurations, including missing AlwaysPullImages admission control, potential unauthorized pod image usage, and possible anonymous API server access.

---

All finding details can be found in the DryRun Security Dashboard.

[valentijnscholten]

@cosmel-dojo Some remarks:

Could you explain a bit about test_mode what it does and why it is needed?
Is the StringIO test really needed, does it test for something that the other filebases tests do not test?
I notice there's already AWS Prowler v3 and v4 parsers. Should these be removed/deprecated/merged into this/one prowler parser?

[cosmel-dojo]

Could you explain a bit about test_mode what it does and why it is needed?
Is the StringIO test really needed, does it test for something that the other filebases tests do not test?
I notice there's already AWS Prowler v3 and v4 parsers. Should these be removed/deprecated/merged into this/one prowler parser?

Hey @valentijnscholten

Thank you for your questions! I've actually made some significant improvements to the parser since my original implementation.

Regarding test_mode:
After careful consideration and following best practices from other parsers in DefectDojo (like AnchoreCTLPoliciesParser), I've completely refactored the parser to remove the special test handling logic. The parser now:

• No longer has a test_mode parameter
• Processes files consistently regardless of context (test or production)
• Follows the Single Responsibility Principle by focusing solely on parsing
• Has cleaner, more maintainable code with fewer conditional branches

This change makes the code simpler, more maintainable, and consistent with other parsers in the codebase.

Regarding the StringIO test:
Yes, the StringIO test is still valuable as it specifically validates that the parser can handle in-memory file-like objects, not just disk files. This ensures:

• The parser works when data comes from memory buffers or network streams
• It properly handles UTF-8 encoding in these scenarios
• It can process both CSV and JSON data properly from in-memory sources

While file-based tests verify most functionality, the StringIO test ensures the parser works in all contexts, including when integrated with other components that might pass in-memory data.

Regarding the AWS Prowler parsers:
The existing aws_prowler and aws_prowler_v3plus parsers are more specialized for specific versions of AWS Prowler output, while this new prowler parser is a universal parser that handles:

• Multiple cloud providers (AWS, Azure, GCP, and Kubernetes) in a single parser
• Both CSV and JSON formats in a consolidated way
• The latest OCSF JSON format along with traditional formats

Rather than deprecating the existing parsers immediately, it makes sense to:

• Keep the existing parsers for backward compatibility with scans already in the system
• Document that new users should use the universal Prowler parser
• Consider a deprecation timeline or migration path for the older parsers in the future

This approach ensures we don't break existing deployments while moving toward a more consolidated, maintainable codebase for Prowler parsing.