Add XML support for Checkmarx CxFlow SAST parser and regression test#15049
Add XML support for Checkmarx CxFlow SAST parser and regression test#15049goutham-hari wants to merge 48 commits into
Conversation
….0.0-3.1.0-dev Release: Merge back 3.0.0 into dev from: master-into-dev/3.0.0-3.1.0-dev
…jo#14962) * Added global required fields notice for WCAG H90 compliance * display_tags.py add and removed blank line * Added setting/env variable DD_SHOW_A11Y_REQUIRED_FIELDS_NOTICE --------- Co-authored-by: symon.vezina <symon.vezina@hrsdc-rhdcc.gc.ca>
….0.1-3.1.0-dev Release: Merge back 3.0.1 into dev from: master-into-dev/3.0.1-3.1.0-dev
* perf(importers): batch vulnerability_id inserts Replace per-row Vulnerability_Id saves with bulk_create in two layers: - fix sanitize_vulnerability_ids to return filtered list (was a no-op bug — reassigned local variable, caller never saw the result) - save_vulnerability_ids now uses bulk_create per finding instead of one INSERT per ID; fixes all callers including the reimporter path - DefaultImporter.store_vulnerability_ids accumulates Vulnerability_Id objects across all findings in a batch; flush_vulnerability_ids() does a single bulk_create at each batch boundary (alongside location_handler.persist()) For a scan with 1000 findings × 5 CVEs each: 5000 INSERT queries reduced to O(batches) bulk_create calls. * perf(reimporter): batch vulnerability_id reconciliation Extend the cross-finding accumulation pattern to DefaultReImporter: - reconcile_vulnerability_ids now accumulates changed findings into pending_vuln_id_deletes / pending_vulnerability_ids instead of issuing per-finding DELETE + INSERT immediately - flush_vulnerability_ids (BaseImporter) runs one bulk DELETE WHERE finding_id IN (...) followed by one bulk_create for all new IDs - flush called at both dedupe batch boundaries (alongside location_handler.persist()) and after the mitigation loop Early-exit path (unchanged IDs) never touches either buffer, so the common case pays zero extra cost. Add two unit tests: cross-finding batch (3 findings, 2 changed + 1 unchanged, verify buffer contents before flush and DB state after) and unchanged-IDs early-exit (verify buffers stay empty). * test(performance): re-baseline importer query counts Remove pending-rebaseline skips from TestDojoImporterPerformanceSmall and TestDojoImporterPerformanceSmallLocations. Update all expected query counts to reflect the batch Vulnerability_Id insert optimisation (counts decrease by 1-20 queries per step depending on the scan size and code path). * fix(test): update TestSaveVulnerabilityIds mock for bulk_create The test mocked Vulnerability_Id.save (individual saves) but save_vulnerability_ids now uses bulk_create. Django's bulk_create validates FK references before issuing SQL, raising ValueError when the finding has no pk. Mock bulk_create instead and assert on the deduplicated object list passed to it. --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
…f tests (DefectDojo#14969) * perf(importers): batch BurpRawRequestResponse inserts + re-enable perf tests Replace per-finding save() calls in process_request_response_pairs with bulk_create at batch boundaries, mirroring the location_handler pattern. Reduces DB round-trips proportionally to findings with req/resp data. Drops the no-op clean() calls (BurpRawRequestResponse has no custom clean). Re-enable TestDojoImporterPerformanceSmall and TestDojoImporterPerformanceSmallLocations with recalibrated query counts after the RBAC→legacy authorization migration. * test(perf): recalibrate tag inheritance ZAP query counts Batch BurpRawRequestResponse inserts reduce per-finding saves for the ZAP parser (which emits req/resp pairs). Update expected counts to match.
…ocker-compose.yml) (DefectDojo#15022) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
…(.github/workflows/release-x-manual-tag-as-latest.yml) (DefectDojo#14990) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Cody Maffucci <46459665+Maffooch@users.noreply.github.com>
…v3.0.3 to v3.1.0 (.github/workflows/detect-merge-conflicts.yaml) (DefectDojo#14992) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Cody Maffucci <46459665+Maffooch@users.noreply.github.com>
….23.1 (docker-compose.override.dev.yml) (DefectDojo#14997) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Cody Maffucci <46459665+Maffooch@users.noreply.github.com>
Bumps [ruff](https://github.com/astral-sh/ruff) from 0.15.15 to 0.15.16. - [Release notes](https://github.com/astral-sh/ruff/releases) - [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md) - [Commits](astral-sh/ruff@0.15.15...0.15.16) --- updated-dependencies: - dependency-name: ruff dependency-version: 0.15.16 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Cody Maffucci <46459665+Maffooch@users.noreply.github.com>
…Dojo#14996) Bumps [pdfmake](https://github.com/bpampuch/pdfmake) from 0.3.8 to 0.3.10. - [Release notes](https://github.com/bpampuch/pdfmake/releases) - [Changelog](https://github.com/bpampuch/pdfmake/blob/master/CHANGELOG.md) - [Commits](bpampuch/pdfmake@0.3.8...0.3.10) --- updated-dependencies: - dependency-name: pdfmake dependency-version: 0.3.10 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Cody Maffucci <46459665+Maffooch@users.noreply.github.com>
Bumps [vcrpy](https://github.com/kevin1024/vcrpy) from 8.1.1 to 8.2.1. - [Release notes](https://github.com/kevin1024/vcrpy/releases) - [Changelog](https://github.com/kevin1024/vcrpy/blob/master/docs/changelog.rst) - [Commits](kevin1024/vcrpy@v8.1.1...v8.2.1) --- updated-dependencies: - dependency-name: vcrpy dependency-version: 8.2.1 dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [sqlalchemy](https://github.com/sqlalchemy/sqlalchemy) from 2.0.50 to 2.0.51. - [Release notes](https://github.com/sqlalchemy/sqlalchemy/releases) - [Changelog](https://github.com/sqlalchemy/sqlalchemy/blob/main/CHANGES.rst) - [Commits](https://github.com/sqlalchemy/sqlalchemy/commits) --- updated-dependencies: - dependency-name: sqlalchemy dependency-version: 2.0.51 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…3 (.github/workflows/test-helm-chart.yml) (DefectDojo#15021) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Bumps [@tailwindcss/cli](https://github.com/tailwindlabs/tailwindcss/tree/HEAD/packages/@tailwindcss-cli) from 4.3.0 to 4.3.1. - [Release notes](https://github.com/tailwindlabs/tailwindcss/releases) - [Changelog](https://github.com/tailwindlabs/tailwindcss/blob/main/CHANGELOG.md) - [Commits](https://github.com/tailwindlabs/tailwindcss/commits/v4.3.1/packages/@tailwindcss-cli) --- updated-dependencies: - dependency-name: "@tailwindcss/cli" dependency-version: 4.3.1 dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps vulners from 3.1.10 to 3.1.11. --- updated-dependencies: - dependency-name: vulners dependency-version: 3.1.11 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…jo#15040) Move the Finding Groups links (Open/All/Closed Findings Groups) into the Findings dropdown in the new-UI sidebar and remove the standalone top-level "Finding Groups" nav item. A subtle divider separates the groups links within the dropdown. This declutters the top-level navigation and keeps related finding views together.
…ectDojo#15006) Trivy legacy-format reports have no "Class" field, so the parser set unsaved_tags entries to None (e.g. ['debian', None]). clean_tags then crashed the whole import with TypeError in TAG_PATTERN.sub, after parsing had already succeeded (regression from DefectDojo#14111 in 2.55.0). - clean_tags: drop None entries instead of crashing (defends every parser) - trivy parser: filter falsy values at all four unsaved_tags sites - regression tests: clean_tags None handling; legacy fixture tags contain no None Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
….0.100-3.1.0-dev Release: Merge back 3.0.100 into dev from: master-into-dev/3.0.100-3.1.0-dev
…efectDojo#14999) Several new-UI templates hardcoded brand colors that should reference the design tokens defined in components/tailwind.css: - calendar.html: active engagement/test events used the legacy Bootstrap primary blue (#337ab7) instead of the brand blue. - benchmark.html: table link color, same legacy #337ab7. - base.html: sidebar background (DefectDojo#3864) and sub-nav link color (#82B0D9) hardcoded the Fuji Blue brand hues. These now use the matching var(--color-dd-primary-*) tokens (500/900/200). Token values are identical to the hardcoded hex, so there is no visual change. Intentionally left as-is: - The sidebar lighter link color (#C6DDF2): the matching shade (dd-primary-100) is not emitted in the compiled CSS, since Tailwind v4 only outputs theme variables referenced by a generated utility. - PDF report templates also contain #337ab7, but CSS custom properties do not resolve in the PDF renderer. - Generic white/black, neutral grays, and the custom #002a4d shade have no design token. FullCalendar 3.10.5 applies the event color verbatim as an inline style, so the variable resolves at render time. Verified in the running new UI: sidebar bg = rgb(0,56,100), sub-nav link = rgb(130,176,217), calendar event = rgb(23,121,197) -- each identical to the hex it replaced.
…gin page (DefectDojo#14998) The new Tailwind login page hardcoded hex color values that exactly duplicate the design tokens defined in components/tailwind.css. Swap them for the corresponding var(--color-*) tokens so the page stays in sync with the design system if the palette changes. The token values are identical to the previously hardcoded hex, so there is no visual change. The control-label color (#333333) and the alpha-channel rgba() shadows are left as-is since they have no exact token equivalent.
…yml) (DefectDojo#14896) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
…fectdojo/chart.yaml) (DefectDojo#14917) * chore(deps): update valkey docker tag from 0.20.2 to v0.22.1 (helm/defectdojo/chart.yaml) * update Helm documentation --------- Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
…ithub/workflows/test-helm-chart.yml) (DefectDojo#15067) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
…github/workflows/k8s-tests.yml) (DefectDojo#15068) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
…ile.integration-tests-debian) (DefectDojo#15069) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
… v3.0.1 (.github/workflows/release-x-manual-helm-chart.yml) (DefectDojo#15070) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
…tDojo#15071) Bumps [django-debug-toolbar](https://github.com/django-commons/django-debug-toolbar) from 6.3.0 to 7.0.0. - [Release notes](https://github.com/django-commons/django-debug-toolbar/releases) - [Changelog](https://github.com/django-commons/django-debug-toolbar/blob/main/docs/changes.rst) - [Commits](django-commons/django-debug-toolbar@6.3.0...7.0.0) --- updated-dependencies: - dependency-name: django-debug-toolbar dependency-version: 7.0.0 dependency-type: direct:development update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [ruff](https://github.com/astral-sh/ruff) from 0.15.16 to 0.15.19. - [Release notes](https://github.com/astral-sh/ruff/releases) - [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md) - [Commits](astral-sh/ruff@0.15.16...0.15.19) --- updated-dependencies: - dependency-name: ruff dependency-version: 0.15.19 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
) Bumps [django-environ](https://github.com/joke2k/django-environ) from 0.13.0 to 0.14.0. - [Release notes](https://github.com/joke2k/django-environ/releases) - [Changelog](https://github.com/joke2k/django-environ/blob/v0.14.0/CHANGELOG.rst) - [Commits](joke2k/django-environ@v0.13.0...v0.14.0) --- updated-dependencies: - dependency-name: django-environ dependency-version: 0.14.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [redis](https://github.com/redis/redis-py) from 8.0.0 to 8.0.1. - [Release notes](https://github.com/redis/redis-py/releases) - [Changelog](https://github.com/redis/redis-py/blob/master/CHANGES) - [Commits](redis/redis-py@v8.0.0...v8.0.1) --- updated-dependencies: - dependency-name: redis dependency-version: 8.0.1 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…tDojo#15075) Bumps [pdfmake](https://github.com/bpampuch/pdfmake) from 0.3.10 to 0.3.11. - [Release notes](https://github.com/bpampuch/pdfmake/releases) - [Changelog](https://github.com/bpampuch/pdfmake/blob/master/CHANGELOG.md) - [Commits](bpampuch/pdfmake@0.3.10...0.3.11) --- updated-dependencies: - dependency-name: pdfmake dependency-version: 0.3.11 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
… (.github/workflows/test-helm-chart.yml) (DefectDojo#15076) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
…/workflows/validate_docs_build.yml) (DefectDojo#15077) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
….23.1 (docker-compose.override.integration_tests.yml) (DefectDojo#15078) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
…m v7.22.0 to v7.23.0 (dockerfile.integration-tests-debian) (DefectDojo#15079) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
7a55ed9 to
007df3b
Compare
DescriptionExtend the Checkmarx CxFlow SAST parser to support both JSON and XML input formats. Changes
MotivationThe Checkmarx CxFlow SAST parser previously only supported JSON export format, with a TODO comment indicating XML support was needed. This change enables users to import Checkmarx reports in both JSON and XML formats, providing greater flexibility in vulnerability management workflows. Implementation Details
TestingThe existing unit tests continue to pass for JSON inputs. XML support leverages the established Checkmarx XML parser, which is independently tested through Related IssuesResolves the |
Maffooch
left a comment
There was a problem hiding this comment.
@goutham-hari please rebase the PR to the dev branch, and correct unit test failures
|
This pull request has conflicts, please resolve those before we can evaluate the pull request. |
Description
The Checkmarx CxFlow SAST parser (
dojo/tools/checkmarx_cxflow_sast/parser.py)previously only supported JSON report imports — XML files hit a TODO stub and
returned no findings.
This PR adds XML support by delegating to the existing
CheckmarxParser(
dojo/tools/checkmarx/parser.py, detailed mode) instead of duplicating itsparsing logic.
Test results
Added
test_xml_parse_file_with_single_vulnerability_has_single_findinginunittests/tools/test_checkmarx_cxflow_sast_parser.py, using the existingsample file
checkmarx/single_finding.xml. Verifies a single finding isreturned with the correct title, "High" severity, active=True, verified=False.
Documentation
No documentation changes — this extends file-format support within the existing
Checkmarx CxFlow SAST parser; no new parser/page is introduced.
Extra information
No pre-approval issue was filed for this one, given the narrow scope (closing a
TODO/format gap in an existing parser rather than adding a new integration).
Happy to open one if a maintainer would prefer that first.