Fill trigger histograms from many files in a directory

[GernotMaier]

Extend simtools-write-trigger-histograms with a directory mode that discovers reduced-event HDF5 files, groups *.part.reduced_event_data.hdf5 files by their shared
filename stem, and produces one trigger-histogram HDF5 file per group. Scheduler backends submit one independent job per group and return immediately.

Implementation changes

• Add --event_data_directory as a new input mode, mutually exclusive with the existing --event_data_files mode.

  • Scan only direct children matching *.reduced_event_data.hdf5.
  • Derive a group key by removing .part immediately before the suffix.
  • Include unpartitioned files as one-file groups; combine matching partitioned and unpartitioned names under the same group key.
  • Fail clearly when no matching files are found.

• In directory mode, write each group to --output_path/.trigger_histograms.hdf5; do not require or use --output_file.

• Preserve existing --event_data_files behavior: explicit patterns remain separate production references in one requested output file.

• Refactor the per-production work so one group can fully write its own product. Use the job-execution submission API rather than the blocking map path in directory mode.

  • Local backend remains synchronous.
  • HTCondor submits one job per discovered group and exits after recording the submission manifest.
  • Each job has a unique output path and owns its HDF5 file, avoiding concurrent writes to a shared product.
  • Keep the existing telescope-selection, binning, invalid-input, metadata, dense-payload, and output-validation behavior per generated file.

• Update the application and production-prerequisites documentation with a directory-mode example and scheduler example.

Public interface

• New: --event_data_directory PATH.
• Directory mode requires --output_path; it derives output names automatically.
• Existing --event_data_files … --output_file … remains supported and unchanged.
• Reject incompatible combinations with actionable CLI errors.

Assumptions

• “Same parameter space” is encoded by the filename convention: identical names after stripping .part.
• Input directories are flat; nested directories are intentionally not scanned.
• The requested “single file” means one final trigger-histogram file per parameter-space group, not one global catalog spanning all groups.
• No post-submission merge job is created; the scheduler manifest is the handoff for monitoring submitted group jobs.

[ctao-sonarqube]

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
89.7% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube