Consistent version naming - #2466
Draft
GernotMaier wants to merge 11 commits into
Draft
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Migrates dependency and database selectors to explicit version, tag, revision, and build-ID terminology across simtools.
Changes:
- Adds dependency catalog schema 0.3 and canonical tag-based interfaces.
- Normalizes legacy CLI, metadata, and build-option names.
- Updates containers, workflows, documentation, and tests.
Reviewed changes
Copilot reviewed 44 out of 44 changed files in this pull request and generated 10 comments.
Show a summary per file
| File | Description |
|---|---|
tests/unit_tests/test_version.py |
Tests release-tag validation. |
tests/unit_tests/test_dependency_versions.py |
Tests catalog migration and matrices. |
tests/unit_tests/test_dependencies.py |
Updates canonical build-option keys. |
tests/unit_tests/simtel/test_simtel_validate_metadata.py |
Tests legacy metadata validation. |
tests/unit_tests/simtel/test_simtel_io_metadata.py |
Tests metadata-name normalization. |
tests/unit_tests/simtel/test_simtel_config_writer.py |
Updates build-ID validation test. |
tests/unit_tests/db/test_mongo_db.py |
Updates database tag fixtures. |
tests/unit_tests/db/test_db_handler.py |
Updates tag-based database calls. |
tests/unit_tests/conftest.py |
Renames database fixture fields. |
tests/unit_tests/configuration/test_configurator.py |
Tests aliases and tag validation. |
tests/unit_tests/application/test_control.py |
Updates version-info build keys. |
tests/integration_tests/conftest.py |
Supports canonical database tag environment. |
tests/conftest.py |
Adds test-resource tag aliases. |
src/simtools/version.py |
Adds release-tag validation. |
src/simtools/simtel/simtel_validate_metadata.py |
Normalizes metadata before validation. |
src/simtools/simtel/simtel_io_metadata.py |
Adds legacy metadata adaptation. |
src/simtools/schemas/sim_telarray_meta_parameters.schema.yml |
Registers canonical metadata names. |
src/simtools/schemas/dependency_versions.schema.yml |
Adds catalog schema 0.3. |
src/simtools/dependency_versions.py |
Adapts catalogs and workflow outputs. |
src/simtools/dependencies.py |
Normalizes build and database metadata. |
src/simtools/db/mongo_db.py |
Renames database version inputs. |
src/simtools/db/db_model_upload.py |
Passes canonical database tags. |
src/simtools/db/db_handler.py |
Uses tag terminology internally. |
src/simtools/configuration/configurator.py |
Normalizes aliases and validates tags. |
src/simtools/configuration/commandline_parser.py |
Registers CLI aliases. |
src/simtools/configuration/arguments.py |
Defines canonical database tag option. |
src/simtools/applications/db_upload_model_repository.py |
Uses database tag arguments. |
src/simtools/applications/db_generate_compound_indexes.py |
Passes database tags. |
src/simtools/applications/db_add_value_from_json_to_db.py |
Renames test database selector. |
src/simtools/applications/db_add_simulation_model_from_repository_to_db.py |
Requires canonical database tags. |
src/simtools/application/control.py |
Reports canonical build keys. |
docs/source/user-guide/applications/simtools-db-upload-model-repository.md |
Adds tag-based examples. |
docs/source/user-guide/applications/simtools-db-add-simulation-model-from-repository-to-db.md |
Documents database tags. |
docs/source/developer-guide/testing_integration.md |
Documents test-resource tags. |
docs/source/components/databases.md |
Documents database tag aliases. |
docker/Dockerfile-simtel_array |
Renames dependency build arguments. |
docker/Dockerfile-corsika7 |
Separates source tags and build IDs. |
dependency_versions.yml |
Migrates catalog data to schema 0.3. |
.github/workflows/CI-unittests.yml |
Exports canonical database tags. |
.github/workflows/CI-test-benchmarks.yml |
Updates benchmark environment names. |
.github/workflows/CI-integrationtests.yml |
Uses canonical catalog tag outputs. |
.github/workflows/build-simtools-prod.yml |
Updates production matrix fields. |
.github/workflows/build-sim_telarray.yml |
Uses tag-based build arguments. |
.github/workflows/build-corsika7.yml |
Uses source tags and build IDs. |
Suppressed comments (2)
src/simtools/schemas/dependency_versions.schema.yml:350
- The sim_telarray component schema only requires non-empty strings, so bare branch names such as
mastervalidate astag,hessio-tag, andstdtools-tag. All three are catalog-managed release tags and should reference the 0.3 tag definition.
required: [tag, source-url, hessio-tag, hessio-source-url, stdtools-tag, stdtools-source-url]
properties:
tag: {type: string, minLength: 1}
dependency_versions.yml:42
- This schema-0.3 CORSIKA record also needs its explicit legacy build identifier; deriving it from
v7.8050is outside the required catalog contract.
- tag: v7.8050
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+215
to
+217
| def is_valid_release_tag(version_string): | ||
| """Return whether a value is a ``v``-prefixed semantic release tag.""" | ||
| return isinstance(version_string, str) and bool(RELEASE_TAG_PATTERN.fullmatch(version_string)) |
Comment on lines
214
to
+218
| def _validate_components(catalog, schema_version): | ||
| """Validate CORSIKA and sim_telarray component records.""" | ||
| for component in catalog["corsika"]: | ||
| if component.get("source-ref") in {"latest", "master", "main"}: | ||
| raise ValueError("CORSIKA source-ref must identify a release.") | ||
| _validate_corsika_components(catalog["corsika"]) | ||
| _validate_simtel_components(catalog["sim-telarray"]) | ||
| _validate_model_and_test_components(catalog, schema_version) |
Comment on lines
+311
to
+313
| if catalog["schema_version"] in {"0.2.0", "0.3.0"}: | ||
| try: | ||
| simtools_version.validate_release_tag(database_tag) |
|
|
||
| @staticmethod | ||
| def get_db_name(db_name=None, db_simulation_model_version=None, model_name=None): | ||
| def get_db_name(db_name=None, db_simulation_model_tag=None, model_name=None): |
|
|
||
| ## Examples | ||
|
|
||
| Upload a released repository version (includes cloning the simulations-models repository): |
Comment on lines
+34
to
+38
| if "version" in component: | ||
| return component["version"] | ||
| tag = _corsika_tag(component) | ||
| if CORSIKA_TAG_PATTERN.fullmatch(tag or "") is None: | ||
| raise ValueError( |
Comment on lines
+332
to
+334
| required: [tag, source-url, config-tag, config-source-url, opt-patch-tag, opt-patch-source-url] | ||
| properties: | ||
| tag: {type: string, pattern: "^v[0-9]+\\.[0-9]+$"} |
| corsika: | ||
| - version: "78010" | ||
| source-ref: v7.8010 | ||
| - tag: v7.8010 |
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Addresses and closes #2463