Skip to content

Add isolde rules - #572

Open
zzacharo wants to merge 4 commits into
masterfrom
add-isolde-rules
Open

Add isolde rules#572
zzacharo wants to merge 4 commits into
masterfrom
add-isolde-rules

Conversation

@zzacharo

@zzacharo zzacharo commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Comment thread cds_migrator_kit/rdm/streams.yaml

Returns the model instance if found, otherwise None.
"""
models = _load_models(self.dojson_model.entry_point_models)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you don't have to load it again, it is loaded somewhere in the app already, if you pass the loaded models in the exception then we could just select by the key

Comment thread scripts/snapshot.sh
@zzacharo
zzacharo force-pushed the add-isolde-rules branch 3 times, most recently from 5590ede to 38ce7b1 Compare August 7, 2026 11:53

__query__ = '693__.a:"CERN ISOLDE" AND (980__:ARTICLE OR 980__:PREPRINT OR 980__:conferencepaper OR 980__:NOTE OR 980__:REPORT) -980__:DELETED -980__.c:MIGRATED -980__:DUMMY'

__ignore_keys__ = {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could we import and concatenate with the keys specified in _config.py?

# YYYY convention but are still migrated as apprn. Post-migration,
# only super-admins can edit those records. New submissions on the
# production app use the strict is_approval_report_number validator.
"validator": always_valid,

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that would require maybe change according to the solution we discussed IRL

cli_logger.info(
f"[apprn] Found approval report number {identifier!r} — storing as apprn"
)
scheme = "apprn"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure about adding them as apprn in the rule, if we' do this we need to change ep-approval workflow, see: restricted entry, public entry. We need to separate the report numbers somehow, for example faser records both have cern-ep-draft- and cern-ep- rep numbers, will this going to keep cern-ep-draft- as cdsrn?

I think we should raise if there's a ep report number and no ep information to check if there is any other record exists in the system. If we say we'll fix it when we have the error (when we found the internal record or etc) it might be more manual work to fix, wdyt?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DIscussed IRL: we should add a step to check for all the identified EP numbers and collect the duplicates

class ISOLDEModel(CdsOverdo):
"""Translation model for ISOLDE."""

__query__ = '693__.a:"CERN ISOLDE" AND (980__:ARTICLE OR 980__:PREPRINT OR 980__:conferencepaper OR 980__:NOTE OR 980__:REPORT) -980__:DELETED -980__.c:MIGRATED -980__:DUMMY'

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
__query__ = '693__.a:"CERN ISOLDE" AND (980__:ARTICLE OR 980__:PREPRINT OR 980__:conferencepaper OR 980__:NOTE OR 980__:REPORT) -980__:DELETED -980__.c:MIGRATED -980__:DUMMY'
__query__ = '693__.a:"CERN ISOLDE" AND (980__:ARTICLE OR 980__:PREPRINT OR 980__:conferencepaper OR 980__:NOTE OR 980__:REPORT) -980__:DELETED -980__:DUMMY'

raise IgnoreKey("title")


_FUNDING_MODEL_MAP = {

@kpsherva kpsherva Aug 19, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should it be moved to config.py in transform folder? also the mapping is a bit strange...

"intnoteatspubl": {"id": "publication-note"},
"intnotehie-isoldepubl": {"id": "publication-note"},
"hie-isolde-project-notes": {"id": "publication-note"},
"intnotebepubl": {"id": "publication-note"},

@kpsherva kpsherva Aug 19, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

records from BE shouldn't be in this data set, unless ISOLDE owns them, same with ATS above (to be checked)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

migration: ISOLDE

3 participants