feat(irr): add IRR data support with RPSL parsing and AS info enrichment - #39
Open
digizeph wants to merge 3 commits into
Open
feat(irr): add IRR data support with RPSL parsing and AS info enrichment#39digizeph wants to merge 3 commits into
digizeph wants to merge 3 commits into
Conversation
Add a new irr module and extend asinfo with IRR and delegated-stats data. Uses rpsl-rs (v3.0) for RPSL parsing and oneio for remote/FTP downloads. New irr module (library-level toolkit): - sources.rs: 14 known IRR registries (RIPE, APNIC, ARIN, LACNIC, AFRINIC, NTTCOM, RADB, ALTDB, BELL, BBOI, JPIRR, TC, CANARIE, REACH) with per-registry dump URLs, transport (HTTPS/FTP), and format (split files vs whole-DB) - types.rs: Typed structs for 7 RPSL object types (AutNum, Route, AsSet, RouteSet, Mntner, Organisation) with serde serialization - extract.rs: Extracts typed objects from rpsl::Object<Raw>, handles comma-separated members, case-insensitive ASN parsing, inline comments - stream.rs: Streaming parser via oneio with read_until for UTF-8 robustness, blank-line object splitting, lossy conversion for legacy IRR data with non-UTF-8 bytes Extended asinfo module: - DelegatedInfo: structured RIR delegated-stats data (registry, country, date, status) attached to every allocated ASN - IrrAsnInfo: per-source IRR data (as-name, descr, mnt-by, route prefixes, route6 prefixes, as-set memberships) as a Vec so callers pick which source(s) to trust - AsInfoProfile enum (Minimum, Default, Full) for curated presets - AsInfoBuilder retained for fine-grained control: with_delegated(), with_irr(), with_irr_sources(IrrSourceConfig), with_irr_route_prefixes() - IRR route prefix collection is opt-in (off by default) to control output size (~90MB savings when disabled) - Unknown ASN names filled from IRR aut-num as-name when available (363 remaining UNKNOWN out of ~999 delegated-stats gap ASNs) Performance (release build, live data): - asn.txt only: ~1s - Default profile: ~30s - Full profile: ~75s (AFRINIC 17.5s, RADB 10.3s are slowest sources) - Whole-DB sources download once (single-pass extraction) - Split-file sources skip route/route6 files when prefixes not requested Output sizes: - Minimum: ~37 MB JSONL (6.3 MB gzipped) - Default: ~50 MB JSONL - Full: ~210 MB JSONL (31 MB gzipped) Integration tests verified with live IRR data: RIPE aut-num (AS13335, AS15169, AS1299), RIPE routes, RADB FTP routes, ARIN whole-DB, APNIC. Closes #38
digizeph
marked this pull request as ready for review
July 30, 2026 23:34
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add IRR (Internet Routing Registry) data support to bgpkit-commons, including a new
irrmodule for parsing RPSL data and extendingasinfowith structured IRR and delegated-stats enrichment.New
irrmoduleLibrary-level RPSL parsing toolkit using rpsl-rs v3.0:
AutNum,Route,Route6,AsSet,RouteSet,Mntner,OrganisationExtended
asinfomoduleDelegatedInfostruct: structured RIR delegated-stats data (registry, country, date, status) for every allocated ASNIrrAsnInfostruct: per-source IRR data (as-name, descr, mnt-by, route prefixes, route6 prefixes, as-set memberships) stored as aVecso callers pick which source(s) to trustAsInfoProfileenum with three presets:Minimum: asn.txt only (~1s load)Default: + as2org, population, hegemony, peeringdb (~30s) — matches current productionFull: + delegated stats + IRR data with route prefixes (~75s)AsInfoBuilderfor fine-grained control:.with_irr_sources(),.with_irr_route_prefixes(), etc.Breaking changes
load_asinfo(bool, bool, bool, bool)→load_asinfo_with_profile(AsInfoProfile)AsInfogainsdelegated: Option<DelegatedInfo>andirr: Vec<IrrAsnInfo>fieldsasinfofeature now depends onirrfeaturePerformance (live data, release build)
Output sizes
Test plan
cargo fmt --checkcleancargo clippy --all-features -- -D warningscleancargo build --no-default-featuresand--features irrboth compileCloses #38