Skip to content

fix(parsing): pin machine-format number parses to the invariant culture - #893

Open
m4bard wants to merge 1 commit into
Listenarrs:canaryfrom
m4bard:fix/796-culture-parse
Open

fix(parsing): pin machine-format number parses to the invariant culture#893
m4bard wants to merge 1 commit into
Listenarrs:canaryfrom
m4bard:fix/796-culture-parse

Conversation

@m4bard

@m4bard m4bard commented Aug 24, 2026

Copy link
Copy Markdown

Fixes #796.

ffprobe, MyAnonamouse, SABnzbd and Torznab all emit numbers in machine format: . is the decimal separator whatever locale the emitting process runs under. Six parses read those strings with the ambient culture, so what the server records depends on the server's locale.

Measured under de-DE and fr-FR, on the values from the issue:

site value de-DE fr-FR
FfprobeMetadataMapper.cs:50 43200.250 seconds 43200250 seconds (500 days) not parsed, duration 0
MyAnonamouseSizeParser.cs:36 1.5 GB 16106127360 0
MyAnonamouseSizeParser.cs:49 1.5 GB 16106127360 0
SabnzbdResponseMapper.cs:201 1.5 M 15728640 B/s 0
SabnzbdResponseMapper.cs:320 1.5 mb 15728640 B 0
TorznabNewznabValueParser.cs:40 1.5 GB 16106127360 0

A default container runs under the invariant culture and is unaffected. It takes a real culture reaching the process, which happens when LANG or LC_ALL is set, and on a desktop install that inherits the operating system's locale. I have not tested the Windows or macOS builds.

The duration one gets written down: it becomes DurationSeconds on the file record, so a twelve hour book is stored as 500 days.

The shape

I asked on the issue which of three shapes you wanted. Rather than leave it sitting, I picked the smallest one, and it is easy to redo as either of the others if you would rather.

Each site now passes NumberStyles and CultureInfo.InvariantCulture. Where the same file already contained a correct parse of the same kind, the fixed site copies it rather than introducing a second convention:

  • SabnzbdResponseMapper.cs:224 (ParseJsonDouble) already pins NumberStyles.Any. GetDouble at :320 is its near-duplicate and now matches it, as does ParseSpeed at :201.
  • MyAnonamouseSizeParser.cs:73 already pins NumberStyles.Float. :36 and :49 now match it.
  • FfprobeMetadataMapper and TorznabNewznabValueParser have no correct parse in-file, so they follow NzbgetHistoryReader.cs:157, which uses NumberStyles.Float.

Two of the four files already held one pinned parse and one bare parse of the same kind of value, so this reads as drift rather than as a decision anyone made.

What I did not do:

A shared helper, Readarr's shape. TryParseExtensions.ParseDouble does source.Replace(',', '.') before parsing. That is deliberate, and it has a cost: "1,234.5" becomes "1.234.5" and stops parsing, and a thousands-separated "1,234" becomes 1.234. I would not want that on a size field. The helper would also have to live in listenarr.domain to be visible to both listenarr.application and listenarr.infrastructure, since that is the only project both of them reference, and the six sites do not agree on NumberStyles anyway. Say the word and I will write it.

A typed ffprobe reader, Sonarr's shape. That is a dependency decision and it touches #791, so not here.

CA1305. It lights up the DateTime and int sites too, so it belongs in its own change.

Not touched

Audiobook.cs and DownloadImportService.Naming.cs belong to #763, and AudibleSeriesWorkflow.cs:342 to #892. Both of those pin the same way. Leaving them alone keeps this from conflicting with them.

The DateTime parses. ISO 8601 and RFC822 both parse correctly under every culture I tried, and the only failure I could produce needed an ambiguous numeric date, where the right answer depends on what the source emits.

MyAnonamousePublishDateParser.cs:72, :77 and :89 are the same shape, reading ageHours, ageMinutes and age when MyAnonamouse sends them as strings rather than numbers. I left them out because I have not seen MAM emit a decimal there, so unlike the six above I have no measured damage to show. They are three more one-line changes if you want them in.

Tests

MachineFormatCultureParsingTests, 28 cases. Each of the six sites, plus the already-correct SabnzbdResponseMapper.cs:224 as a reference, asserted under the invariant culture, en-US, de-DE and fr-FR.

Control, with only the four production files reverted to canary and the tests left in place, 12 of the 28 fail:

FfprobeDuration              de-DE   expected 12:00:00.2500000   actual 500.00:04:10
FfprobeDuration              fr-FR   expected 12:00:00.2500000   actual 00:00:00
MyAnonamouseFormattedSize    de-DE   expected 1610612736         actual 16106127360
MyAnonamouseFormattedSize    fr-FR   expected 1610612736         actual 0
MyAnonamouseUnusableBytes    de-DE   expected 1610612736         actual 16106127360
MyAnonamouseUnusableBytes    fr-FR   expected 1610612736         actual 0
TorznabSize                  de-DE   expected 1610612736         actual 16106127360
TorznabSize                  fr-FR   expected 1610612736         actual 0
SabnzbdSpeed                 de-DE   expected 1572864            actual 15728640
SabnzbdSpeed                 fr-FR   expected 1572864            actual 0
SabnzbdQueueSlotSize         de-DE   expected 1572864            actual 15728640
SabnzbdQueueSlotSize         fr-FR   expected 1572864            actual 0

With the fix in place all 28 pass. The invariant and en-US cases pass on both sides, so a stock container never saw any of this.

Full suite 3067 passed, 0 failed, 127 skipped, on canary d25b3e11 (1.3.3).

One note on the line numbers. The issue cites v1.2.2, three releases back, but all six sites are unmoved: those four files are byte identical to 4555ad21. The two sites the issue set aside have moved. Audiobook.cs:108 is now :101, and DownloadImportService.cs:371 is now DownloadImportService.Naming.cs:54 since that file was split.

ffprobe, MyAnonamouse, SABnzbd and Torznab all emit numbers with '.' as the
decimal separator whatever locale the emitting process runs under. Six parses
read those strings with the ambient culture, so on a server whose culture
treats '.' as the group separator the value was read as a different number, and
on a server whose decimal separator is ',' it did not parse at all.

Measured under de-DE and fr-FR:

  FfprobeMetadataMapper.cs:50    "43200.250" -> 43200250 s / not parsed
  MyAnonamouseSizeParser.cs:36   "1.5 GB"    -> 15 GB     / 0
  MyAnonamouseSizeParser.cs:49   "1.5 GB"    -> 15 GB     / 0
  SabnzbdResponseMapper.cs:201   "1.5 M"     -> 15 MB/s   / 0
  SabnzbdResponseMapper.cs:320   "1.5"       -> 15 MB     / 0
  TorznabNewznabValueParser.cs:40 "1.5 GB"   -> 15 GB     / 0

Each site now passes NumberStyles and CultureInfo.InvariantCulture, matching
the parse that was already correct in the same file where there is one
(SabnzbdResponseMapper.cs:224 uses NumberStyles.Any, MyAnonamouseSizeParser.cs:73
uses NumberStyles.Float) and NzbgetHistoryReader.cs:157 otherwise.

Fixes Listenarrs#796.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Machine-format values are parsed under the server's culture: a 12-hour book records a duration of 43,200,250 seconds

1 participant