Skip to content

fix: use a stable seed in the PyTorch integration - #3072

Merged
mdlinville merged 2 commits into
wandb:mainfrom
marcus-campbell:fix-stable-pytorch-seeds
Sep 14, 2026
Merged

mdlinville merged 2 commits into
wandb:mainfrom
marcus-campbell:fix-stable-pytorch-seeds

Conversation

@marcus-campbell

@marcus-campbell marcus-campbell commented Aug 15, 2026

Copy link
Copy Markdown
Contributor

Description

The PyTorch integration page says this setup ensures deterministic behavior, but it doesn't appear to actually do that. This is because it derives its seeds from hash("..."). By default, Python randomizes string hashes between interpreter processes, so the example can initialize its RNGs differently across runs.

This fix replaces the hash-derived values with one named integer seed.

Before:

torch.backends.cudnn.deterministic = True
random.seed(hash("setting random seeds") % 2**32 - 1)
np.random.seed(hash("improves reproducibility") % 2**32 - 1)
torch.manual_seed(hash("by removing stochasticity") % 2**32 - 1)
torch.cuda.manual_seed_all(hash("so runs are repeatable") % 2**32 - 1)

After:

seed = 42
torch.backends.cudnn.deterministic = True
random.seed(seed)
np.random.seed(seed)
torch.manual_seed(seed)
torch.cuda.manual_seed_all(seed)

Why this matters

A small exploratory search found the same recognizable recipe in at least 24 other repositories. This suggests that the pattern has propagated beyond W&B's own docs, so correcting the bug here can help prevent it from spreading further.

The problem was also noted in the technical-review notes for #2673, but it wasn't fixed as it was out-of-scope for that PR (it was a style guide pass).

The corresponding notebooks are updated in wandb/examples#644.

Testing

  • git diff --check succeeds.
  • Confirmed that the diff contains only the intended seed changes.
  • Confirmed that the old hash-derived seeds vary with PYTHONHASHSEED.
  • Confirmed that the replacement has no interpreter hash-secret dependency.
  • PR tests succeed.

Replace process-randomized string hashes with a fixed seed so the
deterministic-behavior example remains reproducible across Python
processes.
@mdlinville
mdlinville enabled auto-merge (squash) September 14, 2026 23:19

@mdlinville mdlinville left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@mdlinville mdlinville left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you

@mdlinville
mdlinville merged commit 972251b into wandb:main Sep 14, 2026
6 checks passed
@marcus-campbell
marcus-campbell deleted the fix-stable-pytorch-seeds branch September 14, 2026 23:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants