Fix PR #4636 regression: conditionally update rollout params before pre-RL evaluation

[igorts-git]

Description

This PR fixes a regression introduced in PR #4636 where rl_cluster.rollout.update_params was removed prior to Pre-RL evaluation (step 0).

When running an RL training or evaluation job that resumes from a previously saved RL checkpoint (step > 0), omitting update_params causes Pre-RL evaluation to evaluate using the base HuggingFace/SFT checkpoint weights rather than the restored RL checkpoint weights. Conversely, running update_params unconditionally at step 0 causes redundant weight transfers and unnecessary resharding.

This change adds a conditional check so rl_cluster.rollout.update_params is skipped at step 0 and executed whenever resuming from a checkpoint (step > 0).

Tests

Added a few extra log statements (removed in this PR) to validate that :

• Fresh run from step 0: is_restored_run=False, update_params was skipped, and saved an RL checkpoint at step 1.
• Resuming from step 1 checkpoint: Confirmed is_restored_run=True, rl_cluster.rollout.update_params was called prior to Pre-RL evaluation, and verified that training completed successfully.

Checklist

Before submitting this PR, please make sure (put X in square brackets):

• I have performed a self-review of my code. For an optional AI review, add the gemini-review label.
• I have necessary comments in my code, particularly in hard-to-understand areas.
• I have run end-to-end tests tests and provided workload links above if applicable.
• I have made or will make corresponding changes to the doc if needed, including adding new documentation pages to the relevant Table of Contents (toctree directive) as explained in our documentation.

[gemini-code-assist]

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!