Add inference parameters to provider requests - #78
Merged
FernandoCelmer merged 3 commits intoAug 25, 2026
Conversation
FernandoCelmer
left a comment
Member
There was a problem hiding this comment.
🔍 Code Review
Code issues found: 2
| # | Severity | Comment |
|---|---|---|
| 1 | [Blocking] | request_params read from JSON but never passed to constructor — inference params from JSON config are silently dropped |
| 2 | [Suggestion] | temperature/max_tokens typed as float/int in chat.py instead of `float |
Overall the approach is solid — the is not None guard for temperature=0, the **self.inference_params placement (explicit keys win), and the reload() update all look correct. The test covering temperature=0 is a nice touch.
FernandoCelmer
self-requested a review
August 18, 2026 18:50
FernandoCelmer
left a comment
Member
There was a problem hiding this comment.
🔍 Code Review
Code issues found: 3
| # | Severity | Comment |
|---|---|---|
| 1 | [Blocking] | CLI --temperature/--max-tokens silently dropped for JSON providers |
| 2 | [Blocking] | **self.inference_params spread position can mask user intent |
| 3 | [Suggestion] | _ConnectionSnapshot missing inference_params breaks thread-safety guarantee |
Member
There was a problem hiding this comment.
🔍 Code Review
Code issues found: 3
| # | Severity | Comment |
|---|---|---|
| 1 | [Blocking] | inference_params silently dropped for JSON providers |
| 2 | [Blocking] | _ConnectionSnapshot missing inference_params |
| 3 | [Suggestion] | request.params double-applied with default builder |
…late providers - providers/__init__.py: _from_json_path now merges CLI-provided inference_params on top of the ones declared in the config, instead of silently discarding them — the CLI flags previously had zero effect for any JSON config or bundled template. - providers/generic.py: inference_params moved into _ConnectionSnapshot and applied to the built request body inside complete(), instead of being read live off self inside _default_request — closes the race where a concurrent reload() could hand a request a stale/mixed config. Also stops _build_from_json from double-applying request.params when a custom request_builder already bakes them in. - Reformat with the repo's ruff config.
Member
|
Pushed a fix on top of this branch addressing the blocking review comments:
326/326 relevant tests pass locally (excluded |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
.gitignore— Added entries for build artifacts (1 addition)pycodeloop/cli/commands/chat.py— Added--temperatureand--max-tokensCLI options to the interactive chat command, forwarded tobuild_flow(13 additions)pycodeloop/cli/commands/run.py— Same options added to the non-interactiveruncommand (10 additions)pycodeloop/cli/flow.py—resolve_providerandbuild_flowupdated to accept and passtemperature/max_tokens; buildsinference_paramsdict and adds it toprovider_kwargs(16 additions, 1 deletion)pycodeloop/providers/generic.py—GenericProvider.__init__acceptsinference_params: dict | None;_build_from_jsonreadsrequest.paramsfrom JSON config;reload()syncsinference_params;_default_requestconverted from@staticmethodto instance method and spreadsself.inference_paramsinto the request body (9 additions, 1 deletion)tests/providers/test_generic.py— New test verifyinginference_paramsare present in the request built by_default_request(15 additions)Motivation and Context
Adds
--temperatureand--max-tokensCLI flags for user control of LLM sampling parameters without editing JSON config files. Enables JSON config files to declare static inference parameters underrequest.params. Improves user experience for controlling model behavior directly from the command line.Types of changes
Checklist