fix(byoc): send real capability + model_id for usage attribution

[seanhanca]

Problem

BYOC usage reaches OpenMeter via the remote signer's create_signed_ticket
metering event. For every BYOC capability (e.g. nano-banana) that event
was recorded as pipeline=live-video-to-video / model_id=unknown instead of
the real pipeline + model.

Root cause (model attribution lost at _create_byoc_payment)

src/livepeer_gateway/byoc.py → _create_byoc_payment builds the
/generate-live-payment request and:

1. hardcodes type: "lv2v" for every BYOC capability → the go-livepeer signer
   emits pipeline = PipelineLiveVideoToVideo ("live-video-to-video"): the
   coarse mislabel;
2. sends the real capability only as a bare JSON field the signer never
   read (its RemotePaymentRequest had no such field → dropped) and sends
   no model id at all → the signer emits an empty model_id, which the
   pymthouse collector defaults to "unknown".

The real capability is the capability arg; the real model id lives in the
inference request (req.payload / req.parameters). A pymthouse-only fix is
impossible — the data is gone before it reaches Redpanda.

The additive wire contract

Two explicit, optional string fields are added to the /generate-live-payment
request body (the signer's RemotePaymentRequest):

field	meaning
capability	the real BYOC capability (e.g. nano-banana) — already sent today; the signer now reads it
model_id	the specific provider model from the request payload/parameters

type stays "lv2v" so the signer keeps reusing the live fee/pixel routing
(BYOC has no per-job pixel count). Usage attribution is thereby decoupled
from fee routing. Empty/absent fields ⇒ today's exact behavior.

The signer side of this contract is implemented in the cross-linked go-livepeer
PR (see below). The meter already groups by pipeline + model_id, so no
meter change is needed — the values simply become correct.

Changes

• _create_byoc_payment accepts a model_id and includes it in the body
  (omitted when empty → byte-identical to today).
• New _extract_model_id(payload, parameters) helper locates the model id in
  the request (model_id then model, payload before parameters).
• submit_byoc_job and submit_training_job thread the extracted model id
  through. Model source: submit_byoc_job → req.payload/req.parameters
  (byoc.py _extract_model_id call), training → byoc_req.payload
  (which carries model_id).

Zero-regression

• type is unchanged → fee/pixel math unchanged.
• An older signer ignores the new fields.
• When no model id is determinable, model_id is omitted entirely → the body
  is byte-identical to today and the signer falls back.

Test plan

pytest tests/ (full suite green locally, 17 passed). New
tests/test_byoc_usage_attribution.py proves:

• the /generate-live-payment body carries the real capability +
  model_id for a representative BYOC capability (nano-banana);
• type stays "lv2v";
• model_id is omitted when not determinable (backward-compatible);
• model-extraction precedence (payload model_id > payload model >
  parameters);
• end-to-end through submit_byoc_job.

End-to-end verification recipe

With both this PR and the go-livepeer signer change deployed: drive a
nano-banana generation against the preview chain, then query OpenMeter
groupBy=pipeline_model — it should show the real pipeline + model id
(nano-banana / the provider model), not live-video-to-video / unknown.

Cross-link

Signer side (reads these fields): livepeer/go-livepeer PR — livepeer/go-livepeer#3966

Does not modify pymthouse or NaaP; the collector/read unknown fallbacks are
left in place as safety nets.

Made with Cursor

Summary by CodeRabbit

• New Features

  • BYOC signed payments now include an inferred model identifier when available, improving usage attribution.
  • The identifier is resolved from the submitted request content or parameters, with whitespace/absent values handled gracefully.

• Bug Fixes

  • When no model identifier is available, it is omitted to preserve backward-compatible request behavior.
  • Training job payment attribution now correctly prefers the top-level model identifier over any nested model details.

• Tests

  • Added coverage for model identifier resolution and signer request payload generation for BYOC and training submissions.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: b13f9b7a-423a-40e9-b9ca-952963764723

📥 Commits

Reviewing files that changed from the base of the PR and between 5b3a1ef and 57e9e16.

📒 Files selected for processing (2)

• src/livepeer_gateway/byoc.py
• tests/test_byoc_usage_attribution.py

🚧 Files skipped from review as they are similar to previous changes (2)

• tests/test_byoc_usage_attribution.py
• src/livepeer_gateway/byoc.py

---

📝 Walkthrough

Walkthrough

BYOC job submission now extracts an optional provider model identifier from payloads or parameters and includes it in the signer payment request when present. The new tests cover extraction behavior, request-body construction, and forwarding through both BYOC and training submission paths.

Changes

BYOC model attribution

Layer / File(s)	Summary
Model extraction and payment body src/livepeer_gateway/byoc.py, tests/test_byoc_usage_attribution.py	_extract_model_id reads model_id or model from payload or parameters, and _create_byoc_payment adds model_id to the signer request body when available while keeping the fixed request type and capability fields.
Inference submission forwarding src/livepeer_gateway/byoc.py, tests/test_byoc_usage_attribution.py	submit_byoc_job passes the extracted model identifier into payment creation, and the end-to-end test asserts the signer request carries the forwarded model identifier.
Training submission forwarding src/livepeer_gateway/byoc.py, tests/test_byoc_usage_attribution.py	submit_training_job passes the top-level training model identifier into payment creation, and the end-to-end test verifies it wins over nested params.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Poem

🐇 I hop through bytes with nose a-twitch,
A model name now finds its niche.
In signer’s pouch, it rides along,
For BYOC payments, clear and strong.
Ears up, hops out—happy switch!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 29.03% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the main BYOC attribution change: sending the real capability and model_id.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🧹 Nitpick comments (1)

tests/test_byoc_usage_attribution.py (1)

158-209: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Add a training-path attribution regression.

This file verifies inference forwarding, but the PR also changes submit_training_job. Add a test where req.model_id differs from req.params["model_id"] and assert the signer payment body uses the top-level training model ID.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/test_byoc_usage_attribution.py` around lines 158 - 209, Add a
regression test for the training path in submit_training_job, since the current
coverage only checks submit_byoc_job. Create a case where the request’s
top-level model_id differs from req.params["model_id"], then verify the signer
payment request body is built from the top-level training model_id rather than
the nested params value. Use the existing submit_training_job flow and the same
request/signing capture pattern to locate the behavior.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/livepeer_gateway/byoc.py`:
- Line 719: The payment attribution is using a model id extracted from
byoc_req.payload, which can be overridden by req.params and diverge from the
actual training model. Update the signer-attribution path in byoc.py to use the
explicit req.model_id from the training request instead of
_extract_model_id(byoc_req.payload, byoc_req.parameters), while leaving the
orchestrator body construction unchanged. Locate the call site building the
signer request near the byoc_req payload handling and switch it to the top-level
model_id source.

---

Nitpick comments:
In `@tests/test_byoc_usage_attribution.py`:
- Around line 158-209: Add a regression test for the training path in
submit_training_job, since the current coverage only checks submit_byoc_job.
Create a case where the request’s top-level model_id differs from
req.params["model_id"], then verify the signer payment request body is built
from the top-level training model_id rather than the nested params value. Use
the existing submit_training_job flow and the same request/signing capture
pattern to locate the behavior.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: bb3eafc0-fe21-4816-8dce-e10cda18c678

📥 Commits

Reviewing files that changed from the base of the PR and between cc31b56 and 5b3a1ef.

📒 Files selected for processing (2)

• src/livepeer_gateway/byoc.py
• tests/test_byoc_usage_attribution.py

[Copilot]

Pull request overview

This PR fixes BYOC usage attribution by ensuring the gateway includes the real BYOC capability and a best-effort extracted model_id in the remote signer /generate-live-payment request body, so downstream metering can correctly group usage by pipeline + model_id without changing fee routing.

Changes:

• Add _extract_model_id(payload, parameters) helper and thread the extracted model_id through BYOC inference and training payment creation.
• Extend _create_byoc_payment to optionally include model_id in the signer request body (while keeping type="lv2v" unchanged).
• Add focused unit tests covering model-id extraction precedence and the /generate-live-payment wire payload.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
src/livepeer_gateway/byoc.py	Adds model-id extraction and includes optional model_id in signer payment payload for correct attribution.
tests/test_byoc_usage_attribution.py	Adds tests validating the signer request body carries real capability + model_id and remains backward-compatible when absent.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[seanhanca]

Fixed in 57e9e16. _create_byoc_payment now does model_id = model_id.strip() if isinstance(model_id, str) else "" before the if model_id: guard, so whitespace-only or non-string values are omitted and the wire body stays byte-identical to today when no real model id is present.