Skip to content

feat(workflow-operator): score the rows an empty cell leaves usable - #8070

Open
kz930 wants to merge 1 commit into
apache:mainfrom
kz930:scorer-empty-cell
Open

feat(workflow-operator): score the rows an empty cell leaves usable#8070
kz930 wants to merge 1 commit into
apache:mainfrom
kz930:scorer-empty-cell

Conversation

@kz930

@kz930 kz930 commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

What changes were proposed in this PR?

The Machine Learning Scorer handed both configured columns straight to scikit-learn's metric functions, so one empty cell reached them as a None and ended the run with "Classification metrics can't handle a mix of unknown and binary targets". The message named neither the column nor a way out, and no configuration could exclude the row.

The operator now drops the rows missing either value before scoring, the way the visualization operators drop missing values before plotting. The subset names both columns rather than dropping on each separately: dropping on either alone would leave y_true and y_pred misaligned.

Any related issues, documentation, discussions?

Closes #8058

How was this PR tested?

A new case in MachineLearningScorerOpDescSpec pins the drop in the generated code. Run by hand on a table with a blank in the actual-value column and a blank in the predicted-value column: before, the run ended with the ValueError above; after, it reports Accuracy 0.6667 over the three scorable rows.

Was this PR authored or co-authored using generative AI tooling?

Generated-by: Claude Code (Claude Opus 5)

The Machine Learning Scorer handed both configured columns straight to
scikit-learn's metric functions, so one empty cell reached them as a None and
ended the run with "Classification metrics can't handle a mix of unknown and
binary targets". The message named neither the column nor a way out, and no
configuration could exclude the row.

The operator now drops the rows missing either value before scoring, the way
the visualization operators drop missing values before plotting. One subset
naming both columns rather than two separate drops: dropping on either alone
would leave y_true and y_pred misaligned.

Closes apache#8058

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@kz930

kz930 commented Aug 28, 2026

Copy link
Copy Markdown
Contributor Author

/request-review @aglinxinyuan @carloea2

@github-actions

Copy link
Copy Markdown
Contributor

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

  • Contributors with relevant context: @aglinxinyuan
    You can notify them by mentioning @aglinxinyuan in a comment.

@codecov-commenter

codecov-commenter commented Aug 28, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.42%. Comparing base (eddec26) to head (9e68d4d).
⚠️ Report is 8 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff              @@
##               main    #8070      +/-   ##
============================================
- Coverage     93.43%   93.42%   -0.02%     
+ Complexity     4710     4707       -3     
============================================
  Files          1181     1181              
  Lines         47801    47810       +9     
  Branches       5326     5327       +1     
============================================
+ Hits          44663    44666       +3     
- Misses         1698     1700       +2     
- Partials       1440     1444       +4     
Flag Coverage Δ *Carryforward flag
access-control-service 81.00% <ø> (ø)
agent-service 99.32% <ø> (ø) Carriedforward from eddec26
amber 89.58% <100.00%> (-0.03%) ⬇️
computing-unit-managing-service 73.67% <ø> (ø)
config-service 86.73% <ø> (ø)
file-service 87.89% <ø> (ø)
frontend 95.90% <ø> (ø) Carriedforward from eddec26
notebook-migration-service 79.31% <ø> (ø)
pyamber 97.74% <ø> (ø) Carriedforward from eddec26
workflow-compiling-service 77.19% <ø> (ø)

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions

Copy link
Copy Markdown
Contributor

⚠️ Benchmark changes need a look

🟢 4 better · 🔴 1 worse · ⚪ 10 noise (<±5%) · 0 without baseline

Compared against main bdc6d2a benchmarked on this same runner, so the delta is largely free of cross-runner hardware noise. The "7d avg" column still reflects the gh-pages dashboard. Treat <±5% as noise unless repeated.

Dashboard · Run

config throughput MB/s latency max Δ latest / 7d
🟢 bs=10 sw=10 sl=64 407 0.249 24,642/30,836/30,836 us 🟢 -8.0% / 🔴 +120.5%
🔴 bs=100 sw=10 sl=64 895 0.546 109,836/133,462/133,462 us 🔴 +6.4% / 🔴 +40.3%
bs=1000 sw=10 sl=64 1,053 0.643 946,312/1,006,072/1,006,072 us ⚪ within ±5% / 🔴 -11.3%
Baseline details

Latest main bdc6d2a from same runner

config metric PR latest main 7d avg Δ latest Δ 7d
bs=10 sw=10 sl=64 throughput 407 tuples/sec 414 tuples/sec 893.99 tuples/sec -1.7% -54.5%
bs=10 sw=10 sl=64 MB/s 0.249 MB/s 0.252 MB/s 0.546 MB/s -1.2% -54.4%
bs=10 sw=10 sl=64 p50 24,642 us 23,599 us 11,176 us +4.4% +120.5%
bs=10 sw=10 sl=64 p95 30,836 us 33,521 us 14,009 us -8.0% +120.1%
bs=10 sw=10 sl=64 p99 30,836 us 33,521 us 16,898 us -8.0% +82.5%
bs=100 sw=10 sl=64 throughput 895 tuples/sec 925 tuples/sec 1,157 tuples/sec -3.2% -22.7%
bs=100 sw=10 sl=64 MB/s 0.546 MB/s 0.565 MB/s 0.706 MB/s -3.4% -22.7%
bs=100 sw=10 sl=64 p50 109,836 us 103,218 us 88,763 us +6.4% +23.7%
bs=100 sw=10 sl=64 p95 133,462 us 141,293 us 95,096 us -5.5% +40.3%
bs=100 sw=10 sl=64 p99 133,462 us 141,293 us 102,854 us -5.5% +29.8%
bs=1000 sw=10 sl=64 throughput 1,053 tuples/sec 1,040 tuples/sec 1,187 tuples/sec +1.3% -11.3%
bs=1000 sw=10 sl=64 MB/s 0.643 MB/s 0.635 MB/s 0.724 MB/s +1.3% -11.2%
bs=1000 sw=10 sl=64 p50 946,312 us 957,307 us 874,968 us -1.1% +8.2%
bs=1000 sw=10 sl=64 p95 1,006,072 us 1,026,390 us 917,815 us -2.0% +9.6%
bs=1000 sw=10 sl=64 p99 1,006,072 us 1,026,390 us 949,868 us -2.0% +5.9%
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,490.84,200,128000,407,0.249,24641.77,30836.10,30836.10
1,100,10,64,20,2235.25,2000,1280000,895,0.546,109835.96,133461.96,133461.96
2,1000,10,64,20,18994.71,20000,12800000,1053,0.643,946311.67,1006072.10,1006072.10

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Machine Learning Scorer ends the run on an empty cell in either scored column

2 participants