feat(treeplot): lay out the tree without igraph - #7984
Conversation
The generated template imported igraph, which no requirements file declares, so the operator failed with No module named 'igraph' on any environment built from the repository. Declaring it is not open to us. igraph is GPL v2, which is Category X under the ASF 3rd party license policy, and check_binary_deps.py rejects it. igraph did four things here: build a graph from the pairs, read the node names back, run the Reingold-Tilford layout, and return the edge list. Only the layout does real work, and EdgeSeq was imported but never used. The template now computes the layout itself. Depth picks the row, a leaf takes the next free column, and a parent sits centred over its children. Roots are the nodes that never appear as a child, and anything left over sits in a cycle no root reaches, so every node is placed exactly once. The tree keeps its shape. Spacing is uniform per leaf rather than Reingold-Tilford's contour packing, so an unbalanced tree draws slightly wider than it did. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Automated Reviewer SuggestionsBased on the
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #7984 +/- ##
============================================
- Coverage 93.52% 93.51% -0.01%
+ Complexity 4727 4725 -2
============================================
Files 1195 1195
Lines 48492 48492
Branches 5379 5379
============================================
- Hits 45352 45349 -3
Misses 1693 1693
- Partials 1447 1450 +3
*This pull request uses carry forward flags. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
| config | throughput | MB/s | latency | max Δ latest / 7d | |
|---|---|---|---|---|---|
| 🔴 | bs=10 sw=10 sl=64 | 359 | 0.219 | 26,928/41,662/41,662 us | 🔴 +5.9% / 🔴 +193.0% |
| 🔴 | bs=100 sw=10 sl=64 | 796 | 0.486 | 122,933/163,127/163,127 us | 🔴 +8.2% / 🔴 +71.6% |
| ⚪ | bs=1000 sw=10 sl=64 | 935 | 0.571 | 1,074,416/1,138,876/1,138,876 us | ⚪ within ±5% / 🔴 +24.2% |
Baseline details
Latest main 604f109 from same runner
| config | metric | PR | latest main | 7d avg | Δ latest | Δ 7d |
|---|---|---|---|---|---|---|
| bs=10 sw=10 sl=64 | throughput | 359 tuples/sec | 371 tuples/sec | 894.27 tuples/sec | -3.2% | -59.9% |
| bs=10 sw=10 sl=64 | MB/s | 0.219 MB/s | 0.227 MB/s | 0.546 MB/s | -3.5% | -59.9% |
| bs=10 sw=10 sl=64 | p50 | 26,928 us | 27,684 us | 11,164 us | -2.7% | +141.2% |
| bs=10 sw=10 sl=64 | p95 | 41,662 us | 39,334 us | 14,219 us | +5.9% | +193.0% |
| bs=10 sw=10 sl=64 | p99 | 41,662 us | 39,334 us | 16,863 us | +5.9% | +147.1% |
| bs=100 sw=10 sl=64 | throughput | 796 tuples/sec | 845 tuples/sec | 1,158 tuples/sec | -5.8% | -31.3% |
| bs=100 sw=10 sl=64 | MB/s | 0.486 MB/s | 0.516 MB/s | 0.707 MB/s | -5.8% | -31.2% |
| bs=100 sw=10 sl=64 | p50 | 122,933 us | 117,882 us | 88,711 us | +4.3% | +38.6% |
| bs=100 sw=10 sl=64 | p95 | 163,127 us | 150,797 us | 95,062 us | +8.2% | +71.6% |
| bs=100 sw=10 sl=64 | p99 | 163,127 us | 150,797 us | 102,275 us | +8.2% | +59.5% |
| bs=1000 sw=10 sl=64 | throughput | 935 tuples/sec | 925 tuples/sec | 1,188 tuples/sec | +1.1% | -21.3% |
| bs=1000 sw=10 sl=64 | MB/s | 0.571 MB/s | 0.565 MB/s | 0.725 MB/s | +1.1% | -21.2% |
| bs=1000 sw=10 sl=64 | p50 | 1,074,416 us | 1,085,268 us | 874,204 us | -1.0% | +22.9% |
| bs=1000 sw=10 sl=64 | p95 | 1,138,876 us | 1,122,772 us | 916,845 us | +1.4% | +24.2% |
| bs=1000 sw=10 sl=64 | p99 | 1,138,876 us | 1,122,772 us | 947,186 us | +1.4% | +20.2% |
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,557.06,200,128000,359,0.219,26927.86,41662.16,41662.16
1,100,10,64,20,2512.05,2000,1280000,796,0.486,122932.73,163126.78,163126.78
2,1000,10,64,20,21382.04,20000,12800000,935,0.571,1074416.07,1138876.29,1138876.29|
/request-review @aglinxinyuan @carloea2 |
The generated template imported igraph, which no requirements file declares, so the operator failed with No module named 'igraph' on any environment built from the repository. igraph is GPL v2, Category X under the ASF 3rd party license policy, so declaring it is not open to us. Both generators now compute the layout themselves. Depth picks the row, a leaf takes the next free column, and a parent sits centred over its children. Roots are the nodes that never appear as a child, and anything left over sits in a cycle no root reaches, so every node is placed exactly once. The template half is the same change as apache#7984. The standalone half has no upstream counterpart. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
carloea2
left a comment
There was a problem hiding this comment.
The new layout is only checked as generated text. That does not test whether it places a tree correctly or stays safe on cycles and shared children. Please add behavior tests for a normal tree and the non tree cases this code handles.
…emits The layout was only read as generated text, which says nothing about whether it places a tree correctly or stays safe on the shapes an edge list can hold that a tree cannot. The operator's own spec now runs the generated module and calls the layout directly, following FilledAreaPlotOpDescSpec: it resolves a python the same way and cancels when pandas and plotly are not importable, so the pure-JVM job is unaffected while amber-integration, where WorkflowOperator/test runs with the packages installed, executes them. Five shapes, laid out in one driver run: a tree, whose depths and centring are checked against the values a tidy layout owes rather than against what the code returned; a shared child, placed once under the parent that reaches it first; a cycle, which terminates with every node placed once; a self loop; and a forest of two roots in columns that do not overlap. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Added behavior tests that run the layout instead of reading it: a tree checked against the positions a tidy layout owes, plus a shared child, a cycle, a self loop and a forest. They follow FilledAreaPlotOpDescSpec, so they execute in amber-integration and cancel where pandas and plotly are missing. |
carloea2
left a comment
There was a problem hiding this comment.
The new tests execute the generated layout and cover a normal tree, a shared child, cycles, a self loop, and a forest. This addresses my comment.
|
@aglinxinyuan May you take a look at this please? Thank you! |
What changes were proposed in this PR?
The Tree Plot operator's generated template no longer imports igraph. It computes its
own top-down layout instead: depth picks the row, a leaf takes the next free column,
and a parent sits centred over its children. Roots are the nodes that never appear as
a child, and anything unreachable from a root is rooted where it is, so a forest, a
cycle, a shared child and a self-loop all place every node exactly once. The walk is
iterative, so a deep tree cannot hit the recursion limit.
igraph was used for four things:
Graph.TupleList,vs['name'],layout('rt')andget_edgelist(). Only the layout did real work, andEdgeSeqwas imported but neverused. No new dependency is added.
The layout guard is kept and now reads
Tree layout failed. It earns its keep: a celllike
[[1,2], 3]parses to an unhashable node, which now renders the error pageinstead of failing the operator.
The tree keeps its shape. Spacing is uniform per leaf rather than Reingold-Tilford's
contour packing, so an unbalanced tree draws slightly wider than before.
Any related issues, documentation, discussions?
Closes #7969. Supersedes #7970, which declared igraph and was closed because GPL v2 is
Category X under the ASF 3rd party license policy.
How was this PR tested?
TreePlotOpDescSpecpasses with eight tests. Five of them run the layout rather thanread it as text, following
FilledAreaPlotOpDescSpec: the spec resolves a python thesame way, cancels when pandas and plotly are not importable so the pure-JVM
amberjobis unaffected, and executes in
amber-integration, whereWorkflowOperator/testrunswith the packages installed.
The five shapes are laid out in one driver run. A tree, checked against the positions a
tidy layout owes rather than against what the code returned: depth picks the row, the
leaves take consecutive columns, and each parent sits at the mean of its own children. A
shared child, placed once under the parent that reaches it first, leaving the other as a
childless root. A cycle, which terminates with every node placed once and the closing
edge dropped. A self loop. And a forest, whose two roots sit in columns that do not
overlap, each over its own child.
I also dumped the generated template and ran it against a four-level tree with igraph
made unimportable, and it rendered the plot rather than an error page. The root landed at
2.375 over children at 1.25 and 3.5, their children at 0.5, 2, 3 and 4, and the deepest
pair at 0 and 1. A 200k-node chain lays out in 0.29s.
Was this PR authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Claude Opus 5)