Skip to content

feat(treeplot): lay out the tree without igraph - #7984

Open
kz930 wants to merge 3 commits into
apache:mainfrom
kz930:fix/7969-treeplot-drop-igraph
Open

feat(treeplot): lay out the tree without igraph#7984
kz930 wants to merge 3 commits into
apache:mainfrom
kz930:fix/7969-treeplot-drop-igraph

Conversation

@kz930

@kz930 kz930 commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

What changes were proposed in this PR?

The Tree Plot operator's generated template no longer imports igraph. It computes its
own top-down layout instead: depth picks the row, a leaf takes the next free column,
and a parent sits centred over its children. Roots are the nodes that never appear as
a child, and anything unreachable from a root is rooted where it is, so a forest, a
cycle, a shared child and a self-loop all place every node exactly once. The walk is
iterative, so a deep tree cannot hit the recursion limit.

igraph was used for four things: Graph.TupleList, vs['name'], layout('rt') and
get_edgelist(). Only the layout did real work, and EdgeSeq was imported but never
used. No new dependency is added.

The layout guard is kept and now reads Tree layout failed. It earns its keep: a cell
like [[1,2], 3] parses to an unhashable node, which now renders the error page
instead of failing the operator.

The tree keeps its shape. Spacing is uniform per leaf rather than Reingold-Tilford's
contour packing, so an unbalanced tree draws slightly wider than before.

Any related issues, documentation, discussions?

Closes #7969. Supersedes #7970, which declared igraph and was closed because GPL v2 is
Category X under the ASF 3rd party license policy.

How was this PR tested?

TreePlotOpDescSpec passes with eight tests. Five of them run the layout rather than
read it as text, following FilledAreaPlotOpDescSpec: the spec resolves a python the
same way, cancels when pandas and plotly are not importable so the pure-JVM amber job
is unaffected, and executes in amber-integration, where WorkflowOperator/test runs
with the packages installed.

The five shapes are laid out in one driver run. A tree, checked against the positions a
tidy layout owes rather than against what the code returned: depth picks the row, the
leaves take consecutive columns, and each parent sits at the mean of its own children. A
shared child, placed once under the parent that reaches it first, leaving the other as a
childless root. A cycle, which terminates with every node placed once and the closing
edge dropped. A self loop. And a forest, whose two roots sit in columns that do not
overlap, each over its own child.

I also dumped the generated template and ran it against a four-level tree with igraph
made unimportable, and it rendered the plot rather than an error page. The root landed at
2.375 over children at 1.25 and 3.5, their children at 0.5, 2, 3 and 4, and the deepest
pair at 0 and 1. A 200k-node chain lays out in 0.29s.

Was this PR authored or co-authored using generative AI tooling?

Generated-by: Claude Code (Claude Opus 5)

The generated template imported igraph, which no requirements file declares,
so the operator failed with No module named 'igraph' on any environment built
from the repository. Declaring it is not open to us. igraph is GPL v2, which
is Category X under the ASF 3rd party license policy, and check_binary_deps.py
rejects it.

igraph did four things here: build a graph from the pairs, read the node names
back, run the Reingold-Tilford layout, and return the edge list. Only the
layout does real work, and EdgeSeq was imported but never used. The template
now computes the layout itself. Depth picks the row, a leaf takes the next free
column, and a parent sits centred over its children. Roots are the nodes that
never appear as a child, and anything left over sits in a cycle no root
reaches, so every node is placed exactly once.

The tree keeps its shape. Spacing is uniform per leaf rather than
Reingold-Tilford's contour packing, so an unbalanced tree draws slightly wider
than it did.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

  • Contributors with relevant context: @eugenegujing
    You can notify them by mentioning @eugenegujing in a comment.

@codecov-commenter

codecov-commenter commented Aug 26, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.51%. Comparing base (604f109) to head (825b948).

Additional details and impacted files
@@             Coverage Diff              @@
##               main    #7984      +/-   ##
============================================
- Coverage     93.52%   93.51%   -0.01%     
+ Complexity     4727     4725       -2     
============================================
  Files          1195     1195              
  Lines         48492    48492              
  Branches       5379     5379              
============================================
- Hits          45352    45349       -3     
  Misses         1693     1693              
- Partials       1447     1450       +3     
Flag Coverage Δ *Carryforward flag
access-control-service 81.00% <ø> (ø)
agent-service 99.32% <ø> (ø) Carriedforward from b59c740
amber 89.59% <ø> (-0.02%) ⬇️
computing-unit-managing-service 73.67% <ø> (ø)
config-service 86.86% <ø> (ø)
file-service 87.91% <ø> (ø)
frontend 95.86% <ø> (-0.01%) ⬇️ Carriedforward from b59c740
notebook-migration-service 79.31% <ø> (ø)
pyamber 98.54% <ø> (ø) Carriedforward from b59c740
workflow-compiling-service 77.19% <ø> (ø)

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions

github-actions Bot commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

⚠️ Benchmark changes need a look

🟢 0 better · 🔴 6 worse · ⚪ 9 noise (<±5%) · 0 without baseline

Compared against main 604f109 benchmarked on this same runner, so the delta is largely free of cross-runner hardware noise. The "7d avg" column still reflects the gh-pages dashboard. Treat <±5% as noise unless repeated.

Dashboard · Run

config throughput MB/s latency max Δ latest / 7d
🔴 bs=10 sw=10 sl=64 359 0.219 26,928/41,662/41,662 us 🔴 +5.9% / 🔴 +193.0%
🔴 bs=100 sw=10 sl=64 796 0.486 122,933/163,127/163,127 us 🔴 +8.2% / 🔴 +71.6%
bs=1000 sw=10 sl=64 935 0.571 1,074,416/1,138,876/1,138,876 us ⚪ within ±5% / 🔴 +24.2%
Baseline details

Latest main 604f109 from same runner

config metric PR latest main 7d avg Δ latest Δ 7d
bs=10 sw=10 sl=64 throughput 359 tuples/sec 371 tuples/sec 894.27 tuples/sec -3.2% -59.9%
bs=10 sw=10 sl=64 MB/s 0.219 MB/s 0.227 MB/s 0.546 MB/s -3.5% -59.9%
bs=10 sw=10 sl=64 p50 26,928 us 27,684 us 11,164 us -2.7% +141.2%
bs=10 sw=10 sl=64 p95 41,662 us 39,334 us 14,219 us +5.9% +193.0%
bs=10 sw=10 sl=64 p99 41,662 us 39,334 us 16,863 us +5.9% +147.1%
bs=100 sw=10 sl=64 throughput 796 tuples/sec 845 tuples/sec 1,158 tuples/sec -5.8% -31.3%
bs=100 sw=10 sl=64 MB/s 0.486 MB/s 0.516 MB/s 0.707 MB/s -5.8% -31.2%
bs=100 sw=10 sl=64 p50 122,933 us 117,882 us 88,711 us +4.3% +38.6%
bs=100 sw=10 sl=64 p95 163,127 us 150,797 us 95,062 us +8.2% +71.6%
bs=100 sw=10 sl=64 p99 163,127 us 150,797 us 102,275 us +8.2% +59.5%
bs=1000 sw=10 sl=64 throughput 935 tuples/sec 925 tuples/sec 1,188 tuples/sec +1.1% -21.3%
bs=1000 sw=10 sl=64 MB/s 0.571 MB/s 0.565 MB/s 0.725 MB/s +1.1% -21.2%
bs=1000 sw=10 sl=64 p50 1,074,416 us 1,085,268 us 874,204 us -1.0% +22.9%
bs=1000 sw=10 sl=64 p95 1,138,876 us 1,122,772 us 916,845 us +1.4% +24.2%
bs=1000 sw=10 sl=64 p99 1,138,876 us 1,122,772 us 947,186 us +1.4% +20.2%
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,557.06,200,128000,359,0.219,26927.86,41662.16,41662.16
1,100,10,64,20,2512.05,2000,1280000,796,0.486,122932.73,163126.78,163126.78
2,1000,10,64,20,21382.04,20000,12800000,935,0.571,1074416.07,1138876.29,1138876.29

@kz930

kz930 commented Aug 26, 2026

Copy link
Copy Markdown
Contributor Author

/request-review @aglinxinyuan @carloea2

kz930 added a commit to Nicoleee1108/texera_workflow_to_py that referenced this pull request Aug 26, 2026
The generated template imported igraph, which no requirements file declares,
so the operator failed with No module named 'igraph' on any environment built
from the repository. igraph is GPL v2, Category X under the ASF 3rd party
license policy, so declaring it is not open to us.

Both generators now compute the layout themselves. Depth picks the row, a leaf
takes the next free column, and a parent sits centred over its children. Roots
are the nodes that never appear as a child, and anything left over sits in a
cycle no root reaches, so every node is placed exactly once.

The template half is the same change as apache#7984. The standalone
half has no upstream counterpart.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

@carloea2 carloea2 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new layout is only checked as generated text. That does not test whether it places a tree correctly or stays safe on cycles and shared children. Please add behavior tests for a normal tree and the non tree cases this code handles.

…emits

The layout was only read as generated text, which says nothing about whether it
places a tree correctly or stays safe on the shapes an edge list can hold that a
tree cannot.

The operator's own spec now runs the generated module and calls the layout
directly, following FilledAreaPlotOpDescSpec: it resolves a python the same way
and cancels when pandas and plotly are not importable, so the pure-JVM job is
unaffected while amber-integration, where WorkflowOperator/test runs with the
packages installed, executes them.

Five shapes, laid out in one driver run: a tree, whose depths and centring are
checked against the values a tidy layout owes rather than against what the code
returned; a shared child, placed once under the parent that reaches it first; a
cycle, which terminates with every node placed once; a self loop; and a forest of
two roots in columns that do not overlap.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@kz930

kz930 commented Aug 28, 2026

Copy link
Copy Markdown
Contributor Author

Added behavior tests that run the layout instead of reading it: a tree checked against the positions a tidy layout owes, plus a shared child, a cycle, a self loop and a forest. They follow FilledAreaPlotOpDescSpec, so they execute in amber-integration and cancel where pandas and plotly are missing.

@carloea2 carloea2 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new tests execute the generated layout and cover a normal tree, a shared child, cycles, a self loop, and a forest. This addresses my comment.

@kz930

kz930 commented Aug 29, 2026

Copy link
Copy Markdown
Contributor Author

@aglinxinyuan May you take a look at this please? Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

The Tree Plot operator can never run: its igraph dependency is Category X

3 participants