bench: add ASOF join benchmarks - #23833
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #23833 +/- ##
==========================================
+ Coverage 81.88% 81.90% +0.02%
==========================================
Files 1133 1134 +1
Lines 424522 425261 +739
Branches 424522 425261 +739
==========================================
+ Hits 347622 348315 +693
- Misses 56288 56300 +12
- Partials 20612 20646 +34 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
e826548 to
1ff6d2b
Compare
1ff6d2b to
c3d099b
Compare
# Conflicts: # datafusion/expr/src/logical_plan/builder.rs # datafusion/expr/src/logical_plan/plan.rs # datafusion/sql/src/relation/join.rs # datafusion/sql/src/unparser/plan.rs # datafusion/sqllogictest/test_files/asof_join.slt # datafusion/substrait/tests/cases/serialize.rs # docs/source/user-guide/sql/select.md
|
There are many optimizations to do, but I think it good to have a benchmark in first. This PR will be the last one of this stack, all other changes will be non-blocking, optional follow ups to do. Cc @2010YOUY01 and @jayzhan211, let's go! Time to close the issue created 5 years ago 🤓 |
jayzhan211
left a comment
There was a problem hiding this comment.
Thanks @Xuanwo
Similar to nlj, hj, smj it would be nice to have ./bench.sh run asof
Some suggestions for more cases covered
2010YOUY01
left a comment
There was a problem hiding this comment.
Thank you for the fast iteration on this project! Here are some suggestions
I suggest we move the SQL benchmark to the new SQL bench framework, like https://github.com/apache/datafusion/tree/main/benchmarks/sql_benchmarks/nlj, so it would be runnable with
# under benchmarks/
./bench.sh run asof_join
Optional: To better explain the workload covered, we can write comments like
# top comment: axes to vary
#
# size: vary left/right input with very small / medium / large sizes, like 100/100k/10m rows
# sortedness: does the input already have the required order, or do we have to sort?
# group cardinality / size: ...
# per-query comment: just tag the axes
Q1: left small, right large; no existing order; single group ...
We can do this as a follow-up and potentially examine and try to better cover the typical workload.
|
Thanks @jayzhan211 and @2010YOUY01 for the thoughtful reviews! This is much cleaner now 🙌 |
2010YOUY01
left a comment
There was a problem hiding this comment.
I've tested it locally and it works great.
Suggested a small cleanup item — we could do it as a follow-up.
Thank you! And it looks like a simple change, let me do it later . |
Which issue does this PR close?
Rationale for this change
This is the final core benchmark layer of the ASOF JOIN stack. It uses the
standard SQL benchmark framework so the workloads share the existing runner,
result format, and Criterion mode.
What changes are included in this PR?
asof_joinSQL benchmark suite, runnable with./bench.sh run asof_joinorbenchmark_runner.equality-group cardinality and skew, match direction, and payload width.
DATA_DIR/asof_joinwith./bench.sh data asof_join. The benchmark load hook only registers them withWITH ORDER, keeping data generation and input sorting outside the measuredquery.
AsOfJoinExec.Are these changes tested?
Yes:
cargo fmt --all./ci/scripts/doc_prettier_check.sh --write --allow-dirtycargo clippy --all-targets --all-features -- -D warningscargo test -p datafusion-benchmarks --lib --binsDATA_DIR=/tmp/asof_join CARGO_COMMAND='cargo run' ./bench.sh data asof_joinDATA_DIR=/tmp/asof_join CARGO_COMMAND='cargo run' ./bench.sh run asof_join 7cargo run -p datafusion-benchmarks --bin benchmark_runner -- asof_join --iterations 1 --partitions 4 --path /tmp/asof_joincargo run -p datafusion-benchmarks --bin benchmark_runner -- asof_join --query 7 --partitions 4 --path /tmp/asof_join --criterionThe pre-sorted workload's physical plan contains
AsOfJoinExecwithout aninput
SortExec. These smoke runs validate the suite and are not presented asperformance claims.
Are there any user-facing changes?
This adds ASOF workloads to the repository benchmark tooling. It does not
change query semantics or runtime behavior.
This is the final core item tracked by #23738. It does not depend on the
optional floating-point follow-up #24375.