Skip to content

fix!: make a join's null-padded side nullable - #272

Open
alexandrefimov wants to merge 1 commit into
substrait-io:mainfrom
alexandrefimov:fix/join-null-padding
Open

alexandrefimov wants to merge 1 commit into
substrait-io:mainfrom
alexandrefimov:fix/join-null-padding

Conversation

@alexandrefimov

@alexandrefimov alexandrefimov commented Sep 20, 2026

Copy link
Copy Markdown
Contributor

infer_rel_schema keeps a join's fields as its inputs declare them, so a required field on the side the join fills with nulls stays required. Plans that DuckDB and Isthmus write for an ordinary LEFT JOIN hit this.

Make the padded side nullable, per the Join Types in spec v0.99.0: LEFT and LEFT_SINGLE pad the right input, RIGHT and RIGHT_SINGLE the left, and OUTER both. Inner, semi, anti and mark joins pad nothing, a side that is already nullable is unchanged, and lateral joins share the same derivation.

Closes #267

BREAKING CHANGE: Outer and single joins now infer nullable fields on the null-padded side. A schema that repeated a required declaration there changes, along with anything derived from it.

Summary by CodeRabbit

  • Bug Fixes
    • Join schema inference now accurately marks fields as nullable when unmatched rows can produce null values.
    • Left and right joins update nullability only on the opposite side; full outer joins update both sides.
    • Inner, semi, anti, and mark joins preserve existing field nullability.
    • Lateral left joins now correctly reflect nullable fields on the right side.
  • Tests
    • Added coverage for join types and null-padding behavior.

infer_rel_schema kept each input's field types as its input declared them,
so a required field on the side a join fills with nulls stayed required.

LEFT and LEFT_SINGLE pad the right input, RIGHT and RIGHT_SINGLE the left,
and OUTER both; inner, semi, anti and mark joins pad nothing, and a side
that is already nullable is unchanged. Lateral joins share the derivation.

BREAKING CHANGE: Outer and single joins now infer nullable fields on the
null-padded side.
@coderabbitai

coderabbitai Bot commented Sep 20, 2026

Copy link
Copy Markdown

Review Change StackReview Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: b087e6f8-e98e-4d5a-a652-bd4e2b0fe612

📥 Commits

Reviewing files that changed from the base of the PR and between 45c8ca7 and bdafef2.

📒 Files selected for processing (2)
  • src/substrait/type_inference.py
  • tests/test_type_inference.py

Included review availability: Your plan provides up to 10 included reviews per hour; 8 remain after this review.


📝 Walkthrough

Walkthrough

Changes

Join nullability

Layer / File(s) Summary
Null-padded join schema construction
src/substrait/type_inference.py
Join inference makes fields nullable on sides that can produce unmatched-row nulls. Other join types retain input nullability.
Join nullability validation
tests/test_type_inference.py
Tests cover supported join types, lateral left joins, input schema preservation, and overall struct nullability.

Priority: ➖ Normal

Estimated code review effort: 2 (Simple) | ~15 minutes

Change: Bug fix · Severity of issue fixed: Medium

Suggested reviewers: nielspardon

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 41.67% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 12 functions across 2 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed The change satisfies #267. infer_rel_schema now derives null-padded sides for LEFT and LEFT_SINGLE joins, the left side for RIGHT and RIGHT_SINGLE, and both sides for OUTER. Inner, semi, a…
Out of Scope Changes check ✅ Passed The reviewed changes are limited to src/substrait/type_inference.py and related tests in tests/test_type_inference.py. The implementation and tests directly support the null-padding requirements i…
Title check ✅ Passed The title clearly and concisely describes the main change: join null-padded sides become nullable. It is directly related to the changeset.
Description check ✅ Passed The description explains the rationale, specifies null-padding behavior for each join type, covers lateral joins and unchanged cases, identifies the breaking change, and references issue #267.
  • Fix all pre-merge checks with AI
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

infer_rel_schema keeps outer and single join fields required on the null-padded side

1 participant