Skip to content

feat(firestore): add opt-in BSON document read and decoding support - #18372

Draft
ohmayr wants to merge 2 commits into
bson-pr1b-pymongo-writesfrom
bson-pr2-reads-decoding
Draft

ohmayr wants to merge 2 commits into
bson-pr1b-pymongo-writesfrom
bson-pr2-reads-decoding

Conversation

@ohmayr

@ohmayr ohmayr commented Sep 14, 2026

Copy link
Copy Markdown
Contributor

…PR 2)

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

Fixes #<issue_number_goes_here> 🦕

@ohmayr
ohmayr added this pull request to stack #18374 September 14, 2026 21:57

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces support for decoding legacy BSON map representations into native BSON instances within the Firestore SDK, updating the client classes, helper functions, and DocumentSnapshot to handle the new decode_bson configuration. The review feedback highlights critical issues in the implementation: first, DocumentSnapshot fails to dynamically toggle between BSON-decoded and raw formats when raw_fields is None; second, the system tests incorrectly pass decode_bson to get() methods on document references, which will raise runtime TypeErrors; and finally, the return type annotation of decode_dict should be updated to Any to accurately reflect the potential return of native BSON instances.

Comment on lines +417 to +423
if raw_fields is not None:
if self._decode_bson:
self._data_bson = copy.deepcopy(data)
else:
self._data_raw = copy.deepcopy(data)
else:
self._data_raw = copy.deepcopy(data) if data is not None else None

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

When raw_fields is None (which is currently the case for any DocumentSnapshot created outside of _parse_batch_get, such as query or stream results), the snapshot cannot dynamically toggle between BSON-decoded and raw dict formats because it lacks the original protobuf fields to re-decode.

In the current implementation, if raw_fields is None and self._decode_bson is True, self._data_raw is populated with the decoded BSON data. If a user then calls to_dict(decode_bson=False), it will incorrectly return the BSON-decoded data instead of the raw dict.

To fix this, we should:

  1. Correctly initialize self._data_bson or self._data_raw in __init__ based on self._decode_bson when raw_fields is None.
  2. Ensure that all internal paths that instantiate DocumentSnapshot (such as query and stream result processing in base_query.py / query.py) are updated to pass raw_fields so that dynamic BSON decoding works consistently across all read operations.
        if raw_fields is not None:
            if self._decode_bson:
                self._data_bson = copy.deepcopy(data)
            else:
                self._data_raw = copy.deepcopy(data)
        else:
            if self._decode_bson:
                self._data_bson = copy.deepcopy(data) if data is not None else None
            else:
                self._data_raw = copy.deepcopy(data) if data is not None else None

Comment on lines +125 to +127
snapshot = doc_ref.get(decode_bson=True)
assert snapshot.exists
assert snapshot.to_dict(decode_bson=True) == bson_payload

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

DocumentReference.get() does not accept a decode_bson parameter. Calling doc_ref.get(decode_bson=True) will raise a TypeError at runtime.

To test BSON decoding, you should either:

  1. Set decode_bson=True on the client instance.
  2. Or retrieve the snapshot normally and call snapshot.to_dict(decode_bson=True) to decode on-demand.
Suggested change
snapshot = doc_ref.get(decode_bson=True)
assert snapshot.exists
assert snapshot.to_dict(decode_bson=True) == bson_payload
snapshot = doc_ref.get()
assert snapshot.exists
assert snapshot.to_dict(decode_bson=True) == bson_payload

Comment on lines +150 to +152
snapshot = await doc_ref.get(decode_bson=True)
assert snapshot.exists
assert snapshot.to_dict(decode_bson=True) == bson_payload

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

AsyncDocumentReference.get() does not accept a decode_bson parameter. Calling await doc_ref.get(decode_bson=True) will raise a TypeError at runtime.

To test BSON decoding, you should either:

  1. Set decode_bson=True on the client instance.
  2. Or retrieve the snapshot normally and call snapshot.to_dict(decode_bson=True) to decode on-demand.
Suggested change
snapshot = await doc_ref.get(decode_bson=True)
assert snapshot.exists
assert snapshot.to_dict(decode_bson=True) == bson_payload
snapshot = await doc_ref.get()
assert snapshot.exists
assert snapshot.to_dict(decode_bson=True) == bson_payload

return None


def decode_dict(value_fields, client, decode_bson: Optional[bool] = None) -> Union[dict, Vector]:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The return type annotation of decode_dict is currently Union[dict, Vector]. However, with the introduction of BSON decoding support, this function can now return native BSON instances (such as BSONObjectID, BSONDecimal128, etc.) when decode_bson is enabled. The type annotation should be updated to Any to accurately reflect this behavior.

Suggested change
def decode_dict(value_fields, client, decode_bson: Optional[bool] = None) -> Union[dict, Vector]:
def decode_dict(value_fields, client, decode_bson: Optional[bool] = None) -> Any:
References
  1. When finding a precise type hint that satisfies both mypy and unit tests is not cost-effective, using a less specific type (e.g., Any) is an acceptable trade-off, especially if it improves upon the previous state.

@ohmayr
ohmayr force-pushed the bson-pr2-reads-decoding branch from e507507 to 6c5ea1d Compare September 14, 2026 22:05
@ohmayr
ohmayr force-pushed the bson-pr2-reads-decoding branch from 6c5ea1d to 2bf84e6 Compare September 14, 2026 22:59
@ohmayr
ohmayr force-pushed the bson-pr2-reads-decoding branch 2 times, most recently from 134b979 to 983bcde Compare September 14, 2026 23:15
@ohmayr
ohmayr force-pushed the bson-pr2-reads-decoding branch from 983bcde to 57267de Compare September 14, 2026 23:16
@ohmayr
ohmayr force-pushed the bson-pr2-reads-decoding branch from 57267de to cacf194 Compare September 14, 2026 23:25
@ohmayr
ohmayr force-pushed the bson-pr2-reads-decoding branch from cacf194 to e000d4e Compare September 14, 2026 23:29
@ohmayr
ohmayr force-pushed the bson-pr2-reads-decoding branch 2 times, most recently from b50ef69 to e04f8c9 Compare September 14, 2026 23:56
@ohmayr
ohmayr force-pushed the bson-pr2-reads-decoding branch from e04f8c9 to 91ba50f Compare September 15, 2026 00:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant