feat(bigquery): add ArrowQueryPageFetcher for Arrow query result pagination - #14378
jinseopkim0 wants to merge 3 commits into
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces ArrowQueryPageFetcher, a NextPageFetcher implementation designed to paginate query results returned in Arrow format by reading from a gRPC storage read stream. It also adds comprehensive unit tests in ArrowQueryPageFetcherTest.java. The review feedback highlights serialization issues in ArrowQueryPageFetcher: the buffer queue should be marked as transient to prevent duplicate data upon deserialization, and it must be lazily initialized in getNextPage() to avoid a NullPointerException after deserialization.
125ee8e to
21d547f
Compare
|
@gemini-code-assist review |
There was a problem hiding this comment.
Code Review
This pull request introduces the ArrowQueryPageFetcher class in BigQueryImpl.java to support paginating query results returned in Arrow format via the gRPC storage read stream. It also adds a comprehensive suite of unit tests in ArrowQueryPageFetcherTest.java to verify its behavior, including pagination, limit handling, error cases, and serialization. There are no review comments, so no feedback is provided.
|
@gemini-code-assist review |
There was a problem hiding this comment.
Code Review
This pull request introduces the ArrowQueryPageFetcher class in BigQueryImpl.java to handle pagination for query results returned in Arrow format via the gRPC storage read stream. It also updates the read settings configuration to use NoHeaderProvider as a default, and adds a comprehensive suite of unit tests in ArrowQueryPageFetcherTest.java to verify single and multi-page fetching, max results handling, error conditions, and serialization. There are no review comments, and I have no feedback to provide.
This PR introduces
ArrowQueryPageFetcherto handle pagination for queries executing withQueryResultsFormat.ARROW.Summary of Changes
ArrowQueryPageFetcherimplementingNextPageFetcher<FieldValueList>.ArrowDeserializer.loadArrowRowsto deserialize Arrow record batches intoFieldValueListcollections with offset andmaxResultsbounding.ArrowQueryPageFetcherTestcovering single-page, multi-page, max-results, and serialization behaviors.