feat: support async table loads and append mode

CHANGELOG.md

@@ -9,7 +9,17 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Changed
 
+- feat: support async table loads and append mode
 - feat(uploads): support streaming uploads with on-demand part URLs
+- **Breaking:** results and query runs are now scoped to a database via the
+  required `X-Database-Id` header. The generated `ResultsApi` / `QueryRunsApi`
+  methods (`get_result`, `list_results`, `get_query_run`, `list_query_runs`)
+  gain a required `x_database_id` argument, and the hand-written ergonomic
+  helpers match: `hotdata.arrow.ResultsApi.get_result_arrow` /
+  `stream_result_arrow` and `hotdata.query.QueryApi.wait_for_result` take the
+  database id, and `QueryApi.query`'s truncation auto-follow forwards the
+  query's database scope (the `X-Database-Id` header, or the request-body
+  `database_id` when no header is set) to its result and query-run fetches.
 
 ## [0.5.0] - 2026-06-28
 

README.md

@@ -77,17 +77,21 @@ from hotdata.arrow import ResultsApi
 with ApiClient(Configuration(api_key="...", workspace_id="...")) as client:
     results = ResultsApi(client)
 
+    # Results are scoped to a database via the required `X-Database-Id` header,
+    # so pass the id of the database the query ran in.
+    database_id = "your_database_id"
+
     # Buffered: returns a pyarrow.Table.
-    table = results.get_result_arrow(result_id)
+    table = results.get_result_arrow(result_id, database_id)
 
     # Streaming: yields a pyarrow.RecordBatchStreamReader without
     # materializing the full table in memory.
-    with results.stream_result_arrow(result_id) as reader:
+    with results.stream_result_arrow(result_id, database_id) as reader:
         for batch in reader:
             ...
 ```
 
-Both methods accept `offset` and `limit` for pagination. They raise `hotdata.arrow.ResultNotReadyError` if the result is still pending or processing — poll `results.get_result(result_id)` until `status == "ready"` first.
+Both methods accept `offset` and `limit` for pagination. They raise `hotdata.arrow.ResultNotReadyError` if the result is still pending or processing — poll `results.get_result(result_id, database_id)` until `status == "ready"` first.
 
 ## File uploads
 

docs/AsyncQueryResponse.md

@@ -1,6 +1,6 @@
 # AsyncQueryResponse
 
-Response returned when a query is submitted asynchronously (202 Accepted).  Poll GET /query-runs/{id} to track progress. Once status is \"succeeded\", retrieve results via GET /results/{result_id}.
+Response returned when a query is submitted asynchronously (202 Accepted).  Poll GET /query-runs/{id} to track progress, sending the same `X-Database-Id` header used to submit the query — the endpoint is scoped to that database and returns 400 without it. Once status is \"succeeded\", retrieve results via GET /results/{result_id}.
 
 ## Properties
 
@@ -9,7 +9,7 @@ Name | Type | Description | Notes
 **query_run_id** | **str** | Unique identifier for the query run. | 
 **reason** | **str** | Human-readable reason why the query went async (e.g., caching tables for the first time). | [optional] 
 **status** | **str** | Current status of the query run. | 
-**status_url** | **str** | URL to poll for query run status. | 
+**status_url** | **str** | URL to poll for query run status. Requires the same `X-Database-Id` header used to submit the query. | 
 
 ## Example
 

docs/ConnectionsApi.md

@@ -810,7 +810,7 @@ This endpoint does not need any parameter.
 
 Load managed table from upload
 
-Publish a previously-uploaded file as the new contents of a managed table. CSV, JSON, and Parquet uploads are supported; the format is auto-detected from the upload's `Content-Type` and file contents, or set explicitly via the `format` field. Only `mode = "replace"` is supported. Concurrent loads against the same upload return 409.
+Publish a previously-uploaded file as the new contents of a managed table. CSV, JSON, and Parquet uploads are supported; the format is auto-detected from the upload's `Content-Type` and file contents, or set explicitly via the `format` field. If the target table (or its schema) has not been declared yet, it is created automatically as part of the load — declaring tables up front is optional. `mode` selects how the upload is applied: `replace` overwrites the table's contents, `append` inserts the uploaded rows on top of the existing data. Concurrent loads against the same upload return 409. Set `async` to run the load in the background and get back a job ID to poll; add `async_after_ms` to wait briefly for it to finish before falling back to a job ID.
 
 ### Example
 
@@ -894,9 +894,10 @@ Name | Type | Description  | Notes
 | Status code | Description | Response headers |
 |-------------|-------------|------------------|
 **200** | Managed table loaded |  -  |
-**400** | Invalid request (bad mode, non-managed connection, bad parquet) |  -  |
-**404** | Connection or upload not found |  -  |
-**409** | Upload already consumed or in flight |  -  |
+**202** | Load accepted and running in the background; poll the returned job for status and result |  -  |
+**400** | Invalid request (bad mode, non-managed connection, invalid identifier, bad parquet) |  -  |
+**404** | Connection or upload not found, or the table was deleted |  -  |
+**409** | Upload already consumed or in flight, or the uploaded data changes a column's type incompatibly (only widening to a larger compatible type can be applied automatically); the existing data is unchanged and remains queryable |  -  |
 
 [[Back to top]](#) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to Model list]](../README.md#documentation-for-models) [[Back to README]](../README.md)
 

docs/CreateIndexRequest.md

@@ -7,6 +7,7 @@ Request body for POST .../indexes
 Name | Type | Description | Notes
 ------------ | ------------- | ------------- | -------------
 **var_async** | **bool** | When true, create the index as a background job and return a job ID for polling. | [optional] 
+**async_after_ms** | **int** | If set (requires `async` = true), wait up to this many milliseconds for the index build to finish: if it completes in time the index is returned (201), otherwise a 202 with a job ID to poll. Must be between 1000 and the server maximum; a value out of that range, or set without `async` = true, is rejected with 400. | [optional] 
 **columns** | **List[str]** | Columns to index. Required for all index types. | 
 **description** | **str** | User-facing description of the embedding (e.g., \"product descriptions\"). | [optional] 
 **dimensions** | **int** | Output vector dimensions. Some models support multiple dimension sizes (e.g., OpenAI text-embedding-3-small supports 512 or 1536). If omitted, the model's default dimensions are used | [optional] 

docs/DatabasesApi.md

@@ -712,7 +712,7 @@ This endpoint does not need any parameter.
 
 Load database table from upload
 
-Publish a previously-uploaded file as the new contents of a table on the database's default catalog. The database-scoped equivalent of the connection-scoped managed-table load — addressed by `database_id`, so no `default_connection_id` is needed. CSV, JSON, and Parquet uploads are supported; the format is auto-detected or set via `format`. Only `mode = "replace"` is supported. Concurrent loads against the same upload return 409.
+Publish a previously-uploaded file as the new contents of a table on the database's default catalog. The database-scoped equivalent of the connection-scoped managed-table load — addressed by `database_id`, so no `default_connection_id` is needed. CSV, JSON, and Parquet uploads are supported; the format is auto-detected or set via `format`. If the target table (or its schema) has not been declared yet, it is created automatically as part of the load — declaring tables up front is optional. `mode` selects how the upload is applied: `replace` overwrites the table's contents, `append` inserts the uploaded rows on top of the existing data. Concurrent loads against the same upload return 409. Set `async` to run the load in the background and get back a job ID to poll; add `async_after_ms` to wait briefly for it to finish before falling back to a job ID.
 
 ### Example
 
@@ -796,9 +796,10 @@ Name | Type | Description  | Notes
 | Status code | Description | Response headers |
 |-------------|-------------|------------------|
 **200** | Table loaded |  -  |
-**400** | Invalid request (bad mode, bad parquet) |  -  |
-**404** | Database, table, or upload not found |  -  |
-**409** | Upload already consumed or in flight |  -  |
+**202** | Load accepted and running in the background; poll the returned job for status and result |  -  |
+**400** | Invalid request (bad mode, invalid identifier, bad parquet) |  -  |
+**404** | Database or upload not found, or the table was deleted |  -  |
+**409** | Upload already consumed or in flight, or the uploaded data changes a column's type incompatibly (only widening to a larger compatible type can be applied automatically); the existing data is unchanged and remains queryable |  -  |
 
 [[Back to top]](#) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to Model list]](../README.md#documentation-for-models) [[Back to README]](../README.md)
 

docs/IndexesApi.md

@@ -99,6 +99,7 @@ Name | Type | Description  | Notes
 | Status code | Description | Response headers |
 |-------------|-------------|------------------|
 **201** | Index created |  -  |
+**202** | Index build accepted and running in the background; poll the returned job for status |  -  |
 **400** | Invalid request |  -  |
 **404** | Table not found |  -  |
 **500** | Internal server error |  -  |

docs/JobResult.md

@@ -24,6 +24,8 @@ Name | Type | Description | Notes
 **source_column** | **str** | Source text column for an embedding-backed vector index. A query searches it via `vector_distance(<source_column>, …)`; the indexed `columns` hold the generated embedding column instead. Absent for BM25, sorted, and direct (existing-column) vector indexes. | [optional] 
 **status** | [**IndexStatus**](IndexStatus.md) |  | 
 **updated_at** | **datetime** |  | 
+**arrow_schema_json** | **str** | Schema of the loaded table, as JSON. | 
+**row_count** | **int** | Total number of rows in the table after the load. | 
 
 ## Example
 

docs/LoadManagedTableRequest.md

@@ -1,13 +1,15 @@
 # LoadManagedTableRequest
 
-Request body for `POST /v1/connections/{connection_id}/schemas/{schema}/tables/{table}/loads`.  Publishes a previously-uploaded file as the new contents of the named managed table. CSV and JSON uploads are converted to columnar storage on load; Parquet uploads are published directly. `mode` is fixed to `\"replace\"` today; the field is kept in the request body so future modes (e.g. append) are an additive change.
+Request body for the managed-table load endpoints — the connection-scoped `POST /v1/connections/{connection_id}/schemas/{schema}/tables/{table}/loads` and the database-scoped equivalent.  Publishes a previously-uploaded file to the named table. CSV and JSON uploads are converted to columnar storage on load; Parquet uploads are published directly. `mode` selects whether the upload replaces the table's contents or is appended on top of them.
 
 ## Properties
 
 Name | Type | Description | Notes
 ------------ | ------------- | ------------- | -------------
+**var_async** | **bool** | When true, run the load as a background job and return a job ID to poll instead of blocking until it finishes. Recommended for large uploads, which can take longer than an HTTP request should stay open. | [optional] 
+**async_after_ms** | **int** | If set (requires `async` = true), wait up to this many milliseconds for the load to finish: if it completes in time the full result is returned (200), otherwise a 202 with a job ID to poll. Must be between 1000 and the server maximum; a value out of that range, or set without `async` = true, is rejected with 400. | [optional] 
 **format** | **str** | File format of the upload: `\"csv\"`, `\"json\"`, or `\"parquet\"`. Optional — when omitted, the format is auto-detected from the upload's `Content-Type` and, failing that, from the file contents. Provide it explicitly to override detection or when the contents are ambiguous. `\"json\"` expects newline-delimited JSON (one object per line), not a JSON array. | [optional] 
-**mode** | **str** | Load mode. Only `\"replace\"` is supported in this release. | 
+**mode** | **str** | How the upload is applied: `\"replace\"` overwrites the table's contents, `\"append\"` inserts the uploaded rows on top of the existing data. | 
 **upload_id** | **str** | ID of a previously-staged upload (see `POST /v1/files`). The upload is claimed atomically; concurrent loads against the same `upload_id` return 409. | 
 
 ## Example

docs/LoadManagedTableResponse.md

@@ -1,14 +1,14 @@
 # LoadManagedTableResponse
 
-Response body for `POST /v1/connections/{connection_id}/schemas/{schema}/tables/{table}/loads`.
+Result of a managed-table load: row count after the load plus the published table schema. Returned inline (`200`) for a synchronous load, and as the `GET /v1/jobs/{id}` result payload for a completed background load.
 
 ## Properties
 
 Name | Type | Description | Notes
 ------------ | ------------- | ------------- | -------------
 **arrow_schema_json** | **str** | Schema of the loaded table, as JSON. | 
 **connection_id** | **str** |  | 
-**row_count** | **int** | Total rows in the published parquet file. | 
+**row_count** | **int** | Total number of rows in the table after the load. | 
 **schema_name** | **str** |  | 
 **table_name** | **str** |  | 
 

docs/QueryRequest.md

@@ -7,7 +7,7 @@ Request body for POST /query
 Name | Type | Description | Notes
 ------------ | ------------- | ------------- | -------------
 **var_async** | **bool** | When true, execute the query asynchronously and return a query run ID for polling via GET /query-runs/{id}. The query results can be retrieved via GET /results/{id} once the query run status is \"succeeded\". | [optional] 
-**async_after_ms** | **int** | If set with async=true, wait up to this many milliseconds for the query to complete synchronously before returning an async response. Minimum 1000ms. Ignored if async is false. | [optional] 
+**async_after_ms** | **int** | If set (requires `async` = true), first attempt the query synchronously and wait up to this many milliseconds: if it finishes in time the full result is returned, otherwise an async response (a run id to poll) is returned. Must be at least 1000 and at most the server's configured maximum; a value out of that range, or set without `async` = true, is rejected with 400. | [optional] 
 **database_id** | **str** | Database to scope the query to (its id). Alternative to the `X-Database-Id` header — exactly one source must be provided. If both this field and the header are set and they disagree, the request is rejected with a 400. | [optional] 
 **default_catalog** | **str** | Catalog that unqualified table references resolve against within the query's database scope. Must name a catalog visible in the database (`default`, an attached catalog alias, or a system catalog). Defaults to `default` when omitted. | [optional] 
 **default_schema** | **str** | Schema that unqualified table references resolve against within the query's database scope. Defaults to `main` when omitted. Existence is not validated up front — an unknown schema surfaces as a \"table not found\" error at planning time. | [optional] 

docs/QueryRunsApi.md

@@ -9,11 +9,11 @@ Method | HTTP request | Description
 
 
 # **get_query_run**
-> QueryRunInfo get_query_run(id)
+> QueryRunInfo get_query_run(id, x_database_id)
 
 Get query run
 
-Get the status and details of a specific query run by ID.
+Get the status and details of a specific query run by ID, scoped to the database named by the required X-Database-Id header.
 
 ### Example
 
@@ -53,10 +53,11 @@ with hotdata.ApiClient(configuration) as api_client:
     # Create an instance of the API class
     api_instance = hotdata.QueryRunsApi(api_client)
     id = 'id_example' # str | Query run ID
+    x_database_id = 'x_database_id_example' # str | Database the query run belongs to (required)
 
     try:
         # Get query run
-        api_response = api_instance.get_query_run(id)
+        api_response = api_instance.get_query_run(id, x_database_id)
         print("The response of QueryRunsApi->get_query_run:\n")
         pprint(api_response)
     except Exception as e:
@@ -71,6 +72,7 @@ with hotdata.ApiClient(configuration) as api_client:
 Name | Type | Description  | Notes
 ------------- | ------------- | ------------- | -------------
  **id** | **str**| Query run ID | 
+ **x_database_id** | **str**| Database the query run belongs to (required) | 
 
 ### Return type
 
@@ -90,15 +92,18 @@ Name | Type | Description  | Notes
 | Status code | Description | Response headers |
 |-------------|-------------|------------------|
 **200** | Query run details |  -  |
-**404** | Query run not found |  -  |
+**400** | Missing or malformed X-Database-Id header |  -  |
+**404** | Query run or database not found |  -  |
 
 [[Back to top]](#) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to Model list]](../README.md#documentation-for-models) [[Back to README]](../README.md)
 
 # **list_query_runs**
-> ListQueryRunsResponse list_query_runs(limit=limit, cursor=cursor, status=status, saved_query_id=saved_query_id)
+> ListQueryRunsResponse list_query_runs(x_database_id, limit=limit, cursor=cursor, status=status, saved_query_id=saved_query_id)
 
 List query runs
 
+List query runs for the database named by the required X-Database-Id header.
+
 ### Example
 
 * Api Key Authentication (WorkspaceId):
@@ -136,14 +141,15 @@ configuration = hotdata.Configuration(
 with hotdata.ApiClient(configuration) as api_client:
     # Create an instance of the API class
     api_instance = hotdata.QueryRunsApi(api_client)
+    x_database_id = 'x_database_id_example' # str | Database to scope the query runs to (required)
     limit = 56 # int | Maximum number of results (optional)
     cursor = 'cursor_example' # str | Pagination cursor (optional)
     status = 'status_example' # str | Filter by status (comma-separated, e.g. status=running,failed) (optional)
     saved_query_id = 'saved_query_id_example' # str | Filter by saved query ID (optional)
 
     try:
         # List query runs
-        api_response = api_instance.list_query_runs(limit=limit, cursor=cursor, status=status, saved_query_id=saved_query_id)
+        api_response = api_instance.list_query_runs(x_database_id, limit=limit, cursor=cursor, status=status, saved_query_id=saved_query_id)
         print("The response of QueryRunsApi->list_query_runs:\n")
         pprint(api_response)
     except Exception as e:
@@ -157,6 +163,7 @@ with hotdata.ApiClient(configuration) as api_client:
 
 Name | Type | Description  | Notes
 ------------- | ------------- | ------------- | -------------
+ **x_database_id** | **str**| Database to scope the query runs to (required) | 
  **limit** | **int**| Maximum number of results | [optional] 
  **cursor** | **str**| Pagination cursor | [optional] 
  **status** | **str**| Filter by status (comma-separated, e.g. status=running,failed) | [optional] 
@@ -180,6 +187,8 @@ Name | Type | Description  | Notes
 | Status code | Description | Response headers |
 |-------------|-------------|------------------|
 **200** | List of query runs |  -  |
+**400** | Missing or malformed X-Database-Id header |  -  |
+**404** | Database not found |  -  |
 
 [[Back to top]](#) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to Model list]](../README.md#documentation-for-models) [[Back to README]](../README.md)