diff --git a/docs/how-to/index.md b/docs/how-to/index.md index c98e81925..5af26d606 100644 --- a/docs/how-to/index.md +++ b/docs/how-to/index.md @@ -2,6 +2,13 @@ This page contains how-to documentation for installing, configuring, and running CipherStash Proxy. +> [!IMPORTANT] +> This guide is for Proxy 3.x and EQL v3. Proxy 2.x uses EQL v2's separate +> encrypted-column and search-index configuration; use the +> [Proxy 2.2 documentation](https://github.com/cipherstash/proxy/tree/v2.2.4/docs) +> when operating a 2.x deployment. Do not mix configuration instructions from +> the two versions. + ## Table of contents - [Installing Proxy](#installing-proxy) @@ -11,6 +18,7 @@ This page contains how-to documentation for installing, configuring, and running - [Running Proxy locally](#running-proxy-locally) - [Setting up the database schema](#setting-up-the-database-schema) - [Creating columns with the right types](#creating-columns-with-the-right-types) + - [Bloom-filter text matching](#bloom-filter-text-matching) - [Encrypting data in an existing database](#encrypting-data-in-an-existing-database) ## Installing Proxy @@ -27,7 +35,10 @@ services: db: # Your Postgres container config proxy: - image: cipherstash/proxy:latest + # Pin a Proxy 3.0 release — see https://hub.docker.com/r/cipherstash/proxy/tags + # for available releases. `latest` may point at a newer major or minor version + # whose configuration differs from this guide. + image: cipherstash/proxy:3.0 container_name: proxy ports: - 6432:6432 @@ -227,6 +238,34 @@ When deploying CipherStash Proxy into production environments with real data, we To see more examples of how to modify your database schema, check out [the example schema](../sql/schema-example.sql) from [Getting started](#getting-started). +### Bloom-filter text matching + +The `eql_v3_text_match`, `eql_v3_text_search`, and +`eql_v3_text_search_ore` domains carry a Bloom-filter (`bf`) term for fuzzy +text matching. Proxy derives the match-index configuration from the domain; +there is no separate search-config row or per-column SQL configuration in EQL +v3. + +Proxy currently uses these fixed parameters: + +| Parameter | Value | Effect | +|-----------|-------|--------| +| n-gram length | `3` | Text is tokenized into overlapping three-character tokens. Inputs shorter than three characters do not produce a match token. | +| Bloom filter size (`m`) | `2048` bits | Sets the size of the probabilistic match term. | +| Hash count (`k`) | `6` | Sets how many Bloom-filter positions each token occupies. | + +Before tokenization, Proxy strips a leading or trailing `%` and then a leading +or trailing `_`. It applies this preprocessing to stored values as well as +query operands, even when those characters are literal data. Proxy then +generates overlapping three-character tokens and downcases each token. +Wildcard characters within the input are not interpreted specially. + +Bloom matching is probabilistic: matching rows contain every bit set by the +query term, but unrelated values can occasionally be false positives. Changing +the token length, `m`, or `k` would make existing stored and query terms +incompatible, so EQL v3 does not expose those values as per-column tuning +options. + ## Encrypting data in an existing database CipherStash Proxy includes an `encrypt` tool – a CLI application to encrypt existing data, or to apply index changes after changes to the encryption configuration of a protected database. diff --git a/docs/reference/index.md b/docs/reference/index.md index 335d243dc..e3abc8815 100644 --- a/docs/reference/index.md +++ b/docs/reference/index.md @@ -2,6 +2,11 @@ This page contains reference documentation for configuring CipherStash Proxy and its features. +> [!IMPORTANT] +> This reference describes Proxy 3.x and EQL v3. For a Proxy 2.x deployment, +> use the [Proxy 2.2 reference](https://github.com/cipherstash/proxy/tree/v2.2.4/docs/reference); +> EQL v2 configuration is not compatible with EQL v3 domain types. + ## Table of contents - [Proxy config options](#proxy-config-options) @@ -118,9 +123,9 @@ connection_timeout = "300000" # Env: CS_DATABASE__WITH_TLS_VERIFICATION with_tls_verification = "false" -# Encrypt configuration reload interval in sec -# Sets how frequently Encrypted index configuration should be reloaded -# The configuration specifies the encrypted columns in the database +# EQL domain/schema reload interval in sec +# Sets how frequently Proxy refreshes the encryption configuration it derives +# from EQL v3 column domain types in the database schema # Optional # Default: `60` # Env: CS_DATABASE__CONFIG_RELOAD_INTERVAL diff --git a/docs/reference/searchable-json.md b/docs/reference/searchable-json.md index 24e2a9f20..80681904b 100644 --- a/docs/reference/searchable-json.md +++ b/docs/reference/searchable-json.md @@ -2,6 +2,12 @@ This document outlines the supported JSONB functions and operators in CipherStash Proxy for encrypted data. +> [!IMPORTANT] +> This page is for Proxy 3.x and EQL v3. Proxy 2.x requires a separate EQL v2 +> `ste_vec` search configuration; follow the +> [Proxy 2.2 searchable JSON documentation](https://github.com/cipherstash/proxy/blob/v2.2.4/docs/reference/searchable-json.md) +> for a 2.x deployment. + ## Table of Contents @@ -36,76 +42,14 @@ This document outlines the supported JSONB functions and operators in CipherStas EQL v3 encrypted-JSON columns are self-configuring: the `eql_v3_json_search` domain type is the SteVec (searchable encrypted JSON) configuration, so the -column type alone enables JSON search. There is no separate -`add_search_config` call as in EQL v2. +column type alone enables JSON search. Do not create a separate search +configuration for this column. > **Note:** JSONB literals in INSERT and UPDATE statements work directly without explicit `::jsonb` type casts. The proxy infers the JSONB type from the target column and handles encryption transparently. -#### Configuration options - -> **EQL v2 legacy:** In EQL v2 the `ste_vec` index was configured explicitly via -> `add_search_config`, and the options below (and the `add_search_config` examples -> in this section) describe that mechanism. In EQL v3 the `eql_v3_json_search` -> domain type carries a fixed default configuration, so these options are not -> set per-column via SQL. The descriptions are retained to explain the indexing -> behaviour. - -The `ste_vec` index configuration accepts the following options: - -| Option | Type | Default | Description | -|--------|------|---------|-------------| -| `prefix` | string | (required) | Unique prefix for the index, typically `table/column` | -| `term_filters` | array | `[]` | Filters applied to indexed terms (e.g., `[{"kind": "downcase"}]`) | -| `array_index_mode` | string or object | `"all"` | Controls which array selectors are generated during indexing | - -#### Array index mode - -The `array_index_mode` option controls how arrays within JSONB documents are indexed. This affects which JSONPath selectors can be used to query array data. - -**Preset values:** - -- `"all"` (default) - Generates all selector types. This is backwards compatible with existing configurations. -- `"none"` - Disables array indexing entirely. - -**Object form for fine-grained control:** - -```json -{ - "item": true, - "wildcard": true, - "position": false -} -``` - -| Selector | JSONPath | Description | -|----------|----------|-------------| -| `item` | `[@]` | EQL array element selector for functions like `jsonb_array_length` | -| `wildcard` | `[*]` | Standard JSONPath wildcard for iterating array elements | -| `position` | `[0]`, `[1]`, etc. | Positional access to specific array indices | - -**Example with array_index_mode:** - -```sql -SELECT eql_v2.add_search_config( - 'cipherstash', - 'encrypted_jsonb', - 'ste_vec', - 'jsonb', - '{"prefix": "cipherstash/encrypted_jsonb", "array_index_mode": "all"}' -); -``` - -**Example disabling positional indexing:** - -```sql -SELECT eql_v2.add_search_config( - 'events', - 'payload', - 'ste_vec', - 'jsonb', - '{"prefix": "events/payload", "array_index_mode": {"item": true, "wildcard": true, "position": false}}' -); -``` +Proxy derives a unique `table/column` selector prefix, applies no term filters, +and indexes array item, wildcard, and positional selectors. These settings are +fixed in EQL v3 and are not configured per column. ### JSON document structure @@ -734,4 +678,4 @@ The following JSON data types are fully supported: - Non-existent fields return `NULL` - Invalid JSONPath expressions may cause query errors - Type mismatches in comparisons follow PostgreSQL JSONB semantics -- Array functions on non-arrays return empty results \ No newline at end of file +- Array functions on non-arrays return empty results