Skip to content

Correct canonical witness lookup and null handling for v2.2.1 #110

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
0xBreadguy wants to merge 1 commit into
base: main
Choose a base branch
from
+86 −41
Open

Correct canonical witness lookup and null handling for v2.2.1 #110

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 30 additions & 12 deletions docs/dev/rpc/reference/mega_getBlockWitness.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,16 +35,24 @@ Block lookup key.
Payload-attributes hash for an OP payload lookup.
Must be supplied together with `parentHash` and cannot be combined with `blockHash`.

{% hint style="warning" %}
A `blockNumber`-only lookup returns the first stored witness at that height and is not reorg-safe.
Use `blockNumber` together with `blockHash`, or the paired `parentHash` and `attributesHash`, for production verification.
As of mega-reth v2.2.1, a `blockNumber`-only lookup first resolves the node's local canonical hash at that height.
If the hash is unknown, it returns `result: null`, even when stored witnesses exist; retry after the node catches up.
If the hash is known, it returns the witness matching that hash, or `null` if unavailable.
It does not select the first stored row or fall back to an orphaned witness.

{% hint style="info" %}
The local canonical view can change across a reorg.
Use `blockNumber` with `blockHash`, or the paired `parentHash` and `attributesHash`, to pin the requested identity.
These explicit selectors are unchanged by v2.2.1; pinning a block does not guarantee it remains canonical.
{% endhint %}

## Result

**`result`** string
**`result`** string or `null`

`v0:` followed by a base64-encoded zstd-compressed witness blob.
When available, `v0:` followed by a base64-encoded zstd-compressed witness blob.
A missing matching witness, or an unknown canonical hash for a number-only lookup, returns `null` rather than a `-32000` or `-32603` error.
Check for `null` before decoding; see [witness availability and decoding](../../../node/witness.md#response).

## Comparison with Ethereum Standard JSON-RPC

Expand All @@ -64,12 +72,12 @@ The gateway does not cache witness responses. In an outer batch, if any witness

The `| Scope |` column distinguishes method failures from gateway policy errors.

| Code | Scope | Message | When it happens |
| -------- | ---------------- | ------------------- | --------------------------------------------------------------------------- |
| `-32602` | Request | Invalid params | `blockNumber` missing, invalid hex value, or invalid hash field combination |
| `-32603` | Method | Internal error | No witness exists for the requested keys, or the witness service failed |
| `-32005` | Transport/policy | Rate limit exceeded | The caller exceeds the public gateway's simple read budget. |
| `-32099` | Transport/policy | Payload too large | The request body exceeds the 128 KiB public endpoint limit. |
| Code | Scope | Message | When it happens |
| -------- | ---------------- | ------------------- | ------------------------------------------------------------------------------- |
| `-32602` | Request | Invalid params | `blockNumber` missing, invalid hex value, or invalid hash field combination |
| `-32603` | Method | Internal error | Internal failure resolving the canonical hash, reading, or encoding the witness |
| `-32005` | Transport/policy | Rate limit exceeded | The caller exceeds the public gateway's simple read budget. |
| `-32099` | Transport/policy | Payload too large | The request body exceeds the 128 KiB public endpoint limit. |

See also [Error Codes](../error-codes.md).

Expand Down Expand Up @@ -103,9 +111,19 @@ Outcome: success
}
```

An unavailable witness produces this response:

```json
{
"jsonrpc": "2.0",
"id": 1,
"result": null
}
```

## Sources

- Spec: EIP-1474 for JSON-RPC framing and error conventions; this method is an extension or legacy compatibility method.
- Code: `git@github.com:megaeth-labs/mega-reth.git @ ab60376631228edab3a6df180f295280bad26e93: crates/megaeth/rpc/src/witness.rs`
- Code: [mega-reth v2.2.1 witness handler](https://github.com/megaeth-labs/mega-reth/blob/v2.2.1/crates/megaeth/rpc/src/witness.rs) — selector resolution and nullable results.
- Code: `git@github.com:megaeth-labs/mega-rpc.git @ 06aa35aa95d569c227cc25d2aa12834eb0458aa0: workers/src/services/batch/batch-processor.ts`
- Probe: MegaETH Mainnet public endpoint, July 24, 2026
85 changes: 56 additions & 29 deletions docs/node/witness.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,26 +26,31 @@ Any client — an operator running [`stateless-validator`](https://github.com/me
`<keys>` is a JSON object that identifies the block.
`blockNumber` is always required; pair it with `blockHash` to pin the witness to a specific block.

| Field | Type | Required | Description |
| ------------- | ----------------- | -------- | ------------------------------------------------------------------------------------------------------- |
| `blockNumber` | `Quantity` (hex) | Yes | Block number, 0x-prefixed lowercase hex (e.g. `"0x7fd"`). |
| `blockHash` | `Data` (32 bytes) | No | 0x-prefixed lowercase hash of the block to fetch the witness for. Pins the result, so it is reorg-safe. |
| Field | Type | Required | Description |
| ------------- | ----------------- | -------- | ------------------------------------------------------------------------------------------------------------------ |
| `blockNumber` | `Quantity` (hex) | Yes | Block number, 0x-prefixed lowercase hex (e.g. `"0x7fd"`). |
| `blockHash` | `Data` (32 bytes) | No | 0x-prefixed lowercase hash of the block to fetch the witness for. Pins the requested block identity across reorgs. |

### Lookup modes

The combination of fields chosen determines the lookup mode.
**Always pass `blockHash` when one is available.**
The `blockNumber`-only mode does not pin the result to a specific block and can return a witness for the wrong fork.
As of mega-reth v2.2.1, a number-only lookup resolves the node's local canonical hash before selecting a witness.
**Pass `blockHash` when you need a specific block.**

| Mode | Recommendation | When to use |
| --------------------------- | -------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `blockNumber` + `blockHash` | **Preferred** | The caller already knows the canonical block hash (e.g. fetched from `eth_getBlockByNumber` first). The witness is pinned to that exact block, so the result is reorg-safe. |
| `blockNumber` | **Avoid** | Last-resort convenience. The backend returns the first stored witness it finds at that height — there is **no guarantee** the returned witness is for the block you expect. Only use when you cannot obtain a hash and can independently verify the response. |
| Mode | Selection | When to use |
| --------------------------- | ------------------------------------------------------------------------ | ---------------------------------------------------------- |
| `blockNumber` + `blockHash` | Matches the supplied block hash | The caller already knows the block it wants to verify |
| `blockNumber` | Resolves the local canonical hash at that height, then matches that hash | The caller wants the node's canonical block at lookup time |

{% hint style="warning" %}
Calling `mega_getBlockWitness` with `blockNumber` only is unsafe for any caller that needs a specific block.
The server returns the first witness it finds at that height, which is non-deterministic, may correspond to a non-canonical fork, and may change between calls.
Always pair `blockNumber` with `blockHash` unless you are willing to validate the response yourself (e.g. by re-deriving the block hash from the returned witness against an independently-trusted header).
If the canonical hash is unknown, the number-only lookup returns `result: null`, even if stored witnesses exist at that height.
This can happen during resynchronization after an unwind or when the requested height is beyond the local head.
Retry after the node catches up.
When the canonical hash is known, the handler returns its matching witness, or `null` if that witness is unavailable; it does not fall back to the first stored row or an orphaned witness.

{% hint style="info" %}
A number-only lookup follows the node's canonical view, which can change across a reorg.
Supplying `blockHash` pins the requested block identity; it does not guarantee that the block remains canonical.
Hash-keyed and payload-keyed lookups are unchanged by v2.2.1.
For all supported selectors, see the [RPC reference](../dev/rpc/reference/mega_getBlockWitness.md).
{% endhint %}

### Examples
Expand All @@ -64,7 +69,7 @@ Always pair `blockNumber` with `blockHash` unless you are willing to validate th

{% endtab %}

{% tab title="By block number only (unsafe)" %}
{% tab title="By block number (local canonical)" %}

```json
[{ "blockNumber": "0x7fd" }]
Expand All @@ -75,7 +80,7 @@ Always pair `blockNumber` with `blockHash` unless you are willing to validate th

## Response

The response `result` is a single string of the form `<version>:<base64-payload>`.
When a witness is available, `result` is a string of the form `<version>:<base64-payload>`.

```json
{
Expand All @@ -90,16 +95,28 @@ The response `result` is a single string of the form `<version>:<base64-payload>
| `version` | Encoding version. Currently `v0`. Bumped if the witness payload format ever changes — clients must check the prefix. |
| `payload` | Base64-encoded, Zstd-compressed [bincode](https://docs.rs/bincode/2.0.1/bincode) tuple ([`SaltWitness`](#saltwitness--main-state-trie), [`MptWitness`](#mptwitness--withdrawals-storage-trie)). |

When no matching witness is available, the node returns a successful JSON-RPC response with `result: null`:

```json
{
"jsonrpc": "2.0",
"id": 1,
"result": null
}
```

Handle `null` before decoding the payload.
It means the witness is unavailable, not a `-32000` or `-32603` error.
Retry after catch-up for a temporarily unknown canonical hash; missing historical coverage may remain unavailable.

### Errors

| Code | Cause |
| -------- | ------------------------------------------------------------------------------------------------------------- |
| `-32602` | Invalid params — malformed JSON, missing `blockNumber`, unparseable hex, or an invalid parameter combination. |
| `-32603` | Witness not found for the requested keys, or a server-side failure while fetching it. |
| Code | Cause |
| -------- | ------------------------------------------------------------------------------------------------------------------ |
| `-32602` | Invalid params — malformed parameters, missing `blockNumber`, unparseable hex, or an invalid parameter combination |
| `-32603` | Internal failure while resolving the canonical hash, reading, or encoding a witness |

A missing witness and a server-side fetch failure both surface as `-32603`; the error `message` distinguishes them.
To treat a missing witness as an expected outcome, match both `code == -32603` and an error `message` containing `Witness not found`.
Treating `-32603` alone as a hard failure produces false alerts for testnet blocks outside the covered range — see [Historical coverage](#historical-coverage) below.
Do not match a `Witness not found` error message to detect unavailability; check for `result: null`.
The RPC layer in front of the witness service may also return standard JSON-RPC transport codes: `-32700` (parse error), `-32600` (invalid request).

### Historical coverage
Expand All @@ -111,11 +128,11 @@ Witness availability on the public RPC depends on the network:
| Mainnet | All blocks. |
| Testnet | As of 2026-07-24: block `1000` (a permanent test fixture) and blocks `>= 10325789`. Older testnet witnesses are not guaranteed and cannot be backfilled. |

Requesting a testnet block outside this range typically returns `Witness not found`.
A request with no available witness returns `result: null`.

### Decoding pipeline

To turn the response string back into a witness, apply these steps in order:
After checking for an RPC error or `result: null`, decode an available response string in this order:

1. Verify the string starts with the literal prefix `v0:` and strip it.
2. Base64-decode the payload (standard alphabet, padded).
Expand Down Expand Up @@ -271,7 +288,9 @@ Replace `<BLOCK_NUMBER>` (0x-prefixed lowercase hex) and `<BLOCK_HASH>` with val
The pipeline below assumes `jq` and `zstd` are on `PATH` — install them via `brew install jq zstd` on macOS or `apt install jq zstd` on Debian/Ubuntu.

```bash
curl -sS https://mainnet.megaeth.com/rpc \
set -euo pipefail

response=$(curl -fsS https://mainnet.megaeth.com/rpc \
-X POST -H 'Content-Type: application/json' \
-d '{
"jsonrpc": "2.0",
Expand All @@ -281,8 +300,15 @@ curl -sS https://mainnet.megaeth.com/rpc \
"blockNumber": "<BLOCK_NUMBER>",
"blockHash": "<BLOCK_HASH>"
}]
}' \
| jq -r '.result' \
}')

encoded=$(printf '%s' "$response" | jq -er '
if .error then error(.error.message)
elif .result == null then error("Witness unavailable; check sync and coverage before retrying")
else .result end
')

printf '%s' "$encoded" \
| sed 's/^v0://' \
| base64 --decode \
| zstd -d \
Expand All @@ -293,6 +319,7 @@ curl -sS https://mainnet.megaeth.com/rpc \

## Related pages

- [mega-reth v2.2.1 witness handler](https://github.com/megaeth-labs/mega-reth/blob/v2.2.1/crates/megaeth/rpc/src/witness.rs) — canonical lookup and nullable response behavior.
- [Stateless Validation](stateless-validation.md) — the operator guide for the reference client that consumes this RPC.
- [stateless-validator source](https://github.com/megaeth-labs/stateless-validator) — Rust implementation of the witness fetcher and verifier.
- [SALT](https://github.com/megaeth-labs/salt) — the authenticated key-value store that produces `SaltWitness`.
Loading