Skip to content

Add Reference section for the HTTP API #79

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
cdebled merged 5 commits into from
Aug 25, 2026
Merged

Add Reference section for the HTTP API #79

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
b8af55b
Add reference/http-api section documenting the /api/fetch endpoint
cdebled Aug 5, 2026
a444bae
Slim usage/api.mdx down to a practical guide, linking to the new refe…
cdebled Aug 5, 2026
6409ea5
Address review: document the country parameter's two-letter format co…
cdebled Aug 7, 2026
d3a52b7
Merge branch 'main' into http-api-reference
cdebled Aug 25, 2026
579330e
Merge branch 'main' into http-api-reference
cdebled Aug 25, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions src/content/reference/_meta.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ const meta: MetaRecord = {
collapsed: true,
},
},
'http-api': 'HTTP API',

@arrufat arrufat Aug 6, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Heads up that all four PRs in the series (#76, #77, #78, this one) create this same file with a different single key, so the last three to merge each need a small conflict resolution here, and the final key order decides the sidebar order of the Reference section.

@cdebled cdebled Aug 7, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I chose to create independent PRs, all based on the main branch, so as not to constrain which one we want to merge first. But sure, I'll handle the conflicts after each merge (not a complex conflict). As for the order, I'm planning: CLI, HTTP API, MCP tools, Pandascript.

'mcp-tools': 'MCP tools',
pandascript: 'PandaScript',
}
Expand Down
70 changes: 70 additions & 0 deletions src/content/reference/http-api.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
---
title: HTTP API
description: Reference of the Lightpanda Cloud HTTP API endpoints.
---

# HTTP API

Fetch a URL and retrieve the rendered page. See [how to use the HTTP API](/usage/api) for practical documentation.

## Fetch a URL

The `POST /api/fetch` endpoint returns a JSON message containing the page's content.

### Request

The request requires a bearer authorization header with your token and the `Content-Type: application/json` header.

The endpoint accepts the following parameters:

| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
| `url` | string | Yes | — | The URL to fetch. |
| `output_format` | `html` \| `markdown` | No | `html` | — |
| `wait_ms` | uint | No | `5000` | Time to wait before retrieving the page, in milliseconds. `0` also falls back to the default; it doesn't mean "don't wait". |
| `wait_event` | `DOMContentLoaded` \| `load` \| `networkAlmostIdle` \| `networkIdle` | No | `networkIdle` | Event to wait for before retrieving the page. |
| `raw` | bool | No | `false` | If true, the raw HTML or markdown is returned directly, without the JSON wrapper, with `Content-Type: text/html` or `text/plain` respectively. |
| `proxy_name` | string | No | `fast_dc` | — |
| `country` | string | No | — | Two-letter country code, case-insensitive. Only used with the `datacenter` proxy. |

```sh copy
curl -XPOST \
-H "Authorization: Bearer YOUR_TOKEN" \
-H "Content-Type: application/json" \
https://euwest.cloud.lightpanda.io/api/fetch \
--data '{
"url": "https://lightpanda.io",
"output_format": "markdown",
"wait_ms": 50000,
"wait_event": "networkIdle",
"raw": false
}'
```

### Response

If the request sets `raw` to `false`, a JSON response is returned.

Comment thread
cdebled marked this conversation as resolved.
| Field | Type | Description |
|---|---|---|
| `data` | string | The page's content, in the format specified by the `output_format` parameter. |
| `status` | int | The response status code. |
| `headers` | object, optional | The HTTP headers returned by the server. |

```json copy
{
"data": "\n# Example Domain\n\nThis domain is for use in documentation examples without needing permission. Avoid use in operations.\n\n[Learn more](https://iana.org/domains/example)\n",
"status": 200,
"headers": {
"age": "10026",
"allow": "GET, HEAD",
"cf-cache-status": "HIT",
"cf-ray": "9f0dd9374a2320ab-IAD",
"content-encoding": "br",
"content-type": "text/html",
"date": "Thu, 23 Apr 2026 15:19:47 GMT",
"last-modified": "Sat, 18 Apr 2026 00:51:00 GMT",
"server": "cloudflare"
}
}
```
78 changes: 3 additions & 75 deletions src/content/usage/api.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,92 +2,20 @@
title: API
description: Use HTTP API calls to get Lightpanda's results directly.
---
import { Tabs } from 'nextra/components'

# HTTP API

## Connect

<Tabs items={['Cloud']}>
<Tabs.Tab>
Lightpanda Cloud provides an HTTP API to easily retrieve results from Lightpanda without the burden of writing and running a CDP script.

### Authentication

Every HTTP API request must include a bearer authorization header with your token.

```
Authorization: Bearer YOUR_TOKEN
```

Example with curl:
```sh copy
curl -XPOST \
-H "Authorization: Bearer YOUR_TOKEN" \
-H "Content-Type: application/json" \
https://euwest.cloud.lightpanda.io/api/fetch \
--data '{"url":"https://lightpanda.io"}'
```
</Tabs.Tab>
</Tabs>

See [Getting started on Lightpanda Cloud](/run-on-lightpanda-cloud/getting-started) to create an account, generate your token, and pick the URL for your region.

## Fetch a URL

The main API lets you fetch a URL and retrieve the rendered page in HTML or markdown format.
The `POST /api/fetch` endpoint returns a JSON message containing the page's content.

### Request

The request requires the `Content-Type: application/json` header.

The endpoint accepts the following parameters:

* `url`: string, required: the URL to fetch.
* `output_format`: `html` or `markdown`, optional, default `html`.
* `wait_ms`: uint, optional, default `5000`: time to wait before retrieving the page, in milliseconds.
* `wait_event`: `DOMContentLoaded`, `load`, `networkAlmostIdle` or `networkIdle`, optional, default `networkIdle`: event to wait for before retrieving the page.
* `raw`: bool, optional, default `false`: if true, the raw HTML or markdown is returned directly, without the JSON wrapper.
* `proxy_name`: optional, default `fast_dc`.
* `country`: optional: only used with the `datacenter` proxy.

```json copy
{
"url": "https://lightpanda.io",
"output_format": "markdown",
"wait_ms": 50000,
"wait_event": "networkIdle",
"raw": false
}
```

### Response

If the request sets `raw` to `false`, a JSON response is returned.

* `data`: string: the page's content, in the format specified by the `output_format` parameter.
* `status`: uint: the response status code.
* `headers`: array, optional: the HTTP headers returned by the server.

```json copy
{
"data": "\n# Example Domain\n\nThis domain is for use in documentation examples without needing permission. Avoid use in operations.\n\n[Learn more](https://iana.org/domains/example)\n",
"status": 200,
"headers": {
"age": "10026",
"allow": "GET, HEAD",
"cf-cache-status": "HIT",
"cf-ray": "9f0dd9374a2320ab-IAD",
"content-encoding": "br",
"content-type": "text/html",
"date": "Thu, 23 Apr 2026 15:19:47 GMT",
"last-modified": "Sat, 18 Apr 2026 00:51:00 GMT",
"server": "cloudflare"
}
}
```
The API lets you fetch a URL and retrieve the rendered page in HTML or markdown format. Find the full list of parameters and the response format in the [HTTP API reference](/reference/http-api).

### Complete example with curl
For example, to fetch a page as markdown:

```sh copy
$ curl -XPOST \
Expand Down
Loading