diff --git a/README.md b/README.md index c3cacc6..f1d477a 100644 --- a/README.md +++ b/README.md @@ -1,28 +1,60 @@ # Cortex -Cortex is a local-first AI workspace for Windows. It runs Ollama models on the -machine, keeps conversations and memory in local storage, and presents the -React/TypeScript interface inside a Python-owned pywebview/WebView2 window. -The normal launcher owns the backend, native window, and any development +Cortex is a local-first AI workspace for Windows. It runs models on the machine +-- either through Ollama or by loading a local `.gguf` file with its own managed +llama.cpp runtime -- keeps conversations and memory in local storage, and +presents the React/TypeScript interface inside a Python-owned pywebview/WebView2 +window. The normal launcher owns the backend, native window, and any development frontend process; Cortex does not open the user's installed browser. ## Current workspace -The interface is deliberately small: a thread list, a focused transcript, a +The interface is deliberately small: a chat library, a focused transcript, a composer with a local model picker, and settings for model, memory, appearance, and execution controls. -Screenshots are intentionally omitted while the curated capture set is being -finalized. The examples used for the next capture pass are staged locally in an -isolated profile and are not part of the repository. +![The Cortex workspace: chat library with groups on the left, a transcript with +rendered Markdown and a syntax-highlighted code block, per-message copy, +regenerate, and fork controls, and the composer model picker +below.](docs/images/workspace.png) + +Every response carries its own footer -- timestamp, token count, and tokens per +second -- with copy, regenerate, and fork controls that appear on hover. + +Settings keep model selection and generation defaults in one place, and the chat +model doubles as the title model so there is only one choice to make: + +![Cortex settings, AI Model section: top-p, top-k, and repeat penalty sliders, +context window and seed fields, a system instructions box, and a toggle to +bypass Cortex's default system prompt.](docs/images/settings.png) + +`Ctrl`/`Cmd`+`K` opens a command palette that reaches new chat, settings, theme, +model switching, and recent conversations without leaving the keyboard: + +![The Cortex command palette open over the workspace, listing chat actions, +installed models to switch to, and recent +chats.](docs/images/command-palette.png) + +> The application in these images is the real build. The conversations are a +> fixed demo workspace defined in `tools/screenshots/demo_server.py` -- no model +> is contacted during a capture, so the images can be regenerated byte for byte +> with `./tools/screenshots/capture.ps1`. ## What Cortex provides -- **Local chat.** Stream responses from an installed Ollama model with Markdown, +- **Local chat.** Stream responses from a local model with Markdown, syntax-highlighted fenced code, reasoning details, sources, code-block copy controls, retry/regenerate, and forked threads. Long transcripts render in a virtualized list so scroll performance stays smooth regardless of history length. +- **Two local runtimes.** Use models installed through Ollama, or point Cortex at + a folder of `.gguf` files and it serves them through its own managed llama.cpp + runtime -- no separate install. The `llama-server` binary is fetched once, + verified against a pinned SHA-256, and cached. GPU backend selection is + `auto` (try Vulkan, fall back to CPU), `vulkan`, or `cpu`. +- **Bring your own GGUF.** Download a model into the local folder by direct URL + or Hugging Face repo, then select it from the same picker as everything else. + Local files appear as `gguf:`. - **Composer model control.** Inspect the local inventory, switch models without leaving the composer, refresh the inventory, and stage local image or text attachments. @@ -31,6 +63,10 @@ isolated profile and are not part of the repository. for a single conversation from the composer, without changing the standing default. Each response shows its token count and tokens/sec once generation finishes. +- **Prompt control.** Standing system instructions apply to every turn, with no + length cap. A local model can also be run raw: **Bypass Cortex's default + system prompt** leaves the built-in identity and safety instructions out of + the request entirely. It is off by default and takes a deliberate opt-in. - **Model details.** The Models panel shows each installed model's parameter size, quantization, and context length alongside its name, read from Ollama's existing model-detail response. @@ -83,7 +119,7 @@ main.py +-- supervised FastAPI backend (Python) | +-- versioned loopback API, session auth, SSE jobs | +-- SQLite conversations/settings and local memory repositories - | +-- Ollama/model and generation services + | +-- model and generation services (Ollama + managed llama.cpp) | `-- scratch, image, attachment, and code execution lifecycles +-- native pywebview / WebView2 window `-- supervised Vite server (development mode only) @@ -93,6 +129,7 @@ backend/cortex_backend/ API, repositories, services, and worker boundaries contracts/ generated TypeScript API contracts assets/ externalized model prompt assets packaging/ Windows PyInstaller build and WebView2 bootstrapper +tools/ contract generation, qualification spikes, screenshots tests/ Python API, lifecycle, worker, and migration tests ``` @@ -105,14 +142,16 @@ end to end; Cortex does not pull an embedding model at startup. - Windows 10 or later - Python 3.10 or later -- Ollama installed and running at `http://127.0.0.1:11434` -- At least one locally installed generation model +- At least one local model, from either runtime: + - Ollama installed and running at `http://127.0.0.1:11434`, or + - a `.gguf` file in the local models folder, served by the managed llama.cpp + runtime (no Ollama required) - Node.js 22+ and npm for frontend development or source builds ## Quick start -Install a model in Ollama, create a virtual environment, and launch the native -desktop application: +Get a model, create a virtual environment, and launch the native desktop +application: ```powershell ollama pull qwen3:8b @@ -122,10 +161,15 @@ python -m pip install -r requirements.txt python main.py ``` +If you would rather not run Ollama, skip the first line and instead drop a +`.gguf` file into the local models folder shown under **Settings -> System**, or +download one from there by URL or Hugging Face repo. Cortex serves it with its +own llama.cpp runtime. + The first launch checks the local model inventory. Choose a model from the setup -screen or later from the composer picker. If Ollama is unavailable, Cortex still -opens and explains the connection state; generation becomes available after the -service is running and the inventory is refreshed. +screen or later from the composer picker. If neither runtime has a usable model, +Cortex still opens and explains the connection state; generation becomes +available once a model is present and the inventory is refreshed. Useful launcher options: @@ -143,17 +187,17 @@ endpoint can be intentionally changed for a trusted local network setup with ## Development checks -From the repository root: +One script runs the same gates CI runs, from the repository root: ```powershell -python -m pytest -q -python -m compileall -q main.py backend +./scripts/check.ps1 # lint, tests, contracts, frontend -- about two minutes +./scripts/check.ps1 -Tier full # adds compileall, Playwright, and the bundle build +``` -npm.cmd ci --prefix frontend -npm.cmd run lint --prefix frontend -npm.cmd run typecheck --prefix frontend -npm.cmd test --prefix frontend -- --run -npm.cmd run build --prefix frontend +To run them automatically before every push: + +```powershell +git config core.hooksPath .githooks ``` Use `npm.cmd` on Windows when PowerShell execution policy blocks the `npm.ps1` @@ -163,6 +207,14 @@ shim. API contract artifacts are generated with: python tools/generate_contracts.py ``` +The README screenshots are regenerated from a fixed demo workspace, so they stay +in step with the UI without hand-editing images: + +```powershell +npm.cmd run build --prefix frontend +./tools/screenshots/capture.ps1 +``` + ## Windows packaging Build the one-folder package with: diff --git a/docs/images/command-palette.png b/docs/images/command-palette.png new file mode 100644 index 0000000..d587d7b Binary files /dev/null and b/docs/images/command-palette.png differ diff --git a/docs/images/settings.png b/docs/images/settings.png new file mode 100644 index 0000000..75e182f Binary files /dev/null and b/docs/images/settings.png differ diff --git a/docs/images/workspace.png b/docs/images/workspace.png new file mode 100644 index 0000000..fc9df03 Binary files /dev/null and b/docs/images/workspace.png differ diff --git a/tools/screenshots/capture.mjs b/tools/screenshots/capture.mjs new file mode 100644 index 0000000..14af0fd --- /dev/null +++ b/tools/screenshots/capture.mjs @@ -0,0 +1,124 @@ +/** + * Capture the documentation screenshots against the staged demo workspace. + * + * Start tools/screenshots/demo_server.py first, then: + * + * node tools/screenshots/capture.mjs [--port 8799] + * + * Everything is fixed -- viewport, workspace content, and the order of + * interactions -- so a re-run overwrites the images with the same pixels. + * Run from the frontend/ directory so the local playwright install resolves. + */ +import { mkdirSync } from "node:fs"; +import { createRequire } from "node:module"; +import { dirname, resolve } from "node:path"; +import { fileURLToPath } from "node:url"; + +const here = dirname(fileURLToPath(import.meta.url)); +const repoRoot = resolve(here, "..", ".."); +const outDir = resolve(repoRoot, "docs", "images"); + +// Playwright lives in frontend/node_modules, not next to this script, so +// resolve it from there rather than relying on the caller's directory. +const requireFromFrontend = createRequire(resolve(repoRoot, "frontend", "package.json")); +const { chromium } = requireFromFrontend("@playwright/test"); + +const token = process.argv[2]; +if (!token) { + console.error("usage: node capture.mjs [--port N]"); + process.exit(1); +} +const portFlag = process.argv.indexOf("--port"); +const port = portFlag === -1 ? 8799 : Number(process.argv[portFlag + 1]); +const base = `http://127.0.0.1:${port}`; + +const VIEWPORT = { width: 1440, height: 900 }; + +mkdirSync(outDir, { recursive: true }); + +const browser = await chromium.launch(); +const page = await browser.newPage({ + viewport: VIEWPORT, + deviceScaleFactor: 2, // crisp on high-DPI displays + colorScheme: "dark", + // Message times are rendered with Intl in the viewer's locale and zone. + // Pin both, or the captured footers differ from machine to machine. + locale: "en-US", + timezoneId: "UTC", +}); + +const shot = async (name) => { + const path = resolve(outDir, `${name}.png`); + await page.screenshot({ path }); + console.log(` wrote docs/images/${name}.png`); +}; + +// Freeze anything time-based so repeated runs stay identical. +await page.addInitScript(() => { + const fixed = new Date("2026-08-09T17:00:00Z").valueOf(); + const OriginalDate = Date; + // eslint-disable-next-line no-global-assign + Date = class extends OriginalDate { + constructor(...args) { + super(...(args.length ? args : [fixed])); + } + static now() { + return fixed; + } + }; +}); + +console.log("Capturing..."); + +// --- 1. Workspace: sidebar library + a real transcript ----------------------- +await page.goto(`${base}/?bootstrap=${encodeURIComponent(token)}`); +await page.getByLabel("Message Cortex").waitFor({ state: "visible" }); + +// The sidebar is already expanded at this width, so open the hero +// conversation straight from the library. +await page + .getByRole("button", { name: "Reading a 4 GB CSV without exhausting memory", exact: true }) + .click(); +await page.getByText("Read it as a").first().waitFor({ state: "visible" }); +await page.waitForTimeout(600); // let syntax highlighting settle + +// Frame the shot on the final exchange: put the closing question at the top so +// the whole answer, its footer, and the composer all land in one viewport. +await page.evaluate(() => { + const asked = document.querySelectorAll(".message-user"); + asked[asked.length - 1]?.scrollIntoView({ block: "start" }); +}); +await page.waitForTimeout(900); // virtualized list re-renders the window + +// The per-message controls (copy, regenerate, fork) are opacity:0 until the +// card is hovered or focused, so a plain screenshot would omit them entirely. +// Hover the final answer -- the only one where regenerate is enabled. +await page.locator(".message-assistant").last().hover(); +await page.waitForTimeout(400); // 140ms opacity transition +await shot("workspace"); + +// --- 2. Settings: model + generation controls -------------------------------- +await page.goto(`${base}/settings`); +await page.getByRole("heading", { name: "Settings", exact: true }).first().waitFor(); +await page.getByRole("button", { name: "AI Model" }).click(); +await page.getByLabel("System instructions").waitFor({ state: "visible" }); + +// The panel is taller than the viewport. Model selection is already visible in +// the composer of the workspace shot, so frame this one on the lower half -- +// the generation parameters, system instructions, and the system-prompt bypass. +await page.evaluate(() => { + const pane = document.querySelector(".settings-pane"); + if (pane) pane.scrollTop = pane.scrollHeight; +}); +await page.waitForTimeout(400); +await shot("settings"); + +// --- 3. Command palette over the workspace ----------------------------------- +await page.goto(`${base}/`); +await page.getByLabel("Message Cortex").waitFor({ state: "visible" }); +await page.keyboard.press("Control+k"); +await page.waitForTimeout(500); +await shot("command-palette"); + +await browser.close(); +console.log("Done."); diff --git a/tools/screenshots/capture.ps1 b/tools/screenshots/capture.ps1 new file mode 100644 index 0000000..e58d7bf --- /dev/null +++ b/tools/screenshots/capture.ps1 @@ -0,0 +1,66 @@ +<# +.SYNOPSIS + Regenerate the README screenshots from the staged demo workspace. + +.DESCRIPTION + Starts tools/screenshots/demo_server.py on an isolated port, hands the + one-time bootstrap token to the Playwright capture script, writes the + images to docs/images/, and stops the server again. + + The workspace is fixture data and no model is contacted, so re-running this + reproduces the same images. Requires a built frontend bundle: + + npm run build --prefix frontend + +.EXAMPLE + ./tools/screenshots/capture.ps1 +#> +[CmdletBinding()] +param( + [int]$Port = 8799 +) + +$ErrorActionPreference = 'Stop' +$repoRoot = Split-Path -Parent (Split-Path -Parent $PSScriptRoot) + +if (-not (Test-Path (Join-Path $repoRoot 'frontend/dist/index.html'))) { + throw "No built frontend. Run: npm run build --prefix frontend" +} + +$server = $null +try { + Write-Host "Starting staged demo server on port $Port..." -ForegroundColor Cyan + + $stdout = New-TemporaryFile + $stderr = New-TemporaryFile + $server = Start-Process -FilePath 'python' ` + -ArgumentList @('tools/screenshots/demo_server.py', '--port', $Port) ` + -WorkingDirectory $repoRoot ` + -RedirectStandardOutput $stdout ` + -RedirectStandardError $stderr ` + -NoNewWindow -PassThru + + # The bootstrap token is the first line the server prints. + $token = $null + foreach ($attempt in 1..40) { + Start-Sleep -Milliseconds 250 + $line = (Get-Content $stdout -TotalCount 1 -ErrorAction SilentlyContinue) + if ($line) { $token = $line.Trim(); break } + if ($server.HasExited) { + Get-Content $stderr | Write-Host -ForegroundColor Red + throw "Demo server exited before printing a token." + } + } + if (-not $token) { throw "Timed out waiting for the demo server bootstrap token." } + + Write-Host 'Capturing screenshots...' -ForegroundColor Cyan + node (Join-Path $PSScriptRoot 'capture.mjs') $token --port $Port + if ($LASTEXITCODE -ne 0) { throw "Capture failed with exit code $LASTEXITCODE." } + + Write-Host 'Screenshots written to docs/images/.' -ForegroundColor Green +} finally { + if ($server -and -not $server.HasExited) { + Stop-Process -Id $server.Id -Force -ErrorAction SilentlyContinue + Write-Host 'Demo server stopped.' + } +} diff --git a/tools/screenshots/demo_server.py b/tools/screenshots/demo_server.py new file mode 100644 index 0000000..6571298 --- /dev/null +++ b/tools/screenshots/demo_server.py @@ -0,0 +1,382 @@ +"""Serve Cortex with a fixed, staged workspace for documentation screenshots. + +Everything here is deterministic on purpose. No model is contacted and nothing +is generated at capture time, so re-running the capture produces the same +pixels: same chats, same wording, same token counts, same ordering. The UI in +the resulting images is the real application; only the conversation content is +staged fixture data. + + python tools/screenshots/demo_server.py --port 8799 + +Prints the bootstrap token on the first line of stdout so the capture script +can authenticate. +""" + +from __future__ import annotations + +import argparse +from datetime import datetime, timedelta, timezone +from pathlib import Path +import sys + +ROOT = Path(__file__).resolve().parents[2] +sys.path.insert(0, str(ROOT / "backend")) + +import uvicorn # noqa: E402 + +from cortex_backend.api import BackendDependencies, create_app # noqa: E402 +from cortex_backend.core.settings import ( # noqa: E402 + AppearanceSettings, + CortexSettings, + GenerationSettings, + ModelSettings, +) +from cortex_backend.repositories.chats import InMemoryChatRepository # noqa: E402 +from cortex_backend.repositories.memories import InMemoryMemoryRepository # noqa: E402 +from cortex_backend.repositories.settings import InMemorySettingsRepository # noqa: E402 +from cortex_backend.services.attachments import ChatAttachmentService # noqa: E402 +from cortex_backend.services.generation import GenerationService # noqa: E402 +from cortex_backend.services.models import ModelService # noqa: E402 +from cortex_backend.testing.fake_ollama import ( # noqa: E402 + FakeGenerationEngine, + FakeOllamaGateway, + FakeOllamaState, +) + +# A fixed local inventory so the model picker and Models panel read the same +# way in every capture, regardless of what is installed on the machine. +INSTALLED_MODELS = ["qwen3:8b", "granite4.1:8b", "nemotron-3-nano:4b", "translategemma:4b"] + + +def _stats(prompt_tokens: int, output_tokens: int, tokens_per_second: float) -> dict: + eval_ms = round(output_tokens / tokens_per_second * 1000, 1) + return { + "prompt_eval_count": prompt_tokens, + "eval_count": output_tokens, + "prompt_eval_duration_ms": 180.0, + "eval_duration_ms": eval_ms, + "total_duration_ms": round(eval_ms + 180.0, 1), + "tokens_per_second": tokens_per_second, + } + + +CHATS: list[dict] = [ + { + "id": "demo-streaming-csv", + "title": "Reading a 4 GB CSV without exhausting memory", + "timestamp": "2026-08-09T16:42:00Z", + "messages": [ + { + "id": "m-1", + "role": "user", + "content": "I need to total one column of a 4 GB CSV in Python, but loading it keeps exhausting memory. What's the right approach?", + }, + { + "id": "m-2", + "role": "assistant", + "content": """Read it as a **stream** rather than materialising the whole file. `csv.DictReader` already +yields one row at a time, so the only thing holding memory is the row you are currently looking at. + +```python +import csv + +def read_rows(path): + with open(path, newline="", encoding="utf-8") as handle: + yield from csv.DictReader(handle) + +total = 0.0 +for row in read_rows("orders.csv"): + total += float(row["amount"]) + +print(f"{total:,.2f}") +``` + +A few things worth knowing: + +- **Keep the generator lazy.** Wrapping it in `list(...)` undoes the whole benefit. +- **`newline=""` is not optional.** Without it, the `csv` module mishandles quoted fields + that contain line breaks. +- **Only parse what you need.** `float(row["amount"])` per row is cheap; building a `dict` + of every column for later use is not. + +If you would rather stay in pandas, `pd.read_csv(path, chunksize=100_000)` gives you an +iterator of frames and keeps peak memory bounded to roughly one chunk.""", + "stats": _stats(prompt_tokens=38, output_tokens=214, tokens_per_second=47.3), + }, + { + "id": "m-3", + "role": "user", + "content": "Does that still work if the export has a BOM at the start?", + }, + { + "id": "m-4", + "role": "assistant", + "content": """Not quite -- with plain `utf-8` the BOM ends up glued to the first header name, so +`row["amount"]` still works but `row["order_id"]` raises `KeyError`. The actual key is +`"\\ufefforder_id"`. + +Switch the codec to `utf-8-sig`, which consumes the BOM if present and is harmless if not: + +```python +with open(path, newline="", encoding="utf-8-sig") as handle: + yield from csv.DictReader(handle) +``` + +That is the one-line fix. It is worth defaulting to `utf-8-sig` for anything exported from +Excel, which writes the BOM routinely.""", + "stats": _stats(prompt_tokens=272, output_tokens=131, tokens_per_second=46.1), + }, + ], + }, + { + "id": "demo-sqlite-locked", + "title": "SQLite migration fails with database is locked", + "timestamp": "2026-08-09T14:05:00Z", + "messages": [ + { + "id": "m-10", + "role": "user", + "content": "My migration script intermittently fails with `sqlite3.OperationalError: database is locked`. It only happens when the app is running.", + }, + { + "id": "m-11", + "role": "assistant", + "content": """That error means another connection holds a write lock past your busy timeout. SQLite +allows many readers but only one writer, and the default timeout is 5 seconds. + +Two changes usually settle it: + +1. **Enable WAL mode** so readers stop blocking the writer: + + ```sql + PRAGMA journal_mode = WAL; + ``` + +2. **Raise the busy timeout** on the migration connection so it waits instead of failing + immediately: + + ```python + connection = sqlite3.connect(db_path, timeout=30.0) + connection.execute("PRAGMA busy_timeout = 30000") + ``` + +If it still fails, the usual culprit is a long-lived transaction left open by the running +app -- a connection that ran `BEGIN` and has not committed. Look for a code path that opens +a transaction and returns early on an error without rolling back.""", + "stats": _stats(prompt_tokens=44, output_tokens=186, tokens_per_second=45.8), + }, + ], + }, + { + "id": "demo-embeddings", + "title": "What a vector embedding actually is", + "timestamp": "2026-08-09T11:20:00Z", + "messages": [ + { + "id": "m-20", + "role": "user", + "content": "Explain vector embeddings without the usual hand-waving. What is actually being stored?", + }, + { + "id": "m-21", + "role": "assistant", + "thoughts": """The user is explicitly asking to skip the vague explanation, so I should avoid the +"words become numbers that capture meaning" framing and be concrete about what the array +literally is. + +Concrete points worth making: it is a fixed-length array of floats; the length is a +property of the model, not the input; position in the space is what carries information, +not any individual number; similarity is measured by angle, which is why cosine similarity +shows up everywhere. + +I should be careful not to claim individual dimensions are interpretable -- they generally +are not, and that is a common overstatement.""", + "content": """A fixed-length array of floating point numbers. That is the whole storage format -- +for `nomic-embed-text` it is 768 of them, and it is 768 whether you embed one word or three +paragraphs. The length is a property of the model, not the input. + +What makes it useful is **where** the array lands, not what any single number says. The model +is trained so that inputs used in similar contexts land near each other, so position encodes +relationships: + +- "invoice" and "receipt" land close together +- "invoice" and "trombone" land far apart +- direction matters more than magnitude, which is why similarity is nearly always measured + as the cosine of the angle between two vectors rather than the distance between them + +One caveat worth stating plainly: individual dimensions are **not** interpretable. There is no +"formality axis" at index 214. Meaning is distributed across the whole array, which is why you +can compare two embeddings but cannot read one.""", + "stats": _stats(prompt_tokens=41, output_tokens=228, tokens_per_second=44.6), + }, + ], + }, + { + "id": "demo-apache-regex", + "title": "Regex for Apache combined log format", + "timestamp": "2026-08-08T18:30:00Z", + "messages": [ + { + "id": "m-30", + "role": "user", + "content": "Give me a regex that parses the Apache combined log format into named groups.", + }, + { + "id": "m-31", + "role": "assistant", + "content": """```python +import re + +LOG_LINE = re.compile( + r'(?P\\S+) \\S+ (?P\\S+) \\[(?P