v10.0.0

[wayneeseguin]

Modernizes SHIELD for a v10.0.0 release: current Go and dependencies, a
CLI rebuilt on Cobra, AWS SDK v2 in the object-storage plugins, and
support for running the daemon against PostgreSQL and MySQL instead of
only SQLite.

Rebased onto current develop.

Reviewing this

The diff touches 2,604 files, but 2,404 of those are vendor/. There are
148 files of actual change. Reviewing with vendored dependencies
hidden is strongly recommended:

git diff develop...v10.0.0 -- . ':(exclude)vendor'

On the GitHub UI, append ?w=1 to ignore whitespace and use the file
filter to exclude vendor/.

What changed

Toolchain and dependencies

Go 1.25.12 to 1.26.1. Dropped the stale go-restful replace directive.
Retired jhunt/go-log, go-cli, go-querytron, go-s3, and
go-snapshot; go-querytron is inlined into the client package rather
than replaced. Also dropped sqlx, lib/pq, blazer, mapstructure,
yaml.v2, and yaml.v3.

Logging

jhunt/go-log replaced with a log/slog shim across 27 files.

CLI

The shield CLI is decomposed into per-topic Cobra command files, and
the five binaries plus the plugin framework move to Cobra. One follow-up
commit restores environment-variable support that the migration dropped.

Storage plugins

The S3 and Backblaze plugins are rewritten on AWS SDK v2, sharing a new
plugin/s3util package.

Database layer

sqlx is replaced with database/sql, and a dialect adapter lets the
daemon run against PostgreSQL and MySQL as well as SQLite.
cmd/shieldd and cmd/shield-schema register all three drivers.

Tests

Ginkgo v1 to v2, with suites bootstrapped for previously untested
packages.

Multi-database support

This is the most substantial functional addition, and the part most
worth close review. The schema and queries had only ever run against
SQLite, whose dynamic typing and permissive SQL hid a fair amount that
stricter servers reject outright:

• Columns were declared UUID but hold arbitrary strings, including
  "", which PostgreSQL rejects on insert.

• Several columns were declared BOOLEAN with an integer default, or
  INTEGER while carrying a Go bool. stores.last_test_task_uuid was
  declared BOOLEAN but holds a task UUID.

• TEXT columns carried bare literal defaults, and were indexed without
  a prefix length; MySQL rejects both.

• The recent_tasks CTE selected bare columns alongside GROUP BY,
  which only SQLite permits. It now ranks rows with ROW_NUMBER().
  This was also a latent correctness bug on SQLite, which returned an
  arbitrary row per job rather than the most recent one.

• Filter builders emitted WHERE 1, which PostgreSQL rejects as
  non-boolean.

• Task-log appends used ||, which MySQL reads as logical OR.

Because IsNoSuchTable compared driver error strings, it never matched
either server in practice. It tested for exact equality with
pq: relation "x" does not exist, but the registered driver is
pgx/v5/stdlib, which formats the same error as
ERROR: relation "x" does not exist (SQLSTATE 42P01); and it tested for
the prefix Error 1146: Table, while go-sql-driver emits
Error 1146 (42S02): Table. Empty-database detection therefore failed
on both servers. It now matches structured
driver errors via errors.As on *pgconn.PgError (SQLSTATE 42P01) and
*mysql.MySQLError (1146), falling back to a message match only for
go-sqlite3, whose generic error code carries no other signal.

Bugs fixed along the way

Three defects that predate this branch and are unrelated to portability:

• MarkTasksIrrelevant had an unclosed parenthesis and referenced a
  jobs.keepdays column that does not exist. It fails on SQLite too;
  the failure went unnoticed because its only caller discards the error,
  so retention for clear = 'normal' tasks has never worked.

• AnnotateTargetTask concatenated its updates directly onto WHERE
  with no separating space.

• Session cookies were set SameSite=Strict. The OAuth providers
  (github, okta, uaa) return the browser to /auth/:provider/redir as a
  top-level cross-site navigation, on which Strict cookies are withheld
  — so the via cookie set moments earlier was never sent back, and
  shield login could not complete against any OAuth provider. Now
  Lax, which still withholds the cookie from cross-site POSTs and
  subresource requests.

The Secure cookie attribute is set only when the request arrived over
TLS, or when a terminating proxy reports an https scheme. shieldd
only ever calls ListenAndServe, so setting it unconditionally would
mean browsers silently discard the session cookie on any deployment not
fronted by a TLS terminator.

Testing

make test is unchanged and still requires nothing but SQLite; the
PostgreSQL and MySQL specs skip themselves unless a DSN is exported.

To exercise them against real servers:

make test-databases        # brings both up, runs ./db/
make test-databases-down

These run the real migrations and the real queries against a live
PostgreSQL 16 and MySQL 8. Everything in the multi-database section
above was found by running them, not by inspection — the unit specs
that existed asserted hand-written strings copied from the
implementation, so they stayed green while the feature was broken.

Known issues

• TestAgent fails with exit status 127. This failure is present on
  develop and is not introduced here.

• go vet reports unreachable code in plugin/fs/plugin.go and two
  self-assignments in db/tenant_test.go. Both predate this branch.

• The vendored dependency churn is large and unavoidable given the SDK
  and toolchain upgrades.

[wayneeseguin]

Progress Update — 2026-03-30

YAML Library Migration: gopkg.in/yaml.v3 → goccy/go-yaml

Completed migration from gopkg.in/yaml.v3 to github.com/goccy/go-yaml across all SHIELD source files.

Commits:

• a56814de — Fix format string vet errors in core
• 482d75a7 — Add YAML serialization tests for core and CLI
• ae1a9dfe — Migrate from yaml.v3 to goccy/go-yaml

What changed:

• Swapped imports in 5 source files (core/core.go, core/duration.go, agent/config.go, cmd/shield/config.go, cmd/shield/cmd_misc.go)
• Rewrote duration.UnmarshalYAML from *yaml.Node signature to goccy's callback signature
• gopkg.in/yaml.v3 fully removed from go.mod and vendor
• Added 15 new tests (12 Ginkgo specs for duration unmarshaling, 3 stdlib tests for config round-trips and import manifest parsing)
• Fixed 7 pre-existing go vet format-string errors in core/main.go, core/auth.go, core/api_v2.go

Test results: All new tests pass. Existing agent config tests pass. 5 pre-existing agent SSH test failures (missing test binary) are unrelated.

---

Cumulative v10.0.0 Progress

Area	Status
Go 1.26 upgrade	Done
Dependency updates (x/, consul, oauth2, docker)	Done
Cobra CLI decomposition	Done
yaml.v2 → yaml.v3 migration	Done
yaml.v3 → goccy/go-yaml migration	Done
Ginkgo v1 → v2 upgrade	Done
Multi-database dialect adapter (PostgreSQL + MySQL)	Done
go vet clean build	Done

[wayneeseguin]

This is complete and ready to merge.

Worth saying plainly: despite the branch name, none of this is gated on a
v10.0.0 release. It is debt cleanup and quality work — current toolchain
and dependencies, retired abandoned libraries, a CLI on a maintained
framework, AWS SDK v2, and a database layer that works on more than one
database. It stands on its own merits and can land whenever it suits the
release plan, under whatever version number that turns out to be.

Verification

• go build ./... clean.

• gofmt -l reports nothing outside vendor/.

• go vet clean apart from two findings that predate this branch
  (unreachable code in plugin/fs/plugin.go, two self-assignments in
  db/tenant_test.go).

• 16 of 17 test-bearing packages pass. The one failure, TestAgent
  (exit status 127), fails identically on develop and is not
  introduced here.

• The db package passes against SQLite, PostgreSQL 16, and MySQL 8 —
  real servers running the real migrations and the real queries, via
  make test-databases.

• Every commit was checked out individually and confirmed to build and
  test green, so the history is bisectable rather than only green at the
  tip.

A note on the multi-database work

The portability defects listed in the description were found by running
against live servers, not by reading the code. The unit specs that
existed asserted hand-written strings copied from the implementation, so
they passed while the feature did not work at all — missing-table
detection never matched either driver, and the migrations did not survive
the first CREATE TABLE on PostgreSQL. That is the reason the new specs
exercise real servers, and the reason make test deliberately still
requires nothing but SQLite: contributors keep a fast default, while the
claim of multi-database support has something behind it.

Three unrelated pre-existing bugs surfaced along the way and are fixed
here — see the description. The MarkTasksIrrelevant one is worth a
reviewer's attention: it means task retention for clear = 'normal' has
never actually run.

Suggested review approach

Exclude vendor/ — 2,404 of the 2,604 changed files are vendored, and
the remaining 148 are the real diff:

git diff develop...v10.0.0 -- . ':(exclude)vendor'