Hot reload GeoLite2 mmdb databases without a restart - #8
Open
gam6itko wants to merge 3 commits into
Open
Conversation
Poll GeoLite2-City.mmdb and GeoLite2-ASN.mmdb via os.Stat (ModTime+Size) and swap the open maxminddb.Reader when geoipupdate replaces the file, so a container restart is no longer needed. The new reader is opened before any lock is taken; on failure the previous database keeps serving and the reload is retried on the next cycle. The old reader is closed under the same write lock that performs the swap, which guarantees no lookup is still using it. Check interval is configurable via GEOIP_RELOAD_INTERVAL (default 60s). Watchers stop on SIGINT/SIGTERM, alongside a graceful HTTP shutdown. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #7
geoipupdatereplaces the mmdb files on the host, but the app opened them once at startup and kept serving stale data until the container was restarted. Both City and ASN databases are now picked up at runtime.How
A single reusable
ReloadableGeoIPDBbacks both readers, soCityReaderandAsnReadershare one implementation. Their public API is unchanged, and so are the HTTP endpoints and response formats.Changes are detected by re-
os.Stating the path (ModTime + Size) rather than trusting the open file descriptor, sincegeoipupdatewrites a temp file and renames it over the target. Plain polling, no fsnotify.Reload order:
Openfails, the previous database stays active, the error is logged and the next cycle retries;Close()happen under the same write lock. Lookups holdRLockfor the wholeLookup+Decode, so while the write lock is held no lookup is inside the old reader and none can enter it.reader == nilnever occurs.atomic.Pointeris deliberately avoided — the mutex keeps the close lifecycle obviously correct. Nothing is logged while the file is unchanged.Shutdown
mainnow usessignal.NotifyContext+srv.Shutdown, which also fixes an existing bug:log.Fatal(http.ListenAndServe(...))callsos.Exit, sodefer asn.Close()/defer city.Close()never ran.Configuration
GEOIP_RELOAD_INTERVAL, default60s. No docker-compose or Dockerfile changes required. Note that/geolitemust be bind-mounted as a directory, not as individual files — a rename on the host is invisible inside the container otherwise. The current compose files already mount the directory.Tests
A minimal valid MMDB is built in memory, so no large binary fixture is added to the repo. Tests replace the file the way
geoipupdatedoes (write temp +os.Rename) and cover: startup, no reload when unchanged, picking up a replacement, a corrupt replacement leaving the previous database working, recovery on the next valid replacement, concurrent lookups against repeated reloads, watcher shutdown on context cancel, andClose.go test ./...andgo test -race ./...pass.