Engineering blog
Explainers in plain words, and field notes from building an AI platform: what we picked, what broke and how we fixed it, with the numbers.
Vector search and RAG
What is RAG, in plain words
Why an AI model can't answer from your company's documents on its own, the eight steps RAG uses to fix that, and where each one quietly goes wrong.
Field notes · 7 minWhy we moved AI search from Postgres to LanceDB, and what broke
Why our AI search slowed as customers grew, why a hosted store didn't fit, how to move without betting on one database, and what we found after the switch.
Field notes · 6 minChoosing an embedding model: why leaderboards aren't enough
Why a leaderboard can't tell you how a model will do on your data, and how to design a fair test: realistic questions, one change, a blind judge.
Opinion · 6 minpgvector vs LanceDB vs Turbopuffer: when each one makes sense
We've kept our vectors in all three. What each one is, where it runs, how you pay, and the questions that should decide between them.
Field notes · 5 minHybrid search: why rank fusion flattened our scores
Why keyword search still earns its place next to vector search, why rank-only fusion flattened every score to about 0.031, and what we replaced it with.
Field notes · 4 minThe recall you lose to compressed vector indexes
Compressed vector indexes are fast but can miss the right results. Our recall@10 fell to 0.29 on large tenants, and a refine step brought it to 0.83.
Field notes · 6 minHow to test a RAG system: score every stage
Our AI answers counted wrong answers as successes. Why end-to-end checks hide that, how to score each stage of a RAG system, and why tests must start clean.
Field notes · 4 minWhy AI agent loops get expensive, and how we halved ours
Our deepest AI answer mode cost more than it billed. Why an agent loop costs more every round, how we found it, and how a cap and a cache cut the bill.
Field notes · 4 minCaching AI answers: why question text is the wrong key
Why an AI answer cache keyed on question text misses repeats, what our cache's records showed, and how to key on the retrieved documents instead.
Field notes · 4 minTwo prompt tricks we didn't ship, and how we knew
Reordering prompts for caching and adding few-shot examples both made our AI answers give up more often. Why scoring each stage made both calls obvious.
Kubernetes and cloud cost
Kubernetes in plain words: nodes, pods and how big to make them
What a cluster, node, pod and deployment are, what requests and limits really do, and why to size pods from measured load instead of expected load.
Explainer · 6 minWhat is KEDA? Kubernetes autoscaling that can go to zero
KEDA scales Kubernetes workloads on queues, schedules and other events, down to zero pods. How it differs from the HPA, and which work belongs on it.
Field notes · 5 minWhy CPU throttling hides behind a calm CPU graph
A CPU limit is enforced every 100 ms, but graphs show averages. Why throttling hides, how to measure it, and how fixing it made our extraction 14× faster.
Field notes · 6 minRunning a database on Kubernetes: what changes when a pod holds data
Kubernetes' defaults assume any pod can be thrown away. Why that breaks for a database, how we ran LanceDB with that in mind, and a checklist for your first.
Field notes · 3 minShrinking Docker images: find the dead weight first
Why our images grew past a gigabyte, how we found what nothing used, and why smaller images made deploys, pods and CI faster.
Field notes · 4 minA GPU only at night: why batch embedding belongs on a queue
A GPU rented all day sits idle, and live questions barely use one. Why queued night work suits a per-second GPU, with a hard budget and a CPU fallback.
Opinion · 5 minCloud cost is a product problem
Our platform was sized for thousands of users and served 30 to 100 a day. Why infrastructure cost is part of the product, and how to find what you overpay for.
CI/CD
What is CI/CD? Choosing pipelines that don't waste time
CI, continuous delivery and continuous deployment in plain words, how to shape a pipeline, and why most pipeline time goes on moving data, not compiling.
Field notes · 6 minHow we cut CI minutes by 71%: cache by how often things change
Our CI time went on moving data, not compiling. Why a cache that mixes packages with built code goes stale on every commit, and how we fixed it.
Field notes · 5 minRelease by tag: ship the exact image that passed staging
Two builds of the same commit can differ. Why rebuilding for production ships something untested, and how promoting the staging image by tag avoids it.
Field notes · 4 minWhy every change needs a second reader
Why no change should reach production on its author's word alone, how to require review on GitHub, and why the same record is change-management evidence.
Documents
PDF extraction caps your RAG answers: how to test the engines fairly
A RAG answer can't be better than the text extraction hands it. Why, how to run a fair bake-off on your own documents, and what combining engines did.
Field notes · 4 minBenchmark on the hardware you pay for
An int8 model was 40% faster on our research box and 2.6× slower on our production CPUs. Why speed results flip between machines, and how to test.
Field notes · 5 minPricing document processing per page, in tiers
Pages differ widely in what they cost to process. Why that makes per-page tiers the fair way to price extraction, and how to set them up.
Security
A session security checklist for a SaaS app
What sessions and tokens are, why 30-day tokens and a logout that ended nothing come from one cause, and the checklist that fixes it in any app.
Explainer · 8 minSafe file uploads: trust the bytes, not the label
Why a PDF upload can carry script, why the real cause is trusting a file's label, and the fixes: server-decided types, forced downloads, polyglot checks.
Field notes · 5 minMulti-tenant data isolation: what to check before you launch connectors
Before launching connectors, we closed four ways one company could reach another's data. Why connectors open gaps like these, and how to check for them.
Field notes · 4 minWhy a read-only container turned an attack into a patch
Attackers aimed a critical Next.js flaw at our website. A read-only, non-root container stopped the payload, so the fix could be a small, careful patch.
AI on a small team
AI coding agents on a small team: write the rules down
Why AI coding agents drift from a small team's conventions without shared written rules, why the rules should update themselves, and how to set them up.
Opinion · 4 minDesign notes: write down the number, not just the decision
Why a design note should record the measurement, the decision and the trade-off, so nobody reruns the experiment, and a template you can copy.
Field notes · 5 minAI bug triage: a daily first pass, and ticket text you never trust
Why customer tickets should reach the bug board with no copy-paste, why a daily AI first pass helps a small team, and why ticket text is untrusted input.
Frontend and the web
Thirteen repos, one product: why we moved to a monorepo
Twelve service repos and a frontend made every feature a multi-repo job. Why that slowed a small team, and what one monorepo with shared libraries changed.
Field notes · 4 minUpgrading Next.js 13 to 15: why, what it fixes and how
We found critical security issues in our Next.js 13 app. Why we moved to 15 instead of patching, what the upgrade fixes, the steps and what to watch for.
Field notes · 5 minWhy search skips your pages, and how to get quoted by AI search
My city pages lost to their country page. Why thin pages with generic titles get skipped, what search and AI engines need to pick a page, and the steps.