The Reality of 'Local Agents': API Tools vs. Prompt Engineering
Building native cloud APIs is easy. Getting a local Llama-3 model to execute shell commands requires the dark arts of prompt engineering.
Engineering Blog
Opinionated technical deep dives into agent architecture, local LLMs, and prompt economics.
The Context Sandwich: Why 'Infinite Memory' is a Trap
Everyone wants an infinite context window. The math says no. Here's how we compress history to save your system prompt from the 'Lost in the Middle' effect.
The JSON Mode Tax: Why Your Agent Bills Are Bloated
We love JSON mode because it makes agents reliable. But forcing a model to think in JSON burns 3x more tokens.
The Middleman Attack: Why Your AI Wrapper is Logging Your Code
If you aren't using your own API key, you are trusting a middleman with your codebase and agreeing to consumer-level privacy policies.