The Meter Isn’t the Problem
You did the math. Daily allowance divided by average request size, and you figured you had weeks of runway on the free tier. You burned it by Tuesday afternoon.
The tier didn’t shrink. What you fed it did the damage. Paste a three thousand line log file into a prompt and ask “what’s wrong with this,” and you just spent a real chunk of your daily budget on lines the model never needed to see. Do that a dozen times a day and the free tier problem isn’t a stingy provider, it’s you handing the model a haystack every time the task only needed one straw.
This is a discipline problem, not a shopping problem. Which providers to sign up for is a separate conversation. This one is about the habits that make whichever quota you picked go three times further: what to actually send, how to structure prompts so caching kicks in, which tasks deserve the good model and which deserve the intern, how to batch without breaking things, how to survive a 429 without losing your afternoon, and when the right move is to stop calling the cloud at all.
Full example: Clone the runnable versions of every script below at github.com/KingPin/sumguy-examples/llm/stretch-free-llm-tier, including the LiteLLM config and a style guide sized to actually clear the cache minimum.
Context Discipline: Stop Mailing Your Whole Repo
The single biggest waste on any tier, free or paid, is sending more than the model needs to answer the question. Whole repos pasted into a chat. Entire log files when the error is three lines. Full documents when you’re asking about one section. Every one of those tokens counts against your quota whether the model reads them carefully or skims past them, and on a free tier that quota is the whole game.
The fix isn’t clever. It’s just extraction before generation:
- Grep for the error, function name, or symbol first, and send the matching lines plus a few lines of surrounding context, not the file.
- For logs, filter to the time window and severity level that matters, then send that slice.
- For long documents, send the table of contents plus the section you’re asking about, not the whole thing.
- For code review, send the diff, not the file the diff came from. The model doesn’t need to see the 400 unchanged lines to review the 12 that changed.
If you’re using an agent that reads files on your behalf (Claude Code, an IDE assistant, a homegrown script), check what it’s actually sending before you assume it’s being smart about this. A lot of default configurations read an entire file into context “to be safe” when a targeted grep would do. That default is fine on a paid tier where you’re trading dollars for convenience. On a free tier it’s the fastest way to hit a wall by lunchtime.
Prompt Caching: Build a Prefix That Doesn’t Move
Prompt caching lets a provider skip reprocessing the part of your prompt it has already seen, and charges you less (sometimes a lot less) for the repeat. The catch is that caching only works if the cached portion is byte-for-byte identical between calls. Change one word in your system prompt and you’ve invalidated the cache and you’re paying full price again.
Anthropic’s Claude models use explicit cache_control breakpoints: you mark the end of the static portion of your prompt, and everything before that marker gets cached. The default cache lives for 5 minutes; an extended 1 hour cache is available at a higher write cost if your call pattern is spread out more than that. OpenAI’s caching is automatic once a prompt crosses roughly 1,024 tokens, no code changes required, but the cache gets evicted after a short idle window, so bursty, spaced-out calls won’t benefit as much as tight loops. DeepSeek’s disk-based context caching is also automatic and the discount on cache hits is steep, on the order of a 90+ percent cut on the cached portion of input tokens compared to a cache miss.
The part that actually matters for you: structure your prompt so the boring, unchanging stuff comes first, and the part that changes every call comes last.
import anthropic
client = anthropic.Anthropic()
# This block never changes between calls. Put the system role,# your house style guide, and any few-shot examples here, and mark# the end of it with a cache breakpoint.SYSTEM_PROMPT = """You are a code reviewer. Follow these rules:1. Flag only correctness bugs and security issues.2. Never restate code that did not change.3. One bullet per finding, file and line number first."""
STYLE_GUIDE = open("STYLE_GUIDE.md").read()
def review_diff(diff_text): return client.messages.create( model="claude-sonnet-5", max_tokens=512, system=[ { "type": "text", "text": SYSTEM_PROMPT + STYLE_GUIDE, "cache_control": {"type": "ephemeral"}, } ], # Only the diff changes call to call. Everything above it # can be served from cache on every hit after the first. messages=[{"role": "user", "content": f"Review this diff:\n\n{diff_text}"}], )Notice what’s not in the cached block: the diff. That’s the part that’s different every time, so it goes after the breakpoint. If you accidentally interleave a timestamp, a random request ID, or a “current date” string into the static portion, you’ll invalidate the cache on every single call and wonder why your bill (or your quota) isn’t moving the way the docs promised. Keep dynamic content dynamic and static content static, and don’t let them mix.
Tiering the Work: Not Every Task Needs the Smart Model
A rename across forty files, reformatting YAML into TOML, summarizing a diff, writing boilerplate CRUD handlers: none of that needs your best model’s judgment, and none of it should touch your smallest quota. This is the overseer and workhorse pattern from a previous post on this site, and it applies just as directly to free-tier budgeting as it does to cost control.
The overseer (your good model, spent sparingly) scopes the task and reviews the output. The workhorse (free tier, cheap tier, or local) does the mechanical execution. Route by task type, not by habit:
- Workhorse tier: renames, format conversions, boilerplate generation, “summarize this diff in one sentence,” extracting structured fields from unstructured text.
- Overseer tier: anything requiring judgment. Deciding whether an approach is architecturally sound, debugging a subtle race condition, writing the actual design doc.
A model like Qwen3-Coder’s smaller variants or Gemma 4 running through a free API tier handles the mechanical bucket without complaint. Save the free tier’s daily allowance, and your paid model’s context window, for the fifth of the work that actually needs a second opinion. Sending a boilerplate rename job to your most expensive model is like hiring a structural engineer to move a couch. It’ll get moved. You’ll also get a bill that makes you ask why.
Batching: One Structured Call Instead of Ten Small Ones
Every call has fixed overhead: the system prompt (unless it’s cached), the round trip, and on a free tier, one unit of your rate limit regardless of how small the actual question is. If you’re making ten separate calls to classify ten short strings, you’re paying that fixed overhead ten times over.
Combine them into one call with a structured request and a structured response:
import json
items = ["broken pipe error", "disk full warning", "auth token expired", "..."]
prompt = f"""Classify each log line below into one of: network, storage, auth, other.Return a JSON array of {{"line": "...", "category": "..."}} objects, one per input line, in order.
Lines:{json.dumps(items, indent=2)}"""# One call, one rate-limit slot, one round trip, instead of len(items) of each.Batching backfires in two specific cases, so don’t reach for it blindly. First: if any single item in the batch can blow up (malformed input, an edge case that confuses the model), you risk losing the whole batch’s output instead of just one failed call, so keep batches small enough that a partial failure is cheap to redo, and validate the structured output before trusting any of it. Second: batching makes your prompt bigger, and if that pushes you past a cache breakpoint’s stable prefix or past the context window where quality degrades, you’ve traded rate-limit efficiency for worse answers. Batch the boring, uniform stuff (classification, extraction, short transforms). Don’t batch tasks where each item needs the full attention a standalone call would give it.
Rate-Limit Engineering: Backoff, Jitter, and a Router That Doesn’t Panic
You will hit a 429. The question is whether your code treats that as a crisis or as Tuesday.
import randomimport time
from anthropic import Anthropic, RateLimitError
client = Anthropic()
def call_with_backoff(**kwargs): max_retries = 5 base_delay = 1.0 for attempt in range(max_retries): try: return client.messages.create(**kwargs) except RateLimitError as exc: if attempt == max_retries - 1: raise # Honor a Retry-After header when the provider sends one. # It lives on the HTTP response, not on the exception object. retry_after = exc.response.headers.get("retry-after") delay = int(retry_after) if retry_after else base_delay * (2 ** attempt) # Jitter so a fleet of workers doesn't all retry in lockstep. delay += random.uniform(0, delay * 0.5) time.sleep(delay)Exponential backoff with jitter keeps a single script well behaved. But if you’re leaning on more than one free provider, you don’t want to hand-roll retry logic per provider. Put a router in front of them instead, LiteLLM’s proxy or OpenRouter both work as a single OpenAI-compatible endpoint that fans out to multiple backends and fails over automatically.
model_list: # Same model_name = LiteLLM treats these as one pool and rotates # between them, skipping whichever one is currently cooling down. - model_name: free-workhorse litellm_params: model: openrouter/qwen/qwen3-coder api_key: os.environ/OPENROUTER_API_KEY_1 - model_name: free-workhorse litellm_params: model: gemini/gemma-4-27b-it api_key: os.environ/GEMINI_API_KEY - model_name: paid-overseer litellm_params: model: anthropic/claude-sonnet-5 api_key: os.environ/ANTHROPIC_API_KEY
router_settings: # If every free-tier entry is rate-limited or down, escalate rather # than fail the request outright. fallbacks: - free-workhorse: ["paid-overseer"] num_retries: 3 retry_after: 10 cooldown_time: 60 allowed_fails: 2Point your application at free-workhorse and never think about which specific provider handled a given request. LiteLLM cools down a deployment that just 429’d and routes the next call to a healthy one in the pool, then escalates to the paid fallback only if the entire free pool is unavailable. That’s the difference between “the free tier is rate limited, guess we wait” and “the free tier is rate limited, the request went through anyway on a different account.”
One caveat if you’re rotating multiple free-tier accounts behind the same router: check each provider’s terms before you spin up several free accounts for the same person or project. Some providers are fine with it, some explicitly aren’t, and getting banned across the board because you automated your way around a rate limit is a worse outcome than just waiting.
Output Discipline: Don’t Pay for Prose You Throw Away
Input tokens get all the attention because that’s usually where the bloat is, but output tokens count against your quota too, and they’re easier to overspend without noticing. A model asked “is this config valid” that responds with four paragraphs of hedged explanation before finally saying “yes” burned output tokens on filler you’re going to skip past anyway.
Two fixes, both trivial to apply:
- Cap
max_tokens. If you’re expecting a short answer, a classification label, a single boolean, set the cap low enough that the model can’t ramble even if it wants to. This also protects you from a runaway generation eating your whole remaining quota on one confused call. - Use structured output or JSON mode when the answer has a shape. Asking for
{"valid": true, "reason": "..."}instead of free text gets you a parseable answer with none of the preamble, none of the “Sure, I’d be happy to help,” and no restating your own question back at you before answering it.
Structured output has a side benefit beyond token savings: it’s easier to catch a bad answer programmatically. A malformed JSON response fails a parse check immediately. A wandering paragraph of prose might be wrong in a way you don’t notice until three steps later.
Local Fallback for the Grunt Tier
At some point the math stops favoring the cloud entirely, even the free part of it. If eighty percent of your calls are mechanical, low-stakes, and don’t need frontier reasoning, and you already own a GPU that’s idle most of the day, running a small model locally for that eighty percent is the correct answer, not a consolation prize.
A quantized Qwen3-Coder in the 4B to 14B range, or a compact Gemma 4 variant, runs comfortably on consumer hardware through Ollama or llama.cpp and handles renames, boilerplate, summarization, and classification about as well as a free cloud tier does for those tasks, minus the rate limit, minus the network round trip, and minus any risk of an account getting flagged for automated traffic. Reserve the free or paid cloud tier for the twenty percent that actually benefits from a bigger model: architectural judgment calls, tricky debugging, anything where being wrong is expensive.
This is the same overseer and workhorse split as before, just with the workhorse moved onto hardware you already own. The free tier isn’t a competitor to your local setup, it’s backup capacity for the days your local model can’t cut it, or for tasks big enough that your local hardware would take a coffee break’s worth of time to finish. Treating it that way, as a reserve instead of a default, is usually what makes the daily allowance last.
None of this is complicated. Extract before you send, structure prompts so the cache actually hits, route mechanical work away from your best model, batch the uniform stuff, let a router absorb the 429s, cap your output, and push the grunt work onto hardware you already paid for once. Do all six and the same free tier that ran dry by Tuesday afternoon will comfortably carry you through the week.
