How to estimate LLM API costs

The biggest mistake teams make is comparing models by input and output price alone. A pricing page can tell you the list rate, but it cannot tell you what your product will cost in the hands of real users. That only becomes clear once you know how often people use the feature, how much context each request carries, and how long the answers tend to be.

Before you estimate anything, write down a few operating assumptions: monthly active users, requests per user per day, average input tokens, average output tokens, retry rate, and whether part of the prompt can be cached. If those numbers are fuzzy, use a conservative range instead of pretending you already know the exact answer.

A clean estimate starts with cost per request. Take the average input tokens, average output tokens, apply the model's pricing, and include any meaningful adjustments such as cached input or extra retries. That gives you a unit cost you can reason about and compare across models.

This step matters because it keeps the math grounded. If one request costs a few cents instead of a fraction of a cent, the monthly number can swing fast once usage scales. Teams that skip this step usually end up with a budget that looks reasonable on paper but collapses as soon as the product gets adoption.

Once you have cost per request, multiply it by expected request volume for the month. That sounds obvious, but it is the step that turns model pricing into something finance, product, and engineering can all discuss together. A model is not expensive or cheap in the abstract. It is expensive or cheap for a specific workload at a specific volume.

If usage is still early, model a low, expected, and high scenario. The low case tells you whether the feature is safe to test. The high case tells you whether success creates a margin problem. Both numbers are useful, and together they are more honest than a single polished estimate.

A good estimate should survive small changes in assumptions. Try increasing output length, lowering cache share, or adding a few extra retries. If the monthly spend jumps sharply, that is not a bad model. It is a sign that your feature economics are sensitive and should be monitored closely.

This is also where model comparison becomes useful. Sometimes the better decision is not the absolute lowest list price, but the model that stays predictable when prompts get longer or usage spikes. The goal is not perfect forecasting. The goal is choosing a model with eyes open.