Scan and assumptions
I ran tokentoll on openai-cookbook, OpenAI's collection of example code. tokentoll does not execute the code and does not read any billing data. The numbers below are estimates from static analysis, based on these inputs:
- the model names and token parameters that tokentoll detected in the code,
- tokentoll's pricing data at the time of the scan (May 2026),
- an assumed volume of 1,000 calls per call site and month,
- where the prompt length cannot be determined from the code, a fixed assumed input length,
- where a call sets no output limit, an assumed output length equal to a fixed fraction of the model's maximum output length,
- where the model name could not be resolved, the per-SDK default model (gpt-4o for OpenAI).
The cookbook is a collection of examples and not a running service, so the total does not describe any actual spend. It shows how the estimated cost is distributed across call sites at the same assumed volume.
$ tokentoll scan openai-cookbook/
Total estimated monthly cost: $1,884.46
1000 calls/month per call sitetokentoll found 24 LLM API call sites with an estimated total of $1,884 per month.
Estimated cost by model
| Model | Call sites | Monthly cost |
|---|---|---|
| gpt-5 | 4 | $1,282.50 |
| gpt-4o | 13 | $508.77 |
| gpt-4.1 | 1 | $66.54 |
| gpt-4.1-mini | 2 | $26.61 |
| text-embedding-3-small | 4 | $0.04 |
The four gpt-5 call sites account for 68% of the total. They are in the prompt optimization cookbook: baseline generation, optimized generation and two LLM judge calls. Each is estimated at $320 per month.
The 13 gpt-4o call sites together come to $508 per month. Most of them are in evaluation harnesses and example apps that use gpt-4o when no model is specified.
The four embedding call sites come to $0.04 per month in total.
Estimate with gpt-4.1-mini in place of gpt-5
If the four gpt-5 call sites used gpt-4.1-mini, their estimate would drop from $1,282 to about $53, a factor of about 24. Only part of this factor is the difference in price per token. None of the four calls sets an output limit, so tokentoll assumes an output length derived from each model's maximum output length, and that maximum is larger for gpt-5 than for gpt-4.1-mini. Whether gpt-4.1-mini gives acceptable results for these four tasks would have to be tested separately.
Cost estimate in the pull request
A code review shows that a model name has changed. It does not show what the change costs at the expected call volume. tokentoll runs as a GitHub Action and adds this estimate to the pull request:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: Jwrede/tokentoll@v0.7.0When a pull request changes a model or adds an LLM call, the comment lists the
estimated monthly cost before and after the change for each affected call
site. With a policy in
.tokentoll.yml, the comment also contains a PASS, WARN or FAIL verdict, and
the workflow can be set to fail on FAIL. The full workflow and policy setup
are in the README.
Locally, this command compares the current code with the previous commit and lists the LLM call sites that were added, removed or changed, with the cost delta:
$ tokentoll diff HEAD~1About tokentoll
tokentoll statically analyzes Python, JavaScript and TypeScript code for LLM API calls and estimates their cost. It detects calls made through the SDKs of OpenAI, Anthropic and Google, LiteLLM, LangChain, the Vercel AI SDK and others. It needs no API keys and runs inside the CI environment. It also runs as an MCP server, so Claude Code and other MCP hosts can check the cost effect of a change before it is committed.
Current versions skip directories such as examples/ and cookbook/ by
default (use_default_excludes in .tokentoll.yml) and use updated pricing
data, so a new scan of openai-cookbook will not reproduce the numbers above
exactly.
pip install tokentoll
tokentoll scan .GitHub: github.com/Jwrede/tokentoll