LLM cost optimization on AWS: Bedrock FinOps and real bills
[!NOTE] TL;DR - Most Bedrock leaks hide outside your prompt. Pairing explicit prompt caching with non-blocking orchestration cut our AWS pipeline bill by 78.5%, from 5,495.19 USD to 1,178.80 USD per 100,000 runs (AWS Price List, Nov. 2026). - Prefix checkpoints on Claude Sonnet 5.5 drop repeated input reads from 2.00 USD to 0.10 USD per million tokens, while Claude Haiku 5.5 handles triage at 0.10 USD input. - Direct AWS Step Functions calls remove idle Lambda duration charges, and Pydantic AI
UsageLimitswith OpenTelemetry spans stop runaway agent loops.
Your Cost Explorer filter lies by omission. Anthropic models bill under AWS Marketplace rather than Bedrock, where disciplined LLM cost optimization cuts serverless spend by 78.5%, from 5,495.19 USD to 1,178.80 USD per 100,000 runs (AWS Price List, Nov. 2026). In the beginning was the default config, and my client's CFO was already on Slack.
Anatomy of a 5,495 USD surprise
Last autumn I audited an industrial PDF inspection pipeline for a European energy operator processing 100,000 technical pages a month in us-east-1. Nobody noticed the drift for six weeks. Their engineering dashboard filtered Cost Explorer by Service = Amazon Bedrock, showing tiny embedding charges while 5,060.00 USD of Claude Sonnet 5.5 traffic accumulated under AWS Marketplace.
Add 375.19 USD of over-allocated x86 Lambda compute and 60.00 USD of verbose CloudWatch logs, and a two-stage workflow burned 5,495.19 USD monthly (1.10 USD per 20-page report). What I set out to test here was one practical question: how far could we compress unit cost on AWS without hurting extraction recall or operational simplicity? Guesswork wasn't an option.
Prefix amnesia and idle compute
One spontaneously imagines that shortening prompts is the primary lever for LLM cost optimization. Production traces attest to the opposite. Raw Converse API logs exposed four structural leaks: 1) frontier models running binary triage; 2) broken prompt caches; 3) unrequested thinking tokens billed as output; and 4) 2,048 MB Lambdas idling on sockets.
Start with model and thinking defaults. Per Amazon Bedrock pricing (checked Nov. 2026), pinning Claude Sonnet 5.5 to a single Region costs 2.20 USD per million input tokens and 11.00 USD output, 10% above Global routing. Worse, as the AWS Machine Learning Blog (Nov. 12, 2026) documents, Claude Sonnet 5.5 runs adaptive thinking at high effort by default, billing hidden reasoning tokens at 10.00 USD per million on Global endpoints. An 80-token JSON verdict silently carried 520 tokens of internal deliberation.
Then came prefix amnesia. Explicit prompt caching on Bedrock reads checkpoints strictly from top to bottom, locking tool schemas and static system instructions ahead of user messages. It works like a lead printing press bolted into an iron frame. Chisel a fresh timestamp into the upper system plate, and the press scraps every lower block before re-casting the frame at 1.25 times the base input rate. I spent four hours blaming Bedrock's regional routers before spotting my own UTC helper at line two of the system prompt, sabotaging the working memory on which my whole architecture rested.
Compute billing compounded the waste. To mask a 2.8-second x86 cold start from heavy imports, the team kept 15 Provisioned Concurrency environments warm on a 2,048 MB Lambda (0.0000041667 USD per GB-second in us-east-1, Nov. 2026). That idle reservation burned 328.50 USD monthly before reading one PDF. Meanwhile, CloudWatch REPORT logs showed peak RAM at 412 MB while CPUs idled for 24 seconds on Bedrock sockets, reversing the math of our Lambda versus SageMaker serverless inference bills.
Non-blocking Step Functions topology
Building on those four findings, we split orchestration from compute: Step Functions owns network wait time, while ARM64 Lambda runs only when typed Python logic is required. Teams untangling state machines under production load can review this pattern through our AWS serverless architecture consulting.
flowchart LR
S3[S3 PDF Pages] --> SQS[SQS 5m Micro-Batch]
SQS --> SFN[Step Functions Standard]
SFN -->|Direct invokeModel| H55[Bedrock Claude Haiku 5.5 Global]
H55 --> CH{Anomaly Detected?}
CH -->|No: 60% pages| DDB[(DynamoDB Report Store)]
CH -->|Yes: 40% pages| LMB[512 MB ARM64 Lambda: Pydantic AI]
LMB -->|Cached Converse + low effort| S55[Bedrock Claude Sonnet 5.5 Global]
S55 --> DDB
LMB -.->|OTel GenAI Spans| LF[Logfire / OTel Collector]
Holding a 2 GB Lambda open during a 24-second LLM call is like hiring a crane operator to suspend a steel girder while an inspector reads blueprints across town. Following the AWS Step Functions Bedrock integration guide, our workflow parks each request on a passive railway siding at 0.000025 USD per transition (us-east-1, Nov. 2026), releasing compute engines during generation. You buy the click, never the silence.
First-pass triage runs directly from Step Functions against global.anthropic.claude-haiku-5-5 (0.10 USD input and 0.50 USD output per million tokens, Nov. 2026), cutting per-token rates twentyfold against Sonnet 5.5. Clean pages bypass Lambda completely. Stick with Standard workflows over Express here: although Express starts at 1.00 USD per million requests, its duration billing and mandatory CloudWatch vended logs erode savings on slow model calls.
For the 40% of pages needing extraction, we dropped Provisioned Concurrency and moved to 512 MB Graviton2 arm64 containers (0.0000133334 USD per GB-second) running pydantic-ai-slim[bedrock] on Python 3.12. Cold starts fell from 2.8 seconds to 460 milliseconds without idle reservations, avoiding the capacity trap detailed in our SageMaker async inference scale-to-zero analysis.
Typed guardrails in Pydantic AI
State machines eliminate idle waits, yet agent loops still need code-level circuit breakers. Following the Pydantic AI Bedrock documentation (checked Nov. 2026), we wrapped BedrockConverseModel in strict Pydantic schemas enforcing four rules: Global routing, explicit max_tokens caps, low thinking effort, and 5-minute cache checkpoints.
Explicit max_tokens caps stop quota throttling too. If you don't set max_tokens, Pydantic AI sends Claude's full ceiling, which Bedrock reserves upfront against your tokens-per-minute quota before generating a single word. Right-sizing that limit prevents retry storms.
from decimal import Decimal
from typing import Literal
import logfire
from pydantic import BaseModel, Field, field_validator
from pydantic_ai import Agent, InstrumentationSettings, UsageLimits
from pydantic_ai.models.bedrock import BedrockConverseModel, BedrockModelSettings
logfire.configure(send_to_logfire="if-token-present")
# Exclude raw OCR text from spans to prevent AWS X-Ray 64 KB UDP packet drops
OTEL_SETTINGS = InstrumentationSettings(include_content=False, include_binary_content=False)
class FinOpsPolicy(BaseModel):
model_config = {"strict": True}
model_id: str = Field(description="Bedrock Global inference profile ID.")
cache_ttl: Literal["5m", "1h"] = Field(default="5m", description="Explicit prompt cache TTL.")
max_output_tokens: int = Field(default=1024, ge=128, le=4096, description="Caps TPM reservation.")
max_cost_usd: Decimal = Field(default=Decimal("0.08"), gt=Decimal("0"), description="Hard run ceiling.")
@field_validator("model_id")
@classmethod
def require_global_profile(cls, value: str) -> str:
if not value.startswith("global."):
raise ValueError("Use a global. cross-Region profile to avoid the 10% regional markup")
return value
class PageEvent(BaseModel):
model_config = {"strict": True}
document_id: str = Field(description="Upstream document UUID.")
ocr_text: str = Field(min_length=1, description="Extracted page text.")
class DefectFinding(BaseModel):
model_config = {"strict": True}
component_code: str = Field(description="Normalized equipment identifier.")
severity: Literal["low", "medium", "critical"] = Field(description="Audited defect severity.")
evidence_excerpt: str = Field(max_length=280, description="Verbatim OCR span.")
class PageAuditResult(BaseModel):
model_config = {"strict": True}
document_id: str = Field(description="Immutable upstream document UUID.")
findings: list[DefectFinding] = Field(default_factory=list, description="Extracted anomalies.")
POLICY = FinOpsPolicy(model_id="global.anthropic.claude-sonnet-5-5")
# Static rubric stays immutable so Bedrock 5-minute prefix checkpoints hit reliably
STATIC_RUBRIC = (
"Extract verified equipment defects from industrial inspection OCR. "
"Follow the JSON schema strictly and cite verbatim text spans."
)
MODEL_SETTINGS = BedrockModelSettings(
max_tokens=POLICY.max_output_tokens,
thinking="low",
bedrock_cache_instructions=POLICY.cache_ttl,
bedrock_cache_tool_definitions=POLICY.cache_ttl,
bedrock_cache_messages=POLICY.cache_ttl,
)
# Initialize outside lambda_handler to reuse botocore sockets on warm arm64 runs
EXTRACTION_AGENT = Agent(
model=BedrockConverseModel(POLICY.model_id),
output_type=PageAuditResult,
instructions=STATIC_RUBRIC,
model_settings=MODEL_SETTINGS,
instrument=OTEL_SETTINGS,
)
RUN_LIMITS = UsageLimits(
request_limit=4,
tool_calls_limit=6,
input_tokens_limit=24000,
cost_limit=POLICY.max_cost_usd,
count_tokens_before_request=True,
)
def lambda_handler(event: dict[str, str], context: object) -> dict[str, object]:
payload = PageEvent.model_validate(event)
# Dynamic metadata stays in the user message tail to preserve prefix cache hits
user_prompt = f"doc_id={payload.document_id}\npage_ocr:\n{payload.ocr_text}"
result = EXTRACTION_AGENT.run_sync(user_prompt, usage_limits=RUN_LIMITS)
usage = result.usage()
return {
"audit": result.output.model_dump(),
"input_tokens": usage.input_tokens,
"cache_read_tokens": usage.cache_read_tokens,
"estimated_cost_usd": str(usage.cost),
}
Without span-level LLM observability, auditing token drift in CloudWatch feels like walking through a dark cave with a barometer that prints your altitude only at the exit. OpenTelemetry spans act as wired beacons bolted along the tunnel wall, exporting pydantic_ai.cache.hit_ratio and operation.cost to Logfire or Langfuse via the Pydantic Logfire LLM cost guide. When a garbled OCR table triggers a loop, UsageLimits drops an iron gate across the shaft at 0.08 USD.
[!WARNING] X-Ray 64 KB UDP truncation trap: Exporting default GenAI spans to AWS X-Ray drops traces whenever verbose prompts push a UDP segment past 64 KB. Keep
include_content=Falsein production Lambdas so token and cache metrics survive.
Monthly invoice at 100,000 runs
To avoid flattering free-tier distortions, every figure below uses marginal us-east-1 rates from the AWS Price List (checked Nov. 2026), excluding free Lambda and Step Functions tiers. Across 100,000 monthly page runs (40,000 triggering Stage 2 extraction), the invoice dropped by 4,316.39 USD, a 78.5% cut.
| Pipeline Line Item | Naive Baseline (us-east-1) |
Optimized FinOps (us-east-1) |
Monthly Before -> After |
|---|---|---|---|
| Stage 1 Triage (100k calls) | Sonnet 5.5 In-Region, uncached, high thinking (600M in / 60M out) |
Step Functions direct to Haiku 5.5 Global, 85% 5m cache hit | 1,980.00 USD -> 31.08 USD |
| Stage 2 Extraction (40k runs) | Sonnet 5.5 In-Region, broken cache, 2.5 turns (800M in / 120M out) | Sonnet 5.5 Global, 80% 5m cache hit, low effort, 2.0 turns |
3,080.00 USD -> 1,105.28 USD |
| Orchestration & Compute | 2,048 MB x86 Lambda + 15 Provisioned Concurrency units (24/7) | Step Functions Standard (620k transitions) + 512 MB arm64 Lambda |
375.19 USD -> 18.44 USD |
| Telemetry & Log Ingestion | 120 GB verbose CloudWatch raw prompt dumps + X-Ray drops | 18 GB structured JSON logs + stripped OTel GenAI spans | 60.00 USD -> 24.00 USD |
| Total Monthly Spend | 1.10 USD per 20-page report | 0.24 USD per 20-page report | 5,495.19 USD -> 1,178.80 USD |
{
"type": "bar",
"data": {
"labels": [
"Stage 1 Triage LLM",
"Stage 2 Extraction Agent",
"Serverless Compute & State",
"Logs & OTel Traces"
],
"datasets": [
{
"label": "Baseline Naive Setup (USD/mo)",
"data": [1980.00, 3080.00, 375.19, 60.00],
"backgroundColor": ["#ef4444", "#ef4444", "#ef4444", "#ef4444"]
},
{
"label": "Optimized FinOps Setup (USD/mo)",
"data": [31.08, 1105.28, 18.44, 24.00],
"backgroundColor": ["#10b981", "#10b981", "#10b981", "#10b981"]
}
]
},
"options": {
"responsive": true,
"plugins": {
"title": {
"display": true,
"text": "Monthly AWS Spend at 100,000 Page Runs (us-east-1 List Rates, Nov. 2026)"
}
},
"scales": {
"y": {
"title": {
"display": true,
"text": "Monthly Cost (USD)"
}
}
}
}
}
Not every optimization experiment succeeded. During week two, I switched Stage 2 from 5-minute cache checkpoints (2.50 USD per million write tokens on Sonnet 5.5 Global) to 1-hour checkpoints at 4.00 USD. Night-shift uploads arrived 75 minutes apart.
Treating a 1-hour cache TTL as a free upgrade is like firing a brick kiln all night to bake two loaves at dawn. You burn double the fuel upfront to heat the masonry, and the warmth dissipates before a third batch enters the chamber. Because a 1-hour write needs three hits within 60 minutes to beat uncached pricing, our off-peak hit ratio dropped to 31% and raised night input costs by 18%. Although that 1-hour configuration lost money on trickle traffic, our spans confirmed that 5-minute SQS micro-batches locked in an 80% hit ratio on the 1.25x write tier.
Remaining protocol limits
No setup fits every workload intact. Note that explicit caching requires at least 512 prefix tokens on Claude 5.5 models (and 4,096 on Claude Haiku 4.5, AWS docs Nov. 2026). Because Bedrock scopes caches per AWS account rather than per tenant, prepending SHA-256 tenant hashes in multi-tenant SaaS apps fragments hit ratios at low volume. Regulated workloads under strict residency clauses must also skip Global profiles and pay the 10% geographic markup.
Step Functions adds a hard payload boundary too: each state acts like a pneumatic mail canister with a fixed 256 KiB diameter. Stuff a raw multi-page OCR dump inside, and the tube jams with States.DataLimitExceeded until you pass S3Uri pointers instead. To a certain extent, span cost metrics also drift from real invoices because genai-prices estimates won't reflect private AWS Enterprise Discount Program rebates.
Two avenues remain untested on our corpus. Claude Sonnet 5.5 currently rejects the 50%-off Flex tier with a ValidationException (Nov. 2026), and Bedrock Batch disables prompt caching and Converse tool calls. This could easily be tested on overnight backfills by scoring Amazon Nova 2 Lite on Flex (0.15 USD per million input tokens) against our validation set. It remains to be seen whether cheaper Nova tiers hold recall on degraded handwritten tables.
Discipline beyond prompt golf
More generally, sustainable FinOps treats generative inference as an asynchronous state machine governed by prefix invariants and token ceilings, never as a blocking socket inside an oversized function. If your Bedrock and Marketplace lines are drifting faster than traffic, I audit production pipelines and serverless state machines through my AI architecture and FinOps consulting. Anyway, check your Marketplace tab before Friday!
Frequently asked questions
Why are Anthropic Claude charges missing when I filter AWS Cost Explorer by Amazon Bedrock?
Third-party foundation models such as Anthropic Claude bill through AWS Marketplace rather than the native Amazon Bedrock service line (AWS Price List, Nov. 2026). Filtering Cost Explorer solely by Service = Amazon Bedrock hides those token charges under the vendor's name. Include AWS Marketplace in your filters and apply Bedrock cost allocation tags to track full inference spend.
When does Amazon Bedrock explicit prompt caching lose money instead of saving it?
Explicit caching charges a write premium on Claude Sonnet 5.5 Global: 2.50 USD per million tokens for a 5-minute write and 4.00 USD for a 1-hour write (Nov. 2026). A 5-minute cache pays back on its first hit, whereas a 1-hour checkpoint needs three requests within 60 minutes to beat uncached pricing. One dynamic timestamp breaks every downstream checkpoint.
Should an AWS cloud architect choose Step Functions Standard or Express for Bedrock orchestration?
Standard workflows suit multi-second LLM calls because they charge 0.000025 USD per state transition (us-east-1, Nov. 2026) and zero for wait duration. Express workflows start at 1.00 USD per million requests, yet they meter GB-seconds during model waits and add CloudWatch log fees. Keep Express for fast idempotent transforms.
When does hiring an independent AWS consultant pay off for LLM cost optimization?
Default serverless configs quietly inflate generative AI bills through 10% regional markups and 5x output rates on unrequested thinking tokens. An independent AWS consultant correlates CloudWatch REPORT memory logs with OpenTelemetry LLM observability spans (pydantic_ai.cache.hit_ratio and operation.cost) to cut structural waste. Above 2,500 USD in monthly Bedrock spend, non-blocking orchestration and prefix hygiene routinely trim bills by 60% to 78% (audited Nov. 2026). Most audits pay back inside thirty days.
Prepared by Vector, SamAlgo's writing agent, augmented and curated from SamAlgo's research, code and publications.