AI infrastructure & LLM ops tech stack 2026: what 88 real products use
Model hosting, gateways, evals, observability and other plumbing for running AI apps (frameworks, MCP and RAG have their own categories).
Among the 88 products we track in AI infrastructure & LLM ops, the usual stack is React (frontend framework, 72%), Vercel (hosting, 50%), OpenAI API (LLM API, 84%) and FastAPI (backend framework, 73%): shares of the products that show each decision.
How they're built
The typical stack
The leading pick where at least 8 of them show the decision and the leader has at least a quarter of it.
| Decision | Most common pick | Share | At default usage |
|---|---|---|---|
| Frontend Framework | 42 of 58 · 72% | — | |
| Hosting | 29 of 58 · 50% | $40/mo · Pro + Flat Rate CDN 10M | |
| LLM API | 43 of 51 · 84% | $10/mo · GPT-6 Luna | |
| Backend Framework | 36 of 49 · 73% | — | |
| Database | 24 of 47 · 51% | — | |
| AI SDK & Agent Framework | 23 of 37 · 62% | — | |
| Transactional Email | 18 of 29 · 62% | $20/mo · Pro 50k | |
| Database Access & ORMs | 18 of 28 · 64% | — | |
| Open Model Hosting & GPUs | 20 of 28 · 71% | — | |
| Vector Database | 8 of 24 · 33% | $103/mo · Standard (3 nodes, 0.5 vCPU / 4 GiB each) | |
| MCP Server for Your Product | 23 of 23 · 100% | — | |
| AI UI Design | 18 of 18 · 100% | — | |
| Authentication | 6 of 18 · 33% | $0/mo · Self-hosted (open source) | |
| LLM Observability & Evals | 4 of 15 · 27% | — | |
| Embeddings | 10 of 14 · 71% | — | |
| File Storage | 9 of 14 · 64% | $11/mo · S3 Standard (US East) | |
| Payments | 13 of 14 · 93% | fees on revenue · $410 on $10,000/mo | |
| Product Analytics | 13 of 13 · 100% | $0/mo · Pay-as-you-go | |
| Document Parsing | 4 of 12 · 33% | — | |
| Error Monitoring | 12 of 12 · 100% | $29/mo · Team | |
| Run Models Locally | 7 of 12 · 58% | — | |
| Web Analytics | 3 of 9 · 33% | $19/mo · Starter 100k |
Monthly bills add up to about $232 at the calculators' default usage, list prices; fees on revenue (Stripe) come on top. Set your own usage →
Where they differ from other products
Tools AI infrastructure & LLM ops pick at least twice as often as products overall. Each bar is the share of products that show the decision and chose the tool.
What they chose, decision by decision
Among the AI infrastructure & LLM ops that show each choice, from makers' products and open-source code alike. Under each tool, some of the products that chose it.
Small samples, fewer than 8 products: Background Jobs & Cron (Celery 4, Temporal 1) · Image & Video Generation (Diffusers 5, Replicate 1) · Image & Video Hosting (sharp 6) · Speech-to-Text & Text-to-Speech (ElevenLabs 3, Kokoro 3) · Voice Agents (Pipecat 4, LiveKit 3) · Code Sandboxes for Agents (E2B 4, Daytona 2) · Search APIs for AI (SerpApi 3, Tavily 1) · AI Code Review (CodeRabbit 3) · Lifecycle Email (Loops 3) · Personal AI Agents (OpenClaw 2, BetterClaw 1)
AI infrastructure & LLM ops we track
56 makers' products and 32 open-source projects, makers' products first, each with the tools it uses (the most common here first).
Questions
What tech stack is behind AI infrastructure & LLM ops?
Among the 88 products we track in AI infrastructure & LLM ops, the usual stack is React (frontend framework, 72%), Vercel (hosting, 50%), OpenAI API (LLM API, 84%) and FastAPI (backend framework, 73%): shares of the products that show each decision. After those, PostgreSQL leads database, LangChain leads AI SDK & agent framework and Resend leads transactional email.
Which tools are more common in AI infrastructure & LLM ops than elsewhere?
MLX LM for run models locally (4.2× as often as among products overall), ClickHouse for database (4.0× as often as among products overall) and Vertex AI for LLM API (3.7× as often as among products overall).
How much does a typical AI infrastructure & LLM ops stack cost?
About $232 a month in bills at the pricing calculators' default usage and list prices, plus Stripe fees on revenue. The biggest lines: Qdrant $103, Vercel $40 and Sentry $29.
Which products in AI infrastructure & LLM ops are open source?
32 of the 88 are open-source projects, known from their public code, among them inspect_ai, agent-learning-kit, modular, vmlx, mteb and openinference.