Updated
What makers build

AI infrastructure & LLM ops tech stack 2026: what 88 real products use

Model hosting, gateways, evals, observability and other plumbing for running AI apps (frameworks, MCP and RAG have their own categories).

Among the 88 products we track in AI infrastructure & LLM ops, the usual stack is React (frontend framework, 72%), Vercel (hosting, 50%), OpenAI API (LLM API, 84%) and FastAPI (backend framework, 73%): shares of the products that show each decision.

How they're built

The typical stack

The leading pick where at least 8 of them show the decision and the leader has at least a quarter of it.

DecisionMost common pickShareAt default usage
Frontend FrameworkReact42 of 58 · 72%—
HostingVercel29 of 58 · 50%$40/mo · Pro + Flat Rate CDN 10M
LLM APIOpenAI API43 of 51 · 84%$10/mo · GPT-6 Luna
Backend FrameworkFastAPI36 of 49 · 73%—
DatabasePostgreSQL24 of 47 · 51%—
AI SDK & Agent FrameworkLangChain23 of 37 · 62%—
Transactional EmailResend18 of 29 · 62%$20/mo · Pro 50k
Database Access & ORMsSQLAlchemy18 of 28 · 64%—
Open Model Hosting & GPUsHugging Face20 of 28 · 71%—
Vector DatabaseQdrant8 of 24 · 33%$103/mo · Standard (3 nodes, 0.5 vCPU / 4 GiB each)
MCP Server for Your ProductMCP SDK23 of 23 · 100%—
AI UI Designshadcn/ui18 of 18 · 100%—
AuthenticationBetter Auth6 of 18 · 33%$0/mo · Self-hosted (open source)
LLM Observability & EvalsArize Phoenix4 of 15 · 27%—
EmbeddingsSentence Transformers10 of 14 · 71%—
File StorageAmazon S39 of 14 · 64%$11/mo · S3 Standard (US East)
PaymentsStripe13 of 14 · 93%fees on revenue · $410 on $10,000/mo
Product AnalyticsPostHog13 of 13 · 100%$0/mo · Pay-as-you-go
Document Parsingpypdf4 of 12 · 33%—
Error MonitoringSentry12 of 12 · 100%$29/mo · Team
Run Models LocallyOllama7 of 12 · 58%—
Web AnalyticsPlausible Analytics3 of 9 · 33%$19/mo · Starter 100k

Monthly bills add up to about $232 at the calculators' default usage, list prices; fees on revenue (Stripe) come on top. Set your own usage →

Where they differ from other products

Tools AI infrastructure & LLM ops pick at least twice as often as products overall. Each bar is the share of products that show the decision and chose the tool.

What they chose, decision by decision

Among the AI infrastructure & LLM ops that show each choice, from makers' products and open-source code alike. Under each tool, some of the products that chose it.

React42 · 72%Latitude, archestra, neuronpedia
Next.js30 · 52%Latitude, Cekura, HarnessRouter
React Router10 · 17%opencomputer, transformerlab-app, superserve
Streamlit5 · 9%trulens, judgeval, scenario
Hosting58 products
Vercel29 · 50%Latitude, Cekura, HarnessRouter
GitHub Pages16 · 28%Kiln, trulens, ColossalAI
Amazon Web Services7 · 12%Cekura, Edgee, TensorPool
Cloudflare5 · 9%neuronpedia, Kiln, opencomputer
LLM API51 products
OpenAI API43 · 84%Latitude, Cekura, HarnessRouter
Claude20 · 39%Latitude, Cekura, ClawMetry
Gemini API14 · 27%Latitude, Lightning Rod, ClawMetry
LiteLLM13 · 25%Kiln, trulens, pandaprobe
FastAPI36 · 73%archestra, mirage, neuronpedia
Express6 · 12%archestra, opencomputer, OmniRoute
Flask6 · 12%AReaL, diffgram, responsible-ai-toolbox
Hono4 · 8%Latitude, llmgateway, claude-code-hub
Database47 products
PostgreSQL24 · 51%Latitude, archestra, mirage
Supabase11 · 23%Cekura, HarnessRouter, Lamatic.ai
SQLite8 · 17%opencomputer, OmniRoute, transformerlab-app
ClickHouse8 · 17%Latitude, Prefactor, Tracely-ai
LangChain23 · 62%Latitude, Prefactor, mirage
AI SDK10 · 27%Latitude, archestra, mirage
Claude Agent SDK10 · 27%mirage, opencomputer, judgeval
LlamaIndex8 · 22%Latitude, Kiln, trulens
Resend18 · 62%Cekura, HarnessRouter, Lightning Rod
SendGrid8 · 28%Latitude, Lightning Rod, Basalt
Nodemailer5 · 17%Latitude, mirage, neuronpedia
SQLAlchemy18 · 64%trulens, Tracely-ai, labelflow
Drizzle ORM6 · 21%Latitude, archestra, lmring
Prisma4 · 14%mirage, neuronpedia, labelflow
TypeORM1 · 4%llm-gateway
Hugging Face20 · 71%Lightning Rod, neuronpedia, sie
vLLM6 · 21%neuronpedia, ColossalAI, AReaL
Together AI5 · 18%Latitude, Kiln, judgeval
Vector Database24 products
Qdrant8 · 33%mirage, sie, OmniRoute
Chroma7 · 29%mirage, trulens, sie
LanceDB6 · 25%mirage, Kiln, sie
pgvector5 · 21%Latitude, neuronpedia, Tracely-ai
MCP SDK23 · 100%Latitude, archestra, mirage
AI UI Design18 products
shadcn/ui18 · 100%Basalt, Langtail, archestra
Authentication18 products
Better Auth6 · 33%Latitude, archestra, lmring
Auth.js3 · 17%neuronpedia, labelflow, JamAIBase
Arize Phoenix4 · 27%Latitude, openinference, ASSERT
MLflow3 · 20%AReaL, responsible-ai-toolbox, gepa
Weights & Biases2 · 13%weave, mlx-omni-server
Embeddings14 products
Sentence Transformers10 · 71%sie, ColossalAI, feast
Cohere4 · 29%Latitude, archestra, openinference
File Storage14 products
Amazon S39 · 64%Latitude, archestra, mirage
MinIO7 · 50%lmring, Tracely-ai, labelflow
Payments14 products
Stripe13 · 93%Latitude, HarnessRouter, ClawMetry
PostHog13 · 100%Lamatic.ai, Basalt, xpander.ai
Document Parsing12 products
pypdf4 · 33%Kiln, trulens, ColossalAI
Unstructured3 · 25%trulens, ColossalAI, api-for-open-llm
Docling2 · 17%sie, ramalama
PyMuPDF2 · 17%bisheng, lotus
Error Monitoring12 products
Sentry12 · 100%Sailhouse, Prefactor, archestra
Ollama7 · 58%archestra, Kiln, rl-swarm
MLX LM5 · 42%magnitude, LocalAI, ramalama

Small samples, fewer than 8 products: Background Jobs & Cron (Celery 4, Temporal 1) · Image & Video Generation (Diffusers 5, Replicate 1) · Image & Video Hosting (sharp 6) · Speech-to-Text & Text-to-Speech (ElevenLabs 3, Kokoro 3) · Voice Agents (Pipecat 4, LiveKit 3) · Code Sandboxes for Agents (E2B 4, Daytona 2) · Search APIs for AI (SerpApi 3, Tavily 1) · AI Code Review (CodeRabbit 3) · Lifecycle Email (Loops 3) · Personal AI Agents (OpenClaw 2, BetterClaw 1)

AI infrastructure & LLM ops we track

56 makers' products and 32 open-source projects, makers' products first, each with the tools it uses (the most common here first).

LatitudeLatitude is an open-source observability and evaluation platform for AI agents that groups failures found in production traces into trackable issues. It is for developers running LLM agents in production.OpenAI APIClaude CodeReactNext.jsVercel
xpander.aiPlatform for running AI agents in production, with per-agent identity, user permissions, credential vaulting and audit logs. Runs in xpander's cloud or on a company's own Kubernetes, and agents can be used from Slack, Teams, ChatGPT or Claude.VercelPostHogStripeSupabaseNeon
openliveOpensource, on-device voice + vision layer for AI agents. Bring any model or coding agent; the whole speech loop (VAD, STT, TTS, barge-in) runs locally. An open alternative to ElevenLabs Agents, GeminReactNext.jsMCP SDKHonoKokoro

Questions

What tech stack is behind AI infrastructure & LLM ops?

Among the 88 products we track in AI infrastructure & LLM ops, the usual stack is React (frontend framework, 72%), Vercel (hosting, 50%), OpenAI API (LLM API, 84%) and FastAPI (backend framework, 73%): shares of the products that show each decision. After those, PostgreSQL leads database, LangChain leads AI SDK & agent framework and Resend leads transactional email.

Which tools are more common in AI infrastructure & LLM ops than elsewhere?

MLX LM for run models locally (4.2× as often as among products overall), ClickHouse for database (4.0× as often as among products overall) and Vertex AI for LLM API (3.7× as often as among products overall).

How much does a typical AI infrastructure & LLM ops stack cost?

About $232 a month in bills at the pricing calculators' default usage and list prices, plus Stripe fees on revenue. The biggest lines: Qdrant $103, Vercel $40 and Sentry $29.

Which products in AI infrastructure & LLM ops are open source?

32 of the 88 are open-source projects, known from their public code, among them inspect_ai, agent-learning-kit, modular, vmlx, mteb and openinference.