Hosting AI in production

"Vercel vs. AWS vs. Modal vs. self-hosted" sounds like one decision. It's actually two, and conflating them is why hosting comparisons for AI systems get more complicated than they need to be for most people reading one.

Who this is for: an engineer choosing where to run a RAG or agent system, before checking whether the decision is even about hosting a model at all.

The question underneath the question

Every post in this cluster — checklist, chunking, hybrid search, reranking, the model decision tree — has assumed API calls to a hosted model, not a self-hosted one. That's not an oversight; it's the honest default for most RAG and agent systems, and it means "hosting AI" for most of the people asking the question is really "hosting the application that calls an AI API," which is a much smaller and more familiar decision than it sounds.

Ask this first, before any provider comparison: does the system need to run model weights itself, or does it need to call an API that runs them for you? If it's the second — which covers most RAG pipelines, most tool-calling agents, and everything this site's own stack does — the hosting decision is almost entirely about hosting an ordinary web application with occasional long-running requests, not about GPU infrastructure at all.

If you're API-only: this is a normal hosting decision

A Next.js app, a Python service, whatever calls out to a model provider's API — this is the hosting decision you already know how to make for any web application, with one wrinkle: request duration. LLM calls, especially streamed ones, hold connections open longer than a typical API request, which matters for platforms with hard request timeouts and for cost models that charge by compute-time-held rather than by request count.

The same order-of-operations argument that applies to infrastructure generally applies here too: don't provision for a scale or a request pattern you haven't hit yet. A serverless platform with generous timeout limits is the right default until you have a specific, measured reason to move off it — not a guess about what you might need once traffic grows.

If you're actually self-hosting weights: a different decision

This is the minority case, and it's worth being honest about when it's real: data residency requirements that forbid sending content to a third-party API, a fine-tuned open-weight model that only exists as weights you run yourself, or cost at a volume where the math for renting GPU-hours changes relative to per-token API pricing. None of those are hypothetical — they're just less common than the volume of "which host should I use" questions would suggest.

Here the real dimensions are cold-start tolerance (a serverless GPU that scales to zero costs less at idle and costs latency on every cold request), request pattern (spiky traffic favors elasticity, steady traffic favors a provisioned instance you're not paying a cold-start tax on), and whether you need the operational surface of managing GPU infrastructure yourself versus a managed inference platform that abstracts it.

The decision this post can make without a number

Whether you're in the first case or the second is decidable without any cost data at all — it's a question about your own system's requirements, not about provider pricing. Get that answer first. It's the one that determines whether the rest of this decision is "configure a normal web host" or "evaluate GPU infrastructure providers," and those are different enough conversations that collapsing them into one comparison table is what makes this question feel harder than it is for most people asking it.

If you're not sure which case you're actually in, that's usually the first thing worth working out before comparing a single provider price.