Inference Providers — when not to host the model

Some Spaces don't need a GPU at all. If the model is available through HF Inference Providers (Cerebras, Fireworks, Together, Replicate, OpenRouter, etc.), the Space can be a thin Gradio shell that proxies to a hosted endpoint:

Zero VRAM, no @spaces.GPU, no model download.
Works for models too large to fit on ZeroGPU (120B+).
Hardware can be cpu-basic — no GPU at all.

When to use this pattern

Stateless chat or text completion with a big model.
The user wants a public demo of a frontier-scale model that obviously doesn't fit on a single 48 GB MIG.
The user wants to ship something fast without worrying about quantization / sharding.

When NOT to use this pattern

The model isn't available on any Inference Provider. Check with:

curl "https://huggingface.co/api/models/<ns>/<repo>?expand[]=inferenceProviderMapping"

The Space needs custom decoding (special sampling, tool use, retrieval, anything stateful or interactive across calls).
The Space needs multimodal beyond what the provider exposes.
The user explicitly wants to own the inference stack (model loading, decoding, performance tuning).

For those, host the model yourself on ZeroGPU — see zerogpu.md.

Two billing modes

Choose based on who pays for inference.

Mode A — Space creator pays (simple)

Set HF_TOKEN as a Space secret. The Space uses InferenceClient directly. Every visitor's call is billed to the Space creator's account.

import os, gradio as gr
from huggingface_hub import InferenceClient

client = InferenceClient(api_key=os.environ["HF_TOKEN"], provider="fireworks-ai")

def chat(msg, history):
    return client.chat_completion(
        model="<org>/<model>",
        messages=[*history, {"role": "user", "content": msg}],
        max_tokens=512,
    ).choices[0].message.content

gr.ChatInterface(chat).launch()

Use when you want users to "just click and try it" — no sign-in friction. Cost is on you.

Mode B — Visitor pays (recommended for public demos)

gr.LoginButton + gr.load("models/...") with accept_token=button. Each visitor signs in with their HF account; inference is billed to their account.

import gradio as gr

with gr.Blocks(fill_height=True) as demo:
    with gr.Sidebar():
        button = gr.LoginButton("Sign in")
    gr.load("models/<org>/<model>", accept_token=button, provider="fireworks-ai")
demo.launch()

README frontmatter needs:

hf_oauth: true
hf_oauth_scopes:
  - inference-api

This is the recommended pattern for public demos — sustainable cost-wise, and visitors get to use their own provider quotas (which most have paid for or get free).

Hardware

cpu-basic. No GPU. Don't put --flavor zero-a10g — you'd waste a paid grant.

Anti-pattern: `@spaces.GPU` wrapping a provider call

If you do use Inference Providers, do not wrap the call in @spaces.GPU. The decorator reserves a GPU slot on your Space for the full duration=, but the function does no GPU work — just an HTTP call out. You burn your own ZeroGPU quota for nothing.

A provider-proxy Space wants cpu-basic hardware and zero @spaces.GPU.

3.2 KiB Raw Blame History