csh/playbook
1
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

📦 deps(thirdparty): update snapshots

Browse Source
This commit is contained in:
ci[bot]
2026-07-01 16:02:41 +00:00
parent 8301f01888
commit c824ba9d7b
2449 changed files with 555104 additions and 9259 deletions
Download Patch File Download Diff File Expand all files Collapse all files
antigravity-awesome-skills/skills/huggingface-zerogpu/SKILL.md
+306
View File
@@ -0,0 +1,306 @@
---
name: huggingface-zerogpu
description: AI demos and GPU compute with Gradio Spaces and Hugging Face Spaces ZeroGPU. Use when writing or reviewing code that uses `@spaces.GPU`, configuring `python_version` or `requirements.txt` for a ZeroGPU Space, or handling ZeroGPU-specific code constraints — pickle-based process...
risk: unknown
source: https://github.com/huggingface/skills/tree/main/skills/huggingface-zerogpu
source_repo: huggingface/skills
source_type: official
date_added: 2026-07-01
license: Apache-2.0
license_source: https://github.com/huggingface/skills/blob/main/LICENSE
---
# Hugging Face ZeroGPU
## When to Use
Use this skill when you need aI demos and GPU compute with Gradio Spaces and Hugging Face Spaces ZeroGPU. Use when writing or reviewing code that uses `@spaces.GPU`, configuring `python_version` or `requirements.txt` for a ZeroGPU Space, or handling ZeroGPU-specific code constraints — pickle-based process...
Rules and patterns for ML demos on Hugging Face Spaces with **ZeroGPU** hardware. Covers `@spaces.GPU`, duration and quota tuning, process isolation, the CUDA availability model, concurrency safety, and CUDA build constraints.
## Scope
This skill is for **Gradio SDK Spaces using ZeroGPU hardware**. Docker and Static Spaces cannot schedule onto ZeroGPU, and Streamlit apps now run as Docker Spaces — so this skill applies only to Gradio. For general Gradio coding (components, layouts, event listeners), see the `huggingface-gradio` skill in this repo. The authoritative ZeroGPU docs live at https://huggingface.co/docs/hub/spaces-zerogpu — refer to them for the current backing GPU, runtime version lists, and tier thresholds, all of which change over time.
## Reference Files
| Reference | When to read |
|-----------|--------------|
| `references/concurrency.md` | Always read alongside SKILL.md when writing ZeroGPU code — handlers run in parallel by default |
| `references/how-zerogpu-works.md` | When reasoning about cold-starts, worker reuse, why module-scope warmup does not carry to requests, or why returning CUDA tensors hangs |
| `references/how-quota-works.md` | When choosing `duration` values, debugging `illegal duration` vs `quota exceeded` errors, or explaining why default 60s blocks short tasks |
| `references/cuda-and-deps.md` | When installing CUDA-dependent packages (e.g. `flash-attn`), pinning torch side-cars, or reading wheel filename tags |
## Hardware
ZeroGPU exposes two GPU sizes that map to a fraction of the backing card:
| `size` | Slice of backing GPU | Quota cost |
|--------|----------------------|------------|
| `large` *(default)* | Half | 1x |
| `xlarge` | Full | 2x |
Default `large` gives half a physical GPU, so memory bandwidth and compute are significantly lower than the full card's specs. Use `xlarge` only when the workload genuinely needs the extra memory or compute.
> **Backing GPU changes without notice.** ZeroGPU has already migrated across GPU generations several times; older write-ups may name A100 or H200, but those are outdated. For the current backing GPU and exact per-size VRAM, always check the [ZeroGPU docs](https://huggingface.co/docs/hub/spaces-zerogpu) before sizing workloads.
## Basic Pattern
```python
import spaces
import torch
from transformers import pipeline
pipe = pipeline("text-generation", model="...", device="cuda")
@spaces.GPU
def generate(prompt: str) -> str:
return pipe(prompt, max_new_tokens=100)[0]["generated_text"]
```
Key rules:
1. **Instantiate models at module scope** and call `.to("cuda")` eagerly. ZeroGPU handles the actual device mapping transparently (see CUDA availability model below).
2. **Decorate GPU functions with `@spaces.GPU`**. The decorator is a no-op outside ZeroGPU, so it is safe to keep in all environments.
3. **Set `duration` to match the realistic worst-case workload** (default 60s). The platform pre-checks `requested duration` against the user's `remaining quota` — not against the actual run time — so a 10-second task left at the 60s default fails with `quota exceeded` as soon as the user's remaining quota drops below 60s. Smaller declared `duration` also ranks higher in the node-level queue. See "Duration and Quota" below.
4. **`torch.compile` is NOT supported.** Use PyTorch [ahead-of-time compilation (AoTI)](https://huggingface.co/blog/zerogpu-aoti) (torch 2.8+) instead.
5. **Use `size="xlarge"` sparingly.** It allocates the full backing GPU, but costs 2x quota and tends to queue longer.
```python
@spaces.GPU(duration=120)
def generate_image(prompt: str):
return pipe(prompt).images[0]
```
## CUDA Availability Model
Real GPU access is **only** available inside `@spaces.GPU`-decorated functions. Outside those functions, the GPU is not attached to the process.
However, `import spaces` **monkey-patches `torch`** so that:
- `torch.cuda.is_available()` returns `True` globally.
- `.to("cuda")` / `device="cuda"` calls at module scope succeed without error.
This is intentional. Module-scope `model.to("cuda")` calls register tensors with the ZeroGPU backend, which writes them to a disk offload directory at a startup "pack" step and frees the corresponding RAM. When a `@spaces.GPU` call lands, a forked GPU worker process streams those weights from disk into VRAM via a pinned-memory pipeline. Warm workers (reused across requests on the same GPU slot) keep weights resident on the GPU and skip the disk → VRAM step. The user-facing rule: write `device="cuda"` at module scope and it works — see `references/how-zerogpu-works.md` for the full lifecycle.
| Action | Where | Why |
|--------|-------|-----|
| `model.to("cuda")` / `pipe(..., device="cuda")` | **Module scope** | ZeroGPU registers the tensor and manages device migration |
| Actual CUDA computation (inference, etc.) | **Inside `@spaces.GPU`** | Real GPU is only attached during the decorated call |
| Branching on `torch.cuda.is_available()` | Avoid relying on it | Always returns `True` due to the monkey-patch |
Do not run inference or CUDA kernels at module scope — the real GPU is not attached, so operations either silently run on CPU or fail.
### Device selection idiom still works
The standard idiom remains correct under ZeroGPU:
```python
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = AutoModel.from_pretrained("...").to(device)
```
- **ZeroGPU** — `is_available()` is `True` (monkey-patched), so the model is registered for automatic device migration.
- **Dedicated GPU Spaces / local GPU** — `is_available()` is genuinely `True`.
- **CPU Spaces / local CPU** — resolves to `"cpu"`.
Do not hardcode `device="cuda"` — it breaks on CPU-only environments.
### Eager loading is the right default
Load models at module scope, not lazily on first request. The Space process starts before any user arrives, so cold-start cost is paid once. Lazy loading (`global model; if model is None: ...`, `@lru_cache` wrappers, factory functions instantiating on first call) just pushes that cost onto the first user.
## Local Development: Just Install `spaces`
Do **not** wrap `import spaces` in `try/except` and redefine `spaces.GPU` as a no-op fallback for local runs. Off-ZeroGPU, the `spaces` package is already a true no-op:
- Heavyweight behavior (CUDA monkey-patching, client init, startup hooks) is gated on the `SPACES_ZERO_GPU` env var, set only on ZeroGPU.
- `@spaces.GPU` returns the undecorated function unchanged off-ZeroGPU.
- Top-level `import spaces` performs only lightweight imports.
The Gradio SDK base image installs `spaces` on every hardware tier. So even after duplicating a Space onto a dedicated GPU (T4, L4, A10G, etc.) or CPU basic, no code changes are needed — `import spaces` still succeeds and `@spaces.GPU` becomes a transparent passthrough.
### Anti-pattern
```python
try:
import spaces
except ImportError:
class spaces: # type: ignore
@staticmethod
def GPU(func=None, **kwargs):
return func if func else (lambda f: f)
```
Problems:
1. The fallback must mimic every `@spaces.GPU` call shape — bare decorator, `duration=...`, `size=...`, generators, `aoti_*` helpers — and drifts as the `spaces` API grows.
2. It hides `spaces` from `requirements.txt`, even though the Space needs it at deploy time.
3. It solves a non-problem: the real package is already a no-op locally.
### Do this instead
Add `spaces` to dependencies and import it unconditionally:
```python
import spaces
@spaces.GPU
def generate(prompt: str) -> str:
...
```
## Duration and Quota
Three things happen when you declare `@spaces.GPU(duration=N)`:
1. **Tier-max check** — each visitor tier has a per-call `duration` cap. Declaring `duration` larger than the cap fails immediately with `ZeroGPU illegal duration`, regardless of remaining quota. (Tier numbers change over time — see the [ZeroGPU docs](https://huggingface.co/docs/hub/spaces-zerogpu).)
2. **Quota pre-check** — the platform compares `requested duration` against the user's `remaining quota`. If `remaining < requested`, the call fails with `ZeroGPU quota exceeded` — even if the actual work would have fit. The error message shows the explicit numbers, e.g. `"60s requested vs. 30s left"`. A 10-second task left at the default 60s therefore blocks the user once their remaining quota drops below 60s.
3. **Queue priority** — the queue is node-level (requests from all Spaces on the same node compete for GPU slots), and shorter declared `duration` ranks higher.
All three favor declaring the smallest realistic `duration` — including for short tasks. Explicit `@spaces.GPU(duration=15)` on a 10-second task avoids premature `quota exceeded` rejections and ranks higher in the queue.
> **`xlarge` doubles the request.** `requested = N * 2` when `size="xlarge"`, both for the tier-max check and the quota pre-check. So `@spaces.GPU(duration=60, size="xlarge")` is internally a 120s request.
### Dynamic duration for variable workloads
For workloads whose runtime depends on inputs, pass a callable that estimates per request. A static high `duration` locks out low-tier users (whose tier cap may be smaller than the static value) and unnecessarily reserves quota for light inputs.
```python
def estimate_duration(prompt, steps):
return int(steps * 3.5)
@spaces.GPU(duration=estimate_duration)
def generate(prompt, steps):
return pipe(prompt, num_inference_steps=steps).images[0]
```
For the full distinction between `illegal duration` vs `quota exceeded`, runs-per-day limits, the 24h quota window, and pay-as-you-go billing, see `references/how-quota-works.md`.
## Process Isolation and Pickle
`@spaces.GPU`-decorated functions run in a **separate process** managed by the ZeroGPU scheduler. Arguments and return values cross the process boundary via **pickle serialization**.
Consequences:
- **Only picklable objects** can be passed in or returned. Open file handles, database connections, locks, lambdas, and closures over unpicklable state will raise `PicklingError`.
- **Do NOT return CUDA tensors directly.** Unpickling a CUDA tensor in the main process triggers `torch.cuda._lazy_init()`, which ZeroGPU blocks. Convert to CPU first: return `tensor.cpu()` or `tensor.cpu().numpy()`.
- CPU tensors, numpy arrays, PIL Images, and plain Python objects work fine.
- Large objects incur serialization overhead. Prefer lightweight returns (tensors, arrays, file paths, base64 strings) over complex object graphs.
### `gr.State` semantics across the boundary
Because handlers run in a separate process, `gr.State` values are **pickled on every yield** — they are NOT shared by reference.
- The generator receives a **copy** of the state (`id()` differs from the caller's).
- In-place mutations inside the generator are **invisible** to other handlers until the mutated state is explicitly yielded back.
- Yielding `gr.update()` for a `gr.State` slot **skips the update** — other handlers continue to see the pre-yield value.
- Each yield that returns the state object creates a **new copy** via pickle.
Practical guidance:
- **Do NOT assume reference semantics for `gr.State`** on ZeroGPU. Code that mutates state in a generator and expects another handler to see those mutations will silently use stale data.
- **Every yield including a `gr.State` value triggers a full pickle round-trip.** For large state (model sessions, frame buffers), minimize how often you yield it — ideally once at the end. Use `gr.update()` for the state slot on intermediate yields.
- **CUDA tensors inside state must be moved to CPU before yielding** — same `torch.cuda._lazy_init()` issue as above.
## Concurrency
Handlers run **concurrently by default** on ZeroGPU. This is not opt-in. Code that worked in single-user testing can silently corrupt or leak data in production.
Three rules. Full treatment with examples in `references/concurrency.md`.
1. **No mutable global state.** Concurrent requests overwrite each other.
2. **No fixed file paths for outputs.** Concurrent requests clobber the same file. Use `tempfile` for unique paths.
3. **Read-only globals are safe.** Model objects, tokenizers, configs loaded once at startup and only read during requests are safe and encouraged.
## Call Granularity
Each entry into a `@spaces.GPU` function carries non-trivial cost — pickle round-trip across the process boundary, worker warm-up, CUDA re-attach, and a fresh pass through the node-level queue. Calling a decorated function from inside a hot loop multiplies these costs and adds a new failure mode: a later iteration may fail to acquire a GPU slot, stalling the whole job mid-way.
Decorate the outer function that owns the loop, not the per-iteration worker:
```python
# Avoid — N GPU entries for N frames
def process_video(frames):
return [process_frame(f) for f in frames]
@spaces.GPU(duration=...)
def process_frame(frame):
...
# Prefer — one GPU entry for the whole video
@spaces.GPU(duration=...)
def process_video(frames):
return [process_frame(f) for f in frames]
def process_frame(frame):
...
```
If the loop mixes heavy CPU work with GPU work, wrapping the whole loop charges that CPU time against the user's quota. When that cost is material, batching the GPU work so CPU pre/post-processing stays outside the decorator is a situational optimization — not the default.
## CUDA Build Constraints
HF Spaces builds Docker images in a CPU-only environment. **On ZeroGPU, the build phase has no `nvcc`** because the base image is `python:3.13` (dedicated-GPU Spaces use `nvidia/cuda:*-devel-*` and have `nvcc` at build time). A CUDA-dependent package whose only distribution is sdist — e.g. bare `flash-attn` — therefore cannot be installed via `requirements.txt` on ZeroGPU. Only pre-built wheels work.
ZeroGPU **runtime** does have `nvcc` available, mounted from a CUDA devel image at `/cuda-image` since 2025-07 (originally added for AoTI support). This is what makes `torch.export` / AoTI workflows possible inside `@spaces.GPU` calls.
**Bottom line**: install every CUDA-dependent package from a pre-built wheel. If no wheel is available on PyPI, build one externally (e.g. host on HF Hub) and pin the URL. For `flash-attn`, the upstream releases page ships a fairly complete wheel matrix covering most Python × CUDA × torch combinations.
For wheel-tag reading (cxx11 ABI, `cu12torch2.X`, `cp3XX`), torch-family side-car drift, and the kernels-community fallback, see `references/cuda-and-deps.md`.
## Example Caching
`gr.Examples` behavior is environment-dependent. On ZeroGPU specifically:
- `cache_examples` defaults to `True` (Spaces sets `GRADIO_CACHE_EXAMPLES=true`).
- `cache_mode` defaults to `"lazy"` (Spaces sets `GRADIO_CACHE_MODE=lazy` only on ZeroGPU).
ZeroGPU defaults to `lazy` because eager caching pre-runs every example at app startup, but ZeroGPU has **no GPU attached at startup** — only during request handling. Eager caching of GPU-bound examples would fail there.
When `cache_examples=True`, the `run_on_click` / `run_examples_on_click` parameter is silently ignored. If your app relies on click-populates-only behavior, set `cache_examples=False` explicitly to preserve it.
To reproduce ZeroGPU example-caching behavior locally:
```bash
GRADIO_CACHE_EXAMPLES=true GRADIO_CACHE_MODE=lazy python app.py
```
## Dependency Management
### `python_version` pin in README frontmatter
Pinning `python_version` is **effectively required** for ZeroGPU. The runtime default is currently Python 3.10, so a local environment using 3.11+ will fail to install on the Space without an explicit pin. Pin to a ZeroGPU-supported version (3.12 is a reasonable default); the authoritative supported list lives in the [ZeroGPU docs](https://huggingface.co/docs/hub/spaces-zerogpu) — do not hardcode the full list, refer to the docs.
```yaml
# README.md frontmatter
python_version: "3.12"
```
Both `"3.12"` and `"3.12.12"` forms are accepted.
### Do not pin `spaces` in `requirements.txt`
The Space platform pins its own `spaces` version. A conflicting pin in `requirements.txt` causes pip resolution to fail at build time.
> **Rule**: Do not include `spaces` in `requirements.txt`.
How to achieve this depends on your tooling:
- **Hand-written `requirements.txt`**: simply omit `spaces`.
- **uv** (`pyproject.toml`-managed): declare `spaces` in `pyproject.toml` so uv co-resolves transitive constraints (notably `psutil`, which `spaces` pins), then exclude it from the export:
```bash
uv export --no-hashes --no-dev --no-emit-package spaces -o requirements.txt
```
Without `spaces` in `pyproject.toml`, uv cannot see its transitive constraints and may resolve incompatible versions at build time.
- **pip-tools** (`pip-compile`) / **Poetry**: use the equivalent exclude mechanism.
### Pin `torch` to match wheel tags
If you install a CUDA-dependent wheel via direct URL, the wheel filename encodes the `torch` major.minor it was built against (e.g. `cu12torch2.8`). Pin `torch==X.Y.Z` in `requirements.txt` to match — otherwise pip may resolve `torch` to a different version and the Space fails on first import. Details and the kernels-community alternative are in `references/cuda-and-deps.md`.
## Limitations
- Use this skill only when the task clearly matches its upstream product or API scope.
- Verify commands, API behavior, pricing, quotas, credentials, and deployment effects against current official documentation before making changes.
- Do not treat generated examples as a substitute for environment-specific tests, security review, or user approval for destructive or costly actions.
antigravity-awesome-skills/skills/huggingface-zerogpu/references/concurrency.md
+79
View File
@@ -0,0 +1,79 @@
# Concurrency Safety
Gradio handlers run **in parallel by default on ZeroGPU**. Code that works fine in single-user testing can silently corrupt or leak data in production. Always assume handlers execute concurrently.
## No mutable global state
Per-request or per-user data must not live in module-level mutable variables. Concurrent requests will overwrite each other.
```python
# BAD — concurrent requests overwrite each other
results = {}
def process(text):
results["output"] = expensive_compute(text) # race condition
return results["output"]
```
```python
# GOOD — pure function, no shared mutable state
def process(text):
return expensive_compute(text)
```
For state that must persist within a single user session, use `gr.State`:
```python
with gr.Blocks() as demo:
history = gr.State(value=[])
def add_message(msg, hist):
hist.append(msg)
return hist, hist
btn.click(fn=add_message, inputs=[msg, history], outputs=[chatbot, history])
```
Note that on ZeroGPU, `gr.State` is pickled across the worker boundary on every yield — see "Process Isolation and Pickle" in SKILL.md for the implications.
## No fixed file paths for outputs
Hardcoded output filenames cause concurrent requests to overwrite each other's files. This corrupts outputs and, worse, can leak one user's data to another.
```python
# BAD — concurrent calls clobber the same file
def generate_image(prompt):
image = pipe(prompt).images[0]
image.save("output.png")
return "output.png"
```
```python
# GOOD — unique path per invocation
import tempfile
def generate_image(prompt):
image = pipe(prompt).images[0]
with tempfile.NamedTemporaryFile(suffix=".png", delete=False) as f:
image.save(f.name)
return f.name
```
The same applies to any intermediate files (audio, video, CSV exports). Always generate a unique path per invocation.
## Read-only globals are safe
Model objects, tokenizers, and configs loaded once at startup and only read during requests are safe and encouraged. This is the standard ZeroGPU pattern: load at module scope, read inside `@spaces.GPU` handlers.
```python
# SAFE — loaded once at module scope, read-only during requests
model = load_model().to("cuda")
tokenizer = load_tokenizer()
@spaces.GPU
def predict(text):
tokens = tokenizer(text, return_tensors="pt").to("cuda")
return model.generate(**tokens)
```
The "no mutable global state" rule targets *writes* from handlers, not reads. A handler that only reads from a global is concurrency-safe.
antigravity-awesome-skills/skills/huggingface-zerogpu/references/cuda-and-deps.md
+66
View File
@@ -0,0 +1,66 @@
# CUDA Dependencies on ZeroGPU
Detailed guidance for installing CUDA-dependent packages on ZeroGPU. SKILL.md establishes the bottom line — wheels are the recommended path because the ZeroGPU build phase has no `nvcc`. This document covers wheel filename tag reading, the kernels-community fallback, and torch-family side-car drift.
## When no wheel is available on PyPI
Common workarounds, in preference order:
1. **Pre-built wheel via direct URL.** For `flash-attn`, the upstream project ships a fairly complete matrix at https://github.com/Dao-AILab/flash-attention/releases — check there first and pin the matching wheel URL.
2. **Build the wheel yourself and host it** (e.g. on a public HF Hub repo) when no upstream wheel matches the Space environment.
3. **Use a kernels-community kernel** (see below) — handles ABI matching for you, no version pinning needed.
## Reading a CUDA wheel filename
A wheel filename like
```
flash_attn-2.8.0.post2+cu12torch2.8cxx11abiFALSE-cp312-cp312-linux_x86_64.whl
```
encodes four build-time choices:
| Tag | Meaning |
|-----|---------|
| `cu12` | CUDA major version |
| `torch2.8` | torch major.minor the wheel was compiled against |
| `cxx11abiFALSE` | C++ stdlib ABI choice (`TRUE` or `FALSE`) |
| `cp312-cp312` | CPython version (3.12) |
The wheel's compiled C-extension will `ImportError` on ABI/symbol mismatches if any of these drift at install time.
If you hand pip a wheel URL without pinning the surrounding environment, pip may resolve `torch` to a version different from the wheel's build target, and the Space will fail on first import. Therefore:
- Pin `torch==X.Y.Z` in `requirements.txt` to match the wheel's `torch2.X` tag.
- Set `python_version:` in the Space frontmatter to match the `cp3XX` tag.
- Check the runtime's cxx11-ABI choice against the wheel; if unsure, try the opposite ABI wheel.
## Prefer kernels-community when unsure
If you are not sure about the ZeroGPU runtime's torch / Python / ABI combination, prefer a [kernels-community](https://huggingface.co/kernels-community) kernel (e.g. `kernels-community/flash-attn2`) instead of a raw wheel URL. The kernels runtime handles ABI matching on your behalf, so no version pinning is required in your Space.
## torch-family side-car drift
`torchvision`, `torchaudio`, `torchcodec`, and similar side-car packages are built against a specific `torch` major.minor (and CUDA major). On ZeroGPU, the runtime's supported `torch` list lags behind PyPI, so projects often pin a non-latest `torch` — and a bare `uv add <side-car>` can silently resolve to a newer release that targets a different `torch` / CUDA, producing ABI/import failures even though `uv lock` succeeded without warnings.
Concretely observed (2026-04) with `torch==2.9.1` pinned:
- `torchaudio` resolves to `2.11.0`, which targets torch 2.11 / CUDA 13. The `2.11.0` release **dropped the `Requires-Dist: torch==X.Y.Z` line** that every earlier release had, so uv sees no constraint and picks it.
- `torchcodec` resolves to a release targeting torch 2.11. No torchcodec release on PyPI declares a `torch` dependency at all; the compatibility table lives only in the project README.
- `torchvision` happens to resolve correctly because torchvision still declares `Requires-Dist: torch==X.Y.Z`. Which side-cars are affected changes over time — treat every torch-family package as suspect, not just these.
### Verify at add/upgrade time
After any `uv add <torch-side-car>` or `uv lock --upgrade`, verify the resolved version targets the same `torch` major.minor as pinned. Two-step fallback because PyPI metadata is not always sufficient:
1. Query PyPI for the resolved version's `requires_dist`:
```bash
curl -s https://pypi.org/pypi/<pkg>/<version>/json \
| python3 -c "import json,sys,re; rd=json.load(sys.stdin)['info'].get('requires_dist') or []; print('\n'.join(x for x in rd if re.match(r'^torch(?![a-z])', x)) or '(no torch constraint declared)')"
```
If a `torch==X.Y.Z` line appears and matches the pinned torch, good. If it appears and does NOT match, the side-car is wrong — pin it down explicitly.
2. If the query prints `(no torch constraint declared)`, PyPI metadata is silent and cannot be trusted. Fall back to the project's own compatibility table (GitHub README / docs site) — torchcodec, for example, maintains one at https://github.com/pytorch/torchcodec. Pick the side-car version the table maps to the pinned torch major.minor, and pin it explicitly.
### Preventive pin
Once the correct side-car version is known, pin it in `pyproject.toml` alongside torch so uv cannot drift on future `uv lock --upgrade`. The side-car version numbers for a given torch major.minor change each release; always re-verify, do not copy a mapping from an older project.
antigravity-awesome-skills/skills/huggingface-zerogpu/references/how-quota-works.md
+74
View File
@@ -0,0 +1,74 @@
# How ZeroGPU duration and quota are checked
Mechanism for `duration` validation and quota pre-checks. Useful when choosing `duration` values, debugging `illegal duration` vs `quota exceeded` errors, and understanding why the default 60s is pessimistic for short tasks.
For per-tier numerical thresholds (free vs Pro vs Team vs Enterprise quota minutes), the daily quota window length, runs-per-day limits, and pay-as-you-go pricing, see [the ZeroGPU docs](https://huggingface.co/docs/hub/spaces-zerogpu) — those values change over time and are deliberately kept out of this skill.
## What `duration` actually requests
Whatever value is passed to `@spaces.GPU(duration=N)` (or the default 60s when unspecified) becomes the `requested duration` the platform checks against. For `xlarge`, the request is doubled internally:
```
requested = N * 2 if size == "xlarge" else N
```
So `@spaces.GPU(duration=60, size="xlarge")` is internally a 120-second request — both for the tier-max check and the quota pre-check below.
## Two distinct error modes
Two failure messages can come back from the scheduler before the call runs:
| Error | Trigger | What helps |
|---|---|---|
| **`ZeroGPU illegal duration`** | `requested duration > visitor's tier per-call cap` | Lower `duration`. Sign in / upgrade tier. **Waiting does not help.** |
| **`ZeroGPU quota exceeded`** | `remaining quota < requested duration`, OR runs-per-day cap reached | Wait for the quota window to reset. For Pro / Team / Enterprise, pay-as-you-go credits cover the overflow. |
The error wording for `quota exceeded` includes the explicit numbers, e.g.:
```
You have exceeded your Pro ZeroGPU quota
(60s requested vs. 30s left). Try again in 1:23:45.
```
The comparison is **`requested` vs `remaining`** — not `actual run time` vs `remaining`. A 10-second task left at the default 60s requests 60s of quota; once `remaining < 60s` the call fails even though the actual work would have fit.
## Why the default 60s is pessimistic for short tasks
`DEFAULT_SCHEDULE_DURATION` in the `spaces` package is **60 seconds**. So an undecorated `@spaces.GPU` (or `@spaces.GPU()` with no `duration=`) requests 60s of quota.
For a task that actually takes ~10 seconds:
- The user's 60s quota gets reserved up front.
- Once their remaining quota drops below 60s, your Space fails for them — even though they could have run many more 10s tasks if the request matched reality.
- Your call also ranks lower in the queue than equivalent calls declaring smaller durations.
The fix is to declare the realistic duration explicitly:
```python
@spaces.GPU(duration=15)
def fast_task(...):
...
```
For workloads where runtime depends on inputs, use a callable (per-request estimator):
```python
def estimate_duration(prompt, steps):
return int(steps * 3.5)
@spaces.GPU(duration=estimate_duration)
def variable_task(prompt, steps):
...
```
This preserves quota for light inputs and reserves more only when needed.
## Quota window: 24h fixed from first use
The quota window's TTL is set when the first call of a fresh window lands and counts down unconditionally — it is not a sliding window, not a calendar-day reset, and not extended by subsequent use. A user who runs a call at 14:00 sees their next reset at 14:00 the following day, regardless of how heavily or lightly they use the Space in between.
For exact tier thresholds, runs-per-day caps, and pay-as-you-go billing rates, see the [ZeroGPU docs](https://huggingface.co/docs/hub/spaces-zerogpu).
## Queue priority
The queue is **node-level** — requests from every Space scheduled on the same physical node compete for that node's GPU slots. Among queued requests, **shorter declared `duration` ranks higher**. So tight per-request `duration` estimates serve two goals at once: they preserve the user's quota and move the request up the queue.
antigravity-awesome-skills/skills/huggingface-zerogpu/references/how-zerogpu-works.md
+50
View File
@@ -0,0 +1,50 @@
# How ZeroGPU works (mechanism)
Conceptual lifecycle of model weights, processes, and worker reuse on ZeroGPU. Useful when reasoning about cold-starts, why module-scope warmup does not carry over to requests, why returning CUDA tensors hangs the call, or why `gr.State` mutations do not persist across the worker boundary.
For numerical limits (concurrency slots per Space, queue priority by tier, etc.), see [the ZeroGPU docs](https://huggingface.co/docs/hub/spaces-zerogpu) — those values change over time and are deliberately kept out of this skill.
## Two processes, two lifetimes
A ZeroGPU Space runs as **two separate processes**:
- **Main web process** — long-lived. Imports `app.py`, launches Gradio, stays up for the life of the Space. Holds no VRAM and, after the startup "pack" step, holds no model weights in RAM either.
- **GPU worker processes** — short-lived. Forked per `@spaces.GPU` request (or reused if warm). Run the task and are eventually killed by the ZeroGPU scheduler when another Space needs the GPU slot. Your Space code never kills its own worker.
## Module-scope `.to("cuda")` is captured to disk
When `import spaces` is active, `model.to("cuda")` at module scope is intercepted. The call is rewritten to `to("cpu")`, so the tensor data physically lives in main process RAM at this point. A "fake" CUDA-presenting tensor is registered alongside the original CPU tensor.
At a startup "pack" step, the backend writes those original CPU tensors to disk via direct I/O (`O_DIRECT`), then frees the corresponding RAM. After pack, the main process holds no model weights anywhere — the data lives only on disk.
This is why module-scope `pipe(...)` / `model.generate(...)` / `model(...)` calls do not run on a real GPU: there is no GPU attached to the main process, and after pack there are no weights to compute against either. Such calls either fail or silently fall back to CPU on the fake tensors.
## Worker init: disk → pinned memory → VRAM
When a `@spaces.GPU` call lands, the scheduler routes it to a worker:
1. **Cold worker** — forked from the main process. The patched torch is unpatched, real CUDA is initialized, and weights are read from the disk offload directory into pinned host memory and streamed onto VRAM through a double-buffered pipeline (essentially `pin_memory().cuda(non_blocking=True)` per batch). This is the "cold-start" cost.
2. **Warm worker (reused)** — an alive worker bound to the same GPU slot is reused if the scheduler reports it idle. Init is skipped; weights stay on VRAM from the previous call. Subsequent requests within a burst hit this path.
A warm worker is eventually killed by the scheduler when another Space needs the GPU slot. The next call after that point pays the disk → VRAM cost again. Occasional cold-starts on a low-traffic Space are normal.
## Why module-scope warmup does not help
A common instinct is to call `pipe("warmup")` at module scope to "prepare" the model. This does not work on ZeroGPU:
- At module scope, no real GPU is attached. The fake CUDA tensors do not have data after pack, so `pipe(...)` either fails or silently runs on something other than a real GPU.
- Even if you wrap the warmup in `@spaces.GPU`, the worker that ran the warmup will eventually be killed before the first real user request lands — leaving them with a cold worker anyway.
The right answer is to load eagerly at module scope (`pipe = pipeline(..., device="cuda")`) and accept that the first user request after a quiet period will be a cold worker. Cold-start is fast on ZeroGPU because of the pinned-memory disk pipeline; it is not free, but it is not "minutes of model download" either.
## Why returning CUDA tensors hangs the call
The main process never has a CUDA context — it has no GPU attached and its torch never initialized CUDA. When a worker returns a CUDA tensor, unpickling it in the main process triggers `torch.cuda._lazy_init()`, which would attempt to initialize CUDA in the main process. ZeroGPU blocks this, and the call hangs.
The fix is purely client-side: convert to CPU before returning (`.cpu()`, `.cpu().numpy()`, etc.). See "Process Isolation and Pickle" in SKILL.md.
## Why `gr.State` does not share by reference across the boundary
Worker processes are forked separately and exchange data with the main process via pickle. `gr.State` values cross this boundary on every yield, so mutations inside a `@spaces.GPU` generator are local to the worker until the mutated state is explicitly yielded back. The main process gets a fresh deserialized copy each time — `id()` differs, in-place mutations are invisible across the boundary.
See "Process Isolation and Pickle" in SKILL.md for the practical implications and `references/concurrency.md` for related parallel-handler concerns.