📦 deps(thirdparty): update snapshots
This commit is contained in:
ci[bot]
@@ -0,0 +1,409 @@
|
||||
---
|
||||
name: huggingface-lora-space-builder
|
||||
description: Build and publish a Gradio demo on Hugging Face Spaces for a user-provided LoRA. Use when someone asks to create, generate, ship, or publish a Space, demo, Gradio app, or playground for a LoRA — including LoRAs for Qwen-Image, Qwen-Image-Edit, LTX-Video, Wan, FLUX, SDXL, or other...
|
||||
risk: unknown
|
||||
source: https://github.com/huggingface/skills/tree/main/skills/huggingface-lora-space-builder
|
||||
source_repo: huggingface/skills
|
||||
source_type: official
|
||||
date_added: 2026-07-01
|
||||
license: Apache-2.0
|
||||
license_source: https://github.com/huggingface/skills/blob/main/LICENSE
|
||||
---
|
||||
|
||||
# Gradio LoRA Space Builder
|
||||
## When to Use
|
||||
|
||||
Use this skill when you need build and publish a Gradio demo on Hugging Face Spaces for a user-provided LoRA. Use when someone asks to create, generate, ship, or publish a Space, demo, Gradio app, or playground for a LoRA — including LoRAs for Qwen-Image, Qwen-Image-Edit, LTX-Video, Wan, FLUX, SDXL, or other...
|
||||
|
||||
|
||||
Build and publish a Gradio demo on Hugging Face Spaces that runs inference with a user-provided LoRA. Use whenever someone asks to create, generate, ship, or publish "a Space", "a demo", "a Gradio app", or "a playground" for a LoRA — whether the base model is Qwen-Image, Qwen-Image-Edit, LTX, or another diffusion model. Also use when someone describes a LoRA they trained or hosts on the Hub and wants to share it. The default target is ZeroGPU hardware and the default inference library is `diffusers` when the base model supports it.
|
||||
|
||||
The output is a real, published Space (private by default) that the user can try in the browser, not a local script.
|
||||
|
||||
## What "good" looks like for these demos
|
||||
|
||||
The demo should feel handcrafted for this specific LoRA, not a generic template with the LoRA bolted on. Two LoRAs that share a task can still need different demos: a pose-control video LoRA and an outpainting video LoRA both take video in and produce video out, but the inputs the user provides, the preprocessing, and the controls are completely different. Recognizing that is the central job here.
|
||||
|
||||
Concretely, a good demo:
|
||||
|
||||
- Loads fast and runs fast — minimal model loading, sensible step count, no wasted computation per call.
|
||||
- Has a UI with exactly the controls this LoRA needs and nothing else. Excess sliders are a cost, not a feature.
|
||||
- Shows the user what's happening — progress, intermediate outputs where useful, the seed used, a clear error when input is missing.
|
||||
- Honors the LoRA's own recommendations from its model card: trigger words, recommended step count, recommended guidance scale, recommended LoRA scale, example inputs.
|
||||
- Is creative where creativity helps — interactive canvases, before/after sliders, side-by-side previews of intermediate processing — and plain where plainness is right.
|
||||
|
||||
## Workflow
|
||||
|
||||
Work through these phases in order. Information gathered in one phase decides the next.
|
||||
|
||||
1. Gather the LoRA info needed to pick a pipeline and design a UI.
|
||||
2. Pick the base pipeline and inference recipe.
|
||||
3. Design the UI for this specific LoRA's task and inputs.
|
||||
4. Write `app.py`, `requirements.txt`, and `README.md` together; show all three to the user for one batched approval.
|
||||
5. Publish the Space (private).
|
||||
|
||||
Don't drip-feed questions across multiple turns. Batch them.
|
||||
|
||||
---
|
||||
|
||||
## Phase 1 — Gather LoRA info
|
||||
|
||||
Required: a LoRA repo on the Hub (e.g. `username/my-lora`).
|
||||
|
||||
**First, try to read the repo without a token.** If it succeeds, the repo is public — proceed. If it fails with 401/403, the repo is private/gated and you need an authenticated session to read it. **Don't immediately ask for a token.** Check first whether the user is already authenticated.
|
||||
|
||||
```python
|
||||
from huggingface_hub import HfApi, get_token
|
||||
|
||||
cached_token = get_token() # picks up HF_TOKEN env var or cached CLI login
|
||||
if cached_token:
|
||||
try:
|
||||
info = HfApi().whoami(token=cached_token)
|
||||
username = info["name"]
|
||||
# info also has fine-grained token scope info if applicable
|
||||
except Exception:
|
||||
cached_token = None # token exists but is invalid/expired
|
||||
```
|
||||
|
||||
Then:
|
||||
|
||||
- If a valid cached token exists *and* it can read the repo, use it. No prompt needed.
|
||||
- If no cached token, or the cached token can't read this private repo, ask the user for a token — once, with the explanation below.
|
||||
|
||||
When asking for a token (and only when you actually need to ask):
|
||||
|
||||
> I need a Hugging Face access token with **write** scope (to read the LoRA if it's private/gated, and to publish the Space). Create one at https://huggingface.co/settings/tokens. Paste it here.
|
||||
|
||||
The same token will be reused for publishing in the final phase, so this is a one-time ask.
|
||||
|
||||
**Then read what's in the repo:**
|
||||
|
||||
- List the repo files (`huggingface_hub.HfApi().list_repo_files(repo_id)`). Look for `.safetensors`, `README.md`, example images/videos, multiple checkpoints.
|
||||
- Fetch the model card (`huggingface_hub.ModelCard.load(repo_id)`). The `data` dict has structured fields; the `text` has the README body.
|
||||
- If multiple `.safetensors` files exist, pick the right one — see "Picking the LoRA weights file" in `references/zerogpu-and-publishing.md`. Briefly: README-recommended file wins, then `pytorch_lora_weights.safetensors`, then latest training checkpoint, otherwise ask.
|
||||
|
||||
**From the model card, try to determine:**
|
||||
|
||||
- **Base model** — the `base_model` field, or text mentions in the README. Usually present. Use it to pick the pipeline reference file (see Phase 2).
|
||||
- **Task** — `pipeline_tag` if set, otherwise inferred from the base model and README text. The five tasks this skill handles: `text-to-image`, `image-to-image`, `text-to-video`, `image-to-video`, `video-to-video`.
|
||||
- **Trigger words** — often called "trigger word", "instance prompt", "activation word"; sometimes embedded in example prompts.
|
||||
- **Recommended inference recipe** — step count, guidance scale, true CFG scale, LoRA scale, resolution. Many LoRA cards include a Python snippet; trust its *parameters* (steps, guidance, CFG, LoRA scale, dtype). For *loading mechanics*, see `adapting-to-the-lora.md` — prefer `pipe.load_lora_weights(...)` over whatever loading approach the snippet uses.
|
||||
- **Example prompts and example media** — use these as Gradio examples in the UI.
|
||||
- **Sub-task / specific use case** — for image edits and video LoRAs, "what does this LoRA actually do" matters as much as the task category. A relighting LoRA, a face-swap LoRA, and a style LoRA all might be image-to-image, but the UI for each is different.
|
||||
|
||||
**When something can't be inferred, ask the user — once, in a single batched message.** Format the question to make answering trivial. For task category, list the five options as a numbered choice. For sub-task, give a one-line description ("what does this LoRA do? e.g. 'relight portraits', 'apply manga style', 'extend videos to wider aspect ratios'"). Don't ask if you can already infer it confidently from the base model or README.
|
||||
|
||||
If the model card has nothing helpful at all — no base model, no task, no example — surface that clearly: "The model card has no usable info. I'll need you to tell me: (1) base model, (2) what this LoRA does, (3) recommended step count and guidance scale if you know them."
|
||||
|
||||
---
|
||||
|
||||
## Phase 2 — Pick the base pipeline
|
||||
|
||||
Two things to decide here: which reference file to load, and which pipeline class to use. They're not the same question — a base-model family file (e.g. `qwen-image.md`) covers multiple variants, and variants in the same family don't always share a pipeline class. Get this wrong and the Space loads but produces wrong output, or fails at startup.
|
||||
|
||||
**Step 1 — Load the reference file for this base model family.**
|
||||
|
||||
- `references/base-models/qwen-image.md` — covers Qwen-Image and Qwen-Image-Edit family (text-to-image and image-to-image).
|
||||
- `references/base-models/ltx.md` — covers LTX family (text-to-video, image-to-video, video-to-video, including IC-LoRAs).
|
||||
- `references/base-models/krea-2.md` — covers Krea 2 (K2), text-to-image (train on RAW, run inference/LoRAs on the Turbo distilled checkpoint).
|
||||
|
||||
If the base model isn't in one of these files, this skill doesn't have first-class support yet. Tell the user, and ask whether they want to proceed by analogy (use the closest model's recipe and adjust) or stop. Don't guess silently.
|
||||
|
||||
**Step 2 — Verify the pipeline class against the base model's own card. This step is mandatory, not optional.**
|
||||
|
||||
A new base model variant might use the same pipeline class with a different repo path, or a new pipeline class entirely. Don't trust the reference file's table alone — it's best-effort and can lag a recent release. Verify before committing:
|
||||
|
||||
```python
|
||||
from huggingface_hub import ModelCard
|
||||
base_card = ModelCard.load(base_model_id)
|
||||
# Read base_card.text — find the diffusers inference snippet, note the pipeline class it imports.
|
||||
```
|
||||
|
||||
The class imported in the base model card's diffusers snippet is the source of truth. Real examples where this matters:
|
||||
|
||||
- `Qwen-Image-Edit` uses `QwenImageEditPipeline`. `Qwen-Image-Edit-2509` and `Qwen-Image-Edit-2511` use `QwenImageEditPlusPipeline` — different class, different default parameters, takes a list of images instead of one. A LoRA targeting 2511 loaded onto `QwenImageEditPipeline` produces broken output.
|
||||
- LTX-Video uses `LTXPipeline`/`LTXImageToVideoPipeline`/`LTXConditionPipeline`. LTX-2 uses `LTX2Pipeline` from a different module path. LTX-2.3 sometimes needs a native pipeline outside diffusers.
|
||||
|
||||
If the base model card has no diffusers snippet at all, fall back to the reference file's table — and tell the user you're falling back, in case they know something the table doesn't.
|
||||
|
||||
The cost of this verification is one Hub fetch and a few seconds of reading. The cost of skipping it is the failure mode the previous bullet describes — a "working" Space that's quietly using the wrong class.
|
||||
|
||||
**Step 3 — Diffusers vs native pipeline.** Default to `diffusers` when the base model has a diffusers pipeline class. That's the case for Qwen-Image and Qwen-Image-Edit and most of LTX. Some LTX variants (notably LTX-2.3 with certain IC-LoRAs) need a native pipeline; the LTX reference says when. Diffusers gives standard `load_lora_weights` / `set_adapters` semantics; the native path needs LoRA-specific glue.
|
||||
|
||||
---
|
||||
|
||||
## Phase 3 — Design the UI for this LoRA
|
||||
|
||||
Don't reach for a template. Reason from the LoRA's task and inputs to a UI.
|
||||
|
||||
Read `references/tasks.md` for the per-task baseline UI patterns (what the standard inputs/outputs look like for T2I, I2I, T2V, I2V, V2V).
|
||||
|
||||
Then read `references/adapting-to-the-lora.md`, which is about *thinking through what this specific LoRA needs* — beyond the task category. That file is the most important one in this skill. The same task can need very different UIs: a pose-control LTX LoRA needs a video input and a pose-extraction preview; an outpaint LTX LoRA needs an aspect-ratio picker and a black-margin preview; a relighting Flux LoRA needs an image and a brush canvas for indicating where to add light. None of those reduce to "the V2V template" or "the I2I template".
|
||||
|
||||
**Self-check before writing the UI.** Write one sentence describing what a user does with this Space in 10 seconds. If that sentence doesn't distinguish this LoRA from any other LoRA of the same task, the UI isn't shaped enough yet.
|
||||
|
||||
Examples that pass the self-check:
|
||||
|
||||
- "Upload a video, pick a target aspect ratio, click Generate; the model fills the empty margins."
|
||||
- "Draw colored brush strokes where you want light, pick an illumination style, click Generate; the model relights the photo."
|
||||
- "Upload a video of someone moving and an image of a different character; the model produces a video of the character doing the motion."
|
||||
|
||||
Examples that fail:
|
||||
|
||||
- "Type a prompt and click generate." (Generic T2I — say more.)
|
||||
- "Upload an image and an instruction." (Generic edit — what kind of edit?)
|
||||
|
||||
**Gradio component freshness.** Gradio's component set evolves. Before defaulting to plain components, consider whether something newer fits better — for example `gr.ImageSlider` for before/after on edit LoRAs, `gr.BrowserState` for persistent prefs, `@gr.render` for UIs that change based on input. If you're unsure whether a component exists or what its signature is, web-fetch the current Gradio docs at https://www.gradio.app/docs rather than guessing.
|
||||
|
||||
**When stock and Hub custom components aren't enough — creative mode.** If the LoRA's natural input is a shape no Gradio component (built-in or on the Hub) expresses well — point sets, strokes, trajectories, multi-region annotations with metadata, 3D rotation gizmos, timeline scrubbers, anything where the user manipulates a thing on top of media — drop down to custom HTML/JS via `gr.HTML`. See `references/creative-mode.md` for the Gradio primitives (`gr.HTML`, `head=` injection, `elem_id` addressing, the two JS↔Python state-sync approaches), the discipline around defining a JSON wire format, and the pitfalls. Don't reach for creative mode just because it would be cool — reach for it when the LoRA's input shape demands it. And don't skip the Hub custom components rung above (e.g. `gradio_image_annotation`) before going fully bespoke.
|
||||
|
||||
**`gr.Examples` for media-input Spaces.** When no fitting example media is available from the model's own repo, pull from the shared input pools — split by modality so the HF dataset viewer can render proper thumbnails: images at [`linoyts/repo-to-space-example-inputs`](https://huggingface.co/datasets/linoyts/repo-to-space-example-inputs), videos at [`linoyts/repo-to-space-example-videos`](https://huggingface.co/datasets/linoyts/repo-to-space-example-videos). Both are CC0 with `categories` + natural-language `caption` metadata and the same filter/rank recipe in each dataset README. Pick 2–3 that fit the task, preprocess to the shapes the model expects, and bake the copies into the Space. Set `cache_examples=True, cache_mode="lazy"` so the first click caches without running examples at build time (see `references/zerogpu-and-publishing.md`).
|
||||
|
||||
---
|
||||
|
||||
## Phase 4 — Write the Space files
|
||||
|
||||
Before writing, tell the user concretely what's about to happen — name the actual files. Not "I'll write the three files" but something like:
|
||||
|
||||
> "Now I'll write the three files needed to publish a Space: **`app.py`** (the Gradio demo and inference code), **`requirements.txt`** (Python dependencies), and **`README.md`** (Space configuration including ZeroGPU hardware setting). Then I'll show all three for your review before publishing."
|
||||
|
||||
This anchors the user in what's being produced. Don't say "three files" without naming them — it's vague and signals lack of commitment to the deliverable.
|
||||
|
||||
The three files are tightly coupled: `requirements.txt` is determined by what `app.py` imports, and the `README.md` YAML frontmatter sets the SDK version, hardware, and Space title that have to match. Write them together, then show all three to the user for approval in **one batched message** before publishing.
|
||||
|
||||
Read `references/zerogpu-and-publishing.md` for the ZeroGPU rules. The non-obvious ones:
|
||||
|
||||
- Models go on `cuda` at module level (not lazy-loaded inside the GPU function). ZeroGPU has a CUDA emulation that makes this work pre-allocation, and module-level placement is significantly faster than deferred placement.
|
||||
- The function that runs inference is decorated with `@spaces.GPU(duration=...)`. Pick a duration appropriate for the task — short for image generation, longer for video.
|
||||
- Don't use `torch.compile` — it's incompatible with ZeroGPU's process model.
|
||||
|
||||
### `app.py`
|
||||
|
||||
Compose from the pieces decided in Phases 1–3. Don't paste from a template. Each section should be there because it's needed:
|
||||
|
||||
- Imports — `gradio as gr`, `torch`, `spaces`, the pipeline class, anything the preprocessing needs.
|
||||
- Constants — `LORA_REPO`, `BASE_MODEL`, recommended step count, guidance, LoRA scale, trigger word.
|
||||
- Module-level model load — pipeline `from_pretrained`, `.to("cuda")`, `load_lora_weights`. If the LoRA repo is private, pass `token=os.environ["HF_TOKEN"]`.
|
||||
- Preprocessing functions (if any) — pose extraction, padding, mask building, etc. CPU code can run at module level; GPU code needs to be inside a `@spaces.GPU` function.
|
||||
- The inference function — decorated with `@spaces.GPU(duration=...)`. Validates inputs, applies trigger word, builds the pipeline kwargs, returns outputs.
|
||||
- The Gradio Blocks — the UI from Phase 3, wired to the inference function.
|
||||
|
||||
Common things to get right:
|
||||
|
||||
- Return the actually-used seed alongside the result so the user can reproduce.
|
||||
- `gr.Progress(track_tqdm=True)` on the inference function surfaces diffusers' internal progress bar.
|
||||
- Validate inputs — raise `gr.Error("Please upload an image first.")` when a required input is missing, rather than letting the pipeline fail with a cryptic error.
|
||||
- On `gr.Examples`, use `cache_examples=True, cache_mode="lazy"` — plain `cache_examples=True` runs examples at build time and fails on ZeroGPU; lazy mode defers caching to the first user click.
|
||||
|
||||
### `requirements.txt`
|
||||
|
||||
Don't ship a fixed minimal list and hope for the best. The "minimal" list works for plain T2I LoRAs and breaks the moment the base model has a vision-language text encoder, video output, or any non-trivial preprocessing. **Derive `requirements.txt` from what the Space actually needs**, in this order:
|
||||
|
||||
1. **Every top-level non-stdlib import in `app.py`.** If `app.py` does `import cv2`, `requirements.txt` has `opencv-python`. If it does `from controlnet_aux import OpenposeDetector`, `requirements.txt` has `controlnet-aux`. Walk the imports mechanically. (Note the exclusions in the next paragraph — some imports are runtime built-ins and don't need to be listed.)
|
||||
2. **What the base-model reference's "Required dependencies" subsection says.** Each base-model file lists the non-obvious extras the pipeline pulls in — `torchvision` for Qwen-Image (Qwen 2.5-VL text encoder), `imageio[ffmpeg]` for LTX (video export), etc. Include all of them. These are the deps that aren't picked up from imports because the pipeline's components import them transitively at load time.
|
||||
3. **What the LoRA's own model card explicitly mentions installing.** If the LoRA README has its own `pip install` block, lift the deps from there.
|
||||
4. **The diffusers/ML stack:** `diffusers`, `transformers`, `accelerate`, `peft`, `safetensors`. Default to plain (unpinned). Switch `diffusers` to `git+https://github.com/huggingface/diffusers` if the base-model reference says the model needs it (recent releases often do — Qwen-Image-Edit-2511 is a current example).
|
||||
|
||||
**What *not* to list in `requirements.txt`:**
|
||||
|
||||
- **`gradio`** — controlled by the `sdk_version:` field in `README.md`'s YAML frontmatter, not by `requirements.txt`. Listing it in requirements is at best ignored, at worst causes a version conflict with the SDK. Set the version in the README only.
|
||||
- **`torch`** — provided by the Space runtime. Only add if you need a specific version pinned (rare, and usually a sign something else is wrong).
|
||||
- **`spaces`** — provided by the Space runtime. Only add if you need a specific version pinned.
|
||||
- **`huggingface_hub`** — provided by the Space runtime. Only add if you need a specific version pinned.
|
||||
|
||||
These four come pre-installed in the ZeroGPU container. Listing them anyway is the kind of "include rather than skip" instinct that's right for non-baseline deps but wrong for baseline ones, because pinning conflicts with the runtime's managed versions.
|
||||
|
||||
**Bias for everything else: include rather than skip when uncertain.** A package the Space doesn't actually use causes a slightly slower build. A missing required package causes a startup-time crash that's much harder for the user to diagnose. These costs aren't symmetric — the test failure that prompted this rule was exactly the second kind.
|
||||
|
||||
**But two specific deps are *not* safe to add reflexively** because they routinely cause more problems than they solve on ZeroGPU:
|
||||
|
||||
- `xformers` — pinned to specific torch versions, frequent source of conflicts. The ZeroGPU runtime ships torch 2.8+, so any pinned `xformers` version must support that. Additional gotcha on Blackwell: xformers' FA3 dispatch mis-gates the hardware (FA3 kernels are Hopper-only at `sm_90a`, but the dispatcher gates on `device_capability >= (9, 0)`, which also matches Blackwell) and crashes at kernel launch with `CUDA invalid argument`. If a Space using xformers attention hits this, disable FA3 dispatch at module load:
|
||||
|
||||
```python
|
||||
try:
|
||||
from xformers.ops.fmha import _set_use_fa3
|
||||
_set_use_fa3(False)
|
||||
except Exception:
|
||||
pass
|
||||
```
|
||||
|
||||
Only include `xformers` if `app.py` actually uses it.
|
||||
- `flash-attn` — needs a build step, often fails to install. Same torch 2.8+ alignment caveat as `xformers`. Only include if `app.py` actually uses it.
|
||||
|
||||
**Pin other versions only when you have a reason** (e.g. a known incompatibility, or matching a recipe from the model card).
|
||||
|
||||
### `README.md`
|
||||
|
||||
Spaces are configured by the YAML frontmatter at the top of `README.md`. This frontmatter is what selects ZeroGPU.
|
||||
|
||||
```
|
||||
---
|
||||
title: <human-readable title>
|
||||
emoji: 🎨
|
||||
colorFrom: pink
|
||||
colorTo: purple
|
||||
sdk: gradio
|
||||
sdk_version: <current Gradio version>
|
||||
app_file: app.py
|
||||
pinned: false
|
||||
hardware: zero-a10g
|
||||
short_description: <one short line for the Space tile, ~60 chars max>
|
||||
models:
|
||||
- <base model repo>
|
||||
- <lora repo>
|
||||
---
|
||||
|
||||
# <title>
|
||||
|
||||
A short description with links to the LoRA and base model.
|
||||
```
|
||||
|
||||
Key fields:
|
||||
|
||||
- `sdk: gradio` — required for ZeroGPU.
|
||||
- `sdk_version` — match the Gradio version you wrote against. Look up the current version (`pip index versions gradio`, or check https://www.gradio.app) rather than guessing.
|
||||
- `hardware: zero-a10g` — the legacy string for ZeroGPU. The actual hardware is NVIDIA RTX Pro 6000 Blackwell, but the identifier is `zero-a10g`. ZeroGPU is available to PRO, Team, and Enterprise accounts; if the user isn't subscribed, the Space will fall back to CPU. Mention this if you suspect they aren't on PRO.
|
||||
- `models:` — list base and LoRA repos. This enables Hub caching and discovery.
|
||||
- `short_description` — appears on the Space tile. **Keep it short (~60 characters or less).** The Hub's YAML validator rejects long values with a 400 from `https://huggingface.co/api/validate-yaml`, which surfaces as an `HfHubHTTPError` during `create_repo` or `upload_file`. The exact server-side limit isn't documented and may change, so target the visible-tile-length range rather than pushing right up to a cap. If you do hit the 400, the fix is almost always to shorten this field. One sentence describing what the Space does is plenty — the README body below the YAML is where you put longer prose.
|
||||
|
||||
### Single batched approval — order of operations matters
|
||||
|
||||
The discipline here is **write all three files first, then show them all together in one message**. Not "write app.py → talk about it → write requirements → talk about it → write README → talk about it." That rhythm produces three approval moments even if you don't explicitly ask for approval, because the user is being asked to react after each file.
|
||||
|
||||
Concretely:
|
||||
|
||||
1. **Write `app.py`, `requirements.txt`, and `README.md` in succession with no intervening prose.** No commentary between files. No "Now I'll write the next one." No description of what each file does as you produce it. Just the three files, back to back.
|
||||
2. **Then, in a single message, ask for approval covering all three at once.** Something like: "Here's the Space — `app.py` (N lines), `requirements.txt`, and `README.md`. Review and confirm to publish, or tell me what to change."
|
||||
3. The user responds once, covering whatever they want changed across any of the three files.
|
||||
|
||||
What to avoid:
|
||||
|
||||
- Walking through `app.py`'s structure or design choices after writing it but before writing the others. Save commentary for either the pre-writing announcement (Phase 4 opening) or the single approval message after all three exist.
|
||||
- Asking "ready for the next one?" or "want me to continue with requirements?" — those are implicit per-file approvals.
|
||||
- Showing one file inline and offering to "show the next when you're ready" — same trap.
|
||||
- Treating any of the three files as optional or as a follow-up. They are produced together as one deliverable.
|
||||
|
||||
If the user interrupts after seeing the first file with feedback or a question, that's fine — engage with it — but the rule still applies: the next time you produce code, produce all remaining files together, not one at a time.
|
||||
|
||||
---
|
||||
|
||||
## Phase 5 — Publish the Space
|
||||
|
||||
Use the authenticated session from Phase 1. Default to **private**, so the user can vet the Space before flipping it public. Confirm the target username with the user before creating: "I'll publish to `{username}/{space_name}` — confirm?"
|
||||
|
||||
```python
|
||||
from huggingface_hub import HfApi, SpaceHardware
|
||||
|
||||
api = HfApi(token=hf_token)
|
||||
username = api.whoami()["name"]
|
||||
repo_id = f"{username}/{space_name}"
|
||||
|
||||
api.create_repo(
|
||||
repo_id=repo_id,
|
||||
repo_type="space",
|
||||
space_sdk="gradio",
|
||||
space_hardware=SpaceHardware.ZERO_A10G,
|
||||
private=True,
|
||||
exist_ok=True,
|
||||
)
|
||||
|
||||
# Upload files
|
||||
for path in ["app.py", "requirements.txt", "README.md"]:
|
||||
api.upload_file(path_or_fileobj=path, path_in_repo=path,
|
||||
repo_id=repo_id, repo_type="space")
|
||||
```
|
||||
|
||||
If the LoRA repo itself is private/gated, the Space needs the token at runtime to download the LoRA. Set it as a Space secret:
|
||||
|
||||
```python
|
||||
api.add_space_secret(repo_id=repo_id, key="HF_TOKEN", value=HF_TOKEN)
|
||||
```
|
||||
|
||||
…and in `app.py`, load the LoRA with `token=os.environ["HF_TOKEN"]`.
|
||||
|
||||
**After upload**, run the smoke-test below before sharing — the build runs asynchronously and silent failures (wrong `weight_name`, missing dep, wrong pipeline class) only surface at first inference. **Once the smoke-test passes**, share the Space URL (`https://huggingface.co/spaces/{repo_id}`) and tell the user the Space is private — they'll need to be logged in to view it. Note that the build takes a few minutes; the logs are at `https://huggingface.co/spaces/{repo_id}/logs/container` if anything fails.
|
||||
|
||||
**Publish-time failures (before the build starts):**
|
||||
|
||||
- **`HfHubHTTPError: 400 Bad Request` from `https://huggingface.co/api/validate-yaml`** during `create_repo` or `upload_file`. The README YAML failed server-side validation. By far the most common cause is a `short_description` that's too long; sometimes a stray field or malformed value. Fix: shorten `short_description` to ~60 characters and retry. If shortening doesn't fix it, look for typos in field names or invalid values (e.g. unsupported colors in `colorFrom`/`colorTo`, an invalid `hardware` string).
|
||||
- **403 on `create_repo`** with `space_hardware="zero-a10g"`: user isn't on PRO/Team/Enterprise, so they can't request ZeroGPU at creation time. Fix: retry `create_repo` without `space_hardware`, leave `hardware: zero-a10g` in the README YAML — the Space gets created on CPU. The user can then either upgrade to PRO (auto-promotes to ZeroGPU) or apply for a [community GPU grant](https://huggingface.co/docs/hub/spaces-gpus#community-gpu-grants) (request via the Space's hardware settings).
|
||||
- **401/403 on `upload_file`**: token doesn't have write scope. Fix: ask the user for a write-scoped token.
|
||||
|
||||
**Common build failures (after the build starts):**
|
||||
|
||||
- LoRA `weight_name` mismatch in `load_lora_weights` → check the actual filename via `list_repo_files`.
|
||||
- Base model is gated and the token wasn't set as a Space secret.
|
||||
- ZeroGPU not allocated (user not on PRO) → Space falls back to CPU and is unusably slow.
|
||||
- Diffusers version doesn't recognize the pipeline class → pin to git diffusers in `requirements.txt`.
|
||||
- Missing dependency at module load → see `requirements.txt` derivation rules above; the most common case is a transitive dep like `torchvision` for Qwen-Image's text encoder.
|
||||
|
||||
If a build fails, offer to read the logs and propose a fix.
|
||||
|
||||
---
|
||||
|
||||
## Phase 6 — Smoke-test the Space
|
||||
|
||||
Before declaring the Space done and handing the URL to the user, exercise it once end-to-end. Several failure modes (wrong `weight_name`, wrong pipeline class, missing transitive dep, gated-base-model token issue) build cleanly and only surface at first inference. The `gradio` Python package ships a CLI that does exactly this — `gradio info` returns the endpoint signature, `gradio predict` runs an actual inference. Both ship with the `gradio` pip dependency the Space already needs, so they're available in any environment where this skill ran.
|
||||
|
||||
**Step 1 — Wait for the build.** `create_repo` returns immediately, but the container image is still building. Poll `HfApi().get_space_runtime(repo_id).stage` until it reaches `RUNNING`:
|
||||
|
||||
```python
|
||||
import time
|
||||
from huggingface_hub import HfApi
|
||||
api = HfApi(token=hf_token)
|
||||
while True:
|
||||
stage = api.get_space_runtime(repo_id).stage
|
||||
if stage == "RUNNING": break
|
||||
if stage in {"BUILD_ERROR", "RUNTIME_ERROR", "CONFIG_ERROR"}:
|
||||
raise RuntimeError(f"Build failed: {stage}. Logs: https://huggingface.co/spaces/{repo_id}/logs/container")
|
||||
time.sleep(15)
|
||||
```
|
||||
|
||||
If the build fails, fetch the container logs (`https://huggingface.co/spaces/{repo_id}/logs/container`), read the traceback, and propose a fix. Don't run `gradio info` against a Space that isn't running — it'll hang or 503.
|
||||
|
||||
**Step 2 — Verify the endpoint signature.** `gradio info {repo_id} --token {hf_token}` returns the exposed endpoints and their parameter types. Read the output and confirm: (a) the endpoint exists (default is `/predict`, but Blocks Spaces often have a custom name from the Python function name), (b) the parameters in order match what `app.py` declares, (c) file-typed params show `"type": "filepath"` as expected. If any of this is off, the user-facing UI may still appear correct but API calls will fail — fix and re-upload.
|
||||
|
||||
**Step 3 — Run one real inference.** Pick the lightest viable input — the simplest example from the LoRA card, or one of the `gr.Examples` entries. Pass `--token` for private Spaces. For file inputs, the payload uses `{"path": "...", "meta": {"_type": "gradio.FileData"}}`.
|
||||
|
||||
```bash
|
||||
# Text-to-image:
|
||||
gradio predict {repo_id} /predict '{"prompt": "...", "aspect_ratio": "1:1", ...}' --token $HF_TOKEN
|
||||
|
||||
# Image-to-image (file input):
|
||||
gradio predict {repo_id} /predict '{"input_image": {"path": "/tmp/sample.jpg", "meta": {"_type": "gradio.FileData"}}, "prompt": "..."}' --token $HF_TOKEN
|
||||
```
|
||||
|
||||
If you don't have a local sample image for I2I, lift one from the LoRA repo (`hf_hub_download(repo_id, filename="example.png")`) or the base model card.
|
||||
|
||||
**Caveat for creative-mode Spaces.** `gradio info` and `gradio predict` only exercise the Python endpoint — they tell you nothing about whether custom JS in a `gr.HTML` widget works. If the Space uses creative mode (see `references/creative-mode.md`), after the API smoke-test passes, **open the Space URL in a browser and verify the interaction once** before sharing. Server-side green plus broken JS is the most common failure mode for these.
|
||||
|
||||
**Step 4 — Interpret the result.**
|
||||
|
||||
- **Returns successfully and the output looks plausible** → done. Share the URL.
|
||||
- **HTTPError 503 / "Space is sleeping"** → the Space spun down between steps 1 and 3. Wake it (`api.restart_space(repo_id)`) and retry.
|
||||
- **Inference error mentioning `weight_name` / `safetensors`** → the LoRA filename in `app.py` doesn't match the actual file in the LoRA repo. Re-check `list_repo_files`, fix `weight_name=`, re-upload `app.py`.
|
||||
- **Inference error mentioning a missing pipeline class or attribute** → diffusers version too old. Switch `requirements.txt` to `git+https://github.com/huggingface/diffusers` and re-upload.
|
||||
- **`ImportError` at module load** → missing dep. Add it to `requirements.txt` and re-upload. The runtime logs (`/logs/run`) name the missing package.
|
||||
- **OOM** → reduce default resolution or step count, or pick a smaller base variant.
|
||||
- **Timeout / hangs** → bump `@spaces.GPU(duration=...)` and re-upload.
|
||||
|
||||
The smoke-test exists to convert these from "user discovers it and reports back" to "you discover it and fix it before sharing." Don't skip it because the build went green — green-build-broken-inference is the most common failure mode for Spaces with a non-trivial pipeline.
|
||||
|
||||
---
|
||||
|
||||
## What to avoid
|
||||
|
||||
- A generic "one demo for all LoRAs" template. The whole point of this skill is to tailor.
|
||||
- Lazy-loading the model inside the GPU function. Slow on ZeroGPU, and hides startup errors until first request.
|
||||
- `torch.compile`. Not supported on ZeroGPU.
|
||||
- `cache_examples=True` without `cache_mode="lazy"` on ZeroGPU.
|
||||
- Uploading the LoRA weights into the Space repo. Pull from the LoRA's own Hub repo at runtime.
|
||||
- Asking for the HF token only at the end, then discovering the LoRA was private all along and you couldn't read the model card.
|
||||
- Exposing every diffusers knob. Pick the 1–3 controls that matter for this LoRA.
|
||||
- Long preambles in the chat reply once the Space is published. The Space URL is the deliverable; keep the wrap-up brief.
|
||||
|
||||
## Limitations
|
||||
|
||||
- Use this skill only when the task clearly matches its upstream product or API scope.
|
||||
- Verify commands, API behavior, pricing, quotas, credentials, and deployment effects against current official documentation before making changes.
|
||||
- Do not treat generated examples as a substitute for environment-specific tests, security review, or user approval for destructive or costly actions.
|
||||
+113
@@ -0,0 +1,113 @@
|
||||
# Adapting the demo to the specific LoRA
|
||||
|
||||
The task category (T2I, I2I, T2V, I2V, V2V) gets you a starting shape. It does not get you a UI. Two LoRAs in the same task category can need very different demos. This file is about the reasoning that gets you from "I know the task" to "I know what this demo should look like."
|
||||
|
||||
## The core question
|
||||
|
||||
For each LoRA, ask: **what does this LoRA actually need from the user, and what's the most natural way for the user to provide it?**
|
||||
|
||||
These are different questions. The model might need a pose video as conditioning. That doesn't mean the user has to provide a pose video — they can provide a regular video and the demo extracts pose from it. The model might need a black-bordered video for outpainting. That doesn't mean the user uploads a pre-bordered video — they upload a normal video and pick an aspect ratio, and the demo adds the borders.
|
||||
|
||||
The job of the demo is to translate between what the user has (a video, a photo, an idea) and what the model wants (pose conditioning, padded frames, masked latents, a prompt prefixed with a trigger word).
|
||||
|
||||
## Reading what the LoRA needs
|
||||
|
||||
Sources, in order of usefulness:
|
||||
|
||||
1. **The model card's example code snippet — for *parameters*.** If the README has a Python block showing how to call the LoRA, trust it for inference parameters: pipeline class, step count, guidance scale, true CFG, LoRA scale, dtype, resolution, negative prompt. Trust it for inputs: if it passes `image=...`, the demo takes an image; if it passes `image=...` and `mask_image=...`, the demo needs both.
|
||||
|
||||
**For *loading mechanics*, treat the snippet as a signal, not a directive.** Prefer the standard diffusers path: `pipe.load_lora_weights(repo_id, weight_name=...)`. This is the maintained, well-tested path that handles DoRA, rsLoRA, custom target modules, and most format variants when diffusers + PEFT are recent enough. If the model card uses something else — `PeftModel.from_pretrained(pipe.transformer, ...)`, `diffsynth_engine`, custom imports, manual state-dict surgery — that's a flag to investigate, not to copy.
|
||||
|
||||
Reasons model card authors reach for non-standard loading paths often don't transfer: training-time conventions, environment quirks (older diffusers/PEFT versions, CPU offload patterns), or even malformed configs that pass silently in their setup but crash elsewhere. (Real example: an `adapter_config.json` with `task_type: "DIFFUSION"` works locally on some PEFT versions but crashes on current PEFT, because PEFT's `TaskType` enum only contains NLP tasks; diffusers' loader bypasses this validation by reading the safetensors directly.)
|
||||
|
||||
Custom inference paths absolutely *can* work on ZeroGPU when needed (LTX-2.3 native pipeline is a real example). But default to diffusers because it's the standard, current, maintained path. Only adopt the model card's loading approach if `load_lora_weights` demonstrably can't handle this LoRA — and when you do, port it to the ZeroGPU constraints (module-level `.to('cuda')`, no `enable_model_cpu_offload`).
|
||||
|
||||
2. **Trigger words and prompt patterns.** "Use the trigger word X at the start of the prompt" → automatically prepend `X` to the user's prompt rather than asking them to type it. "Prompts should describe the scene as Y" → add prompt formatting examples or a placeholder. If the LoRA expects a structured input embedded in the prompt (like bounding-box coordinates or named regions), the UI should produce that structure for the user, not require them to type it.
|
||||
|
||||
3. **Example media in the repo.** Example outputs tell you what the LoRA does. Example inputs tell you what the user has to provide. Example *paired* inputs+outputs (input video → output video, input image → output image) tell you the transformation. Lift these into `gr.Examples` so the user can click through them.
|
||||
|
||||
4. **The model's task family.** A pose-conditioned model wants pose maps. A depth-conditioned model wants depth. An outpaint model wants padded frames with masked regions. Each implies preprocessing.
|
||||
|
||||
5. **Recommended hyperparameters.** Step count, guidance scale, true CFG, LoRA scale. Bake the recommended values in as defaults. Expose a slider only if the LoRA's behavior is sensitive to the value across a range (e.g. LoRA scale 0.7–1.3 produces meaningfully different results); otherwise it's just clutter.
|
||||
|
||||
When the model card has none of these, you have three options:
|
||||
|
||||
- **Infer from precedent.** If similar LoRAs exist for the same base model, study their demos.
|
||||
- **Ask the user.** Once, batched, with concrete questions: "Does this LoRA have a trigger word? What's a recommended step count? Got an example prompt?"
|
||||
- **Use sensible defaults from the base model.** Worse than the previous two; only fall back to this when nothing else is available.
|
||||
|
||||
## Verifying the pipeline class
|
||||
|
||||
The pipeline class is decided in Phase 2, not here. The procedure (read the base model's own card, trust its diffusers snippet over reference tables) is in `SKILL.md` under "Phase 2 — Pick the base pipeline." Mentioning it here as a pointer because skipping that verification is one of the most common ways to ship a broken Space — when in doubt, re-read Phase 2.
|
||||
|
||||
## From "what the LoRA needs" to UI shape
|
||||
|
||||
For each thing the LoRA expects as input, decide:
|
||||
|
||||
- **Where does the user get this from?** A pose video — extract from a regular video, or accept a pre-extracted pose video, or both? An aspect ratio — picker, sliders for width/height, or auto from input? A reference image — separate slot, or embedded in a "drag and drop here" area?
|
||||
|
||||
- **Can a more natural input be transformed into this?** Almost always yes. A user has a video, not a pose video. A user wants "wider", not "padded with black bars at these specific coordinates". The demo bridges the gap.
|
||||
|
||||
- **Should the user see the intermediate?** Often yes — a pose extraction preview, a letterbox preview, a generated mask preview. Showing the intermediate builds trust ("yes, the model is conditioning on what I expected") and helps the user iterate. But a preview that takes 10 seconds to generate every time the user changes a setting is a worse UX than no preview.
|
||||
|
||||
- **What's the smallest set of controls that lets the user actually drive this LoRA?** Anything beyond that is clutter. A LoRA that's only good at one specific transformation might need just an input slot and a Generate button.
|
||||
|
||||
## Examples of the reasoning, applied
|
||||
|
||||
**Pose-control video LoRA (V2V).**
|
||||
The model conditions on pose video. The user has a regular video. The demo takes a video, extracts pose, optionally takes an appearance reference image (for "the character looks like *this*, doing *that* motion"), runs inference, returns a video. The pose extraction is shown as a preview so the user knows what's being used. Aspect-ratio picker is irrelevant — the output matches the source.
|
||||
|
||||
**Outpaint video LoRA (V2V).**
|
||||
The model fills black-margined frames. The user has a video and wants it wider/taller. The demo takes a video, takes a target aspect ratio (dropdown: 16:9, 9:16, 1:1, etc.), pads frames with black bars to that aspect ratio, shows a preview of the padded first frame, runs inference. No appearance reference — the LoRA's job is to extend, not transform. If the LoRA's model card mentions that gamma correction helps for dark scenes, expose that as an Advanced toggle.
|
||||
|
||||
**Relight image LoRA (I2I).**
|
||||
The model relights based on prompt. The user has a photo and an idea of what lighting they want. The demo takes an image, takes a brush canvas where the user paints colored strokes indicating where light should come from, takes an illumination style dropdown ("golden hour", "neon", "studio"), takes an optional background change, builds the prompt, runs inference. Brush color and position become structured prompt content.
|
||||
|
||||
**Style image LoRA (T2I).**
|
||||
The model produces images in a specific style. The user has a prompt. The demo takes a prompt (with the trigger word auto-prepended), takes an aspect ratio, runs inference. That's the whole UI. No reference image, no brush canvas, no preprocessing — the LoRA does the work.
|
||||
|
||||
**Bounding-box drag-drop LoRA (I2I).**
|
||||
The model moves and resizes objects between two bounding boxes drawn on the image. The user has an image and an intent ("move that vase from here to there"). The demo takes an image, lets the user draw two boxes (red for source, green for target) directly on the image with a custom canvas component, runs inference. The boxes become the structured input the model expects.
|
||||
|
||||
**Identity-preserving I2V.**
|
||||
The model animates a character while preserving identity. The user has a still photo and a motion intent. The demo takes an image (the character), takes a prompt (the motion), runs inference. If the model also accepts a driving video for motion, expose that as an alternative input mode rather than a second mandatory input.
|
||||
|
||||
The pattern in all of these: start from what the user *has* and what they *want*, and build the UI that bridges to what the model needs.
|
||||
|
||||
## Things that change UI shape
|
||||
|
||||
Signals from the model card or the LoRA's behavior that should change the UI:
|
||||
|
||||
- **Few-step inference (≤ 8 steps).** Hide the steps slider — at this regime the model is recipe-locked. CFG is often 1.0 too. Lock these defaults rather than exposing them.
|
||||
- **Recommended LoRA scale ≠ 1.0** or scale-sensitive behavior. Expose a LoRA-scale slider centered on the recommended value.
|
||||
- **Multiple reference inputs.** Two image slots, clearly labeled (e.g. "appearance" and "pose source") with help text explaining the role of each.
|
||||
- **Optional inputs.** Make optional clearly optional — placeholder text, "(optional)" in the label, the demo runs without it.
|
||||
- **Multi-stage pipelines** (e.g. extract + generate, generate + refine). Show stage progress (`progress(0.3, desc="Extracting pose...")` then `progress(0.6, desc="Generating...")`). Otherwise the user stares at a blank progress bar for a long generation.
|
||||
- **Output is video > 5s.** Bump `@spaces.GPU(duration=...)` higher and warn the user in the UI that generation takes longer.
|
||||
- **LoRA expects structured prompt content** (coordinates, region tags, named entities). Build a small UI that produces that structure rather than asking the user to type it raw.
|
||||
|
||||
## Things that don't change UI shape
|
||||
|
||||
- The base model identity, beyond determining the pipeline class. A Qwen-Image style LoRA and a Flux style LoRA can have identical UIs.
|
||||
- Pure performance details (dtype, device map, attention impl). These belong in the model load code, not the UI.
|
||||
- The LoRA's training data composition. Interesting, not load-bearing.
|
||||
|
||||
## When LoRA loading fails
|
||||
|
||||
The right move when `pipe.load_lora_weights(...)` fails is to read the error and recognize the *category* of failure. Each category has a different fix path. Don't guess — different errors imply different things about whether the LoRA is salvageable on the diffusers path at all.
|
||||
|
||||
**Don't preemptively call conversion utilities.** `load_lora_weights` already calls the appropriate converters internally for the formats it knows about. Calling `convert_state_dict_to_diffusers` or similar before there's an error is redundant in the common case and risky if you guess wrong — you can mangle a state dict that would have loaded fine.
|
||||
|
||||
Failure categories:
|
||||
|
||||
- **Config-validation failure** (errors mentioning `task_type`, `peft_type`, "Invalid task type", `PeftConfig`, or anything from `peft/config.py`): the safetensors weights themselves may be fine; `adapter_config.json` is the problem. Fix paths: pass `weight_name=` explicitly so `load_lora_weights` reads the safetensors directly without going through PEFT's strict config parser; or download the safetensors and load via state dict. Falling back to `PeftModel.from_pretrained` will *not* help — that path crashes on the same config.
|
||||
|
||||
- **Missing keys / unexpected keys** ("Loading adapter weights from state_dict led to missing keys" or "led to unexpected keys"): the state dict's key naming doesn't match what diffusers' loader expects. This often means the LoRA was trained with a non-diffusers convention (kohya, ComfyUI, OneTrainer, custom training scripts), or with custom target modules. Fix paths: try diffusers' built-in conversion utilities (`convert_state_dict_to_diffusers`, base-model-specific converters in `diffusers.loaders`); if those don't help, the format may not be supported yet — surface this clearly to the user rather than silently using a partial load.
|
||||
|
||||
- **Shape mismatch errors**: the LoRA's tensor shapes don't match the base model's. Usually means the LoRA was trained against a different base model variant than the one being used (e.g. trained on Qwen-Image-Edit but loaded against Qwen-Image, or trained on FLUX.1-dev but loaded against FLUX.1-schnell). Fix: check the model card's `base_model` field carefully and switch to the correct base.
|
||||
|
||||
- **OOM during load or first inference**: not really a loading failure — the LoRA loaded but the combined base + LoRA + activations don't fit. Fix paths involve `pipe.enable_vae_tiling()`, smaller resolutions, FP8/quantized base variants. Not in scope for this section.
|
||||
|
||||
- **Missing keys for *some* weights only** (e.g. text encoder LoRA missing but transformer LoRA present): often a partial-coverage LoRA that only targets one component. May actually be intentional and may still work — generate a test image and see if the LoRA effect is present.
|
||||
|
||||
When none of these fit and `load_lora_weights` simply doesn't work, falling back to a non-diffusers path becomes a real option. At that point the model card's snippet becomes more useful — but port it to ZeroGPU constraints (no `enable_model_cpu_offload`, module-level `.to('cuda')`, models on `cuda` outside `@spaces.GPU`).
|
||||
+63
@@ -0,0 +1,63 @@
|
||||
# Krea 2 reference
|
||||
|
||||
Krea 2 (K2) is a flow-matching **text-to-image** model: a 12B dense DiT (grouped-query attention) with a **Qwen3-VL** text encoder (multi-layer feature aggregation) and the **Qwen-Image VAE** (`AutoencoderKLQwenImage`). Pipeline class: `Krea2Pipeline`. It ships as two checkpoints designed to work together:
|
||||
|
||||
| Repo | Role | Use it for |
|
||||
|------|------|------------|
|
||||
| [`krea/Krea-2-Turbo`](https://huggingface.co/krea/Krea-2-Turbo) | 8-step **distilled** | **Inference / demos** |
|
||||
| [`krea/Krea-2-Raw`](https://huggingface.co/krea/Krea-2-Raw) | base, non-distilled | **LoRA training** — *not* for inference |
|
||||
|
||||
**For a LoRA Space, load Turbo.** Krea 2 LoRAs are *trained on RAW but run on Turbo* (they "express strongly on Turbo"), and RAW is explicitly not meant for inference — expect poor quality if you load it directly. So a demo should almost always use `krea/Krea-2-Turbo`. The official LoRA cards confirm this ("To be used on `krea/Krea-2-Turbo`").
|
||||
|
||||
> Needs a `diffusers` build that includes `Krea2Pipeline` (both repos are `library_name: diffusers`). If `from diffusers import Krea2Pipeline` fails, the installed `diffusers` predates the integration — update it.
|
||||
|
||||
## Required dependencies
|
||||
|
||||
- `diffusers` with `Krea2Pipeline`.
|
||||
- `transformers` recent enough for **Qwen3-VL** (the text encoder is a `Qwen3VLModel`, e.g. `Qwen/Qwen3-VL-4B-Instruct`).
|
||||
- **`torchvision`** — the Qwen3-VL processor pulls it in transitively; missing it is a startup `ImportError`. Always include it.
|
||||
- `sentencepiece` if you see tokenizer-related ImportErrors at startup.
|
||||
|
||||
## Default load + LoRA (Turbo)
|
||||
|
||||
```python
|
||||
import torch
|
||||
from diffusers import Krea2Pipeline
|
||||
|
||||
pipe = Krea2Pipeline.from_pretrained("krea/Krea-2-Turbo", torch_dtype=torch.bfloat16).to("cuda")
|
||||
pipe.transformer.load_lora_adapter("user/my-krea2-lora", weight_name="my_lora.safetensors")
|
||||
pipe.transformer.set_adapters("default", weights=1.0)
|
||||
|
||||
# include the LoRA's trigger word(s) from its card
|
||||
image = pipe(
|
||||
"a deer grazing in a forest, <trigger words>",
|
||||
num_inference_steps=8, guidance_scale=0.0,
|
||||
generator=torch.Generator("cuda").manual_seed(0),
|
||||
).images[0]
|
||||
```
|
||||
|
||||
Krea 2 LoRAs load through the **transformer's** adapter API (`pipe.transformer.load_lora_adapter(...)` + `pipe.transformer.set_adapters("default", weights=1.0)`), per the official LoRA cards — not the pipeline-level `pipe.load_lora_weights`. Honor the LoRA's **trigger word(s)** and recommended weight (1.0 by default).
|
||||
|
||||
## Inference recipe
|
||||
|
||||
- **Turbo (the demo default):** `num_inference_steps=8`, **`guidance_scale=0.0`** (guidance disabled), LoRA weight `1.0`.
|
||||
- **RAW:** training only — don't ship a RAW inference demo; its quality is intentionally low (it's the malleable base you fine-tune on, then run on Turbo).
|
||||
|
||||
## `guidance_scale` convention (gotcha)
|
||||
|
||||
Krea 2 enables guidance whenever **`guidance_scale > 0`** and computes velocity as `cond + guidance_scale * (cond − uncond)` (≡ the usual CFG formulation with scale `1 + guidance_scale`). So Turbo disables guidance with **`guidance_scale=0.0`** — not `1.0`.
|
||||
|
||||
## Resolution
|
||||
|
||||
`height`/`width` must be divisible by **16** (`vae_scale_factor * patch_size`); the pipeline rounds up to a multiple of 16 (with a warning) otherwise. Default 1024×1024.
|
||||
|
||||
## ZeroGPU duration
|
||||
|
||||
Standard T2I: place modules at module scope, `pipe.to("cuda")`, no `torch.compile`. Turbo 8-step 1024² is fast (≈ 20–40s) — set `@spaces.GPU(duration=...)` comfortably above that. The 12B DiT + Qwen3-VL encoder fits ZeroGPU; if you OOM on the default size, try `@spaces.GPU(size="xlarge")`.
|
||||
|
||||
## Things to watch
|
||||
|
||||
- **Load Turbo, not RAW, for demos.** RAW is the training base and not meant for inference.
|
||||
- **`guidance_scale=0` disables guidance** (Krea convention, unlike pipelines where 1.0 is "off"). Turbo = 8 steps, `guidance_scale=0.0`.
|
||||
- **LoRAs use `pipe.transformer.load_lora_adapter` + `set_adapters("default", …)`** (transformer-level), and have **trigger words** — read the card.
|
||||
- **New pipeline → update diffusers** if `from diffusers import Krea2Pipeline` fails.
|
||||
+205
@@ -0,0 +1,205 @@
|
||||
# LTX reference
|
||||
|
||||
The LTX family covers `LTX-Video` (0.9.x series), `LTX-2`, and `LTX-2.3`. All target text-to-video, image-to-video, and video-to-video.
|
||||
|
||||
Diffusers support varies across versions:
|
||||
|
||||
| Base model series | Diffusers support | LoRA loading via diffusers? |
|
||||
|---------------------------|------------------------------------------|-----------------------------|
|
||||
| `Lightricks/LTX-Video` (0.9.x) | Yes — `LTXPipeline`, `LTXImageToVideoPipeline`, `LTXConditionPipeline` | Yes |
|
||||
| `Lightricks/LTX-2` | Yes — `LTX2Pipeline` from `diffusers.pipelines.ltx2` | Yes (recent diffusers) |
|
||||
| `Lightricks/LTX-2.3` | Partial — diffusers support is still limited overall, so the original `Lightricks/LTX-2.3` repo can still be used via the native path. Diffusers-converted variants exist at `dg845/LTX-2.3-Diffusers` and `dg845/LTX-2.3-Distilled-Diffusers`, supporting regular LoRAs via the standard pipelines, IC LoRAs via `LTX2InContextPipeline`, and HDR IC-LoRAs via `LTX2HDRPipeline` (PR #13572, merged 2026-05-15; needs `git+https://github.com/huggingface/diffusers`) | Yes on the diffusers variants — standard `load_lora_weights` + `set_adapters`. Native path still required for some configurations. |
|
||||
|
||||
When in doubt about LTX-2.3, check the LoRA's model card for an example snippet. If it imports from `ltx_video` or a native module rather than `diffusers`, use the native path.
|
||||
|
||||
## Pipelines (diffusers path)
|
||||
|
||||
> **Before using this table, verify against the base model's own card on the Hub.** This table is best-effort and can lag a recent release (LTX moves fast). The diffusers snippet on the base model's Hub page is source of truth. See `SKILL.md` Phase 2 for the procedure.
|
||||
|
||||
| Task | Pipeline | Pretrained ID |
|
||||
|-------------------------|-----------------------------------|------------------------------------------------------|
|
||||
| Text-to-video | `LTXPipeline` | `Lightricks/LTX-Video` |
|
||||
| Image-to-video | `LTXImageToVideoPipeline` | `Lightricks/LTX-Video` |
|
||||
| Video-to-video / keyframes | `LTXConditionPipeline` | `Lightricks/LTX-Video-0.9.5` or later |
|
||||
| Spatial upscale | `LTXLatentUpsamplePipeline` | `Lightricks/ltxv-spatial-upscaler-0.9.8` |
|
||||
| LTX-2 T2V/I2V | `LTX2Pipeline` | `Lightricks/LTX-2` |
|
||||
| LTX-2.3 in-context (IC LoRAs) | `LTX2InContextPipeline` | `dg845/LTX-2.3-Distilled-Diffusers` (preferred for demos; non-distilled `dg845/LTX-2.3-Diffusers` also works) |
|
||||
|
||||
`LTXConditionPipeline` is the workhorse for V2V — it takes a `LTXVideoCondition` (or list of them) plus optional first-frame image, and produces a video conditioned on the input.
|
||||
|
||||
## Required dependencies
|
||||
|
||||
LTX pipelines need extras beyond the standard diffusers/transformers/peft set, because video output requires file-format support:
|
||||
|
||||
- **`imageio`** and **`imageio-ffmpeg`** — required by `diffusers.utils.export_to_video`. Without them, video export fails at runtime even though model loading succeeds. Always include both.
|
||||
- **`sentencepiece`** — required by the T5 text encoder some LTX variants use. Include if you see tokenizer-related ImportErrors at startup.
|
||||
- **`av`** — `pyav`, useful when reading input videos in non-trivial formats. Include for V2V pipelines that take video input via `load_video`.
|
||||
|
||||
For LTX-2 and LTX-2.3, the latest diffusers from git is often required since pipeline classes (`LTX2Pipeline`, conditioning APIs) land before pip releases:
|
||||
|
||||
```
|
||||
git+https://github.com/huggingface/diffusers
|
||||
```
|
||||
|
||||
For LTX-2.3 native path:
|
||||
|
||||
```
|
||||
git+https://github.com/Lightricks/LTX-Video.git
|
||||
```
|
||||
|
||||
If `from_pretrained` fails with class-not-found errors for a recent LTX variant, switch to git diffusers.
|
||||
|
||||
## Default load (T2V)
|
||||
|
||||
```python
|
||||
import torch
|
||||
from diffusers import LTXPipeline
|
||||
from diffusers.utils import export_to_video
|
||||
|
||||
pipe = LTXPipeline.from_pretrained(
|
||||
"Lightricks/LTX-Video",
|
||||
torch_dtype=torch.bfloat16,
|
||||
)
|
||||
pipe.to("cuda")
|
||||
pipe.load_lora_weights("user/my-ltx-lora")
|
||||
```
|
||||
|
||||
## Default load (V2V via LTXConditionPipeline)
|
||||
|
||||
```python
|
||||
import torch
|
||||
from diffusers import LTXConditionPipeline
|
||||
from diffusers.pipelines.ltx.pipeline_ltx_condition import LTXVideoCondition
|
||||
from diffusers.utils import load_video, export_to_video
|
||||
|
||||
pipe = LTXConditionPipeline.from_pretrained(
|
||||
"Lightricks/LTX-Video-0.9.5",
|
||||
torch_dtype=torch.bfloat16,
|
||||
)
|
||||
pipe.to("cuda")
|
||||
pipe.load_lora_weights("user/my-ltx-vlora")
|
||||
|
||||
video = load_video("input.mp4")
|
||||
condition = LTXVideoCondition(video=video, frame_index=0)
|
||||
|
||||
frames = pipe(
|
||||
conditions=[condition],
|
||||
prompt="...",
|
||||
negative_prompt="worst quality, jittery, blurry",
|
||||
width=768, height=512,
|
||||
num_frames=121,
|
||||
num_inference_steps=50,
|
||||
).frames[0]
|
||||
export_to_video(frames, "out.mp4", fps=24)
|
||||
```
|
||||
|
||||
## LTX-2 (diffusers path)
|
||||
|
||||
```python
|
||||
from diffusers.pipelines.ltx2 import LTX2Pipeline
|
||||
pipe = LTX2Pipeline.from_pretrained("Lightricks/LTX-2", torch_dtype=torch.bfloat16)
|
||||
```
|
||||
|
||||
LTX-2 supports a two-stage pipeline (base + latent upsample) for production quality. For a demo, the single-stage pipeline is usually sufficient and faster.
|
||||
|
||||
**Don't default two-stage on for IC-LoRAs — check the card.** The common LTX-2.3 two-stage recipe runs stage 2 as an x2 latent-upsample + refine **with the IC-LoRA disabled** (`disable_lora()`), re-rendering on the bare base — which re-degrades exactly the LoRA-conditioned detail (text, identity). Whether two-stage is appropriate is per-LoRA, per the card: e.g. the *In/Out-painting* card explicitly says "both tasks use a two-stage pipeline," whereas the *Ingredients* card recommends a single-pass 30-step recipe and never mentions two stages. Single-stage is the safer demo default unless the card calls for two-stage.
|
||||
|
||||
## Inference defaults
|
||||
|
||||
- **Resolution**: typically multiples of 32. Common sizes: 768×512, 1216×704, 704×480.
|
||||
- **Frame count**: typically `8k+1` (e.g. 121, 161, 257). Compute from `duration_seconds * fps` and round.
|
||||
- **fps**: 24 by default; some LoRAs are trained at different rates (16, 30) — check the model card.
|
||||
- **`num_inference_steps`**: 30–50 for non-distilled. Distilled checkpoints (look for "distilled" in the name) often run at 8–12.
|
||||
- **`negative_prompt`**: LTX is sensitive to negative prompts. A good default: `"worst quality, inconsistent motion, blurry, jittery, distorted"`.
|
||||
- **Distilled IC-LoRA — disable audio guidance too, not just `guidance_scale`.** `LTX2InContextPipeline` computes `do_classifier_free_guidance = guidance_scale > 1 OR audio_guidance_scale > 1`, and **`audio_guidance_scale` defaults to `7.0`**. So `guidance_scale=1.0` is not enough — CFG stays on via audio (and `stg_scale` defaults on), which doubles/mis-batches the forward against the in-context reference tokens (wrong recipe, sometimes a runtime error). For a distilled IC-LoRA pass all four: `pipe(..., guidance_scale=1.0, stg_scale=0.0, audio_guidance_scale=1.0, audio_stg_scale=0.0)`.
|
||||
|
||||
## Frame-count helper
|
||||
|
||||
```python
|
||||
def num_frames_for_duration(seconds, fps=24, base=8):
|
||||
raw = seconds * fps
|
||||
return ((int(raw) - 1) // base) * base + 1
|
||||
```
|
||||
|
||||
## Native pipeline path (LTX-2.3 fallback)
|
||||
|
||||
The native repo is a fallback when the diffusers path doesn't work, or for specific conditionings that diffusers may not support yet. Try `LTX2InContextPipeline` on the diffusers path first (see the IC-LoRAs section above).
|
||||
|
||||
**The native repo depends on the model generation** — pick by the LoRA's base model (and what its card's snippet imports):
|
||||
|
||||
| Base model | Native repo | Package(s) / import | Pipeline classes |
|
||||
|---|---|---|---|
|
||||
| `LTX-Video` (0.9.x) | `github.com/Lightricks/LTX-Video` | `ltx_video` | `from ltx_video.pipelines import LTXPipeline` |
|
||||
| `LTX-2`, `LTX-2.3` | `github.com/Lightricks/LTX-2` | `ltx-core` + `ltx-pipelines` (editable) | `ltx_pipelines.ic_lora.ICLoraPipeline`, `TI2VidOneStagePipeline`, … |
|
||||
|
||||
For the 0.9.x series the pattern looks like:
|
||||
|
||||
```bash
|
||||
# in requirements.txt
|
||||
git+https://github.com/Lightricks/LTX-Video.git
|
||||
```
|
||||
|
||||
```python
|
||||
from ltx_video.pipelines import LTXPipeline as NativeLTXPipeline
|
||||
# ... model loading per the native repo's README
|
||||
```
|
||||
|
||||
The native path doesn't use `load_lora_weights`. Instead, LoRAs are usually wired in at pipeline construction time, often via a list of LoRA configurations or by pointing at a fused checkpoint.
|
||||
|
||||
When the LoRA's model card has a Python snippet using the native repo, copy its construction pattern verbatim. The native API changes more often than diffusers' does, so don't paraphrase.
|
||||
|
||||
### LTX-2.x native path on ZeroGPU — gotchas
|
||||
|
||||
For `LTX-2` / `LTX-2.3` (the `ltx-core` + `ltx-pipelines` repo), clone + `pip install -e packages/ltx-core packages/ltx-pipelines` at runtime in `app.py` and pin a commit. Three things bite on ZeroGPU:
|
||||
|
||||
- **Native loader bypasses ZeroGPU virtualization → "No CUDA GPUs available" at startup.** The native safetensors loader does `safe_open(path, device="cuda")` and copies host→device inside safetensors' own C++ (`cudaMemcpy`), **bypassing `torch.Tensor.to`** — the call ZeroGPU patches to virtualize + pack module-scope weights. Nothing packs and module-scope placement raises *"No CUDA GPUs are available."* Fix: monkeypatch the loader to open on CPU then move with `.to`:
|
||||
```python
|
||||
with safetensors.safe_open(shard, framework="pt", device="cpu") as f:
|
||||
value = f.get_tensor(name).to(device=device) # torch path → ZeroGPU-virtualisable
|
||||
```
|
||||
Also: patch attention to **SDPA** (FA3 crashes on Blackwell ZeroGPU), and **never call `torch.cuda.*` / `get_device_capability()` at module scope** (it poisons virtualization).
|
||||
- **Native two-stage pins two 22B transformers → offload-disk overflow.** `ICLoraPipeline` builds two independent `ModelLedger`s (stage-1 with the LoRA, stage-2 without), each loading its **own** full transformer. The CLI loads them sequentially, but ZeroGPU pins all weights at module scope, so enabling stage 2 pins **both** (~143G) and overflows the offload disk (`OSError: [Errno 28] No space left on device`, ~96G cap). For a demo: pin stage 1, then have stage 2 **reuse stage 1's pinned modules** (`for n in [...]: setattr(s2, n, getattr(s1, n))`) so only one transformer is resident (~71G).
|
||||
- **`ICLoraPipeline` is distilled-only.** Per the LTX-2 `ltx-pipelines` guide, IC-LoRA inference runs in distilled-only mode (fixed 8-step sigmas, no `num_inference_steps`/`guidance_scale`/`negative_prompt`); the docs do not describe non-distilled IC-LoRA or IC-LoRA + CFG/STG (guidance lives only on full-model pipelines like `TI2VidOneStagePipeline`, and in two-stage is stage-1-only). So if a card recommends a *non-distilled* recipe (e.g. 30 steps + guidance + STG on a `…-dev` base), there is no stock IC-LoRA pipeline for it — use the supported distilled `ICLoraPipeline` and note the recipe difference, or fall back to the diffusers `LTX2InContextPipeline` (which exposes steps/guidance).
|
||||
|
||||
## IC-LoRAs (in-context conditioning)
|
||||
|
||||
LTX IC-LoRAs condition the model on a reference video alongside a first-frame image. The diffusers path uses `LTX2InContextPipeline` with standard `load_lora_weights` + `set_adapters`.
|
||||
|
||||
Common flavors:
|
||||
|
||||
- **Pose / depth / canny IC-LoRAs**: the reference video must be preprocessed into the control signal (pose skeletons, depth maps, edge maps) before being passed as conditioning. Passing a raw video with appearance information leaks color/style through.
|
||||
- **Outpaint IC-LoRAs**: the input video is padded with black margins to a target aspect ratio and passed as conditioning. The model fills the black regions.
|
||||
- **Frame interpolation / extension IC-LoRAs**: the input is a sparse set of keyframes; the model fills in between or extends.
|
||||
- **Audio-driven lip-sync IC-LoRAs**: take a reference video and a new audio track; the model re-syncs lip motion to match the new audio. UI needs both a video input and an audio input (e.g. for video translation, voiceover replacement, multilingual dubbing).
|
||||
|
||||
Each of these implies different preprocessing in the demo and different UI shape (an aspect-ratio picker for outpaint, a pose preview for pose-control, a frame-pair input for keyframe extension). Read `references/adapting-to-the-lora.md` and shape the UI to the specific IC-LoRA.
|
||||
|
||||
## Quantization gotcha (LTX-2.3, native path only)
|
||||
|
||||
On the native path, some LTX-2.3 IC-LoRAs ship with FP8 quantization policies that fuse LoRA deltas into transformer weights at model-load time, using a Triton CUDA kernel. ZeroGPU has no real CUDA at module-load time (only the emulation layer) — this can crash the Space at startup. The diffusers path via `LTX2InContextPipeline` does not hit this: LoRAs are loaded through PEFT as separate adapter weights and applied at runtime rather than fused into the base transformer at load.
|
||||
|
||||
Two workarounds when the user hits this on the native path:
|
||||
|
||||
- **Skip quantization for the LoRA-fusion stage.** LTX-2.3 has a two-stage pipeline; apply quantization only to the second (non-LoRA) stage. Set `stage_1_quantization=None`.
|
||||
- **Pre-fuse the LoRA into a standalone checkpoint.** Download base + LoRA, fuse on a dev GPU, push the fused checkpoint to the Hub under the user's namespace, and have the Space load the fused checkpoint with no LoRA.
|
||||
|
||||
The second option is preferable when the user plans to demo the same LoRA repeatedly — Space startup is much faster.
|
||||
|
||||
## ZeroGPU duration guidance for LTX
|
||||
|
||||
- Short T2V (3 seconds, 24fps, distilled): 60–90 seconds.
|
||||
- Standard T2V (5 seconds, 50 steps): 120–180 seconds.
|
||||
- I2V: similar to T2V at the same duration.
|
||||
- V2V with preprocessing (pose extraction, etc.): add 10–30 seconds for preprocessing overhead. Set duration to 180+.
|
||||
- LTX-2.3 two-stage: 240–360 seconds.
|
||||
|
||||
Set `@spaces.GPU(duration=...)` to comfortably exceed expected generation time.
|
||||
|
||||
## Things to watch
|
||||
|
||||
- **Latest pipelines often need git diffusers.** Pin to `git+https://github.com/huggingface/diffusers` in `requirements.txt` for any LTX variant released in the last few weeks.
|
||||
- **`git+diffusers` is a moving target — verify LTX2 output quality.** Bare `@main` inherits whatever's there. The LTX2 text connector carried a token-reversal regression (PR #13564) that scrambled prompt tokens/registers — degrading prompt adherence and fine detail (e.g. garbled on-screen text), worst on short prompts — until **PR #13931 (merged 2026-06-19)** fixed it. If LTX2 output looks weak or garbled, confirm the installed diffusers commit is **at or past #13931**, and prefer pinning to a known-good commit (`git+https://github.com/huggingface/diffusers@<sha>`) over bare `@main` for reproducibility.
|
||||
- **Don't `torch.compile`.** LTX is fast enough on ZeroGPU without it; compile is incompatible anyway.
|
||||
- **`enable_vae_tiling()` for higher resolutions.** LTX VAE memory grows with resolution; enable tiling for outputs above 768px on either axis.
|
||||
- **Negative prompts matter more than for image models.** Don't ship with an empty negative_prompt unless the LoRA's model card says so.
|
||||
- **Frame-rate mismatch can produce glitches.** If the LoRA was trained at 24 fps and the demo passes 30, motion looks wrong. Use the LoRA's recommended fps.
|
||||
+144
@@ -0,0 +1,144 @@
|
||||
# Qwen-Image and Qwen-Image-Edit reference
|
||||
|
||||
The Qwen-Image family is fully supported in `diffusers`. Both base and edit variants accept LoRAs via the standard `load_lora_weights` interface.
|
||||
|
||||
## Pipelines
|
||||
|
||||
> **Before using this table, verify against the base model's own card on the Hub.** This table is best-effort and can lag a recent release. The diffusers snippet on the base model's Hub page is source of truth for which pipeline class to import. See `SKILL.md` Phase 2 for the procedure.
|
||||
|
||||
| Base model | Pipeline class | Task |
|
||||
|---------------------------------------|-------------------------------|-------------------------------------|
|
||||
| `Qwen/Qwen-Image` | `QwenImagePipeline` | Text-to-image |
|
||||
| `Qwen/Qwen-Image-Edit` | `QwenImageEditPipeline` | Image editing (instruction-driven) |
|
||||
| `Qwen/Qwen-Image-Edit-2509` | `QwenImageEditPlusPipeline` | Image editing, multi-image input |
|
||||
| `Qwen/Qwen-Image-Edit-2511` | `QwenImageEditPlusPipeline` | Image editing, latest variant |
|
||||
|
||||
The 2509 and 2511 variants use a *different* pipeline class than the original `QwenImageEditPipeline` — they take a list of input images and have different default parameters. Don't assume that variants in the same family share a pipeline class. Loading a 2511-trained LoRA onto `QwenImageEditPipeline` produces broken output; the failure is silent (no exception), so verifying against the base model card is the only way to catch it.
|
||||
|
||||
The 2511 variant integrates several popular community LoRAs into the base, which can mean a LoRA trained against earlier Qwen-Image-Edit may behave subtly differently when loaded against 2511; if the LoRA's model card specifies which Edit variant it was trained on, match it.
|
||||
|
||||
## Required dependencies
|
||||
|
||||
Qwen-Image and Qwen-Image-Edit pipelines need extras beyond the standard diffusers/transformers/peft set, because the text encoder is `Qwen2_5_VLForConditionalGeneration` (Qwen 2.5-VL):
|
||||
|
||||
- **`torchvision`** — required by `Qwen2VLVideoProcessor`, which the text encoder's processor pulls in transitively. Missing this is a startup-time `ImportError` ("Qwen2VLVideoProcessor requires the Torchvision library"). Always include in `requirements.txt` for any Qwen-Image Space.
|
||||
- **`sentencepiece`** — required by some Qwen tokenizer paths. Include if you see tokenizer-related ImportErrors at startup.
|
||||
|
||||
The 2511 variant in particular often requires the latest `diffusers` from git, since `QwenImageEditPlusPipeline` and 2511-specific fixes land before pip releases:
|
||||
|
||||
```
|
||||
git+https://github.com/huggingface/diffusers
|
||||
```
|
||||
|
||||
If `from_pretrained("Qwen/Qwen-Image-Edit-2511", ...)` fails with a class-not-found or attribute error, switch the requirement to git.
|
||||
|
||||
## Default load (T2I)
|
||||
|
||||
```python
|
||||
import torch
|
||||
from diffusers import QwenImagePipeline
|
||||
|
||||
pipe = QwenImagePipeline.from_pretrained(
|
||||
"Qwen/Qwen-Image",
|
||||
torch_dtype=torch.bfloat16,
|
||||
)
|
||||
pipe.to("cuda")
|
||||
pipe.load_lora_weights("user/my-qwen-lora")
|
||||
```
|
||||
|
||||
`pytorch_lora_weights.safetensors` is the conventional filename. If the repo has a different name, pass `weight_name="..."`.
|
||||
|
||||
For multiple adapters or when you want to control LoRA scale at inference time, use `set_adapters`:
|
||||
|
||||
```python
|
||||
pipe.load_lora_weights("user/my-qwen-lora", adapter_name="mylora")
|
||||
pipe.set_adapters(["mylora"], adapter_weights=[0.9])
|
||||
```
|
||||
|
||||
## Default load (Image Edit)
|
||||
|
||||
For original `Qwen-Image-Edit`:
|
||||
|
||||
```python
|
||||
import torch
|
||||
from diffusers import QwenImageEditPipeline
|
||||
|
||||
pipe = QwenImageEditPipeline.from_pretrained(
|
||||
"Qwen/Qwen-Image-Edit",
|
||||
torch_dtype=torch.bfloat16,
|
||||
)
|
||||
pipe.to("cuda")
|
||||
pipe.load_lora_weights("user/my-qwen-edit-lora")
|
||||
```
|
||||
|
||||
For `Qwen-Image-Edit-2509` and `Qwen-Image-Edit-2511`:
|
||||
|
||||
```python
|
||||
import torch
|
||||
from diffusers import QwenImageEditPlusPipeline
|
||||
|
||||
pipe = QwenImageEditPlusPipeline.from_pretrained(
|
||||
"Qwen/Qwen-Image-Edit-2511", # or 2509
|
||||
torch_dtype=torch.bfloat16,
|
||||
)
|
||||
pipe.to("cuda")
|
||||
pipe.load_lora_weights("user/my-qwen-edit-lora")
|
||||
```
|
||||
|
||||
`QwenImageEditPlusPipeline` accepts `image=<PIL>` or `image=[<PIL>, <PIL>, ...]` for multi-image edits. `QwenImageEditPipeline` accepts a single image. Default parameters differ slightly between the two — see "Inference defaults" below.
|
||||
|
||||
## Inference defaults
|
||||
|
||||
For non-distilled Qwen-Image:
|
||||
- `num_inference_steps`: 50 by default; the LoRA's model card may recommend lower.
|
||||
- `true_cfg_scale`: typical 4.0.
|
||||
- `width`/`height`: multiples of 16, ideally 1024 or 1328 along the long axis.
|
||||
|
||||
For Qwen-Image-Edit (original):
|
||||
- `num_inference_steps`: 30–50 typical, often less for distilled variants.
|
||||
- `true_cfg_scale`: 4.0 typical.
|
||||
- Input image gets internally resized; passing a reasonable resolution (1024px on the long side) is fine.
|
||||
|
||||
For Qwen-Image-Edit-2509 / 2511 (`QwenImageEditPlusPipeline`):
|
||||
- `num_inference_steps`: 40 typical for 2511; 50 for 2509.
|
||||
- `true_cfg_scale`: 4.0.
|
||||
- `guidance_scale`: 1.0 (the new pipeline uses `true_cfg_scale` as the active CFG; standard `guidance_scale` is kept at 1.0).
|
||||
- Input is a list of one or more PIL images.
|
||||
|
||||
For Lightning / few-step LoRAs (e.g. `lightx2v/Qwen-Image-Lightning-*`):
|
||||
- `num_inference_steps`: 4 or 8 (read the LoRA's model card — they ship 4-step and 8-step variants).
|
||||
- `true_cfg_scale`: usually 1.0 (CFG disabled).
|
||||
- Often comes with a custom scheduler config — see the lightx2v README for the exact `FlowMatchEulerDiscreteScheduler` config to use.
|
||||
|
||||
## Resolution buckets
|
||||
|
||||
Qwen-Image uses 16-pixel-aligned resolutions. When the user picks an aspect ratio, compute `width` and `height` as multiples of 16. A helper:
|
||||
|
||||
```python
|
||||
def round_to_bucket(w, h, multiple=16):
|
||||
return (w // multiple) * multiple, (h // multiple) * multiple
|
||||
```
|
||||
|
||||
For image-edit pipelines, resize the input image to the nearest bucket while preserving aspect; don't crop.
|
||||
|
||||
## ZeroGPU duration guidance
|
||||
|
||||
- Standard 50-step Qwen-Image T2I at 1024×1024: 60–90 seconds.
|
||||
- 4-step Lightning Qwen-Image: 15–25 seconds.
|
||||
- Qwen-Image-Edit 30 steps: 60–90 seconds.
|
||||
|
||||
Set `@spaces.GPU(duration=...)` accordingly.
|
||||
|
||||
## Common LoRA patterns on Qwen-Image
|
||||
|
||||
- **Style LoRAs (T2I).** Standard load. Trigger word usually present. UI: prompt + aspect ratio.
|
||||
- **Subject / character LoRAs (T2I).** Standard load. Trigger word almost always present. UI: prompt with the trigger pre-prepended in code, possibly an example prompt highlighting the trigger.
|
||||
- **Lighting / aesthetic LoRAs (T2I).** Often paired with a recommended LoRA scale ≠ 1.0 — check the model card.
|
||||
- **Edit LoRAs (image-to-image, on Qwen-Image-Edit).** Specific instructions baked in. The LoRA might require a specific instruction phrasing — match the pattern from the model card. UI: input image + instruction textbox.
|
||||
- **Lightning-distilled LoRAs.** Lock step count and CFG to recommended values; hide the sliders.
|
||||
|
||||
## Things to watch
|
||||
|
||||
- **VAE memory.** For 1328×1328 outputs, consider `pipe.enable_vae_tiling()` and `pipe.enable_vae_slicing()` after loading. Keep them off for smaller resolutions to avoid quality loss.
|
||||
- **Don't compile the transformer on ZeroGPU.** `torch.compile` won't work; the speedup options on ZeroGPU are limited to reducing steps or using FP8-distilled variants.
|
||||
- **Negative prompts work** but defaults are often empty string. Don't expose a negative prompt in the UI unless the LoRA's behavior actually benefits from it.
|
||||
+262
@@ -0,0 +1,262 @@
|
||||
# Creative mode: custom HTML/JS UIs in Gradio
|
||||
|
||||
When a LoRA's natural input shape doesn't fit any standard Gradio component or Hub custom component, you can drop down to plain HTML/CSS/JS inside a Gradio app. This file is about *how* — the Gradio primitives that make it work, and the discipline that keeps it from turning into a tangled mess.
|
||||
|
||||
This is the third rung of the component ladder (see `tasks.md` → "Picking components"):
|
||||
|
||||
1. **Stock Gradio components.** First choice. Almost always enough for T2I and a lot of I2I.
|
||||
2. **Hub custom components.** `gradio_image_annotation`, `gradio_imageslider`, `gradio_modal`, `gradio_rangeslider`, etc. No JS, just a `pip install` and an import.
|
||||
3. **Creative mode (this file).** Custom HTML/JS, when the user's input shape is something none of the above expresses well — point sets, trajectories, brush strokes, region selections with metadata, timeline scrubbing, 3D rotation gizmos, color picking on regions, drag-resize handles on media, keyframed inputs, anything where the user manipulates *a thing on top of media*.
|
||||
|
||||
Skipping rung 2 is a common mistake. If a Hub custom component fits, use it — `gradio_image_annotation` already covers bbox drawing, label assignment, and basic editing without a single line of JS.
|
||||
|
||||
But rung 2 has its own discipline (see "Hub custom components are fragile" below). The most common failure mode for these Spaces is **the custom component silently fails to render**: the Python side imports fine, the page loads, the API smoke-test in Phase 6 of `SKILL.md` passes — and yet the widget just isn't on the page. A user opening the Space sees the surrounding layout (buttons, accordions, outputs) but no upload zone, no annotator, nothing. If you don't actually look at the rendered page in a browser, you'll ship a broken Space and not know it.
|
||||
|
||||
## Hub custom components are fragile
|
||||
|
||||
Treat any `gradio_*` package from the Hub as load-bearing-and-untested-against-your-Gradio-version until you've seen it render. The most common failure modes:
|
||||
|
||||
- **Version mismatch silent breakage.** A Hub component built against Gradio N may load (because its declared `gradio<N+2,>=N` range covers your `sdk_version`) but mount to an empty DOM node on a slightly newer Gradio. No traceback in the build logs. No Python error. The component simply doesn't appear, and the rest of the column flows up around the gap. This is what produces "Generate button at the top of an empty left column" Spaces.
|
||||
- **Stale releases.** Many Hub custom components were last published 1–2 years ago. The Gradio frontend has changed since. Check the package's release date against the Gradio version you're targeting; if there's a multi-major-version gap, expect breakage.
|
||||
- **Mismatched param shapes.** The component's Python signature accepts a parameter the Svelte side no longer reads. Your `disable_edit_boxes=True` does nothing, or worse, throws on the JS side and the whole component fails to mount.
|
||||
|
||||
Discipline before committing to a Hub component:
|
||||
|
||||
1. **Check the package's last release date** (PyPI page or `pip index versions <pkg>`). If it's older than the Gradio release in your `sdk_version`, treat it as suspect.
|
||||
2. **Smoke-test the component in isolation** — a five-line Gradio app with just that one component, locally, *before* integrating it into the full app. If it renders, fine. If it's missing or blank, you've caught the breakage cheaply.
|
||||
3. **When a Hub component fails to render, don't iterate on its kwargs.** Drop to either (a) split stock components (e.g. `gr.Image` for upload + a sibling widget for the box coords) or (b) rung 3 (custom HTML/JS via `gr.HTML`). Twiddling `disable_edit_boxes` / `use_default_label` / `sources` is not going to bring a Svelte component back from a JS-side mount failure.
|
||||
|
||||
The same discipline applies to less obviously "custom" components if they were added recently — `gr.ImageSlider`, for example, can render unexpectedly when paired with a custom component on the same page.
|
||||
|
||||
## When creative mode is the right call
|
||||
|
||||
Reach for it when the user's *natural* input is structurally outside what stock components express:
|
||||
|
||||
- **Spatial input on top of media.** Drawing arrows on a frame, painting strokes on an image, dropping points along a trajectory, selecting irregular regions, drawing curves.
|
||||
- **Multi-shape annotation.** Source-and-destination box pairs, multiple labeled regions, ordered sequences of points/boxes that are semantically distinct.
|
||||
- **Continuous-with-snapping controls.** 3D rotation gizmos, dial/wheel controls, timeline scrubbers — anything where a slider would technically work but feel wrong.
|
||||
- **Composite controls bound together.** A canvas plus a color picker plus a brush-size dial that all feed the same structured input, where binding three separate components and reasoning about their joint state is uglier than rolling one widget.
|
||||
- **Live preview that depends on multiple inputs.** Something the user wants to see *immediately* as they manipulate, where a server roundtrip per change is too slow.
|
||||
|
||||
If the input is "a number," "a string," or "an image," you don't need this. Don't build a custom canvas because it would look cool — build it because the LoRA's input shape demands it.
|
||||
|
||||
## The Gradio primitives
|
||||
|
||||
Before reading the patterns below, it's worth re-checking the current Gradio docs for anything that landed recently:
|
||||
|
||||
- `gr.HTML` — https://www.gradio.app/docs/gradio/html
|
||||
- Custom components — https://www.gradio.app/guides/custom-components-in-five-minutes
|
||||
- `Blocks.launch(head=, css=)` — https://www.gradio.app/docs/gradio/blocks#blocks-launch
|
||||
|
||||
WebFetch these if you're unsure about a signature. The custom-HTML surface area evolves and lagging on it produces Spaces that "work" in stale ways.
|
||||
|
||||
The primitives that creative mode is built from:
|
||||
|
||||
### `gr.HTML` for arbitrary markup
|
||||
|
||||
Drop any HTML into the page. The block becomes a regular Gradio component, but its content is whatever you write. You're responsible for everything inside it: layout, styling, interactivity.
|
||||
|
||||
```python
|
||||
gr.HTML("""
|
||||
<div id="my-widget" style="...">
|
||||
<canvas id="my-canvas" width="512" height="512"></canvas>
|
||||
<div id="my-status"></div>
|
||||
</div>
|
||||
""")
|
||||
```
|
||||
|
||||
### `demo.launch(head=..., css=...)`
|
||||
|
||||
Inject `<script>` and `<style>` tags into the page `<head>`. This is how you load external JS libraries (Three.js, p5, fabric, anime.js, …) or define page-wide CSS that needs to be in `<head>` rather than inline.
|
||||
|
||||
```python
|
||||
head = '<script src="https://cdnjs.cloudflare.com/ajax/libs/three.js/r128/three.min.js"></script>'
|
||||
css = '.fillable {max-width: 1200px !important}'
|
||||
demo.launch(head=head, css=css)
|
||||
```
|
||||
|
||||
Use the highest-quality CDN you can (cdnjs, jsdelivr, unpkg). Pin the version (`/three.js/r128/`, not `/three.js/latest/`). Loading from the LoRA author's personal server is a recipe for the Space breaking when their server moves.
|
||||
|
||||
### `elem_id` and `elem_classes` for addressing
|
||||
|
||||
Every Gradio component accepts `elem_id="..."` and `elem_classes=[...]`. JS uses these to find the rendered DOM nodes:
|
||||
|
||||
```python
|
||||
prompt_box = gr.Textbox(elem_id="my-prompt", elem_classes=["hidden-input"])
|
||||
```
|
||||
|
||||
```javascript
|
||||
const promptBox = document.getElementById("my-prompt");
|
||||
// Note: the Gradio component wraps an <input> or <textarea>;
|
||||
// you usually want the inner element:
|
||||
const inner = promptBox.querySelector("input, textarea");
|
||||
```
|
||||
|
||||
### Two ways JS pushes state into Python
|
||||
|
||||
This is the part that trips people up. Pick whichever fits your case and stick with it for the whole widget — mixing them in one app is confusing.
|
||||
|
||||
**Approach A — Hidden Gradio input + DOM event dispatch.** Define a hidden `gr.Textbox` (or `gr.JSON`, `gr.Number`, etc.). From JS, set its inner `<input>`/`<textarea>` value and dispatch synthetic `input` and `change` events so Gradio's reactivity fires.
|
||||
|
||||
```python
|
||||
state_json = gr.Textbox(value="{}", elem_id="state-json", visible=False)
|
||||
```
|
||||
|
||||
```javascript
|
||||
function setGradioValue(elemId, value) {
|
||||
const container = document.getElementById(elemId);
|
||||
if (!container) return;
|
||||
const el = container.querySelector("input, textarea");
|
||||
if (!el) return;
|
||||
const proto = el.tagName === "TEXTAREA"
|
||||
? HTMLTextAreaElement.prototype
|
||||
: HTMLInputElement.prototype;
|
||||
const setter = Object.getOwnPropertyDescriptor(proto, "value").set;
|
||||
setter.call(el, value);
|
||||
el.dispatchEvent(new Event("input", { bubbles: true }));
|
||||
el.dispatchEvent(new Event("change", { bubbles: true }));
|
||||
}
|
||||
|
||||
setGradioValue("state-json", JSON.stringify(myState));
|
||||
```
|
||||
|
||||
The native setter dance (`Object.getOwnPropertyDescriptor(...).set.call(el, value)`) is necessary because React intercepts plain `el.value = ...` assignments and they won't trigger a re-render. The DOM event dispatch is what makes Gradio see the change.
|
||||
|
||||
This approach works in any Gradio version, with any component type. Downsides: lots of glue, fragile to DOM structure changes, easy to write race conditions.
|
||||
|
||||
**Approach B — `gr.HTML` subclass with `html_template`/`js_on_load`.** Newer Gradio supports subclassing `gr.HTML` and providing custom props plus a `js_on_load` script. The script gets a `props` object with custom values and a `trigger()` function for emitting events back to Python. State binding is value-based (Gradio reads the `value` prop just like any other component).
|
||||
|
||||
```python
|
||||
class PointPicker(gr.HTML):
|
||||
def __init__(self, value=None, image_url=None, **kwargs):
|
||||
super().__init__(
|
||||
value=value or {"points": []},
|
||||
html_template="<canvas id='pp-canvas' width='512' height='512'></canvas>",
|
||||
js_on_load="""
|
||||
const canvas = document.getElementById('pp-canvas');
|
||||
canvas.addEventListener('click', (e) => {
|
||||
const rect = canvas.getBoundingClientRect();
|
||||
const x = (e.clientX - rect.left) / rect.width;
|
||||
const y = (e.clientY - rect.top) / rect.height;
|
||||
props.value = {points: [...(props.value?.points || []), {x, y}]};
|
||||
trigger('change', props.value);
|
||||
});
|
||||
""",
|
||||
image_url=image_url,
|
||||
**kwargs,
|
||||
)
|
||||
```
|
||||
|
||||
Cleaner, value-based, no hidden inputs. Downsides: requires a Gradio version recent enough to support the subclass shape, and the JS lives inside a Python string, which IDEs don't lint.
|
||||
|
||||
**Picking between A and B:** if you're starting fresh and on current Gradio, prefer B — the binding semantics are saner. Reach for A when you need to integrate with already-existing Gradio components (sliders that should mirror the canvas state, prompt boxes the canvas writes into) or when the UI shell is large enough that putting it inside a `gr.HTML` subclass is awkward.
|
||||
|
||||
### Two ways Python pushes state into JS
|
||||
|
||||
- **For B (subclass):** return `gr.update(prop=value)` from an event handler. The widget's polling loop sees the prop change and reacts.
|
||||
- **For A (hidden input):** write to a hidden Gradio Textbox from your event handler. JS polls the Textbox value (or hooks into Gradio's mutation observer) and reacts.
|
||||
|
||||
Either way, expect a small polling interval (50–200ms is typical). Don't poll faster — it will dominate CPU on weaker machines.
|
||||
|
||||
### `js=` on event handlers
|
||||
|
||||
For tiny JS-only transforms (e.g. "scroll to result on click"), event handlers accept a `js=` arg that runs in the browser without a server roundtrip:
|
||||
|
||||
```python
|
||||
btn.click(fn=infer, inputs=[...], outputs=[...], js="() => { window.scrollTo(0, 0); }")
|
||||
```
|
||||
|
||||
Useful for UI polish (scroll, focus, show/hide loaders), not for state management.
|
||||
|
||||
## The communication contract
|
||||
|
||||
The single most important discipline in creative mode: **define the JSON shape that flows between JS and Python, and treat it as the API.**
|
||||
|
||||
Write it down at the top of `app.py` as a comment, even one line. Pick names that survive translation between camelCase JS and snake_case Python (or commit to one and convert at the boundary):
|
||||
|
||||
```python
|
||||
# State shape on the JS↔Python wire:
|
||||
# {
|
||||
# "src": {"x1": float, "y1": float, "x2": float, "y2": float} | null,
|
||||
# "dst": {"x1": float, "y1": float, "x2": float, "y2": float} | null,
|
||||
# "label": string | null
|
||||
# }
|
||||
```
|
||||
|
||||
Once it's written down, both sides have a target to conform to and you'll catch shape drift early.
|
||||
|
||||
Common shape archetypes — none of these are required, just illustrative:
|
||||
|
||||
- **Point set:** `{"points": [{"x": 0.3, "y": 0.4}, ...]}`
|
||||
- **Stroke set:** `{"strokes": [{"color": "#ff0", "size": 12, "points": [...]}, ...]}`
|
||||
- **Regions:** `{"regions": [{"label": "subject", "bbox": {...}}, ...]}`
|
||||
- **Transform:** `{"azimuth": 45, "elevation": 0, "distance": 1.0}`
|
||||
- **Trajectory with time:** `{"keyframes": [{"t": 0.0, "x": ...}, {"t": 0.5, ...}]}`
|
||||
|
||||
All coordinates normalized to `[0, 1]` is almost always the right call — it survives image resizing without rescaling math.
|
||||
|
||||
## External libraries
|
||||
|
||||
Use what helps. Stay light when you can.
|
||||
|
||||
- **Pure canvas + DOM** is enough for: drawing rectangles/points/lines, brush strokes, drag-and-drop handles, image overlays.
|
||||
- **SVG** for: vector overlays, especially when the marks need to scale crisply or be styled with CSS.
|
||||
- **Three.js** for: 3D rotation gizmos, depth visualizations, anything where you need WebGL rendering of a small scene. Pulled in via `head=` script tag.
|
||||
- **fabric.js / paper.js / konva** for: complex shape-on-canvas interactions when raw canvas is getting ugly. Worth it once you've added more than ~200 lines of canvas glue.
|
||||
- **p5.js** for: rapid prototyping of generative-art-style canvases. Heavier than necessary for production.
|
||||
|
||||
When in doubt, raw canvas. Pulling in fabric for a single rectangle is overkill; pulling in three.js because you wanted "fancier sliders" is a smell.
|
||||
|
||||
## Pitfalls
|
||||
|
||||
These bite repeatedly. Read before writing.
|
||||
|
||||
- **Silent mount failure of Hub custom components.** Covered in detail above. Worth restating because it's the failure mode that the API smoke-test cannot catch: the component imports, the page loads, and the widget is just absent from the DOM. Always verify in a browser; never trust "the build went green" as proof a Hub custom component renders.
|
||||
|
||||
- **Init race with Gradio mount.** The `js_on_load` (B) or a top-level `<script>` injected via `head` may run before Gradio has rendered the DOM nodes you're trying to hook into. Guard with a short `setTimeout` or wait for the element to exist:
|
||||
```javascript
|
||||
function init() {
|
||||
const el = document.getElementById("my-widget");
|
||||
if (!el) { setTimeout(init, 50); return; }
|
||||
// ... do work
|
||||
}
|
||||
init();
|
||||
```
|
||||
|
||||
- **Double init.** If the user navigates between tabs or Gradio re-renders, your init can fire twice. Stash a flag: `if (window.__myWidgetInited) return; window.__myWidgetInited = true;`.
|
||||
|
||||
- **Base64 image transfer cost.** Sending a full-resolution image as a base64 string through a hidden Textbox is fine for previews but punishingly slow for 4K images. Downscale on the JS side before stuffing into state, or pass the image through a real `gr.Image` component and only send the *interaction state* (boxes, points) through the hidden channel.
|
||||
|
||||
- **File inputs don't roundtrip through hidden Textboxes.** A hidden `gr.File` won't accept arbitrary JS-set values. If the user uploads via a custom `<input type="file">`, you have two options: (a) read the file in JS, base64-encode, and write to a hidden Textbox; (b) wire the custom file input to programmatically click a real `gr.File` (`document.querySelector("#real-file input").click()` is brittle but works).
|
||||
|
||||
- **Polling cadence.** 100ms is a reasonable default; 50ms feels snappy but burns CPU on weaker machines; 500ms feels laggy. Don't poll inside `requestAnimationFrame` for state sync — that's 60Hz and overkill.
|
||||
|
||||
- **Mobile / touch.** If the widget lives in a Space people will open on their phones, handle `touchstart`/`touchmove`/`touchend` alongside mouse events. The two-event-system thing is annoying but unavoidable.
|
||||
|
||||
- **ZeroGPU duration when payloads grow.** If the custom UI lets users submit large state (many points, big strokes, multiple boxes), the server-side processing time can grow with input size. Re-check `@spaces.GPU(duration=...)` against worst-case payloads.
|
||||
|
||||
- **Custom run buttons.** If you build your own "Run" button in HTML and have it trigger a real Gradio button via `.click()`, make sure state-sync (write to hidden inputs) happens *before* the click, with enough delay for events to propagate (`setTimeout(() => realBtn.click(), 50)` is the usual fix).
|
||||
|
||||
- **CSS isolation.** Spaces inject their own CSS. If your widget styles get clobbered, increase specificity (`#my-widget .foo` instead of `.foo`) or use `!important` sparingly. Don't fight it with inline styles for everything — that gets unreadable fast.
|
||||
|
||||
## Smoke-test caveat — applies to rung 2 AND rung 3
|
||||
|
||||
`gradio info` / `gradio predict` (Phase 6 of `SKILL.md`) only exercise the Python endpoint. They tell you nothing about what actually renders. For any Space that uses a Hub custom component (rung 2) or custom HTML/JS (rung 3), the Python side can be perfectly correct *and* the user can see a broken UI.
|
||||
|
||||
The two distinct failure modes:
|
||||
|
||||
- **Rung 2 silent mount failure.** Hub custom component imports cleanly, gets a slot in the Gradio config, and just doesn't appear in the rendered DOM. You see the components around it but the widget itself is gone — leaving e.g. a Generate button at the top of an otherwise empty column. No error anywhere. See "Hub custom components are fragile" above.
|
||||
- **Rung 3 broken JS.** Server-side green, but `js_on_load` errored, or a CDN script failed to load, or your event handlers never bound. The widget mounts but doesn't do anything when clicked.
|
||||
|
||||
After the API smoke-test in Phase 6 passes, **open the Space URL in a browser and verify both that every component is visible and that one full interaction (upload → click Generate → see result) works** before sharing. This is not optional for these Spaces — it's the only check that catches the failure modes above.
|
||||
|
||||
If the user is in an environment where you can't drive a browser yourself, ask them to open the Space URL and confirm the upload zone is visible and accepts an image before declaring the Space done. The cost of asking is one extra message; the cost of skipping is shipping a Space that looks empty.
|
||||
|
||||
## Real-world examples
|
||||
|
||||
These two Spaces use the patterns above for very different LoRAs. They're useful as concrete proof the patterns work, not as templates to copy:
|
||||
|
||||
- **3D camera control via Three.js** (Approach B, `gr.HTML` subclass): https://huggingface.co/spaces/multimodalart/qwen-image-multiple-angles-3d-camera
|
||||
- **Bounding-box drag/resize via canvas** (Approach A, hidden inputs + custom Run button): https://huggingface.co/spaces/linoyts/FLUX.2-klein-Move
|
||||
|
||||
A short look at each is worth it before designing a new widget — they show what "production polish" feels like in this register (snap-to-nearest animations, status overlays, cursor changes on hover, mobile-touch handling). But your widget will look different, because your LoRA wants different inputs.
|
||||
@@ -0,0 +1,124 @@
|
||||
# Tasks: per-task baseline UI patterns
|
||||
|
||||
This file describes the *baseline* UI shape for each task category. Use it after the LoRA's task is identified, as a starting skeleton. Then read `adapting-to-the-lora.md` to shape the actual UI to the specific LoRA — the baseline is rarely the right final answer.
|
||||
|
||||
The five tasks: text-to-image, image-to-image, text-to-video, image-to-video, video-to-video.
|
||||
|
||||
## Common to all tasks
|
||||
|
||||
- Layout: two-column `gr.Row` with `equal_height=True`. Inputs on the left, outputs on the right.
|
||||
- One primary `gr.Button("Generate", variant="primary", size="lg")`. No secondary buttons unless they do meaningfully different things.
|
||||
- An `Advanced` accordion (`gr.Accordion("Advanced", open=False)`) for power-user controls that most users will never touch (seed, randomize seed, advanced sampler params).
|
||||
- Always include a seed control with a "Randomize seed" checkbox, and return the actually-used seed alongside the result so users can reproduce.
|
||||
- Wire up `prompt.submit` as well as button click for text inputs, so Enter works.
|
||||
- Use `gr.Progress(track_tqdm=True)` so diffusers' internal progress bar surfaces.
|
||||
|
||||
## Text-to-image (T2I)
|
||||
|
||||
**Inputs:** prompt (`gr.Textbox`, `lines=2`, with example placeholder). Aspect ratio or resolution control. Optional: negative prompt for models that support it.
|
||||
|
||||
**Outputs:** `gr.Image` (or `gr.Gallery` if returning more than one).
|
||||
|
||||
**Standard advanced controls:** seed, randomize seed, num inference steps, guidance scale. Hide steps and guidance entirely for few-step LoRAs (Lightning, Turbo, schnell).
|
||||
|
||||
**Aspect ratio handling:** offer a dropdown of common ratios with width/height auto-derived. Snap dimensions to the model's native bucket size (16 for most diffusion transformers, 8 for older UNet-based). Show width/height as read-only display.
|
||||
|
||||
**Examples:** lift example prompts from the LoRA model card into a `gr.Examples` block. Use `cache_examples=True, cache_mode="lazy"` so caching defers to first click instead of failing at build time on ZeroGPU.
|
||||
|
||||
## Image-to-image (I2I)
|
||||
|
||||
**Inputs:** input image (`gr.Image(type="pil")` or `numpy` depending on what your processing wants), instruction or prompt (`gr.Textbox`).
|
||||
|
||||
**Outputs:** `gr.Image`. For edit LoRAs, consider `gr.ImageSlider` (built-in) for before/after comparison instead of a separate image.
|
||||
|
||||
**Resolution handling:** for instruction-edit pipelines (Qwen-Image-Edit, Flux Kontext, Flux.2 Klein), resize the input to the nearest multiple of the model's bucket size, preserving aspect. Don't crop unless the LoRA expects a specific aspect.
|
||||
|
||||
**Validation:** raise `gr.Error("Please upload an image first.")` when the input is empty. Consider disabling Generate until an image is loaded (`run_button.interactive = False`, flip on `input_image.change`).
|
||||
|
||||
**Sub-task variants change a lot here.** Read `adapting-to-the-lora.md` — relighting, face swap, object move, style transfer, instruction edits, inpainting all live under "I2I" but call for different UIs.
|
||||
|
||||
## Text-to-video (T2V)
|
||||
|
||||
**Inputs:** prompt. Duration slider (1–10 seconds typical, depending on the model's max). Resolution/aspect picker.
|
||||
|
||||
**Outputs:** `gr.Video(autoplay=True)`. Set `format="mp4"` and pick fps explicitly (24 is a safe default; some models prefer 16 or 30).
|
||||
|
||||
**Standard advanced controls:** seed, randomize seed, fps. Steps usually locked for distilled video models.
|
||||
|
||||
**Duration awareness:** set `@spaces.GPU(duration=...)` to comfortably exceed expected generation time. For 5-second 720p video, 180+ seconds of GPU time is realistic. Tell the user in the UI that generation takes a while ("Generating a 5s video takes about 2 minutes").
|
||||
|
||||
**Frame count math:** most video diffusion models want frame counts that are `8k+1` or similar. Compute `num_frames` from `duration * fps` and round to the nearest valid value rather than passing arbitrary frames. The base-model reference file says what's valid for each model.
|
||||
|
||||
## Image-to-video (I2V)
|
||||
|
||||
**Inputs:** input image (the first frame, or a stylistic reference depending on the LoRA), prompt describing the motion. Duration.
|
||||
|
||||
**Outputs:** `gr.Video`.
|
||||
|
||||
**Aspect ratio:** auto-detect from the input image and snap to the model's nearest bucket. Show the chosen resolution as info text.
|
||||
|
||||
**Variants:** some I2V LoRAs use the input image as the literal first frame; others use it as a stylistic reference and generate a new first frame from the prompt. The model card usually says which. The UI for both is similar; the difference is in how `image=` is passed to the pipeline and whether a "use as first frame" toggle makes sense.
|
||||
|
||||
## Video-to-video (V2V)
|
||||
|
||||
**Inputs:** at minimum, a source video. Almost always also a prompt. Often additional inputs depending on what the LoRA does (reference image for appearance, mask, control video, etc.).
|
||||
|
||||
**Outputs:** `gr.Video`. For LoRAs that do preprocessing on the input (pose extraction, depth estimation, padding), show the preprocessed intermediate as a second smaller video alongside the result, so the user sees what the model actually saw.
|
||||
|
||||
**This is where adaptation matters most.** "V2V" alone tells you almost nothing about the UI. Pose-control, depth-control, canny-control, outpainting, inpainting, style transfer, motion transfer, frame interpolation, and upscaling are all V2V and all need different UIs. Always read `adapting-to-the-lora.md` and the per-base-model file before designing.
|
||||
|
||||
**Common patterns:**
|
||||
|
||||
- Preprocessing preview: a small `gr.Video(height=240)` showing the extracted pose / depth / canny / padded video. Update it on input change so the user sees the preprocessed result before clicking Generate.
|
||||
- Two-input layout for motion-transfer LoRAs: source video + appearance image, clearly labeled.
|
||||
- Aspect-ratio picker only when the LoRA actually changes aspect (outpainting). For pose/depth/canny control, output aspect matches input.
|
||||
|
||||
## Picking components
|
||||
|
||||
Walk this ladder in order. Stop at the first rung that fits the LoRA's input shape.
|
||||
|
||||
**1. Stock Gradio components.** First choice almost always:
|
||||
|
||||
- `gr.ImageSlider` — built-in before/after comparison for edit LoRAs.
|
||||
- `gr.ImageEditor` — upload + paint on top of an image. The right pick for any LoRA whose "input shape" is "a region of the image" expressed by painting — object removal trained on red-highlighted regions, relight trained on colored brush strokes, scribble-conditioned edits. Constrain the brush with `gr.Brush(default_color="#ff0000", colors=["#ff0000"])` so the user can only paint in the color the LoRA was trained on; the editor returns `{"background", "layers", "composite"}` and the `composite` is what you feed the pipeline. Used in production by `linoyts/QIE-2509-Object-Remover-Bbox-v3` (qie-2509-object-remover) — don't reach for `gradio_image_annotation` for these tasks just because the LoRA was trained "with bboxes"; the user-facing shape is a painted region, not a literal box.
|
||||
- `@gr.render` — UI that changes shape based on input (e.g. show extra controls only when an input is uploaded).
|
||||
- `gr.Examples` — clickable example inputs. Almost always worth including. Lift from the LoRA's model card.
|
||||
- `gr.BrowserState` — persist user preferences (preferred aspect ratio, last seed, etc.) across sessions.
|
||||
- `gr.DeepLinkButton` — share a specific generation as a URL.
|
||||
|
||||
**2. Hub custom components.** A `pip install` and an import, no JS to maintain:
|
||||
|
||||
- `gradio_image_annotation` — bbox/point annotation on top of an image. Right when the LoRA literally needs box *coordinates* as structured input (e.g. drag-and-drop "move from box A to box B" LoRAs, region-tagged edits). Wrong when the LoRA wants a painted region — use `gr.ImageEditor` instead.
|
||||
- `gradio_imageslider` — alternative before/after slider with extra controls.
|
||||
- `gradio_modal` — modal dialogs.
|
||||
- `gradio_rangeslider` — dual-handle range slider.
|
||||
|
||||
Browse the rest at https://www.gradio.app/custom-components/gallery before going further down the ladder.
|
||||
|
||||
**3. Creative mode (custom HTML/JS).** When stock and Hub custom components both come up short — point sets, strokes, trajectories, region selections with metadata, 3D rotation gizmos, timeline scrubbers, anything where the user manipulates a thing on top of media. See `creative-mode.md` for the Gradio primitives, the JS↔Python communication contract, and the pitfalls. Don't skip rung 2 to get here — `gradio_image_annotation` already covers a lot of what looks like it needs custom HTML.
|
||||
|
||||
Themes: default to `gr.themes.Citrus()`.
|
||||
|
||||
Before defaulting to a plain component or guessing at a custom one, web-fetch the current Gradio docs at https://www.gradio.app/docs.
|
||||
|
||||
## Gradio 6.x gotchas
|
||||
|
||||
The current `sdk_version` (6.x at time of writing — verify with `pip index versions gradio`) changed a few things that older recipes get wrong. The failures are easy to miss because they happen at the Space's first import, not when you write the file locally.
|
||||
|
||||
- **`theme=` and `css=` moved from `gr.Blocks(...)` to `demo.launch(...)`.** Passing them to `Blocks` now emits a deprecation warning and the styling silently doesn't apply. Always:
|
||||
|
||||
```python
|
||||
with gr.Blocks(title="...") as demo:
|
||||
...
|
||||
|
||||
if __name__ == "__main__":
|
||||
demo.launch(theme=gr.themes.Citrus(), css=CSS)
|
||||
```
|
||||
|
||||
Spaces runs `app.py` as `__main__`, so the `launch()` call executes.
|
||||
|
||||
- **Some component kwargs were removed.** `gr.Image` no longer accepts `show_download_button` (the same change affects a handful of other components). The Space fails at import with `TypeError: __init__() got an unexpected keyword argument 'show_download_button'` — not surfaced until the container actually starts. When in doubt, web-fetch the current docs for the specific component before passing non-obvious kwargs.
|
||||
|
||||
- **The Gradio version the Space runs at is set by `sdk_version:` in the README YAML, *not* by `requirements.txt`.** Pinning `gradio` in `requirements.txt` is at best ignored and at worst causes a runtime mismatch; set the version once, in the README, and write `app.py` against that version.
|
||||
|
||||
If the first build fails on a `TypeError` or signature mismatch in a Gradio component, this is the most common cause — read `/logs/container` (build) or `/logs/run` (runtime), look at the line in `app.py`, and check the current component signature.
|
||||
+194
@@ -0,0 +1,194 @@
|
||||
# ZeroGPU and publishing
|
||||
|
||||
ZeroGPU is the default hardware target. It's a shared serverless GPU pool: GPU is allocated on each request, held for the duration of a `@spaces.GPU` function call, and released. The key implications for the demo's code shape:
|
||||
|
||||
## ZeroGPU rules
|
||||
|
||||
**Models go on `cuda` at module level, not lazy-loaded inside the GPU function.**
|
||||
|
||||
```python
|
||||
import torch
|
||||
import spaces
|
||||
from diffusers import QwenImagePipeline
|
||||
|
||||
pipe = QwenImagePipeline.from_pretrained("Qwen/Qwen-Image", torch_dtype=torch.bfloat16)
|
||||
pipe.to("cuda")
|
||||
pipe.load_lora_weights("user/my-lora")
|
||||
|
||||
@spaces.GPU(duration=60)
|
||||
def generate(prompt):
|
||||
return pipe(prompt).images[0]
|
||||
```
|
||||
|
||||
ZeroGPU uses a CUDA emulation mode that lets `.to("cuda")` work outside `@spaces.GPU` functions during startup. Module-level placement is significantly faster than deferred placement because CUDA transfers are optimized for startup-time placement. Lazy-loading inside `@spaces.GPU` is discouraged.
|
||||
|
||||
**The `@spaces.GPU` decorator wraps the function that needs the GPU.**
|
||||
|
||||
The default duration is 60 seconds. Set it higher for longer tasks: `@spaces.GPU(duration=120)` or `@spaces.GPU(duration=300)`. Set it lower if the task reliably finishes faster — lower duration means higher queue priority. For tasks where duration varies with input, pass a function: `@spaces.GPU(duration=lambda *args: ...)`.
|
||||
|
||||
**GPU size: `large` (default, 48GB VRAM) or `xlarge` (96GB, full Blackwell).** Specify with `@spaces.GPU(size="xlarge")` when a single inference needs more than 48GB — large base video models, high-resolution generation, or heavy multi-stage pipelines. `xlarge` consumes 2× the daily quota per second and queues longer, so only reach for it when `large` actually OOMs.
|
||||
|
||||
Typical durations:
|
||||
|
||||
- Few-step T2I (4-8 steps): 30-60 seconds.
|
||||
- Standard T2I (20-50 steps): 60-90 seconds.
|
||||
- I2I / instruction edits: 60-90 seconds.
|
||||
- Short video (3-5 seconds): 120-180 seconds.
|
||||
- Long video / multi-stage: 180-300 seconds.
|
||||
|
||||
**Don't use `torch.compile`.** It's incompatible with ZeroGPU's process model (the GPU process forks per call). The decorator is a no-op outside ZeroGPU, so `pipe(...)` runs uncompiled in both environments.
|
||||
|
||||
**Validate inputs at the top of the GPU function.** Raising `gr.Error(...)` inside a `@spaces.GPU` function still consumes some GPU quota for the allocation. Validate before doing real work, or move validation into a non-decorated function called by the UI.
|
||||
|
||||
**Use `cache_examples=True` with `cache_mode="lazy"` on `gr.Examples`.** Plain `cache_examples=True` runs the function at build time, before a GPU is allocated, and will fail. `cache_mode="lazy"` defers caching to the first time a user clicks each example — the GPU is available, and subsequent clicks return the cached result instantly.
|
||||
|
||||
**Don't initialize CUDA from outside the controlled paths.** `pipe.to("cuda")` is fine (CUDA emulation handles it). Calling `torch.cuda.something()` directly at module level can break the process model — when in doubt, do it inside the GPU function or skip it.
|
||||
|
||||
**ZeroGPU requires PRO/Team/Enterprise.** A free-tier user can create a Space with `hardware: zero-a10g` in the README, but it'll fall back to CPU. If the user isn't on a supporting plan, mention this and point them at two paths: upgrade to PRO (unlocks ZeroGPU directly), or apply for a [community GPU grant](https://huggingface.co/docs/hub/spaces-gpus#community-gpu-grants) (request free paid GPU hardware via the Space's hardware settings, subject to approval).
|
||||
|
||||
## HF Hub patterns
|
||||
|
||||
### Authentication — check first, ask only if needed
|
||||
|
||||
Don't ask for a token reflexively. Check whether the user is already authenticated, and only prompt if there's no usable session.
|
||||
|
||||
```python
|
||||
from huggingface_hub import HfApi, get_token
|
||||
|
||||
def resolve_auth():
|
||||
"""Returns (token, username) or (None, None) if no usable auth."""
|
||||
cached = get_token() # picks up HF_TOKEN env var or cached CLI login
|
||||
if not cached:
|
||||
return None, None
|
||||
try:
|
||||
info = HfApi().whoami(token=cached)
|
||||
return cached, info["name"]
|
||||
except Exception:
|
||||
return None, None # token exists but is invalid/expired
|
||||
```
|
||||
|
||||
Decision tree:
|
||||
|
||||
- **User already authenticated and the LoRA repo is public**: use the existing token. Confirm the username with the user before publishing ("I'll publish to `{username}` — confirm?").
|
||||
- **User already authenticated and the LoRA repo is private**: try `api.repo_info(repo_id, token=cached)`. If it succeeds, the existing token has the right scope — proceed. If it fails (token doesn't have access to that repo), ask for a token with broader access.
|
||||
- **No cached token**: ask the user. One ask, with the explanation: "I need a Hugging Face access token with **write** scope. Create one at https://huggingface.co/settings/tokens. Paste it here." The same token will be reused for publishing.
|
||||
|
||||
The default flow on a Hugging Face Space, in a logged-in user's local environment with `huggingface-cli login`, or in any environment with `HF_TOKEN` set, will *not* require asking the user for a token. Asking is the fallback, not the default.
|
||||
|
||||
### Reading the LoRA repo
|
||||
|
||||
```python
|
||||
from huggingface_hub import HfApi, ModelCard
|
||||
|
||||
api = HfApi(token=hf_token) # token may be None for public repos
|
||||
|
||||
try:
|
||||
info = api.repo_info(repo_id) # 401/403 → private/gated; need token
|
||||
except Exception as e:
|
||||
# Handle private/gated repo case
|
||||
...
|
||||
|
||||
files = api.list_repo_files(repo_id)
|
||||
card = ModelCard.load(repo_id, token=hf_token)
|
||||
base_model = card.data.get("base_model")
|
||||
pipeline_tag = card.data.get("pipeline_tag")
|
||||
readme_text = card.text
|
||||
```
|
||||
|
||||
### Picking the LoRA weights file
|
||||
|
||||
Many LoRA repos contain a single `.safetensors` file and the choice is trivial. But some contain several — variants (4-step / 8-step distillations, FP16 vs BF16, different ranks), training-history checkpoints (`epoch-10.safetensors`, `epoch-20.safetensors`), or genuinely different methods (`lora.safetensors` + `lora_dora.safetensors`). Pick in this order, stopping at the first match:
|
||||
|
||||
1. **The README recommends a specific file.** This is the strongest signal — if the author bothered to name a file, that's the choice. Look for filenames inside inference snippets (especially `weight_name="..."` arguments), in "recommended" or "best" callouts, in comparison tables ranking variants, or in any prose like "use X for Y." If the README clearly points at one file, use it without asking.
|
||||
|
||||
2. **No README recommendation, and `pytorch_lora_weights.safetensors` exists at the repo root.** Use it. This is the diffusers convention and a safe default.
|
||||
|
||||
3. **Neither, but the multiple files look like training checkpoints** (filenames with patterns like `epoch-N`, `step-N`, `checkpoint-N`, or a numeric progression like `lora-1.safetensors`, `lora-2.safetensors`, `lora-3.safetensors`). Default to the highest-numbered / latest one, but mention the choice in the response so the user can override: "Repo has epoch-10, epoch-20, epoch-30; using epoch-30 — let me know if you want a different one."
|
||||
|
||||
4. **Otherwise** — files look like alternative variants (`*-4steps` vs `*-8steps`, `*-fp16` vs `*-bf16`, `lora` vs `lora_dora`), or names are opaque (`v2.safetensors`, `final.safetensors`, `output.safetensors`), or there's no clear "latest." Ask, with a one-line description of each option based on what the filenames suggest. Don't pick blindly — the wrong choice produces a working Space that's silently using the wrong weights.
|
||||
|
||||
This reasoning happens once, in Phase 1. The chosen filename is then passed to `load_lora_weights` via `weight_name="..."` in `app.py`.
|
||||
|
||||
### Loading a private LoRA in `app.py`
|
||||
|
||||
```python
|
||||
import os
|
||||
pipe.load_lora_weights("user/private-lora", token=os.environ["HF_TOKEN"])
|
||||
```
|
||||
|
||||
### Creating and publishing the Space
|
||||
|
||||
```python
|
||||
from huggingface_hub import HfApi, SpaceHardware
|
||||
|
||||
api = HfApi(token=hf_token)
|
||||
username = api.whoami()["name"]
|
||||
repo_id = f"{username}/{space_name}"
|
||||
|
||||
api.create_repo(
|
||||
repo_id=repo_id,
|
||||
repo_type="space",
|
||||
space_sdk="gradio",
|
||||
space_hardware=SpaceHardware.ZERO_A10G,
|
||||
private=True,
|
||||
exist_ok=True,
|
||||
)
|
||||
|
||||
# Set HF token as a Space secret if the LoRA or base model is private/gated
|
||||
api.add_space_secret(repo_id=repo_id, key="HF_TOKEN", value=hf_token)
|
||||
|
||||
# Upload files
|
||||
for path in ["app.py", "requirements.txt", "README.md"]:
|
||||
api.upload_file(
|
||||
path_or_fileobj=path,
|
||||
path_in_repo=path,
|
||||
repo_id=repo_id,
|
||||
repo_type="space",
|
||||
)
|
||||
```
|
||||
|
||||
The Space starts building automatically once files are pushed.
|
||||
|
||||
### `SpaceHardware.ZERO_A10G`
|
||||
|
||||
The string value is `"zero-a10g"`. This is a legacy name from when ZeroGPU ran on A10Gs; the actual hardware is NVIDIA RTX Pro 6000 Blackwell, but the identifier stuck. Both `SpaceHardware.ZERO_A10G` and the literal `"zero-a10g"` work. Prefer the enum for clarity.
|
||||
|
||||
If `create_repo` rejects the hardware (typically because the user isn't on PRO), retry without `space_hardware=`, set the README's `hardware: zero-a10g` anyway, and tell the user the Space will run on CPU until they either upgrade to PRO or apply for a [community GPU grant](https://huggingface.co/docs/hub/spaces-gpus#community-gpu-grants) (request form lives in the Space's hardware settings).
|
||||
|
||||
### Updating an existing Space
|
||||
|
||||
If the user already has a Space they want to update (rather than creating fresh), `create_repo` with `exist_ok=True` is a no-op on the existing repo. `upload_file` overwrites. Existing secrets and hardware settings are preserved. Don't delete and recreate the Space — they'll lose stars, comments, and any custom config.
|
||||
|
||||
## After publishing
|
||||
|
||||
The Space URL is `https://huggingface.co/spaces/{repo_id}`. Build logs are at `https://huggingface.co/spaces/{repo_id}/logs/container`. Runtime logs at `https://huggingface.co/spaces/{repo_id}/logs/run`.
|
||||
|
||||
When sharing the URL with the user:
|
||||
|
||||
- Note that the Space is private — they need to be logged in to view it.
|
||||
- Note that the build takes a few minutes the first time.
|
||||
- Offer to look at the logs if anything fails.
|
||||
- Don't add a long postamble — they want to click the link, not read more text.
|
||||
|
||||
**Confirm a redeploy is actually live before testing it.** An `app.py`-only push does **not** change the Space's reported `runtime.stage` — the old replica keeps serving "RUNNING" while the new build swaps in, so a `gradio_client` test can silently hit **stale code**. To be sure: push → `api.restart_space(repo)` → poll until the stage leaves and returns to RUNNING → grep the boot logs (`/logs/run`) for a unique `[VERSION] …` marker you printed at module scope → then test. Also set `demo.launch(show_error=True)` so `gradio_client` surfaces the real traceback instead of a generic `AppError`.
|
||||
|
||||
## Publish-time failures (before the build starts)
|
||||
|
||||
These happen during `create_repo` or `upload_file`, *before* the Space build pipeline runs. Diagnose by reading the exception, not the container logs (the container hasn't started yet).
|
||||
|
||||
- **`HfHubHTTPError: 400 Bad Request` from `https://huggingface.co/api/validate-yaml`.** The README's YAML frontmatter failed server-side validation. By far the most common cause is `short_description` exceeding the server's length cap (the cap isn't documented and may change; targeting ~60 characters keeps you well clear). Other causes include typos in field names (`hardware` vs `hardwre`), invalid color values in `colorFrom`/`colorTo`, an unrecognized `hardware` string, or a malformed `models:` list. Fix: open `README.md`, shorten `short_description`, double-check the other YAML fields, retry. If the user gave you a long description for the Space, put the long version in the README body below the YAML — that's the right home for prose.
|
||||
|
||||
- **`HfHubHTTPError: 403 Forbidden` on `create_repo` with `space_hardware="zero-a10g"`.** The user's account can't request ZeroGPU at creation time (typically because they're not on PRO/Team/Enterprise). Fix: retry `create_repo` without the `space_hardware` argument; keep `hardware: zero-a10g` in the README YAML. The Space gets created on CPU. Point the user at two paths to get off CPU: upgrade to PRO (auto-promotes the Space to ZeroGPU), or apply for a [community GPU grant](https://huggingface.co/docs/hub/spaces-gpus#community-gpu-grants) (request via the Space's hardware settings).
|
||||
|
||||
- **`HfHubHTTPError: 401/403` on `upload_file`.** Token lacks write scope. Fix: ask the user for a write-scoped token (or use a fine-grained token with write permission on this specific Space).
|
||||
|
||||
- **`RepositoryNotFoundError` on `upload_file` immediately after `create_repo`.** Race condition; very rare. Fix: small `time.sleep(1)` between create and upload, or retry the upload.
|
||||
|
||||
## Common build failures
|
||||
|
||||
- **`weight_name` mismatch in `load_lora_weights`.** The actual file in the repo is named differently. Fix: `api.list_repo_files(repo_id)` to find the real filename; pass `weight_name=` explicitly.
|
||||
- **Gated base model, no token.** The base model (e.g. `black-forest-labs/FLUX.1-dev`) requires accepting a license. Fix: ensure the user has accepted the license on the Hub, and the token is set as a Space secret.
|
||||
- **Diffusers version too old for the pipeline class.** The base model was released after the latest pinned diffusers. Fix: change `requirements.txt` from `diffusers` to `git+https://github.com/huggingface/diffusers`.
|
||||
- **CUDA OOM on first request.** The model is too big for the 48GB VRAM available on the default `large` size. Solutions, in order of preference: pick a smaller or quantized variant (FP8, smaller checkpoint); request `@spaces.GPU(size="xlarge")` to get the full 96GB (costs 2× quota and queues longer); enable model offloading (`pipe.enable_model_cpu_offload()` — conflicts with ZeroGPU's process model, last resort only).
|
||||
- **`cache_examples=True` failure.** Build-time GPU isn't available on ZeroGPU. Fix: add `cache_mode="lazy"` so caching happens on first user click instead of at build.
|
||||
- **Free-tier user, hardware not allocated.** Space falls back to CPU. The build succeeds but inference is unusably slow. Fix: user upgrades to PRO, or removes `hardware: zero-a10g` and lives with CPU.
|
||||
Reference in New Issue
Block a user