csh/playbook
1
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

📦 deps(thirdparty): update snapshots

Browse Source
This commit is contained in:
ci[bot]
2026-07-01 16:02:41 +00:00
parent 8301f01888
commit c824ba9d7b
2449 changed files with 555104 additions and 9259 deletions
Download Patch File Download Diff File Expand all files Collapse all files
antigravity-awesome-skills/skills/huggingface-lora-space-builder/references/base-models/krea-2.md
+63
View File
@@ -0,0 +1,63 @@
# Krea 2 reference
Krea 2 (K2) is a flow-matching **text-to-image** model: a 12B dense DiT (grouped-query attention) with a **Qwen3-VL** text encoder (multi-layer feature aggregation) and the **Qwen-Image VAE** (`AutoencoderKLQwenImage`). Pipeline class: `Krea2Pipeline`. It ships as two checkpoints designed to work together:
| Repo | Role | Use it for |
|------|------|------------|
| [`krea/Krea-2-Turbo`](https://huggingface.co/krea/Krea-2-Turbo) | 8-step **distilled** | **Inference / demos** |
| [`krea/Krea-2-Raw`](https://huggingface.co/krea/Krea-2-Raw) | base, non-distilled | **LoRA training** — *not* for inference |
**For a LoRA Space, load Turbo.** Krea 2 LoRAs are *trained on RAW but run on Turbo* (they "express strongly on Turbo"), and RAW is explicitly not meant for inference — expect poor quality if you load it directly. So a demo should almost always use `krea/Krea-2-Turbo`. The official LoRA cards confirm this ("To be used on `krea/Krea-2-Turbo`").
> Needs a `diffusers` build that includes `Krea2Pipeline` (both repos are `library_name: diffusers`). If `from diffusers import Krea2Pipeline` fails, the installed `diffusers` predates the integration — update it.
## Required dependencies
- `diffusers` with `Krea2Pipeline`.
- `transformers` recent enough for **Qwen3-VL** (the text encoder is a `Qwen3VLModel`, e.g. `Qwen/Qwen3-VL-4B-Instruct`).
- **`torchvision`** — the Qwen3-VL processor pulls it in transitively; missing it is a startup `ImportError`. Always include it.
- `sentencepiece` if you see tokenizer-related ImportErrors at startup.
## Default load + LoRA (Turbo)
```python
import torch
from diffusers import Krea2Pipeline
pipe = Krea2Pipeline.from_pretrained("krea/Krea-2-Turbo", torch_dtype=torch.bfloat16).to("cuda")
pipe.transformer.load_lora_adapter("user/my-krea2-lora", weight_name="my_lora.safetensors")
pipe.transformer.set_adapters("default", weights=1.0)
# include the LoRA's trigger word(s) from its card
image = pipe(
"a deer grazing in a forest, <trigger words>",
num_inference_steps=8, guidance_scale=0.0,
generator=torch.Generator("cuda").manual_seed(0),
).images[0]
```
Krea 2 LoRAs load through the **transformer's** adapter API (`pipe.transformer.load_lora_adapter(...)` + `pipe.transformer.set_adapters("default", weights=1.0)`), per the official LoRA cards — not the pipeline-level `pipe.load_lora_weights`. Honor the LoRA's **trigger word(s)** and recommended weight (1.0 by default).
## Inference recipe
- **Turbo (the demo default):** `num_inference_steps=8`, **`guidance_scale=0.0`** (guidance disabled), LoRA weight `1.0`.
- **RAW:** training only — don't ship a RAW inference demo; its quality is intentionally low (it's the malleable base you fine-tune on, then run on Turbo).
## `guidance_scale` convention (gotcha)
Krea 2 enables guidance whenever **`guidance_scale > 0`** and computes velocity as `cond + guidance_scale * (cond − uncond)` (≡ the usual CFG formulation with scale `1 + guidance_scale`). So Turbo disables guidance with **`guidance_scale=0.0`** — not `1.0`.
## Resolution
`height`/`width` must be divisible by **16** (`vae_scale_factor * patch_size`); the pipeline rounds up to a multiple of 16 (with a warning) otherwise. Default 1024×1024.
## ZeroGPU duration
Standard T2I: place modules at module scope, `pipe.to("cuda")`, no `torch.compile`. Turbo 8-step 1024² is fast (≈ 20–40s) — set `@spaces.GPU(duration=...)` comfortably above that. The 12B DiT + Qwen3-VL encoder fits ZeroGPU; if you OOM on the default size, try `@spaces.GPU(size="xlarge")`.
## Things to watch
- **Load Turbo, not RAW, for demos.** RAW is the training base and not meant for inference.
- **`guidance_scale=0` disables guidance** (Krea convention, unlike pipelines where 1.0 is "off"). Turbo = 8 steps, `guidance_scale=0.0`.
- **LoRAs use `pipe.transformer.load_lora_adapter` + `set_adapters("default", …)`** (transformer-level), and have **trigger words** — read the card.
- **New pipeline → update diffusers** if `from diffusers import Krea2Pipeline` fails.
antigravity-awesome-skills/skills/huggingface-lora-space-builder/references/base-models/ltx.md
+205
View File
@@ -0,0 +1,205 @@
# LTX reference
The LTX family covers `LTX-Video` (0.9.x series), `LTX-2`, and `LTX-2.3`. All target text-to-video, image-to-video, and video-to-video.
Diffusers support varies across versions:
| Base model series | Diffusers support | LoRA loading via diffusers? |
|---------------------------|------------------------------------------|-----------------------------|
| `Lightricks/LTX-Video` (0.9.x) | Yes — `LTXPipeline`, `LTXImageToVideoPipeline`, `LTXConditionPipeline` | Yes |
| `Lightricks/LTX-2` | Yes — `LTX2Pipeline` from `diffusers.pipelines.ltx2` | Yes (recent diffusers) |
| `Lightricks/LTX-2.3` | Partial — diffusers support is still limited overall, so the original `Lightricks/LTX-2.3` repo can still be used via the native path. Diffusers-converted variants exist at `dg845/LTX-2.3-Diffusers` and `dg845/LTX-2.3-Distilled-Diffusers`, supporting regular LoRAs via the standard pipelines, IC LoRAs via `LTX2InContextPipeline`, and HDR IC-LoRAs via `LTX2HDRPipeline` (PR #13572, merged 2026-05-15; needs `git+https://github.com/huggingface/diffusers`) | Yes on the diffusers variants — standard `load_lora_weights` + `set_adapters`. Native path still required for some configurations. |
When in doubt about LTX-2.3, check the LoRA's model card for an example snippet. If it imports from `ltx_video` or a native module rather than `diffusers`, use the native path.
## Pipelines (diffusers path)
> **Before using this table, verify against the base model's own card on the Hub.** This table is best-effort and can lag a recent release (LTX moves fast). The diffusers snippet on the base model's Hub page is source of truth. See `SKILL.md` Phase 2 for the procedure.
| Task | Pipeline | Pretrained ID |
|-------------------------|-----------------------------------|------------------------------------------------------|
| Text-to-video | `LTXPipeline` | `Lightricks/LTX-Video` |
| Image-to-video | `LTXImageToVideoPipeline` | `Lightricks/LTX-Video` |
| Video-to-video / keyframes | `LTXConditionPipeline` | `Lightricks/LTX-Video-0.9.5` or later |
| Spatial upscale | `LTXLatentUpsamplePipeline` | `Lightricks/ltxv-spatial-upscaler-0.9.8` |
| LTX-2 T2V/I2V | `LTX2Pipeline` | `Lightricks/LTX-2` |
| LTX-2.3 in-context (IC LoRAs) | `LTX2InContextPipeline` | `dg845/LTX-2.3-Distilled-Diffusers` (preferred for demos; non-distilled `dg845/LTX-2.3-Diffusers` also works) |
`LTXConditionPipeline` is the workhorse for V2V — it takes a `LTXVideoCondition` (or list of them) plus optional first-frame image, and produces a video conditioned on the input.
## Required dependencies
LTX pipelines need extras beyond the standard diffusers/transformers/peft set, because video output requires file-format support:
- **`imageio`** and **`imageio-ffmpeg`** — required by `diffusers.utils.export_to_video`. Without them, video export fails at runtime even though model loading succeeds. Always include both.
- **`sentencepiece`** — required by the T5 text encoder some LTX variants use. Include if you see tokenizer-related ImportErrors at startup.
- **`av`** — `pyav`, useful when reading input videos in non-trivial formats. Include for V2V pipelines that take video input via `load_video`.
For LTX-2 and LTX-2.3, the latest diffusers from git is often required since pipeline classes (`LTX2Pipeline`, conditioning APIs) land before pip releases:
```
git+https://github.com/huggingface/diffusers
```
For LTX-2.3 native path:
```
git+https://github.com/Lightricks/LTX-Video.git
```
If `from_pretrained` fails with class-not-found errors for a recent LTX variant, switch to git diffusers.
## Default load (T2V)
```python
import torch
from diffusers import LTXPipeline
from diffusers.utils import export_to_video
pipe = LTXPipeline.from_pretrained(
"Lightricks/LTX-Video",
torch_dtype=torch.bfloat16,
)
pipe.to("cuda")
pipe.load_lora_weights("user/my-ltx-lora")
```
## Default load (V2V via LTXConditionPipeline)
```python
import torch
from diffusers import LTXConditionPipeline
from diffusers.pipelines.ltx.pipeline_ltx_condition import LTXVideoCondition
from diffusers.utils import load_video, export_to_video
pipe = LTXConditionPipeline.from_pretrained(
"Lightricks/LTX-Video-0.9.5",
torch_dtype=torch.bfloat16,
)
pipe.to("cuda")
pipe.load_lora_weights("user/my-ltx-vlora")
video = load_video("input.mp4")
condition = LTXVideoCondition(video=video, frame_index=0)
frames = pipe(
conditions=[condition],
prompt="...",
negative_prompt="worst quality, jittery, blurry",
width=768, height=512,
num_frames=121,
num_inference_steps=50,
).frames[0]
export_to_video(frames, "out.mp4", fps=24)
```
## LTX-2 (diffusers path)
```python
from diffusers.pipelines.ltx2 import LTX2Pipeline
pipe = LTX2Pipeline.from_pretrained("Lightricks/LTX-2", torch_dtype=torch.bfloat16)
```
LTX-2 supports a two-stage pipeline (base + latent upsample) for production quality. For a demo, the single-stage pipeline is usually sufficient and faster.
**Don't default two-stage on for IC-LoRAs — check the card.** The common LTX-2.3 two-stage recipe runs stage 2 as an x2 latent-upsample + refine **with the IC-LoRA disabled** (`disable_lora()`), re-rendering on the bare base — which re-degrades exactly the LoRA-conditioned detail (text, identity). Whether two-stage is appropriate is per-LoRA, per the card: e.g. the *In/Out-painting* card explicitly says "both tasks use a two-stage pipeline," whereas the *Ingredients* card recommends a single-pass 30-step recipe and never mentions two stages. Single-stage is the safer demo default unless the card calls for two-stage.
## Inference defaults
- **Resolution**: typically multiples of 32. Common sizes: 768×512, 1216×704, 704×480.
- **Frame count**: typically `8k+1` (e.g. 121, 161, 257). Compute from `duration_seconds * fps` and round.
- **fps**: 24 by default; some LoRAs are trained at different rates (16, 30) — check the model card.
- **`num_inference_steps`**: 30–50 for non-distilled. Distilled checkpoints (look for "distilled" in the name) often run at 8–12.
- **`negative_prompt`**: LTX is sensitive to negative prompts. A good default: `"worst quality, inconsistent motion, blurry, jittery, distorted"`.
- **Distilled IC-LoRA — disable audio guidance too, not just `guidance_scale`.** `LTX2InContextPipeline` computes `do_classifier_free_guidance = guidance_scale > 1 OR audio_guidance_scale > 1`, and **`audio_guidance_scale` defaults to `7.0`**. So `guidance_scale=1.0` is not enough — CFG stays on via audio (and `stg_scale` defaults on), which doubles/mis-batches the forward against the in-context reference tokens (wrong recipe, sometimes a runtime error). For a distilled IC-LoRA pass all four: `pipe(..., guidance_scale=1.0, stg_scale=0.0, audio_guidance_scale=1.0, audio_stg_scale=0.0)`.
## Frame-count helper
```python
def num_frames_for_duration(seconds, fps=24, base=8):
raw = seconds * fps
return ((int(raw) - 1) // base) * base + 1
```
## Native pipeline path (LTX-2.3 fallback)
The native repo is a fallback when the diffusers path doesn't work, or for specific conditionings that diffusers may not support yet. Try `LTX2InContextPipeline` on the diffusers path first (see the IC-LoRAs section above).
**The native repo depends on the model generation** — pick by the LoRA's base model (and what its card's snippet imports):
| Base model | Native repo | Package(s) / import | Pipeline classes |
|---|---|---|---|
| `LTX-Video` (0.9.x) | `github.com/Lightricks/LTX-Video` | `ltx_video` | `from ltx_video.pipelines import LTXPipeline` |
| `LTX-2`, `LTX-2.3` | `github.com/Lightricks/LTX-2` | `ltx-core` + `ltx-pipelines` (editable) | `ltx_pipelines.ic_lora.ICLoraPipeline`, `TI2VidOneStagePipeline`, … |
For the 0.9.x series the pattern looks like:
```bash
# in requirements.txt
git+https://github.com/Lightricks/LTX-Video.git
```
```python
from ltx_video.pipelines import LTXPipeline as NativeLTXPipeline
# ... model loading per the native repo's README
```
The native path doesn't use `load_lora_weights`. Instead, LoRAs are usually wired in at pipeline construction time, often via a list of LoRA configurations or by pointing at a fused checkpoint.
When the LoRA's model card has a Python snippet using the native repo, copy its construction pattern verbatim. The native API changes more often than diffusers' does, so don't paraphrase.
### LTX-2.x native path on ZeroGPU — gotchas
For `LTX-2` / `LTX-2.3` (the `ltx-core` + `ltx-pipelines` repo), clone + `pip install -e packages/ltx-core packages/ltx-pipelines` at runtime in `app.py` and pin a commit. Three things bite on ZeroGPU:
- **Native loader bypasses ZeroGPU virtualization → "No CUDA GPUs available" at startup.** The native safetensors loader does `safe_open(path, device="cuda")` and copies host→device inside safetensors' own C++ (`cudaMemcpy`), **bypassing `torch.Tensor.to`** — the call ZeroGPU patches to virtualize + pack module-scope weights. Nothing packs and module-scope placement raises *"No CUDA GPUs are available."* Fix: monkeypatch the loader to open on CPU then move with `.to`:
```python
with safetensors.safe_open(shard, framework="pt", device="cpu") as f:
value = f.get_tensor(name).to(device=device) # torch path → ZeroGPU-virtualisable
```
Also: patch attention to **SDPA** (FA3 crashes on Blackwell ZeroGPU), and **never call `torch.cuda.*` / `get_device_capability()` at module scope** (it poisons virtualization).
- **Native two-stage pins two 22B transformers → offload-disk overflow.** `ICLoraPipeline` builds two independent `ModelLedger`s (stage-1 with the LoRA, stage-2 without), each loading its **own** full transformer. The CLI loads them sequentially, but ZeroGPU pins all weights at module scope, so enabling stage 2 pins **both** (~143G) and overflows the offload disk (`OSError: [Errno 28] No space left on device`, ~96G cap). For a demo: pin stage 1, then have stage 2 **reuse stage 1's pinned modules** (`for n in [...]: setattr(s2, n, getattr(s1, n))`) so only one transformer is resident (~71G).
- **`ICLoraPipeline` is distilled-only.** Per the LTX-2 `ltx-pipelines` guide, IC-LoRA inference runs in distilled-only mode (fixed 8-step sigmas, no `num_inference_steps`/`guidance_scale`/`negative_prompt`); the docs do not describe non-distilled IC-LoRA or IC-LoRA + CFG/STG (guidance lives only on full-model pipelines like `TI2VidOneStagePipeline`, and in two-stage is stage-1-only). So if a card recommends a *non-distilled* recipe (e.g. 30 steps + guidance + STG on a `…-dev` base), there is no stock IC-LoRA pipeline for it — use the supported distilled `ICLoraPipeline` and note the recipe difference, or fall back to the diffusers `LTX2InContextPipeline` (which exposes steps/guidance).
## IC-LoRAs (in-context conditioning)
LTX IC-LoRAs condition the model on a reference video alongside a first-frame image. The diffusers path uses `LTX2InContextPipeline` with standard `load_lora_weights` + `set_adapters`.
Common flavors:
- **Pose / depth / canny IC-LoRAs**: the reference video must be preprocessed into the control signal (pose skeletons, depth maps, edge maps) before being passed as conditioning. Passing a raw video with appearance information leaks color/style through.
- **Outpaint IC-LoRAs**: the input video is padded with black margins to a target aspect ratio and passed as conditioning. The model fills the black regions.
- **Frame interpolation / extension IC-LoRAs**: the input is a sparse set of keyframes; the model fills in between or extends.
- **Audio-driven lip-sync IC-LoRAs**: take a reference video and a new audio track; the model re-syncs lip motion to match the new audio. UI needs both a video input and an audio input (e.g. for video translation, voiceover replacement, multilingual dubbing).
Each of these implies different preprocessing in the demo and different UI shape (an aspect-ratio picker for outpaint, a pose preview for pose-control, a frame-pair input for keyframe extension). Read `references/adapting-to-the-lora.md` and shape the UI to the specific IC-LoRA.
## Quantization gotcha (LTX-2.3, native path only)
On the native path, some LTX-2.3 IC-LoRAs ship with FP8 quantization policies that fuse LoRA deltas into transformer weights at model-load time, using a Triton CUDA kernel. ZeroGPU has no real CUDA at module-load time (only the emulation layer) — this can crash the Space at startup. The diffusers path via `LTX2InContextPipeline` does not hit this: LoRAs are loaded through PEFT as separate adapter weights and applied at runtime rather than fused into the base transformer at load.
Two workarounds when the user hits this on the native path:
- **Skip quantization for the LoRA-fusion stage.** LTX-2.3 has a two-stage pipeline; apply quantization only to the second (non-LoRA) stage. Set `stage_1_quantization=None`.
- **Pre-fuse the LoRA into a standalone checkpoint.** Download base + LoRA, fuse on a dev GPU, push the fused checkpoint to the Hub under the user's namespace, and have the Space load the fused checkpoint with no LoRA.
The second option is preferable when the user plans to demo the same LoRA repeatedly — Space startup is much faster.
## ZeroGPU duration guidance for LTX
- Short T2V (3 seconds, 24fps, distilled): 60–90 seconds.
- Standard T2V (5 seconds, 50 steps): 120–180 seconds.
- I2V: similar to T2V at the same duration.
- V2V with preprocessing (pose extraction, etc.): add 10–30 seconds for preprocessing overhead. Set duration to 180+.
- LTX-2.3 two-stage: 240–360 seconds.
Set `@spaces.GPU(duration=...)` to comfortably exceed expected generation time.
## Things to watch
- **Latest pipelines often need git diffusers.** Pin to `git+https://github.com/huggingface/diffusers` in `requirements.txt` for any LTX variant released in the last few weeks.
- **`git+diffusers` is a moving target — verify LTX2 output quality.** Bare `@main` inherits whatever's there. The LTX2 text connector carried a token-reversal regression (PR #13564) that scrambled prompt tokens/registers — degrading prompt adherence and fine detail (e.g. garbled on-screen text), worst on short prompts — until **PR #13931 (merged 2026-06-19)** fixed it. If LTX2 output looks weak or garbled, confirm the installed diffusers commit is **at or past #13931**, and prefer pinning to a known-good commit (`git+https://github.com/huggingface/diffusers@<sha>`) over bare `@main` for reproducibility.
- **Don't `torch.compile`.** LTX is fast enough on ZeroGPU without it; compile is incompatible anyway.
- **`enable_vae_tiling()` for higher resolutions.** LTX VAE memory grows with resolution; enable tiling for outputs above 768px on either axis.
- **Negative prompts matter more than for image models.** Don't ship with an empty negative_prompt unless the LoRA's model card says so.
- **Frame-rate mismatch can produce glitches.** If the LoRA was trained at 24 fps and the demo passes 30, motion looks wrong. Use the LoRA's recommended fps.
antigravity-awesome-skills/skills/huggingface-lora-space-builder/references/base-models/qwen-image.md
+144
View File
@@ -0,0 +1,144 @@
# Qwen-Image and Qwen-Image-Edit reference
The Qwen-Image family is fully supported in `diffusers`. Both base and edit variants accept LoRAs via the standard `load_lora_weights` interface.
## Pipelines
> **Before using this table, verify against the base model's own card on the Hub.** This table is best-effort and can lag a recent release. The diffusers snippet on the base model's Hub page is source of truth for which pipeline class to import. See `SKILL.md` Phase 2 for the procedure.
| Base model | Pipeline class | Task |
|---------------------------------------|-------------------------------|-------------------------------------|
| `Qwen/Qwen-Image` | `QwenImagePipeline` | Text-to-image |
| `Qwen/Qwen-Image-Edit` | `QwenImageEditPipeline` | Image editing (instruction-driven) |
| `Qwen/Qwen-Image-Edit-2509` | `QwenImageEditPlusPipeline` | Image editing, multi-image input |
| `Qwen/Qwen-Image-Edit-2511` | `QwenImageEditPlusPipeline` | Image editing, latest variant |
The 2509 and 2511 variants use a *different* pipeline class than the original `QwenImageEditPipeline` — they take a list of input images and have different default parameters. Don't assume that variants in the same family share a pipeline class. Loading a 2511-trained LoRA onto `QwenImageEditPipeline` produces broken output; the failure is silent (no exception), so verifying against the base model card is the only way to catch it.
The 2511 variant integrates several popular community LoRAs into the base, which can mean a LoRA trained against earlier Qwen-Image-Edit may behave subtly differently when loaded against 2511; if the LoRA's model card specifies which Edit variant it was trained on, match it.
## Required dependencies
Qwen-Image and Qwen-Image-Edit pipelines need extras beyond the standard diffusers/transformers/peft set, because the text encoder is `Qwen2_5_VLForConditionalGeneration` (Qwen 2.5-VL):
- **`torchvision`** — required by `Qwen2VLVideoProcessor`, which the text encoder's processor pulls in transitively. Missing this is a startup-time `ImportError` ("Qwen2VLVideoProcessor requires the Torchvision library"). Always include in `requirements.txt` for any Qwen-Image Space.
- **`sentencepiece`** — required by some Qwen tokenizer paths. Include if you see tokenizer-related ImportErrors at startup.
The 2511 variant in particular often requires the latest `diffusers` from git, since `QwenImageEditPlusPipeline` and 2511-specific fixes land before pip releases:
```
git+https://github.com/huggingface/diffusers
```
If `from_pretrained("Qwen/Qwen-Image-Edit-2511", ...)` fails with a class-not-found or attribute error, switch the requirement to git.
## Default load (T2I)
```python
import torch
from diffusers import QwenImagePipeline
pipe = QwenImagePipeline.from_pretrained(
"Qwen/Qwen-Image",
torch_dtype=torch.bfloat16,
)
pipe.to("cuda")
pipe.load_lora_weights("user/my-qwen-lora")
```
`pytorch_lora_weights.safetensors` is the conventional filename. If the repo has a different name, pass `weight_name="..."`.
For multiple adapters or when you want to control LoRA scale at inference time, use `set_adapters`:
```python
pipe.load_lora_weights("user/my-qwen-lora", adapter_name="mylora")
pipe.set_adapters(["mylora"], adapter_weights=[0.9])
```
## Default load (Image Edit)
For original `Qwen-Image-Edit`:
```python
import torch
from diffusers import QwenImageEditPipeline
pipe = QwenImageEditPipeline.from_pretrained(
"Qwen/Qwen-Image-Edit",
torch_dtype=torch.bfloat16,
)
pipe.to("cuda")
pipe.load_lora_weights("user/my-qwen-edit-lora")
```
For `Qwen-Image-Edit-2509` and `Qwen-Image-Edit-2511`:
```python
import torch
from diffusers import QwenImageEditPlusPipeline
pipe = QwenImageEditPlusPipeline.from_pretrained(
"Qwen/Qwen-Image-Edit-2511", # or 2509
torch_dtype=torch.bfloat16,
)
pipe.to("cuda")
pipe.load_lora_weights("user/my-qwen-edit-lora")
```
`QwenImageEditPlusPipeline` accepts `image=<PIL>` or `image=[<PIL>, <PIL>, ...]` for multi-image edits. `QwenImageEditPipeline` accepts a single image. Default parameters differ slightly between the two — see "Inference defaults" below.
## Inference defaults
For non-distilled Qwen-Image:
- `num_inference_steps`: 50 by default; the LoRA's model card may recommend lower.
- `true_cfg_scale`: typical 4.0.
- `width`/`height`: multiples of 16, ideally 1024 or 1328 along the long axis.
For Qwen-Image-Edit (original):
- `num_inference_steps`: 30–50 typical, often less for distilled variants.
- `true_cfg_scale`: 4.0 typical.
- Input image gets internally resized; passing a reasonable resolution (1024px on the long side) is fine.
For Qwen-Image-Edit-2509 / 2511 (`QwenImageEditPlusPipeline`):
- `num_inference_steps`: 40 typical for 2511; 50 for 2509.
- `true_cfg_scale`: 4.0.
- `guidance_scale`: 1.0 (the new pipeline uses `true_cfg_scale` as the active CFG; standard `guidance_scale` is kept at 1.0).
- Input is a list of one or more PIL images.
For Lightning / few-step LoRAs (e.g. `lightx2v/Qwen-Image-Lightning-*`):
- `num_inference_steps`: 4 or 8 (read the LoRA's model card — they ship 4-step and 8-step variants).
- `true_cfg_scale`: usually 1.0 (CFG disabled).
- Often comes with a custom scheduler config — see the lightx2v README for the exact `FlowMatchEulerDiscreteScheduler` config to use.
## Resolution buckets
Qwen-Image uses 16-pixel-aligned resolutions. When the user picks an aspect ratio, compute `width` and `height` as multiples of 16. A helper:
```python
def round_to_bucket(w, h, multiple=16):
return (w // multiple) * multiple, (h // multiple) * multiple
```
For image-edit pipelines, resize the input image to the nearest bucket while preserving aspect; don't crop.
## ZeroGPU duration guidance
- Standard 50-step Qwen-Image T2I at 1024×1024: 60–90 seconds.
- 4-step Lightning Qwen-Image: 15–25 seconds.
- Qwen-Image-Edit 30 steps: 60–90 seconds.
Set `@spaces.GPU(duration=...)` accordingly.
## Common LoRA patterns on Qwen-Image
- **Style LoRAs (T2I).** Standard load. Trigger word usually present. UI: prompt + aspect ratio.
- **Subject / character LoRAs (T2I).** Standard load. Trigger word almost always present. UI: prompt with the trigger pre-prepended in code, possibly an example prompt highlighting the trigger.
- **Lighting / aesthetic LoRAs (T2I).** Often paired with a recommended LoRA scale ≠ 1.0 — check the model card.
- **Edit LoRAs (image-to-image, on Qwen-Image-Edit).** Specific instructions baked in. The LoRA might require a specific instruction phrasing — match the pattern from the model card. UI: input image + instruction textbox.
- **Lightning-distilled LoRAs.** Lock step count and CFG to recommended values; hide the sliders.
## Things to watch
- **VAE memory.** For 1328×1328 outputs, consider `pipe.enable_vae_tiling()` and `pipe.enable_vae_slicing()` after loading. Keep them off for smaller resolutions to avoid quality loss.
- **Don't compile the transformer on ZeroGPU.** `torch.compile` won't work; the speedup options on ZeroGPU are limited to reducing steps or using FP8-distilled variants.
- **Negative prompts work** but defaults are often empty string. Don't expose a negative prompt in the UI unless the LoRA's behavior actually benefits from it.