📦 deps(thirdparty): update snapshots

antigravity-awesome-skills/skills/train-sentence-transformers/references/losses_sparse_encoder.md

@@ -0,0 +1,106 @@
+# Sparse-Encoder Losses (SPLADE)
+
+All losses live in `sentence_transformers.sparse_encoder.losses`.
+
+This reference targets the **SPLADE** architecture (Transformer + SpladePooling). The sparse-encoder package also exports `CSRLoss` and `CSRReconstructionLoss` for the CSR architecture (Transformer + Pooling + SparseAutoEncoder); those are out of scope here — see the sbert.net docs if you're training a CSR model.
+
+Choosing a loss means (a) pick a base loss (contrastive, regression, distillation) and (b) wrap it in `SpladeLoss` to add FLOPS regularization.
+
+## Top-line decision table
+
+| You have | Use |
+|---|---|
+| `(anchor, positive)` or triplet, SPLADE architecture | `SpladeLoss(loss=SparseMultipleNegativesRankingLoss(model), ...)` |
+| Same, want effective batch size of 256+ | `CachedSpladeLoss(...)` |
+| `(text1, text2, score)` labeled pairs | `SparseCoSENTLoss` or `SparseCosineSimilarityLoss` |
+| Distillation from cross-encoder teacher | `SparseMarginMSELoss` |
+| Listwise distillation | `SparseDistillKLDivLoss` |
+| Explicit triplet | `SparseTripletLoss` |
+
+## The core wrapper: `SpladeLoss`
+
+`SpladeLoss` adds **FLOPS regularization** on top of another sparse loss. FLOPS regularization penalizes non-zero activations, keeping embeddings genuinely sparse.
+
+```python
+loss = SpladeLoss(
+    model=model,
+    loss=SparseMultipleNegativesRankingLoss(model=model),
+    query_regularizer_weight=5e-5,
+    document_regularizer_weight=3e-5,
+)
+```
+
+- `query_regularizer_weight`: how much to penalize non-zero terms in query embeddings.
+- `document_regularizer_weight`: same for documents.
+- Typical range: 1e-5 to 1e-4. Higher = sparser embeddings, lower recall; lower = denser, possibly better recall.
+- `SparseEncoderTrainer` automatically registers a `SpladeRegularizerWeightSchedulerCallback` whenever the loss is a `SpladeLoss`. The callback ramps the weights from 0 up to the target over the first ~33% of training; the default shape is `SchedulerType.QUADRATIC` (not linear). The ramp length and shape are configured on the callback (`SpladeRegularizerWeightSchedulerCallback(loss=..., warmup_ratio=..., scheduler_type=...)`), not on `SpladeLoss`; to override, instantiate the callback yourself and pass it via `callbacks=[...]`. This ramp is important; starting with full regularization from step 0 kills learning.
+
+Use `CachedSpladeLoss` for the GradCache variant.
+
+## Contrastive losses (no labels)
+
+### `SparseMultipleNegativesRankingLoss`
+
+Sparse analog of bi-encoder MNRL. In-batch contrastive.
+
+```python
+inner = SparseMultipleNegativesRankingLoss(model=model)
+loss = SpladeLoss(model=model, loss=inner, query_regularizer_weight=5e-5, document_regularizer_weight=3e-5)
+```
+
+- **Always wrap in `SpladeLoss`** for SPLADE architectures.
+- Set `batch_sampler=BatchSamplers.NO_DUPLICATES` on training args.
+
+### `SparseTripletLoss`
+
+Classic triplet margin loss on explicit `(anchor, positive, negative)`.
+
+## Labeled regression losses
+
+### `SparseCoSENTLoss`
+
+Pairwise ranking loss for `(text1, text2, score)`. Mirrors bi-encoder `CoSENTLoss`.
+
+### `SparseCosineSimilarityLoss`
+
+MSE on cosine similarity. Simpler, usually worse than CoSENT.
+
+### `SparseAnglELoss`
+
+Angle-based loss in complex space. Alternative to CoSENT.
+
+## Distillation losses
+
+### `SparseMSELoss`
+
+Embedding MSE. Student sparse embedding should match teacher embedding.
+
+- **Data**: `(text, teacher_embedding)`.
+- Teacher can be a dense bi-encoder or another sparse model.
+
+### `SparseMarginMSELoss`
+
+Margin MSE from a cross-encoder teacher.
+
+- **Data**: `(query, positive, negative, score_diff)` where `score_diff = teacher_score(query, positive) - teacher_score(query, negative)`.
+- Typical recipe for training SPLADE from cross-encoder labels (ms-marco distillation).
+- Wrap in `SpladeLoss(model, loss=SparseMarginMSELoss(model), ...)` for SPLADE.
+
+### `SparseDistillKLDivLoss`
+
+Listwise KL-div distillation — student's softmax distribution over candidates should match teacher's.
+
+## Independent regularizer
+
+### `FlopsLoss`
+
+Standalone FLOPS regularizer. Usually you use this via `SpladeLoss`, not directly.
+
+For regularizer-weight tuning and dense-output recovery, see `troubleshooting.md` ("SPLADE embeddings are dense"). MLM-head requirement: `base_model_selection.md` (SPARSE section). Active-dim sparsity targets and how to monitor them: `evaluators_sparse_encoder.md` (Sparsity tracking).
+
+## Gotchas
+
+- **`SparseMultipleNegativesRankingLoss` without `SpladeLoss` wrapping on a SPLADE model**: no FLOPS regularization -> dense outputs defeating the purpose of SPLADE. Always wrap.
+- **`CachedSpladeLoss` + `gradient_checkpointing=True`**: crash. Pick one.
+- **Starting training with full FLOPS regularization at step 0**: the model outputs zero everywhere and gets stuck. The built-in scheduler avoids this — don't override it unless you know why.
+- **`query_regularizer_weight` == `document_regularizer_weight`**: usually wrong. Queries should be sparser than documents (fewer terms per query). Since higher regularization drives more zeros, give the query weight the larger value. `query_regularizer_weight=5e-5`, `document_regularizer_weight=3e-5` is a good starting ratio.