📦 deps(thirdparty): update snapshots
This commit is contained in:
ci[bot]
@@ -0,0 +1,269 @@
|
||||
---
|
||||
name: n8n-binary-and-data
|
||||
description: Handle n8n files and binary data across uploads, downloads, transforms, multimodal inputs, agent tools, and chat surfaces.
|
||||
risk: unknown
|
||||
source: https://github.com/czlonkowski/n8n-skills/tree/main/skills/n8n-binary-and-data
|
||||
source_repo: czlonkowski/n8n-skills
|
||||
source_type: community
|
||||
date_added: "2026-07-21"
|
||||
author: Romuald Czlonkowski
|
||||
license: MIT
|
||||
license_source: https://github.com/czlonkowski/n8n-skills/blob/main/LICENSE
|
||||
---
|
||||
|
||||
# n8n Binary and Data
|
||||
|
||||
## When to Use
|
||||
|
||||
Use this skill when an n8n workflow reads, transforms, stores, uploads, downloads, or transmits files and binary fields, including multimodal agent inputs and chat attachments.
|
||||
|
||||
Treat uploaded files and generated URLs as potentially sensitive. Obtain approval before sending data to a new external host, use the narrowest retention and access scope available, avoid logging bytes or base64 payloads, and do not embed credentials in URLs or workflow fields.
|
||||
|
||||
Every n8n item carries two independent slots: `$json` for structured data and `$binary` for file bytes. They travel side by side through the workflow. File contents — the actual PDF, image, or zip — live in `$binary`, never in `$json`. Get that split wrong and you read an empty field, lose a file mid-flow, or hand an AI agent a tool input it can't use.
|
||||
|
||||
This skill covers where binary lives, how to read and write it, how to keep it from being silently stripped, the hard wall between binary and the AI-agent tool boundary, and why chat surfaces need a URL instead of raw bytes.
|
||||
|
||||
---
|
||||
|
||||
## The three rules that prevent 90% of binary bugs
|
||||
|
||||
1. **File contents are in `$binary`, not `$json`.** After an HTTP download, a "Read Files", or an email-attachment trigger, the bytes sit in `$binary.<key>`. `$json` holds metadata at most. Reading `$json.data` for file contents gives you nothing.
|
||||
|
||||
2. **Binary cannot cross the AI-agent tool boundary — in either direction.** Tool arguments and tool return values are JSON only. An uploaded image can't be passed into a tool as a file, and a tool can't return raw bytes. Pre-stage to storage and pass a key or URL through JSON instead. See `references/AGENT_TOOL_BINARY.md`.
|
||||
|
||||
3. **Chat surfaces render images by URL, not by `$binary`.** Slack, Discord, Teams, Telegram, embedded webhook chat — none of them read the binary slot. The image has to live somewhere a URL can fetch it. See `references/CDN_REQUIREMENT.md`.
|
||||
|
||||
---
|
||||
|
||||
## The two slots
|
||||
|
||||
Each item is shaped like this:
|
||||
|
||||
```json
|
||||
{
|
||||
"json": { "customerId": 42, "status": "sent" },
|
||||
"binary": {
|
||||
"invoice": {
|
||||
"data": "<base64-encoded bytes>",
|
||||
"mimeType": "application/pdf",
|
||||
"fileName": "invoice-42.pdf",
|
||||
"fileExtension": "pdf"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The key inside `binary` (`invoice` here) is the **binary property name**. Most file-handling nodes have a `binaryPropertyName` parameter that points at it — the producer names the slot, the consumer references it by that name. The default key across most nodes is `data`, so when nothing tells you otherwise, assume `$binary.data`.
|
||||
|
||||
`$json` and `$binary` are separate namespaces. An expression like `{{ $binary.invoice.fileName }}` reads file metadata; `{{ $json.customerId }}` reads data. They never mix.
|
||||
|
||||
This split also explains a webhook gotcha: a Webhook trigger receiving `multipart/form-data` puts the uploaded file in `$binary` and the accompanying form fields in `$json.body` — so an uploaded file is not somewhere under `$json` at all. (The `$json.body` nesting for webhooks is **n8n-expression-syntax** territory.)
|
||||
|
||||
See `references/BINARY_BASICS.md` for the full slot anatomy, mime types, and size limits.
|
||||
|
||||
---
|
||||
|
||||
## Producing binary
|
||||
|
||||
You rarely build a `$binary` slot by hand — nodes populate it for you:
|
||||
|
||||
| Source | How binary appears |
|
||||
|---|---|
|
||||
| HTTP Request with `responseFormat: "file"` | Response body lands in `$binary.data` (or the name you set) |
|
||||
| Read/Write Files from Disk | File contents read into `$binary` |
|
||||
| Storage downloads (S3, Google Drive, Dropbox, etc.) | Downloaded file in `$binary.<key>` |
|
||||
| Email triggers with attachments | Each attachment arrives in `$binary` |
|
||||
| Provider AI media nodes (image/audio gen) | Set `options.binaryPropertyOutput` so the bytes land where the next node looks |
|
||||
|
||||
For an HTTP download, the one field that matters is `responseFormat`. Confirm it with `get_node` on `nodes-base.httpRequest` — leaving it as the default JSON/string format is the classic reason a downloaded file ends up as garbled text in `$json` instead of clean bytes in `$binary`.
|
||||
|
||||
---
|
||||
|
||||
## Reading and writing binary in a Code node
|
||||
|
||||
Most workflows never need to crack open the bytes — they just pass binary through to a consumer (email attachment, file upload, Slack file). When you do need the raw bytes, do it in a Code node.
|
||||
|
||||
**Read** with `getBinaryDataBuffer` — do not try to base64-decode `$binary.<key>.data` by hand:
|
||||
|
||||
```javascript
|
||||
// Code node, "Run Once for Each Item"
|
||||
const buffer = await this.helpers.getBinaryDataBuffer(0, 'data'); // (itemIndex, propertyName)
|
||||
const text = buffer.toString('utf-8');
|
||||
const length = buffer.length;
|
||||
|
||||
return [{
|
||||
json: { ...$json, length },
|
||||
binary: $input.item.binary, // pass the binary through, or it's gone
|
||||
}];
|
||||
```
|
||||
|
||||
**Write** by building the slot yourself — base64 the bytes plus a mime type and file name:
|
||||
|
||||
```javascript
|
||||
const text = 'Hello, world!';
|
||||
return [{
|
||||
json: { ok: true },
|
||||
binary: {
|
||||
report: {
|
||||
data: Buffer.from(text).toString('base64'),
|
||||
mimeType: 'text/plain',
|
||||
fileName: 'report.txt',
|
||||
fileExtension: 'txt',
|
||||
},
|
||||
},
|
||||
}];
|
||||
```
|
||||
|
||||
The Code-node sandbox, helpers, and execution modes are the domain of **n8n-code-javascript** (and **n8n-code-python**) — use those for the language-level detail. The one binary-specific thing to remember here: a Code node that returns `[{ json: {...} }]` without re-attaching `binary` **silently drops the file**. See `references/BINARY_BASICS.md`.
|
||||
|
||||
---
|
||||
|
||||
## Keeping binary alive across transforms
|
||||
|
||||
JSON-only nodes — Edit Fields (Set), Code, IF, and others — can drop the `$binary` slot from their output. The workflow validates clean and runs without error; the file just isn't there downstream when the email node goes to attach it.
|
||||
|
||||
Two ways to keep it:
|
||||
|
||||
- **Pass-through option on the transforming node.** Edit Fields has `includeOtherFields`; a Code node can return `binary: $input.item.binary` explicitly. Cheapest fix when it's available.
|
||||
- **Fan out and Merge by position.** Route the source into both the transform and a bypass branch, then recombine with a Merge in `combineByPosition` mode. The JSON comes from the transform side, the binary survives on the bypass side.
|
||||
|
||||
```
|
||||
[Source with binary] ─┬─→ [Edit Fields: change JSON] ─┐
|
||||
│ (binary stripped here) ├─→ [Merge: combineByPosition] ─→ [Email: attach]
|
||||
└──────────────────────────────────┘
|
||||
(bypass — binary passes through untouched)
|
||||
```
|
||||
|
||||
`combineByPosition` pairs item N from each input, so the field counts must line up. The connection wiring and the alternatives for many-strip-point chains (upload-early, sub-workflow) are in `references/MERGE_FOR_CONTEXT.md`.
|
||||
|
||||
---
|
||||
|
||||
## The agent-tool binary boundary
|
||||
|
||||
This is the sharpest edge. An AI Agent talks to its tools (Custom Code Tool, Call n8n Workflow Tool, HTTP Request Tool, MCP tools) over JSON. Binary does not fit through that pipe in either direction. The fix is the same shape both ways: **stage the bytes in storage, pass a key/URL through JSON, fetch on the other side.**
|
||||
|
||||
**Inbound — a user uploads a file the agent's tool must operate on:**
|
||||
|
||||
1. The chat trigger gives you a `files[]` array. Split it out and upload each file to private storage under a hashed key.
|
||||
2. Re-merge that branch before the agent runs (it's a synchronization barrier, not decoration), and set `executeOnce: true` on the agent so N files don't trigger N agent runs.
|
||||
3. Inject the keys into the agent's system prompt, listing both the original name (human context) and the storage key (what the tool needs), with an explicit "use EXACTLY this key".
|
||||
4. The tool receives the key as a string argument and downloads the file from storage itself.
|
||||
|
||||
**Outbound — a tool generates a file the agent must return:**
|
||||
|
||||
1. The tool sub-workflow generates the binary, uploads it to storage, and returns JSON like `{ "ok": true, "key": "...", "url": "https://...", "mimeType": "image/png" }`.
|
||||
2. The agent embeds the URL in its reply (or passes the key to another tool).
|
||||
|
||||
`passthroughBinaryImages: true` on the agent only changes what the **LLM sees** for vision — it does **not** let tools receive the file, and it's image-only (no PDFs, audio, or video). You still need the upload-and-pass-key pattern for any tool. Full patterns, hash strategy, storage choices, and the long-running-tool variant are in `references/AGENT_TOOL_BINARY.md`.
|
||||
|
||||
> Building the tool itself? See **n8n-code-tool** for the Custom Code Tool contract and **n8n-workflow-patterns** for the AI-Agent-with-tools shape.
|
||||
|
||||
---
|
||||
|
||||
## The CDN requirement for chat surfaces
|
||||
|
||||
When a workflow generates an image and the user wants it shown inside a chat message:
|
||||
|
||||
- **Binary on the item isn't enough.** The chat client renders messages that reference images by URL (or pushes bytes through the platform's own file-upload API). It never reads `$binary`.
|
||||
- **The bytes have to live somewhere a URL can fetch over HTTPS.** Upload to an object store or drive first, then embed the returned URL.
|
||||
- **n8n has no built-in CDN.** The user provides the storage.
|
||||
|
||||
Ask which storage they already use rather than defaulting to S3 — object storage (S3, R2, GCS, Azure Blob, Backblaze B2, Supabase Storage) and drive-style services (Dropbox, Google Drive, OneDrive, Box) all work and all change the URL shape. Cloudflare R2 is the lowest-friction starting point if they have nothing. For sensitive content, use a signed URL with an expiry rather than a permanently public one. See `references/CDN_REQUIREMENT.md`.
|
||||
|
||||
---
|
||||
|
||||
## What's NOT available
|
||||
|
||||
- **`$fromAI()` cannot carry binary.** It fills tool parameters with strings, numbers, booleans, and objects — never file bytes. Pass a storage key instead.
|
||||
- **Tool arguments and returns are JSON only.** There is no "binary parameter" on an agent tool, in or out.
|
||||
- **n8n ships no CDN or public file host.** Serving a file over a URL is always something the user's storage does, not n8n.
|
||||
- **`getBinaryDataBuffer` is a Code-node helper.** It isn't available in the Custom Code Tool sandbox (see **n8n-code-tool**).
|
||||
|
||||
---
|
||||
|
||||
## Where Data Tables live
|
||||
|
||||
For persistent tabular storage — reference-counting staged files, tracking which keys are live, dedup — that's the `n8n_manage_datatable` surface, owned by **n8n-mcp-tools-expert**. This skill does not cover Data Tables.
|
||||
|
||||
---
|
||||
|
||||
## Anti-patterns
|
||||
|
||||
| Anti-pattern | What goes wrong | Fix |
|
||||
|---|---|---|
|
||||
| Reading file contents from `$json` | Bytes live in `$binary`; `$json` is empty or metadata only | Read `$binary.<key>`, or `getBinaryDataBuffer` in a Code node |
|
||||
| HTTP download without `responseFormat: "file"` | Bytes arrive as mangled text in `$json`, not clean binary | Set `responseFormat: "file"` on the HTTP Request node |
|
||||
| Code node returns `[{json:{...}}]`, no `binary` | The file is silently dropped downstream | Re-attach `binary: $input.item.binary` in the return |
|
||||
| JSON transform (Edit Fields/IF) eats the binary | Email/upload node finds nothing to attach | Pass-through option, or fan out + Merge by position |
|
||||
| Passing an uploaded file into a tool via `$fromAI` | `$fromAI` can't carry binary; the tool gets nothing | Pre-stage to storage, inject the key in the system prompt, tool fetches by key |
|
||||
| Assuming `passthroughBinaryImages` lets tools see the file | It only affects what the LLM sees, and only for images | Still need the upload-and-pass-key pattern for tools |
|
||||
| Tool returns raw binary to the agent | Tool output is JSON; bytes don't survive (and bloat context) | Upload, return `{ key, url }` in JSON |
|
||||
| Posting `$binary` to a chat surface and expecting an image | Chat clients render by URL, not raw bytes | Upload to storage/CDN, embed the URL or use the platform file API |
|
||||
| Hardcoding base64 in a Code node | Huge workflow JSON, slow, leaky | Reference via `$binary`, or upload and reference by URL |
|
||||
|
||||
---
|
||||
|
||||
## Reference files
|
||||
|
||||
| File | Read when |
|
||||
|---|---|
|
||||
| `references/BINARY_BASICS.md` | First time handling binary, or reading/writing the `$binary` slot, mime types, size limits |
|
||||
| `references/AGENT_TOOL_BINARY.md` | An agent tool needs an uploaded file, or produces one — the boundary in either direction |
|
||||
| `references/MERGE_FOR_CONTEXT.md` | Binary disappears after a JSON transform and you need to re-attach it |
|
||||
| `references/CDN_REQUIREMENT.md` | Showing images in a chat surface or anywhere that needs URL-referenced images |
|
||||
|
||||
---
|
||||
|
||||
## Integration with Other Skills
|
||||
|
||||
**n8n-code-javascript / n8n-code-python**: the Code node is where you read/write raw bytes (`getBinaryDataBuffer`, `Buffer.from(...).toString('base64')`). Those skills own the sandbox, helpers, and execution-mode detail — this skill owns the rule that binary must be re-attached on return.
|
||||
|
||||
**n8n-code-tool**: the Custom Code Tool sandbox is narrower — no `$binary`, no `getBinaryDataBuffer`, no `$fromAI`. When a tool needs a file, this skill's storage-key pattern is how it gets one.
|
||||
|
||||
**n8n-workflow-patterns**: the agent-tool binary boundary sits inside the AI-Agent-with-tools pattern; the CDN flow is a generate → upload → reply chain.
|
||||
|
||||
**n8n-node-configuration**: `responseFormat`, `binaryPropertyName`, `includeOtherFields`, `binaryPropertyOutput` are all conditional fields — use `get_node` to confirm the exact names on the user's version.
|
||||
|
||||
**n8n-expression-syntax**: addressing `$binary.<key>.fileName` vs `$json.body` (webhook uploads in particular) is expression territory.
|
||||
|
||||
**n8n-validation-expert**: a dropped binary slot is a silent failure — `validate_workflow` won't flag it. Confirm presence by inspecting the execution.
|
||||
|
||||
**n8n-mcp-tools-expert**: owns `n8n_manage_datatable` (Data Tables) and `n8n_executions` — use the latter to confirm a `binary` slot actually survived a given node.
|
||||
|
||||
**n8n-error-handling**: storage uploads and downloads fail; the inbound/outbound staging steps need error branches so a missing key doesn't 404 silently.
|
||||
|
||||
**using-n8n-mcp-skills**: the index of how these skills fit together.
|
||||
|
||||
---
|
||||
|
||||
## Verifying binary survived
|
||||
|
||||
Validation won't catch a stripped binary slot — it's a silent failure. Confirm it ran correctly:
|
||||
|
||||
1. `n8n_test_workflow` (or trigger a real run) to produce an execution.
|
||||
2. `n8n_executions` to pull that execution, and inspect per-node output for the `binary` slot — it shows presence and metadata even if the base64 is too large to render.
|
||||
3. The node where `binary` last appears is the node before the strip. That's where the pass-through or Merge goes.
|
||||
|
||||
---
|
||||
|
||||
## Quick Reference Checklist
|
||||
|
||||
- [ ] File contents read from `$binary.<key>` — never `$json`
|
||||
- [ ] HTTP downloads use `responseFormat: "file"`
|
||||
- [ ] Code nodes re-attach `binary` on return when the file must continue
|
||||
- [ ] JSON transforms either pass binary through or Merge it back (`combineByPosition`)
|
||||
- [ ] No attempt to pass binary into/out of an agent tool — keys/URLs through JSON instead
|
||||
- [ ] `passthroughBinaryImages` used only for LLM vision, not as a tool channel
|
||||
- [ ] Chat-surface images uploaded to storage; the URL is embedded, not the bytes
|
||||
- [ ] Storage backend chosen with the user (not defaulted to S3); signed URLs for sensitive content
|
||||
- [ ] Binary presence confirmed by inspecting the execution, not by validation
|
||||
|
||||
---
|
||||
|
||||
**Remember**: two slots, side by side. Data rides in `$json`, files ride in `$binary` — and the moment a file has to cross an agent tool or reach a chat surface, it travels as a URL, not as bytes.
|
||||
|
||||
## Limitations
|
||||
|
||||
- Storage limits, binary modes, and node-specific field names vary across n8n versions and hosting configurations.
|
||||
- An n8n validation pass cannot prove that file bytes survived a live execution; inspect execution data with a safe sample.
|
||||
- This skill does not choose a storage provider or authorize uploading sensitive data to one.
|
||||
+227
@@ -0,0 +1,227 @@
|
||||
# Agent Tools and Binary
|
||||
|
||||
The hard wall: an AI Agent and its tools talk to each other in JSON. Binary doesn't fit through that pipe in either direction, and it catches people twice.
|
||||
|
||||
1. **Inbound** — a user uploads a file. The agent can *see* an image via vision, but tool calls don't carry the file.
|
||||
2. **Outbound** — a tool generates a file. Its result back to the agent is JSON, so it can't return raw bytes.
|
||||
|
||||
The workaround has the same shape both ways: **stage the bytes in storage, pass a key or URL through the JSON boundary, fetch on the other side.**
|
||||
|
||||
## Contents
|
||||
|
||||
- [Why the boundary exists](#why-the-boundary-exists)
|
||||
- [Inbound: an uploaded file into a tool](#inbound-an-uploaded-file-into-a-tool)
|
||||
- [The two pieces of plumbing that look optional](#the-two-pieces-of-plumbing-that-look-optional)
|
||||
- [What the system prompt and the tool argument look like](#what-the-system-prompt-and-the-tool-argument-look-like)
|
||||
- [passthroughBinaryImages](#passthroughbinaryimages)
|
||||
- [Outbound: a tool that produces a file](#outbound-a-tool-that-produces-a-file)
|
||||
- [Storage choices](#storage-choices)
|
||||
- [Hashing, cleanup, long-running tools](#hashing-cleanup-long-running-tools)
|
||||
- [Surface-specific seams](#surface-specific-seams)
|
||||
- [Common mistakes](#common-mistakes)
|
||||
|
||||
---
|
||||
|
||||
## Why the boundary exists
|
||||
|
||||
A tool call is a function call the LLM makes by emitting JSON arguments; the result comes back as a JSON observation. Tool parameters are filled by `$fromAI()`, which only produces strings, numbers, booleans, and objects — never file bytes. And a tool's return is a string/JSON the model reads as text. Base64-stuffing a 2 MB image into a JSON field would bloat every tool call and the agent's context window, and some runtimes reject oversized observations outright. So in practice: **binary never crosses the boundary.**
|
||||
|
||||
---
|
||||
|
||||
## Inbound: an uploaded file into a tool
|
||||
|
||||
The user pastes an image into chat. The chat trigger exposes a `files[]` array. If the agent only needs to *look* at the image, `passthroughBinaryImages: true` on the agent handles that (vision). But the moment a **tool** must operate on the file — OCR, image edit, document parse — the tool can't receive it directly. You pre-stage it.
|
||||
|
||||
```
|
||||
[Chat Trigger]
|
||||
│ files[]
|
||||
▼
|
||||
[IF: files empty?]
|
||||
├── empty ────────────────────────────────────────────► [AI Agent]
|
||||
└── not empty:
|
||||
[Split Out files]
|
||||
↓
|
||||
[Crypto: hash → storage key]
|
||||
↓
|
||||
[HTTP Request / S3 / Drive: upload to PRIVATE storage by key]
|
||||
↓
|
||||
[Merge: combineByPosition] ← synchronization barrier, see below
|
||||
↓
|
||||
[AI Agent] ← executeOnce: true; system prompt is told the keys
|
||||
│ tool call: imageKey = "sess12-abc123.png"
|
||||
▼
|
||||
[Call n8n Workflow Tool → sub-workflow]
|
||||
↓
|
||||
[Download from storage by key]
|
||||
↓
|
||||
[Operate on bytes: edit / OCR / parse]
|
||||
↓
|
||||
[Upload result, return JSON { key, url }]
|
||||
```
|
||||
|
||||
Building this with the community MCP server, the wiring goes in as `n8n_update_partial_workflow` operations — `addNode` for each step, `addConnection` to thread them, and `updateNode`/`patchNodeField` to set `executeOnce` and the system prompt. The agent's tool is a `Call n8n Workflow Tool` node pointed at the sub-workflow; the sub-workflow itself is a normal workflow that starts with an Execute Workflow Trigger.
|
||||
|
||||
> The Execute Workflow Trigger's input mode matters here. The default typed-input mode carries only named JSON fields and **drops `$binary`** at the boundary; for a sub-workflow that needs to receive binary directly, use the passthrough input mode. (When the sub-workflow downloads by key instead of receiving bytes, this is moot — which is exactly why the key pattern is cleaner.)
|
||||
|
||||
---
|
||||
|
||||
## The two pieces of plumbing that look optional
|
||||
|
||||
Both of these are silent-failure traps — leave them out and the workflow runs, then misbehaves.
|
||||
|
||||
**The Merge is a synchronization barrier, not decoration.** The chat trigger fans out to the IF branch and the upload branch in parallel. Without merging the upload branch back before the agent, the agent fires while uploads are still in flight. The system prompt's key template then renders against partial state, the model gets keys that don't exist in storage yet, and the tool's download 404s. The Merge forces the agent to wait for the upload to finish.
|
||||
|
||||
**`executeOnce: true` on the AI Agent node.** When files split out and merge back, the merged item count equals the file count. Without `executeOnce`, the agent runs once per file — N agent runs, N replies, N times the token cost — for what is one logical user message. Set it on the agent node:
|
||||
|
||||
```json
|
||||
{ "executeOnce": true }
|
||||
```
|
||||
|
||||
(Apply with `patchNodeField` on the agent node, or include it in the `updateNode` payload.)
|
||||
|
||||
---
|
||||
|
||||
## What the system prompt and the tool argument look like
|
||||
|
||||
The agent has to know which keys exist *for this turn*. Inject them into the system prompt, listing both the original name (human context for the model) and the storage key (what the tool needs):
|
||||
|
||||
```
|
||||
## File Handling
|
||||
Files passed in this turn:
|
||||
{{ JSON.stringify($('Chat Trigger').first().json.files.map((f, i) => ({
|
||||
originalFileName: f.fileName,
|
||||
storageKey: $('Crypto').all()[i].json.hash + '.' + f.fileExtension
|
||||
})), null, 2) }}
|
||||
|
||||
CRITICAL: Use EXACTLY the `storageKey` value above when calling a tool. Do not paraphrase or reconstruct it.
|
||||
```
|
||||
|
||||
Two details earn their keep:
|
||||
|
||||
1. **Both names are listed.** The original (`photo.png`) tells the model what kind of file it is; the storage key is what the tool can actually resolve.
|
||||
2. **The "use EXACTLY".** Without it, the model paraphrases — "the user's image", "photo.png" — and the tool can't find the file.
|
||||
|
||||
On the tool side, the storage-key parameter is bound with `$fromAI` and described so the model fills it correctly:
|
||||
|
||||
```
|
||||
$fromAI('imageKey', 'Storage key of an existing uploaded image to operate on, taken verbatim from the system prompt (e.g. "sess12-abc123.png"). Leave empty to generate a new image. Do not invent or reconstruct keys.', 'string')
|
||||
```
|
||||
|
||||
The description is the model's only guidance on the value's shape — match it to the storage backend the workflow actually uses, and name only that one shape (not a menu of possibilities).
|
||||
|
||||
**Generate vs edit in one tool.** If the tool serves both "make a new image" and "edit this one", branch inside the sub-workflow on whether `imageKey` is empty — empty means generate, present means download-then-edit. One tool with an internal IF is usually clearer for the model than two near-identical tools. If the model keeps misfiring on that discriminator, the viable alternative is two `Call n8n Workflow Tool` nodes pointing at the **same** sub-workflow with different parameter wiring (one hardcodes an empty key, the other lets the model fill it) — one sub-workflow, two front doors with sharply different descriptions.
|
||||
|
||||
---
|
||||
|
||||
## passthroughBinaryImages
|
||||
|
||||
Set `passthroughBinaryImages: true` on the agent when the model should be able to *see* uploaded images (multimodal vision). It adds the image to the LLM's prompt context.
|
||||
|
||||
Two limits to keep straight:
|
||||
|
||||
- **Image-only.** It does nothing for PDFs, audio, or video. For those, the model only knows what the system prompt tells it (name, type, storage key) and must call a tool to extract content. For PDFs, that means an OCR/parse tool.
|
||||
- **It does not feed tools.** Tools still receive only their `$fromAI` parameters, regardless of this flag. Vision and tool access are separate channels:
|
||||
- `passthroughBinaryImages: true` → the model can *see and reason about* the image.
|
||||
- Pre-staged storage + key in the prompt → the model can ask a tool to *do something* with the file.
|
||||
|
||||
You usually want both at once.
|
||||
|
||||
---
|
||||
|
||||
## Outbound: a tool that produces a file
|
||||
|
||||
A tool generates a PDF, image, or document. Its result to the agent is JSON, so it returns a *reference*, not the bytes.
|
||||
|
||||
```
|
||||
[Agent calls tool]
|
||||
▼
|
||||
[Sub-workflow]
|
||||
↓ generate or transform binary
|
||||
↓ (provider AI node: set options.binaryPropertyOutput so bytes land in the slot)
|
||||
[Upload to storage by key]
|
||||
↓
|
||||
[Respond with JSON: { ok, key, url, mimeType, sizeBytes, expiresAt }]
|
||||
▼
|
||||
[Agent receives JSON — embeds the URL in its reply, or passes the key to another tool]
|
||||
```
|
||||
|
||||
A useful return shape:
|
||||
|
||||
```json
|
||||
{
|
||||
"ok": true,
|
||||
"key": "sess12-9f3c1a.png",
|
||||
"url": "https://storage.example.com/files/sess12-9f3c1a.png",
|
||||
"mimeType": "image/png",
|
||||
"sizeBytes": 184320,
|
||||
"expiresAt": "2026-06-25T12:00:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
Then tell the agent how to present it, in the system prompt — and be explicit about images vs video, because the model will copy the image pattern onto video and produce a broken thumbnail:
|
||||
|
||||
```
|
||||
## Display Protocol
|
||||
Show generated images inline using markdown: 
|
||||
Share generated VIDEO as a plain link, NOT an embed: [title](url)
|
||||
```
|
||||
|
||||
(The `![]()` markdown is the canvas chat trigger's syntax — production surfaces differ; see [Surface-specific seams](#surface-specific-seams).)
|
||||
|
||||
**When you don't need any of this:** if one node generates binary and another consumes it *in the same workflow* with no agent involved, just pass binary through normally — there's no boundary. And a plain webhook API that returns a file can use `Respond to Webhook` with binary in the body. The upload-and-return-key dance is specifically for the agent-calls-tool-and-tool-produces-a-file case.
|
||||
|
||||
---
|
||||
|
||||
## Storage choices
|
||||
|
||||
**Ask which service before building.** n8n has native nodes for many backends, and defaulting to S3 is presumptuous.
|
||||
|
||||
- **Object storage:** Amazon S3, Cloudflare R2, Google Cloud Storage, Azure Blob, Backblaze B2, Supabase Storage. Most expose S3-compatible APIs (the S3 node with the right endpoint, or HTTP Request with AWS auth) or ship a dedicated node. Keys, optional public buckets, signed URLs, lifecycle rules for TTL.
|
||||
- **Drive-style:** Dropbox, Google Drive, OneDrive, Box. File IDs and share links instead of keys, folder permissions instead of bucket ACLs, no built-in TTL (cleanup is its own workflow).
|
||||
- **Self-hosted / FTP / SFTP:** when the user has on-prem infrastructure.
|
||||
- **Caller-supplied URL:** the agent's caller provides the storage location as input.
|
||||
|
||||
A common production split: a **private** bucket/folder for inbound user files, and a **public** (or signed-URL) bucket/folder for outbound results so the agent can return a fetchable URL. The choice changes credential setup, URL shape, and how the tool's `$fromAI` description should explain the key/URL format — don't pick on the user's behalf.
|
||||
|
||||
---
|
||||
|
||||
## Hashing, cleanup, long-running tools
|
||||
|
||||
**Hash strategy differs by direction:**
|
||||
|
||||
- **Inbound** files may be referenced repeatedly within a session, so use a stable key — re-uploading the same file lands at the same key and the agent's reference doesn't break. A session-and-filename composite hash works.
|
||||
- **Outbound** artifacts are single-use, so use a fresh random key every time, or concurrent generations overwrite each other. Pattern: `<session-suffix>-<random-hex>.<ext>`.
|
||||
|
||||
Two `Crypto` nodes in one of these workflows is usually deliberate, not a copy-paste error — one for the inbound stable hash, one for the outbound unique suffix.
|
||||
|
||||
**Cleanup** keeps the bill down. Object storage has lifecycle rules (auto-delete after 7–30 days). Drive-style backends need a scheduled cleanup workflow. For precise control, track live keys in a Data Table (the `n8n_manage_datatable` surface — see **n8n-mcp-tools-expert**) and delete unreferenced files.
|
||||
|
||||
**Long-running tools** (video generation, large batches): agent tool calls have no agent-layer timeout — a sub-workflow tool returns whenever it returns and the agent waits. The one real timeout is on the **HTTP Request node** itself (default ~5 minutes). If the tool is an HTTP Request Tool calling a slow external API, bump `options.timeout` past the expected duration, or the HTTP call aborts mid-job while the work keeps running and the agent gets nothing. Error-branch these steps so a failed upload or a storage 404 surfaces instead of vanishing — see **n8n-error-handling**.
|
||||
|
||||
---
|
||||
|
||||
## Surface-specific seams
|
||||
|
||||
The examples above use the canvas Chat Trigger's conventions: `$('Chat Trigger').first().json.files[]` inbound, `![]()` markdown outbound. **These shapes are not universal.** Production surfaces (Slack, Discord, Microsoft Teams, Telegram, WhatsApp Business, custom webhooks) each differ on:
|
||||
|
||||
- **Inbound file event shape** — where the file lives in the trigger payload, and whether the file URL needs a bearer/bot token to download.
|
||||
- **Outbound rendering** — markdown image, Block Kit image block, adaptive card, Discord embed, or a dedicated file-upload API that pushes bytes natively.
|
||||
|
||||
Before wiring an inbound or outbound binary path on a real surface, check the platform's official API docs and the n8n node docs for two things: the exact path to the file in the trigger event (and whether downloading it needs auth), and the exact shape the platform expects for an image/file in a reply. Get those right and the patterns here carry over; guess from the canvas examples and the workflow ships looking correct, then fails on real messages.
|
||||
|
||||
---
|
||||
|
||||
## Common mistakes
|
||||
|
||||
| Mistake | Consequence | Fix |
|
||||
|---|---|---|
|
||||
| Passing binary through `$fromAI()` | Can't carry binary; tool gets nothing | Pass a key/URL, re-fetch on the other side |
|
||||
| Forgetting to inject keys into the system prompt | Agent hallucinates names or refuses | List original + storage key, "use EXACTLY" |
|
||||
| Skipping the Merge synchronization barrier | Agent fires before uploads finish; tool 404s | Merge the upload branch back before the agent |
|
||||
| Forgetting `executeOnce: true` when files split | N files → N agent runs → N replies | Set `executeOnce: true` on the agent |
|
||||
| Forgetting `options.binaryPropertyOutput` on provider AI nodes | Produced bytes don't land where upload looks | Set it explicitly on image/audio/video gen nodes |
|
||||
| Public bucket for inbound user files | Privacy hole | Private bucket, session-scoped keys, short TTL |
|
||||
| Returning binary in the tool response | Bloated context, some runtimes reject | Upload, return `{ key, url }` |
|
||||
| Assuming `passthroughBinaryImages` feeds tools | Tools still get only `$fromAI` params | Use the upload-and-pass-key pattern |
|
||||
| Default HTTP timeout on a slow generation endpoint | Call aborts mid-job, agent gets nothing | Bump `options.timeout` past expected duration |
|
||||
| Embedding video as `![]()` | Broken thumbnail on most surfaces | Use `[title](url)` link form for video |
|
||||
@@ -0,0 +1,187 @@
|
||||
# Binary Basics
|
||||
|
||||
The `$binary` slot in depth: its shape, which nodes fill and read it, how to handle the bytes in a Code node, mime types, size limits, and how to confirm a file actually made it through.
|
||||
|
||||
## Contents
|
||||
|
||||
- [The slot shape](#the-slot-shape)
|
||||
- [Which nodes produce binary](#which-nodes-produce-binary)
|
||||
- [Which nodes consume binary](#which-nodes-consume-binary)
|
||||
- [Reading binary in a Code node](#reading-binary-in-a-code-node)
|
||||
- [Writing binary in a Code node](#writing-binary-in-a-code-node)
|
||||
- [Mime types](#mime-types)
|
||||
- [File-size limits](#file-size-limits)
|
||||
- [Inspecting binary in an execution](#inspecting-binary-in-an-execution)
|
||||
- [When binary is the trigger input](#when-binary-is-the-trigger-input)
|
||||
|
||||
---
|
||||
|
||||
## The slot shape
|
||||
|
||||
Every item has two top-level keys. `json` is your data; `binary` is your files. They are independent — a transform that rewrites `json` doesn't automatically carry `binary`, and vice versa.
|
||||
|
||||
```json
|
||||
{
|
||||
"json": { "customerId": 42, "status": "sent" },
|
||||
"binary": {
|
||||
"invoice": {
|
||||
"data": "<base64-encoded bytes>",
|
||||
"mimeType": "application/pdf",
|
||||
"fileName": "invoice-42.pdf",
|
||||
"fileExtension": "pdf",
|
||||
"fileSize": "12 kB"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The key inside `binary` — `invoice` here — is the **binary property name**. It can be anything; `data` is the default that most nodes use. File-handling nodes expose a `binaryPropertyName` parameter that points at this key, so the producer names the slot and every consumer references it by that exact name. Get the name wrong on the consumer and it looks for a slot that doesn't exist.
|
||||
|
||||
The four fields that matter:
|
||||
|
||||
| Field | What it is |
|
||||
|---|---|
|
||||
| `data` | The bytes, base64-encoded |
|
||||
| `mimeType` | How consumers should interpret the bytes (`application/pdf`, `image/png`, …) |
|
||||
| `fileName` | Used by email attachments, uploads, downloads to disk |
|
||||
| `fileExtension` | Often derived from `fileName`; some nodes use it directly |
|
||||
|
||||
---
|
||||
|
||||
## Which nodes produce binary
|
||||
|
||||
You almost never assemble the slot by hand — a node populates it:
|
||||
|
||||
| Node | What to set | Result |
|
||||
|---|---|---|
|
||||
| HTTP Request | `responseFormat: "file"` | Response body in `$binary.data` (or the name in `options`) |
|
||||
| Read/Write Files from Disk (read) | the file path | File contents in `$binary` |
|
||||
| S3 / Google Drive / Dropbox (download) | the file reference | Downloaded file in `$binary.<key>` |
|
||||
| Email triggers (IMAP, Gmail trigger) | attachment handling on | Each attachment in `$binary` |
|
||||
| Provider AI media nodes (image/audio gen) | `options.binaryPropertyOutput` | Generated bytes in the named slot |
|
||||
|
||||
The single most common bug here: an **HTTP Request download left on the default response format**. Without `responseFormat: "file"`, n8n tries to parse the body as JSON or text and you end up with a corrupted string in `$json` instead of clean bytes in `$binary`. Confirm the field with `get_node` on `nodes-base.httpRequest` — the response-handling options sit under different shapes across versions.
|
||||
|
||||
Provider AI nodes (image generation, text-to-speech) are the other recurring trap: many don't emit binary unless you set `options.binaryPropertyOutput` explicitly. Without it, the next node has nothing to upload.
|
||||
|
||||
---
|
||||
|
||||
## Which nodes consume binary
|
||||
|
||||
Consumers reference the slot by its property name:
|
||||
|
||||
| Node | How it references binary |
|
||||
|---|---|
|
||||
| Email (Send) | attachment field points at `binaryPropertyName` |
|
||||
| Slack (send file) | references the binary property |
|
||||
| HTTP Request (multipart/form-data) | references binary in the body parameters |
|
||||
| Storage upload (S3, R2, Drive) | references binary as the request body |
|
||||
| Write Files to Disk | writes the named binary property to a path |
|
||||
|
||||
The pattern is always the same: producer names a property, consumers point at that name. Most "the file didn't attach" bugs are a property-name mismatch between the two ends — verify both with `get_node` and by inspecting the execution.
|
||||
|
||||
---
|
||||
|
||||
## Reading binary in a Code node
|
||||
|
||||
Most workflows never read the bytes — they pass binary straight through to a consumer. When you genuinely need the bytes (hashing, parsing, text extraction), use `getBinaryDataBuffer` in a Code node. Do **not** grab `$binary.<key>.data` and base64-decode it yourself; the helper handles n8n's storage modes (in-memory vs filesystem) for you.
|
||||
|
||||
```javascript
|
||||
// Code node, "Run Once for Each Item"
|
||||
const buffer = await this.helpers.getBinaryDataBuffer(0, 'data'); // (itemIndex, propertyName)
|
||||
|
||||
const text = buffer.toString('utf-8'); // for text-like files
|
||||
const length = buffer.length;
|
||||
|
||||
return [{
|
||||
json: { ...$json, length },
|
||||
binary: $input.item.binary, // ← pass the file through, or it's gone after this node
|
||||
}];
|
||||
```
|
||||
|
||||
`getBinaryDataBuffer(itemIndex, propertyName)` returns a Node `Buffer`. Treat it like any buffer — slice it, hash it, decode it. The language-level specifics (which helpers exist, execution modes, `$input` vs `$json`) belong to the **n8n-code-javascript** skill; the only binary-specific rule is the one in the comment above: **if you don't return `binary`, the file is dropped at this node.**
|
||||
|
||||
> Reading a PDF's text is not as simple as `buffer.toString('utf-8')` — PDF is a binary container, not UTF-8 text. You need a real parse step (an OCR/extract node, or a dedicated library in an environment that has one). The buffer gives you the bytes; turning them into readable text is a separate problem.
|
||||
|
||||
---
|
||||
|
||||
## Writing binary in a Code node
|
||||
|
||||
Build the slot yourself: base64 the bytes, then add a mime type and file name so consumers know what they're getting.
|
||||
|
||||
```javascript
|
||||
const text = 'Hello, world!';
|
||||
|
||||
return [{
|
||||
json: { ok: true },
|
||||
binary: {
|
||||
report: {
|
||||
data: Buffer.from(text).toString('base64'),
|
||||
mimeType: 'text/plain',
|
||||
fileName: 'report.txt',
|
||||
fileExtension: 'txt',
|
||||
},
|
||||
},
|
||||
}];
|
||||
```
|
||||
|
||||
Skip `mimeType` and downstream consumers may refuse the file or render it wrong (an email won't attach it cleanly, Slack shows a generic file icon instead of an inline image). Always set it.
|
||||
|
||||
---
|
||||
|
||||
## Mime types
|
||||
|
||||
`mimeType` is the contract between producer and consumer. A wrong value doesn't error — it makes the consumer misbehave: refuse to attach, render as a download instead of inline, or show a broken thumbnail.
|
||||
|
||||
| File type | Mime type |
|
||||
|---|---|
|
||||
| PDF | `application/pdf` |
|
||||
| PNG | `image/png` |
|
||||
| JPEG | `image/jpeg` |
|
||||
| Plain text | `text/plain` |
|
||||
| JSON | `application/json` |
|
||||
| CSV | `text/csv` |
|
||||
| XLSX | `application/vnd.openxmlformats-officedocument.spreadsheetml.sheet` |
|
||||
| ZIP | `application/zip` |
|
||||
|
||||
When the source doesn't tell you the type, sniff it from the leading bytes — PDF starts with `%PDF-`, PNG with `\x89PNG`, JPEG with `\xFF\xD8\xFF`. A few lines of magic-byte checking in a Code node is a reliable fallback when you can't trust the upstream metadata.
|
||||
|
||||
---
|
||||
|
||||
## File-size limits
|
||||
|
||||
Execution data is stored in n8n's database, and large base64 blobs bloat it and slow the instance down. Rough guidance:
|
||||
|
||||
| Size per slot | Verdict |
|
||||
|---|---|
|
||||
| A few MB | Fine |
|
||||
| Tens of MB | Works, but slower; watch instance memory |
|
||||
| 100 MB+ | Offload to external storage and pass a URL/ID instead |
|
||||
|
||||
For large files, the pattern is: upload to object storage as soon as the bytes exist, thread the URL or key through the workflow as plain JSON, and re-fetch only at the node that actually needs the bytes. This keeps the per-item payload small and the execution fast. (If a self-hosted instance uses filesystem binary-data mode rather than in-memory, the database pressure is lower, but the same offload advice holds for genuinely large files.)
|
||||
|
||||
---
|
||||
|
||||
## Inspecting binary in an execution
|
||||
|
||||
`validate_workflow` will not tell you whether binary survived a node — a dropped slot is a silent failure. The only reliable check is the execution itself:
|
||||
|
||||
1. Run the workflow (`n8n_test_workflow`, or trigger it for real).
|
||||
2. Pull the execution with `n8n_executions` and look at per-node output for the `binary` slot.
|
||||
3. The slot shows presence and metadata (name, mime type, size) even when the base64 is too large to render in full. Its presence or absence on each node is what you're checking.
|
||||
|
||||
The node where `binary` last appears, then vanishes on the next, is exactly where a pass-through or a Merge needs to go. (See `MERGE_FOR_CONTEXT.md`.)
|
||||
|
||||
---
|
||||
|
||||
## When binary is the trigger input
|
||||
|
||||
For workflows that receive a file — a multipart webhook upload, an email attachment, a watched folder — the binary arrives at the **trigger's output**:
|
||||
|
||||
- Reference it by its binary property name from the trigger onward.
|
||||
- Pass it through every downstream node that needs it (each is a potential strip point).
|
||||
|
||||
If binary doesn't show up at the trigger output, check:
|
||||
|
||||
- **Content-type handling.** A Webhook receiving `multipart/form-data` puts files in `$binary` and form fields in `$json.body`; one receiving JSON has no binary at all. Expression-level detail on `$json.body` for webhooks lives in **n8n-expression-syntax**.
|
||||
- **The trigger's binary settings.** Some triggers skip attachments unless explicitly told to download them.
|
||||
@@ -0,0 +1,109 @@
|
||||
# The CDN / URL Requirement for Chat Surfaces
|
||||
|
||||
When a workflow generates an image and the user wants it shown inside a chat message — Slack, Discord, Teams, Telegram, embedded webhook chat — the image in `$binary` is not enough. Chat clients render messages that reference images by **URL** (or push bytes through the platform's own file-upload API). None of them read the `$binary` slot. The bytes have to live somewhere a URL can fetch them over HTTPS, and n8n does not bundle a CDN — the user provides the storage.
|
||||
|
||||
## Contents
|
||||
|
||||
- [Why $binary doesn't display](#why-binary-doesnt-display)
|
||||
- [What the user needs](#what-the-user-needs)
|
||||
- [What the workflow does](#what-the-workflow-does)
|
||||
- [How to tell the user](#how-to-tell-the-user)
|
||||
- [Signing and expiration](#signing-and-expiration)
|
||||
- [File naming](#file-naming)
|
||||
- [Cleanup](#cleanup)
|
||||
|
||||
---
|
||||
|
||||
## Why $binary doesn't display
|
||||
|
||||
A chat message is HTML or a JSON block. An embedded image is a reference to a URL:
|
||||
|
||||
```html
|
||||
<img src="https://cdn.example.com/img/abc123.png">
|
||||
```
|
||||
|
||||
Some surfaces accept bytes directly through a platform file API instead of a URL — Slack's two-step `files.getUploadURLExternal` + `files.completeUploadExternal`, Discord attachments, Telegram `sendPhoto`. Either way, the bytes have to be reachable: either at a URL the client fetches, or handed to the platform's upload endpoint. The raw `$binary` slot inside an n8n execution is neither — it's internal to the workflow run.
|
||||
|
||||
---
|
||||
|
||||
## What the user needs
|
||||
|
||||
A place that serves the image over a fetchable URL. Ask what they already have, but lead with a recommendation:
|
||||
|
||||
1. **A real object store / CDN (recommended).** Cloudflare R2, AWS S3 (+ CloudFront), Google Cloud Storage, Azure Blob, Backblaze B2, Vercel Blob, Supabase Storage, Bunny CDN. Direct URL embedding works once the object is public, edge caching keeps latency low, and signed-URL flows are first-class. Cloudflare R2 is the lowest-friction starting point if they have nothing — a few minutes to set up, generous free tier, no egress fees.
|
||||
2. **Drive-style services (fallback).** Dropbox, Google Drive, OneDrive, Box can produce shareable links, but the URL shape and whether it renders as an `<img src>` varies, and some need the share link converted to a direct-download URL first. Confirm the service can serve an inline-renderable URL before committing to it.
|
||||
3. **Self-hosted.** The user serves from their own domain. Fine if it already exists; don't propose standing one up just for this.
|
||||
|
||||
The right choice depends on the user's existing infrastructure, cost tolerance, and how sensitive the content is.
|
||||
|
||||
---
|
||||
|
||||
## What the workflow does
|
||||
|
||||
The shape is always generate → upload → reply-with-URL:
|
||||
|
||||
```
|
||||
[Generate image] → [Upload to storage] → [Set: imageUrl = response URL] → [Send chat reply referencing imageUrl]
|
||||
```
|
||||
|
||||
Concretely, uploading to an S3-compatible store (R2 here) via the HTTP Request node:
|
||||
|
||||
```
|
||||
[AI node: generate image] ← set options.binaryPropertyOutput so bytes land in $binary
|
||||
↓ binary on the item
|
||||
[HTTP Request: PUT to R2]
|
||||
url: https://<account>.r2.cloudflarestorage.com/<bucket>/<key>
|
||||
authentication: AWS-style signed (or the S3 node with the R2 endpoint)
|
||||
contentType: binaryData
|
||||
binaryPropertyName: data
|
||||
↓
|
||||
[Set: { imageUrl: "https://pub-<id>.r2.dev/<key>" }]
|
||||
↓
|
||||
[Send to chat surface: imageUrl embedded — markdown, Block Kit image block, adaptive card, etc.]
|
||||
```
|
||||
|
||||
Upload mechanics vary by provider; most expose S3-compatible APIs usable through n8n's S3 node or HTTP Request with AWS auth. Confirm the upload node's field names (`contentType`, `binaryPropertyName`) with `get_node`, and **error-branch the upload** so a failed write surfaces instead of producing a reply that references a URL that was never written — see **n8n-error-handling**. The exact reply shape per platform is surface-specific (see `AGENT_TOOL_BINARY.md`).
|
||||
|
||||
---
|
||||
|
||||
## How to tell the user
|
||||
|
||||
Don't quietly ship a workflow that generates images "but they don't display." Surface the requirement before building:
|
||||
|
||||
> "I can generate the image, but the chat surface can't display raw binary — it embeds images by URL. So I'll need to upload the image somewhere that serves a public URL first. What do you use for image/file storage today (R2, S3, GCS, Dropbox, Google Drive, …)? If you don't have anything set up, Cloudflare R2 is the lowest-friction starting point."
|
||||
|
||||
There is no fallback that hides this — n8n won't host the file. If the user has no storage, pause until they pick a service and provision a bucket and credentials, then resume. (Posting the URL as a plain link rather than an inline image is a lighter option if inline rendering isn't critical — but that link still has to come from somewhere.)
|
||||
|
||||
---
|
||||
|
||||
## Signing and expiration
|
||||
|
||||
| URL type | Trade-off | Use for |
|
||||
|---|---|---|
|
||||
| **Public** | Anyone with the URL can fetch it; simplest | Non-sensitive content (already-public assets) |
|
||||
| **Signed, with expiry** | Per-request URL that expires (e.g. 1 hour) | Sensitive or user-specific content |
|
||||
|
||||
For internal chat with scoped channels, public is usually fine — the URL only lives inside messages a known set of users sees. For compliance-sensitive content, default to signed URLs with a short expiry. A permanently public, unguessable-but-non-expiring URL is a slow leak for anything private.
|
||||
|
||||
---
|
||||
|
||||
## File naming
|
||||
|
||||
| Scheme | Example | Note |
|
||||
|---|---|---|
|
||||
| UUID / random | `img/abc-123-def-456.png` | Unguessable; good default |
|
||||
| Content hash | `img/sha256-abc123….png` | Free deduplication |
|
||||
| User-prefixed | `users/<userId>/<name>.png` | Easy per-user cleanup |
|
||||
|
||||
Avoid user-controlled filenames (path traversal, collisions) and sequential IDs (predictable, scrapeable).
|
||||
|
||||
---
|
||||
|
||||
## Cleanup
|
||||
|
||||
Without it, storage costs grow:
|
||||
|
||||
- **Lifecycle rules** — object stores (S3, R2, GCS, Azure Blob) auto-delete objects after N days. 7–30 days is usually plenty for chat use cases.
|
||||
- **Scheduled cleanup workflow** — for drive-style backends that have no TTL, run a workflow that lists and deletes old files.
|
||||
|
||||
Ask the user's retention preference rather than picking a window for them — chat artifacts are often disposable, but some surfaces (audit, support transcripts) need them kept.
|
||||
+130
@@ -0,0 +1,130 @@
|
||||
# Merge for Keeping Binary in Context
|
||||
|
||||
A common, maddening bug: an item carries both `json` and `binary`, it runs through a JSON-only node (Edit Fields, Code, IF), the binary slot quietly disappears, and the email node three steps later has nothing to attach. No error, no validation warning — just a missing file.
|
||||
|
||||
The fix is to keep the binary on a branch that doesn't touch it, and recombine. This is the same Merge node covered in **n8n-node-configuration**'s gotchas; here it's used specifically to re-attach binary.
|
||||
|
||||
## Contents
|
||||
|
||||
- [The pattern](#the-pattern)
|
||||
- [Wiring it with n8n-mcp](#wiring-it-with-n8n-mcp)
|
||||
- [Configuring the Merge](#configuring-the-merge)
|
||||
- [Why it works](#why-it-works)
|
||||
- [Cheaper alternative: pass-through on the transform](#cheaper-alternative-pass-through-on-the-transform)
|
||||
- [When Merge isn't enough](#when-merge-isnt-enough)
|
||||
- [Verifying after merge](#verifying-after-merge)
|
||||
- [Common mistakes](#common-mistakes)
|
||||
|
||||
---
|
||||
|
||||
## The pattern
|
||||
|
||||
Split the stream at the source: one branch does the JSON work, the other carries the original item (binary intact) untouched. Merge them back.
|
||||
|
||||
```
|
||||
[Source with binary] ─┬─→ [Edit Fields: change JSON] ─┐
|
||||
│ (binary stripped here) │
|
||||
│ ├─→ [Merge: combineByPosition] ─→ [Email: attach]
|
||||
│ │
|
||||
└──────────────────────────────────┘
|
||||
(bypass — binary passes through unchanged)
|
||||
```
|
||||
|
||||
- **Transform branch:** does the JSON work; may lose binary. That's fine — this branch only contributes the JSON.
|
||||
- **Bypass branch:** the original item, with binary. No node needed; just route the connection straight into the Merge.
|
||||
|
||||
The merged item gets its JSON from the transform branch and its binary from the bypass branch.
|
||||
|
||||
---
|
||||
|
||||
## Wiring it with n8n-mcp
|
||||
|
||||
The source already feeds the transform branch. You add the bypass connection and the Merge with `n8n_update_partial_workflow`:
|
||||
|
||||
```json
|
||||
{
|
||||
"operations": [
|
||||
{ "type": "addNode", "node": {
|
||||
"name": "Merge",
|
||||
"type": "n8n-nodes-base.merge",
|
||||
"parameters": { "mode": "combine", "combineBy": "combineByPosition" }
|
||||
}},
|
||||
{ "type": "addConnection", "source": "Edit Fields", "target": "Merge", "targetInput": 0 },
|
||||
{ "type": "addConnection", "source": "Source", "target": "Merge", "targetInput": 1 },
|
||||
{ "type": "addConnection", "source": "Merge", "target": "Send Email" }
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
The exact parameter names (`mode`, `combineBy`, `combineByPosition`, and how `numberOfInputs` is expressed) have shifted across Merge node versions — confirm the current shape with `get_node` on `nodes-base.merge` for the user's version before committing the structure. The principle is stable; the field names move.
|
||||
|
||||
Two wiring details that bite (both detailed in **n8n-node-configuration**'s Merge section):
|
||||
|
||||
- The Merge defaults to **2 inputs**. If you wire 3+ branches, set the input count to match or the extra branch silently drops.
|
||||
- Connection input indexes are **0-based**. The bypass branch above lands on `targetInput: 1` (the second input).
|
||||
|
||||
---
|
||||
|
||||
## Configuring the Merge
|
||||
|
||||
For re-attaching binary, you want position-based combination:
|
||||
|
||||
| Mode | What it does | Use for binary re-attach? |
|
||||
|---|---|---|
|
||||
| `combineByPosition` | Pairs item N from input 1 with item N from input 2 | ✅ Yes |
|
||||
| `combineBySql` / `combineByFields` | Joins on a key | Only if the two branches share a join key |
|
||||
| `combineAll` | Cartesian product (N×M items) | ❌ No — explodes the item count |
|
||||
| `append` | Concatenates inputs end to end | ❌ No — doesn't pair items |
|
||||
|
||||
`combineByPosition` is the right default: it keeps the item count at N and pairs each transformed JSON item with its corresponding binary-bearing original. For this to work, both branches must emit items in the same order and count — which they do when they share a single source.
|
||||
|
||||
---
|
||||
|
||||
## Why it works
|
||||
|
||||
A Merge combines both `json` and `binary` from the items it pairs. When one input holds the JSON you want and the other holds the binary you want, the merged item carries both. The binary survives because it traveled on the branch that never touched it.
|
||||
|
||||
---
|
||||
|
||||
## Cheaper alternative: pass-through on the transform
|
||||
|
||||
If the transforming node can preserve binary itself, do that instead — it's one node, not three:
|
||||
|
||||
- **Edit Fields (Set):** enable `includeOtherFields` so the node carries unmentioned fields and the binary slot forward.
|
||||
- **Code node:** return `binary: $input.item.binary` explicitly in the returned item (see `BINARY_BASICS.md`).
|
||||
- **IF / Filter:** these route items rather than rebuild them, and generally preserve binary on the items they pass — but verify in the execution rather than assuming.
|
||||
|
||||
Reach for Merge only when the transforming node genuinely can't carry the binary, or when the JSON and binary come from genuinely different upstream nodes.
|
||||
|
||||
---
|
||||
|
||||
## When Merge isn't enough
|
||||
|
||||
If the chain has many strip points, threading binary through all of them — and Merging at each one — becomes more work than it's worth. Two better routes:
|
||||
|
||||
- **Upload early.** Push the bytes to object storage as soon as they exist, carry the URL/key as plain JSON through the whole chain (JSON survives every transform trivially), and re-fetch only at the node that needs the bytes. This is also the right move for large files (see `BINARY_BASICS.md`).
|
||||
- **Push the binary work into a sub-workflow.** Hand the file to a sub-workflow that does the binary handling and returns the final result. The Execute Workflow Trigger's input mode matters: the default typed-input mode carries only named JSON fields and drops `$binary`, so use the passthrough input mode if the sub-workflow must receive bytes directly.
|
||||
|
||||
Past a couple of strip points, one of these is usually less work — and less fragile — than keeping every node in a long chain honest about binary.
|
||||
|
||||
---
|
||||
|
||||
## Verifying after merge
|
||||
|
||||
A merged-but-missing binary won't show in validation. Confirm in the execution:
|
||||
|
||||
1. Run with `n8n_test_workflow`, then pull the execution with `n8n_executions`.
|
||||
2. On the Merge node's output, check the merged item has the `json` from the transform branch **and** the `binary` from the bypass branch.
|
||||
3. If binary is missing: check the Merge mode (some modes don't pair the way you expect) and confirm the bypass branch actually carried binary into the Merge in the first place.
|
||||
|
||||
---
|
||||
|
||||
## Common mistakes
|
||||
|
||||
| Mistake | Symptom | Fix |
|
||||
|---|---|---|
|
||||
| Noticing the strip too late | The original binary is already gone | Inspect the execution after each node during development |
|
||||
| "Merging" a single-source chain with no bypass | Nothing to merge with; binary still missing | Split the stream at the source so binary rides a bypass branch |
|
||||
| `combineAll` where you meant `combineByPosition` | N×M items instead of N | Choose the mode deliberately |
|
||||
| Bypass branch on the wrong input index | Wrong pairing, or the branch drops | Connections are 0-based; verify with `n8n_get_workflow` |
|
||||
| Forgetting to raise the Merge input count past 2 | A third branch silently drops | Set the input count to match the wired branches |
|
||||
Reference in New Issue
Block a user