AI SDK agents on Neon Functions

A Neon Function is a long-lived Node.js 24 process, which makes it a natural host for a Vercel AI SDK agent: the handler keeps streaming for the life of the request (15-minute budget, see Timeouts), so multi-step tool loops and image/video generation don't get cut off the way they do on lambda-style serverless. Point the model at the Neon AI Gateway (see the neon-ai-gateway skill) and there are no extra provider keys to manage — one Neon credential reaches the whole catalog.

The AI SDK is the recommended way to build agents on Functions from TypeScript: one set of primitives (streamText, generateText, tool calling, structured output) over every catalog model. For a memory- and workflow-heavy agent with built-in tracing, use Mastra instead (see references/mastra-studio.md); both point at the same gateway.

The pattern below is a complete agent: it streams chat and, when asked, generates an image, uploads it to Object Storage, and indexes it in Postgres.

1. Declare the gateway and the function

The agent needs the AI Gateway (and, for the image example, an Object Storage bucket). Declare both in neon.ts alongside the function — neon deploy provisions them and injects the credentials at runtime (see the neon-ai-gateway and neon-object-storage skills):

// neon.ts
import { defineConfig } from "@neon/config/v1";

export default defineConfig({
  preview: {
    aiGateway: true,
    buckets: { images: {} },
    functions: {
      agent: { name: "ai agent", source: "src/index.ts" },
    },
  },
});

2. The handler: stream a tool-calling agent

The function's default export is a web-standard { fetch } handler. The @neon/ai-sdk-provider reads the injected gateway credentials automatically, so neon("<model>") is all the model config you need — it routes each model to the right dialect (Anthropic → Messages, OpenAI/Codex → Responses, everything else → MLflow). Return result.toUIMessageStreamResponse() so the AI SDK's useChat hooks can consume the stream:

// src/index.ts
import { neon } from "@neon/ai-sdk-provider";
import { streamText, tool, stepCountIs, type ModelMessage } from "ai";
import { z } from "zod";
import { drizzle } from "drizzle-orm/node-postgres";
import { Pool } from "pg";
import { todos } from "./db/schema";

const pool = new Pool({ connectionString: process.env.DATABASE_URL, max: 5 });
const db = drizzle(pool);

export default {
  async fetch(request: Request) {
    if (request.method !== "POST") {
      return new Response("POST chat messages here", { status: 405 });
    }
    const { messages } = (await request.json()) as { messages: ModelMessage[] };

    const result = streamText({
      model: neon("claude-sonnet-4-6"), // swap to gpt-5-mini, gemini-2-5-flash, …
      system: "You are a concise assistant with access to the user's todos.",
      messages,
      tools: {
        countOpenTodos: tool({
          description: "Count the user's open todos.",
          inputSchema: z.object({}),
          execute: async () => ({ open: await db.$count(todos) }),
        }),
      },
      // Let the model call tools and then summarize, instead of stopping after
      // the first tool call. The loop runs in-process — no host timeout.
      stopWhen: stepCountIs(5),
      onError({ error }) {
        console.error("[streamText] error:", error);
      },
    });

    return result.toUIMessageStreamResponse({
      onError: (error) => (error instanceof Error ? error.message : String(error)),
    });
  },
};

tool({ inputSchema, execute }) is the AI SDK v5+ shape (the parameter is inputSchema, not the old parameters). The tool's execute runs inside the function, right next to Postgres — no extra network hop.

3. Generate images and persist them

The gateway exposes the OpenAI Responses image_generation built-in tool (GPT-5 models only; the image comes back inline as base64). Persist generated assets to Object Storage and index them in Postgres so they branch together — the recommended storage client is the Files SDK neon adapter (see the neon-object-storage skill):

import { neon } from "@neon/ai-sdk-provider";
import { streamText } from "ai";
import { Files } from "files-sdk";
import { neon as neonFiles } from "files-sdk/neon";
import { randomUUID } from "node:crypto";

const files = new Files({ adapter: neonFiles({ bucket: "images" }) });

const result = streamText({
  model: neon("gpt-5-mini"),
  system: "Use image_generation when the user asks for a picture, then describe it.",
  messages,
  tools: {
    image_generation: neon.tools.imageGeneration({
      outputFormat: "jpeg",
      quality: "low", // the gateway caps a response near 640 KB — keep images small
      size: "1024x1024",
    }),
  },
  async onStepFinish({ toolResults }) {
    for (const tr of toolResults) {
      if (tr.toolName !== "image_generation") continue;
      const base64 = imageResultBase64(tr.output);
      if (!base64) continue;
      const key = `generated/${randomUUID()}.jpg`;
      await files.upload(key, Buffer.from(base64, "base64"), { contentType: "image/jpeg" });
      // …insert a row keyed by `key` into Postgres; serve later via files.url(key)
    }
  },
});

Keep generated images small: the gateway caps a single response near 640 KB and has an upstream timeout, so request a compressed JPEG rather than a full-size PNG.

4. Call it directly from the client (don't proxy the stream)

So the long stream isn't cut off by your web host's serverless limits, have the browser call the function directly and authenticate at the top of the handler — see Functions as an agent backend for the JWT-verify + CORS pattern and the AI SDK DefaultChatTransport wiring.

5. Run and deploy

neon dev      # injects DATABASE_URL + the gateway/storage creds; hot reload
neon deploy   # provisions the gateway + bucket and deploys the function

curl -N -X POST "$(neon functions get agent -o json | jq -r .invocation_url)" \
  -H "content-type: application/json" \
  -d '{"messages":[{"role":"user","content":"How many open todos do I have?"}]}'

6.5 KiB Raw Blame History