Posted inDevelopment

Building Multi-Model AI Streaming Chat with Vercel AI SDK and Next.js

Development tutorial - IT technology blog
Development tutorial - IT technology blog

The headache of multi-model AI integration

Have you ever stayed up all night just to rewrite streaming logic because a client wanted to switch from GPT-4 to Claude 3.5? Every provider has a different data structure and streaming process. If done manually, you’d have to manage message arrays, handle network errors, and maintain a complex UI state yourself.

Vercel AI SDK was born to solve this problem. It acts as an abstraction layer, allowing you to communicate with any AI model using a single set of code. Let’s build a chat application that supports the three “giants”: OpenAI, Anthropic, and Google.

Comparing Implementation Options

Before diving into the code, let’s look at the currently popular choices:

  • Traditional Fetch API: Simple but a nightmare for streaming. You have to handle ReadableStream and manage state manually, which can easily cause memory leaks if not handled carefully.
  • LangChain: Extremely powerful for complex tasks like RAG or Agents. However, it’s quite heavy and has a steep learning curve if you only need a simple chat interface.
  • Vercel AI SDK: The optimal choice for the Next.js ecosystem. This library provides the useChat hook, which reduces UI handling code by 80% and supports smooth data streaming directly from the server.

Why use Vercel AI SDK?

The biggest advantage is its Unified nature. You only need to write the processing logic once. Switching from GPT-4o to Claude 3.5 Sonnet can sometimes take just 5 seconds by changing a single variable.

However, this tool is tightly coupled with JavaScript/TypeScript. If your backend runs on Python or Go, you won’t be able to leverage its powerful React hooks. But for Next.js developers, it is truly a powerful assistant.

Detailed Implementation Guide

1. Initialize the Next.js Project

First, create a new Next.js project using the App Router:

npx create-next-app@latest my-ai-chat
cd my-ai-chat

Then, install the core library and the corresponding providers:

npm install ai @ai-sdk/openai @ai-sdk/anthropic @ai-sdk/google zod

2. Set up Environment Variables

Create a .env.local file in the root directory. You need to enter the API Keys from the OpenAI, Anthropic, and Google AI Studio dashboards here.

OPENAI_API_KEY=sk-...
ANTHROPIC_API_KEY=sk-ant-...
GOOGLE_GENERATIVE_AI_API_KEY=...

3. Build the API Route (Backend)

We will handle the AI call logic at app/api/chat/route.ts. The special thing is that the backend will automatically select the model based on the request sent from the client.

import { openai } from '@ai-sdk/openai';
import { anthropic } from '@ai-sdk/anthropic';
import { google } from '@ai-sdk/google';
import { streamText } from 'ai';

export const maxDuration = 30;

export async function POST(req: Request) {
  const { messages, modelType } = await req.json();

  let selectedModel;
  if (modelType === 'claude') {
    selectedModel = anthropic('claude-3-5-sonnet-20240620');
  } else if (modelType === 'gemini') {
    selectedModel = google('models/gemini-1.5-pro-latest');
  } else {
    selectedModel = openai('gpt-4o');
  }

  const result = await streamText({
    model: selectedModel,
    messages,
  });

  return result.toDataStreamResponse();
}

While debugging complex JSON requests, you can use a JSON Formatter to inspect the data structure faster instead of looking at a mess of text.

4. Build the Chat Interface (Frontend)

In the app/page.tsx file, we use the useChat hook. This hook automatically manages the message list and input state for you.

'use client';

import { useChat } from 'ai/react';
import { useState } from 'react';

export default function Chat() {
  const [modelType, setModelType] = useState('openai');
  const { messages, input, handleInputChange, handleSubmit } = useChat({
    body: { modelType },
  });

  return (
    <div className="flex flex-col w-full max-w-md py-24 mx-auto stretch">
      <select 
        value={modelType} 
        onChange={(e) => setModelType(e.target.value)}
        className="mb-4 p-2 border rounded text-black"
      >
        <option value="openai">GPT-4o</option>
        <option value="claude">Claude 3.5 Sonnet</option>
        <option value="gemini">Gemini 1.5 Pro</option>
      </select>

      {messages.map(m => (
        <div key={m.id} className="mb-4">
          <span className="font-bold">{m.role === 'user' ? 'You: ' : 'AI: '}</span>
          {m.content}
        </div>
      ))}

      <form onSubmit={handleSubmit}>
        <input
          className="fixed bottom-0 w-full max-w-md p-2 mb-8 border rounded shadow-xl text-black"
          value={input}
          placeholder="Ask something..."
          onChange={handleInputChange}
        />
      </form>
    </div>
  );
}

Pro Tip: Avoiding Timeout Errors

A common error when working with streaming is maxDuration. On the Vercel Hobby (free) plan, Serverless Functions are terminated after 10 seconds. With “slow” models like GPT-4, the response might take 15-20 seconds to complete.

To fix this, always declare export const maxDuration = 30; (the maximum limit for the Pro plan is 300 seconds). Additionally, leverage the isLoading variable from useChat to display a loading effect, ensuring users don’t feel like the app has frozen.

Conclusion

Vercel AI SDK is more than just a library; it’s the new standard for building AI applications on the web. Instead of wasting time reading dozens of different API documentation pages, you can focus entirely on the user experience.

Next, you can experiment with the Tool Calling feature. This technique allows the AI to call your JavaScript functions to fetch weather data or query a database in real-time.

Share:
Tags: