Gemini Live WebSocket Transport

GeminiLiveWebsocketTransport for the JavaScript SDK: connect directly to Google’s Gemini Live service over WebSocket.

Overview

The GeminiLiveWebsocketTransport class implements a fully functional Pipecat Transport, providing a framework for implementing real-time communication directly with the Gemini Live service. Like all transports, it handles media device management, audio/video streams, and state management for the connection.

Transports of this type are designed primarily for development and testing purposes. For production applications, you will need to build a server component with a server-friendly transport, like the DailyTransport, to securely handle API keys.

Usage

Basic Setup

import {
  GeminiLiveWebsocketTransport,
  GeminiLLMServiceOptions,
} from "@pipecat-ai/gemini-live-websocket-transport";
import { PipecatClient } from "@pipecat-ai/client-js";

const options: GeminiLLMServiceOptions = {
  api_key: "YOUR_API_KEY",
  initial_messages: [
    // Set up initial system and user messages.
    // Without the user message, the bot will not respond immediately
    // and wait for the user to speak first.
    {
      role: "model",
      content: "You are a confused jellyfish.",
    },
    { role: "user", content: "Blub blub!" },
  ],
  settings: {
    temperature: 0.7,
    maxOutput_tokens: 1000,
  },
};

const transport = new GeminiLiveWebsocketTransport(options);
let pcClient = new PipecatClient({
  transport: new GeminiLiveWebsocketTransport(options),
  callbacks: {
    // Event handlers
  },
});
pcClient.connect();

API Reference

Constructor Options

GeminiLLMServiceOptions

interface GeminiLLMServiceOptions {
  api_key: string; // Required: Your Gemini API key
  initial_messages?: Array<{
    // Optional: Initial conversation context
    content: string;
    role: string;
  }>;
  settings?: {
    // Optional: Generation parameters
    candidate_count?: number;
    maxOutput_tokens?: number;
    temperature?: number;
    top_p?: number;
    top_k?: number;
    presence_penalty?: number;
    frequency_penalty?: number;
    response_modalities?: string;
    speech_config?: {
      voice_config?: {
        prebuilt_voice_config?: {
          voice_name: "Puck" | "Charon" | "Kore" | "Fenrir" | "Aoede";
        };
      };
    };
  };
}

TransportConnectionParams

connect() optionally takes a partial GeminiLLMServiceOptions, which is merged over the configuration given to the constructor — useful for supplying the API key or overriding generation_config per session. Omit it and the transport connects with the constructor’s configuration as-is.

Events

The GeminiLiveWebSocketTransport implements the various PipecatClient event handlers. Check out the docs or samples for more info.

More Information

Demo

Gemini Live Basic Demo

Source

GeminiLiveWebsocketTransport

Package

@pipecat-ai/gemini-live-websocket-transport