Google Speech-to-Text

Streaming STT with GeminiSTTService on the Gemini Live API and GoogleSTTService on Cloud Speech-to-Text V2: hints and 125+ languages.

Overview

Google provides two STT service implementations:

• GeminiSTTService for streaming transcription with a Gemini Live transcription model (e.g. gemini-3.5-transcribe-live) over the Gemini Live API, with automatic language detection, language hints, and adaptation phrases
• GoogleSTTService for real-time speech recognition using Google Cloud’s Speech-to-Text V2 API with support for 125+ languages, multiple models, voice activity detection, and advanced features like automatic punctuation and word-level confidence scores

The two services authenticate differently: GeminiSTTService uses a Google AI (Gemini) API key, while GoogleSTTService uses Google Cloud service account credentials.

Gemini STT API Reference

Pipecat’s API methods for Gemini STT integration

Google STT API Reference

Pipecat’s API methods for Google Cloud STT integration

Gemini Voice Example

Complete voice bot example with Gemini services

Gemini Transcription Example

Transcription example with GeminiSTTService

Google Cloud STT Example

Complete example with Google Cloud services

Google AI Documentation

Official Google Gemini API documentation

Google Cloud Documentation

Official Google Cloud Speech-to-Text documentation

Google Cloud Console

Create service accounts and manage API access

Installation

To use either Google STT service, install the required dependency:

uv add "pipecat-ai[google]"

GeminiSTTService additionally requires google-genai >= 2.9.0 for transcription support. Upgrade with uv add "google-genai>=2.9.0" if you have an older version installed.

Prerequisites

Google AI Setup

Before using GeminiSTTService, you need:

1. Google AI Account: Sign up at Google AI Studio
2. API Key: Generate an API key from the AI Studio console
3. Model Access: Ensure you have access to the gemini-3.5-transcribe-live model

Required environment variables:

• GOOGLE_API_KEY: Your Google AI (Gemini) API key

Google Cloud Setup

Before using GoogleSTTService, you need:

1. Google Cloud Account: Sign up at Google Cloud Console
2. Project Setup: Create a project and enable the Speech-to-Text API
3. Service Account: Create a service account with Speech-to-Text permissions
4. Authentication: Set up credentials via service account key or Application Default Credentials

Required environment variables:

• GOOGLE_APPLICATION_CREDENTIALS: Path to your service account key file (recommended)
• Or use Application Default Credentials for cloud deployments

GeminiSTTService

GeminiSTTService streams raw PCM audio to a Gemini Live transcription model (e.g. gemini-3.5-transcribe-live) over the Gemini Live API, delivering interim and final transcription frames as results arrive.

The model detects utterance boundaries itself, and when the pipeline’s VAD signals end of speech the service sends an audio-stream-end signal to flush the utterance, so the final transcript is produced promptly instead of waiting for the model to decide the utterance ended. Without an upstream VAD the model finalizes on its own schedule.

Audio is sent at the pipeline’s input sample rate; the model performs best with 16 kHz mono PCM.

api_key · str · required

Google AI (Gemini) API key for authentication.

http_options · HttpOptions · default: "None"

Optional HTTP options passed to the google-genai client.

sample_rate · int · default: "None"

Audio sample rate in Hz. When None, uses the pipeline’s configured sample rate. The model performs best with 16 kHz mono PCM.

settings · GeminiSTTService.Settings · default: "None"

Runtime-configurable settings for the STT service. Defaults to the gemini-3.5-transcribe-live model with automatic language detection. See Settings below.

ttfs_p99_latency · float · default: "GEMINI_TTFS_P99"

P99 latency from speech end to final transcript in seconds. Override for your deployment. See https://github.com/pipecat-ai/stt-benchmark

Settings

Runtime-configurable settings passed via the settings constructor argument using GeminiSTTService.Settings(...). These can be updated mid-conversation with STTUpdateSettingsFrame. See Service Settings for details.

Language configuration maps to the Live API’s AudioTranscriptionConfig: when language or languages are set they are sent as language hints; otherwise the model detects the language automatically. Language hints and automatic detection are mutually exclusive — if both are configured, hints take precedence.

Parameter	Type	Default	Description
model	str	"gemini-3.5-transcribe-live"	Gemini transcription model to use. (Inherited from base STT settings.)
language	Language | str	None	Single language hint. (Inherited from base STT settings.)
languages	list[Language]	[]	List of Language enums used as language hints (e.g. [Language.ES_ES]).
language_auto	bool | None	None	Enable automatic language detection. None auto-detects unless hints are given. Set to False to disable.
adaptation_phrases	list[str]	None	Phrases to bias recognition toward, improving accuracy for domain-specific terms (e.g. ["Pipecat", "oatmilk"]).

Usage

import os
from pipecat.services.google.gemini_live.stt import GeminiSTTService

stt = GeminiSTTService(
    api_key=os.getenv("GOOGLE_API_KEY")
)

With Language Hints

from pipecat.services.google.gemini_live.stt import GeminiSTTService
from pipecat.transcriptions.language import Language

stt = GeminiSTTService(
    api_key=os.getenv("GOOGLE_API_KEY"),
    settings=GeminiSTTService.Settings(
        languages=[Language.EN_US, Language.ES_ES]
    )
)

With Adaptation Phrases

Use adaptation phrases to improve recognition accuracy for domain-specific terms:

stt = GeminiSTTService(
    api_key=os.getenv("GOOGLE_API_KEY"),
    settings=GeminiSTTService.Settings(
        adaptation_phrases=["Pipecat", "oatmilk", "WebRTC"]
    )
)

Notes

• Requires google-genai >= 2.9.0: The service will raise an ImportError if an older version is installed. Upgrade with uv add "google-genai>=2.9.0".
• VAD Integration: When the pipeline’s VAD signals end of speech (via VADUserStoppedSpeakingFrame), the service flushes the utterance to receive the final transcript promptly. Without a VAD, the model finalizes on its own schedule.
• Language configuration: Language hints (languages) and automatic detection (language_auto) are mutually exclusive. If both are configured, language hints take precedence.
• Audio requirements: The model performs best with 16 kHz mono PCM audio. Audio is sent at the pipeline’s input sample rate.
• Connection management: Settings changes (model, languages, adaptation phrases) require a reconnection. The service automatically handles reconnection on transient errors.
• Authentication: Uses Google AI (Gemini) API keys, not Google Cloud service accounts. This is a different authentication method from GoogleSTTService.

Event Handlers

Supports the standard service connection events:

Event	Description
on_connected	Called when successfully connected to the Gemini Live API
on_disconnected	Called when disconnected from the Gemini Live API
on_connection_error	Called when a fatal connection error occurs

@stt.event_handler("on_connected")
async def on_connected(service):
    print("Connected to Gemini Live API")

@stt.event_handler("on_disconnected")
async def on_disconnected(service):
    print("Disconnected from Gemini Live API")

GoogleSTTService

credentials · str · default: "None"

JSON string containing Google Cloud service account credentials.

credentials_path · str · default: "None"

Path to service account credentials JSON file.

location · str · default: "global"

Google Cloud location (e.g., "global", "us-central1"). Non-global locations use regional endpoints.

sample_rate · int · default: "None"

Audio sample rate in Hz. When None, uses the pipeline’s configured sample rate.

params · GoogleSTTService.InputParams · default: "None" · deprecated

Configuration parameters for the STT service. Deprecated in v0.0.105. Use settings=GoogleSTTService.Settings(...) instead.

settings · GoogleSTTService.Settings · default: "None"

Runtime-configurable settings for the STT service. See Settings below.

ttfs_p99_latency · float · default: "GOOGLE_TTFS_P99"

P99 latency from speech end to final transcript in seconds. Override for your deployment.

You must provide either credentials (JSON string), credentials_path (file path), or have Application Default Credentials configured. At least one authentication method is required.

Settings

Runtime-configurable settings passed via the settings constructor argument using GoogleSTTService.Settings(...). These can be updated mid-conversation with STTUpdateSettingsFrame. See Service Settings for details.

Parameter	Type	Default	Description
model	str	"latest_long"	Speech recognition model to use. (Inherited from base STT settings.)
language	Language | str	None	Language for speech recognition. (Inherited from base STT settings.)
languages	Language | List[Language]	[Language.EN_US]	Single language or list of recognition languages. First language is primary.
use_separate_recognition_per_channel	bool	False	Process each audio channel separately.
enable_automatic_punctuation	bool	True	Add punctuation to transcripts.
enable_spoken_punctuation	bool	False	Include spoken punctuation in transcript.
enable_spoken_emojis	bool	False	Include spoken emojis in transcript.
profanity_filter	bool	False	Filter profanity from transcript.
enable_word_time_offsets	bool	False	Include timing information for each word.
enable_word_confidence	bool	False	Include confidence scores for each word.
enable_interim_results	bool	True	Stream partial recognition results.
enable_voice_activity_events	bool	False	Detect voice activity in audio.
adaptation	dict | SpeechAdaptation	None	Phrase sets biasing recognition toward domain-specific terms. Each phrase_sets entry is either a resource name or an inline phrase set. See Speech Adaptation for details. Support varies by model/language.

Usage

from pipecat.services.google.stt import GoogleSTTService

stt = GoogleSTTService(
    credentials_path=os.getenv("GOOGLE_APPLICATION_CREDENTIALS"),
)

With Credentials JSON String

import json
from pipecat.services.google.stt import GoogleSTTService

stt = GoogleSTTService(
    credentials=json.dumps(credentials_dict),
    location="us-central1",
)

With Custom Parameters

from pipecat.services.google.stt import GoogleSTTService
from pipecat.transcriptions.language import Language

stt = GoogleSTTService(
    credentials_path=os.getenv("GOOGLE_APPLICATION_CREDENTIALS"),
    settings=GoogleSTTService.Settings(
        languages=[Language.EN_US, Language.ES],
        model="latest_long",
        enable_automatic_punctuation=True,
        enable_word_time_offsets=True,
        enable_word_confidence=True,
    ),
)

Speech Adaptation

Bias recognition toward domain-specific terms using inline phrase sets or references to pre-created phrase sets:

from pipecat.services.google.stt import GoogleSTTService

stt = GoogleSTTService(
    credentials_path=os.getenv("GOOGLE_APPLICATION_CREDENTIALS"),
    settings=GoogleSTTService.Settings(
        adaptation={
            "phrase_sets": [
                # Reference a pre-created phrase set
                "projects/my-project/locations/global/phraseSets/catalog",
                # Inline phrase set with boost values
                {
                    "phrases": [
                        {"value": "pipecat", "boost": 15.0},
                        {"value": "voice pipeline"},
                    ]
                },
            ]
        }
    ),
)

Updating Settings at Runtime

Google Cloud STT supports dynamic settings updates via STTUpdateSettingsFrame:

from pipecat.frames.frames import STTUpdateSettingsFrame
from pipecat.transcriptions.language import Language

await worker.queue_frame(
    STTUpdateSettingsFrame(
        delta=GoogleSTTService.Settings(
            languages=[Language.FR],
            model="latest_short",
            enable_automatic_punctuation=False,
        )
    )
)

Notes

• Streaming time limit: Google Cloud STT has a 5-minute streaming limit per connection. The service automatically handles stream reconnection at 4 minutes to provide seamless transcription without interruption.
• Multi-language support: Pass a list of Language values to languages for multi-language recognition. The first language is the primary language.
• Regional endpoints: Use the location parameter to route requests through regional endpoints (e.g., "us-central1", "europe-west1") for data residency requirements. The default "global" endpoint works for most use cases.
• Stream abort on inactivity: If no audio is sent for ~10 seconds (e.g., when audio frames are blocked), Google automatically closes the stream. The service recovers by automatically reconnecting.
• Authentication priority: The service checks for credentials in this order: credentials (JSON string), credentials_path (file), then Application Default Credentials.
• Speech adaptation: Support varies by model and language — see Google’s language support documentation.

The InputParams / params= pattern is deprecated as of v0.0.105. Use Settings / settings= instead. See the Service Settings guide for migration details.

Event Handlers

Supports the standard service connection events:

Event	Description
on_connected	Connected to Google Cloud Speech-to-Text
on_disconnected	Disconnected from Google Cloud Speech-to-Text

@stt.event_handler("on_connected")
async def on_connected(service):
    print("Connected to Google STT")