Azure OpenAI LLM

AzureLLMService reaches Azure OpenAI language models through the OpenAI-compatible interface it inherits from OpenAILLMService.

Overview

AzureLLMService provides access to Azure OpenAI’s language models through an OpenAI-compatible interface. It inherits from OpenAILLMService and supports streaming responses, function calling, and context management with enterprise-grade security and compliance.

Azure LLM API Reference

Pipecat’s API methods for Azure OpenAI integration

Example Implementation

Complete example with function calling

Azure OpenAI Documentation

Official Azure OpenAI documentation and setup

Azure Portal

Create OpenAI resources and get credentials

Installation

To use Azure OpenAI services, install the required dependency:

uv add "pipecat-ai[azure]"

Prerequisites

Azure OpenAI Setup

Before using Azure OpenAI LLM services, you need:

1. Azure Account: Sign up at Azure Portal
2. OpenAI Resource: Create an Azure OpenAI resource in your subscription
3. Model Deployment: Deploy your chosen model (GPT-4, GPT-4o, etc.)
4. Credentials: Get your endpoint and deployment name, plus either:
   • An API key for key-based authentication, or
   • Microsoft Entra ID credentials for token-based authentication

Environment Variables

• AZURE_CHATGPT_ENDPOINT: Your Azure OpenAI endpoint URL
• AZURE_CHATGPT_MODEL: Your model deployment name
• AZURE_CHATGPT_API_KEY: Your Azure OpenAI API key (required unless using token_provider)

Configuration

endpoint · str · required

Azure OpenAI endpoint URL. Ending it in /openai/v1 selects the v1 API surface (recommended), where api_version does not apply. Example: "https://my-resource.openai.azure.com/openai/v1".

api_key · str | None · default: "None"

Azure OpenAI API key for key-based authentication. Required unless token_provider is given.

token_provider · AzureTokenProvider | None · default: "None"

Async callable supplying a Microsoft Entra ID bearer token, used instead of api_key when given. Build one with azure.identity.aio.get_bearer_token_provider() and the https://ai.azure.com/.default scope.

model · str · default: "None" · deprecated

Deprecated in v0.0.105. Use settings=AzureLLMService.Settings(model=...) instead.

api_version · str | None · default: "None" · deprecated

Deprecated in v1.8.0. Use an endpoint ending in /openai/v1 instead. Azure API version applied to endpoints outside the v1 API surface. Defaults to 2025-04-01-preview.

settings · AzureLLMService.Settings · default: "None"

Runtime-configurable settings. See Settings below.

Since AzureLLMService inherits from OpenAILLMService, it also accepts the following parameters:

params · InputParams · default: "None" · deprecated

Deprecated in v0.0.105. Use settings=AzureLLMService.Settings(...) instead.

retry_timeout_secs · float · default: "5.0"

Request timeout in seconds. Used when retry_on_timeout is enabled to determine when to retry.

retry_on_timeout · bool · default: "False"

Whether to retry the request once if it times out. The retry attempt has no timeout limit.

Settings

Runtime-configurable settings passed via the settings constructor argument using AzureLLMService.Settings(...). These can be updated mid-conversation with LLMUpdateSettingsFrame. See Service Settings for details.

AzureLLMService uses the same settings as OpenAILLMService. See the OpenAI LLM Settings section for the full parameter reference.

Usage

Basic Setup

from pipecat.services.azure.llm import AzureLLMService

llm = AzureLLMService(
    api_key=os.getenv("AZURE_CHATGPT_API_KEY"),
    endpoint="https://my-resource.openai.azure.com/openai/v1",
    settings=AzureLLMService.Settings(
        model=os.getenv("AZURE_CHATGPT_MODEL"),
    ),
)

With Microsoft Entra ID Authentication

from azure.identity.aio import DefaultAzureCredential, get_bearer_token_provider
from pipecat.services.azure.llm import AzureLLMService

llm = AzureLLMService(
    token_provider=get_bearer_token_provider(
        DefaultAzureCredential(), "https://ai.azure.com/.default"
    ),
    endpoint="https://my-resource.openai.azure.com/openai/v1",
    settings=AzureLLMService.Settings(
        model="my-deployment",
    ),
)

With Custom Settings

from pipecat.services.azure.llm import AzureLLMService

llm = AzureLLMService(
    api_key=os.getenv("AZURE_CHATGPT_API_KEY"),
    endpoint="https://my-resource.openai.azure.com/openai/v1",
    settings=AzureLLMService.Settings(
        model=os.getenv("AZURE_CHATGPT_MODEL"),
        temperature=0.7,
        max_completion_tokens=1000,
        frequency_penalty=0.5,
    ),
)

Updating Settings at Runtime

Model settings can be changed mid-conversation using LLMUpdateSettingsFrame:

from pipecat.frames.frames import LLMUpdateSettingsFrame
from pipecat.services.openai.base_llm import OpenAILLMSettings

await worker.queue_frame(
    LLMUpdateSettingsFrame(
        delta=OpenAILLMSettings(
            temperature=0.3,
            max_completion_tokens=500,
        )
    )
)

Notes

• Deployment name vs model name: The model parameter should be your Azure deployment name, not the underlying model name (e.g., use "my-gpt4-deployment" instead of "gpt-4").
• v1 API surface: Endpoints ending in /openai/v1 use Azure’s v1 API, which tracks new features without requiring a dated api_version. This is the recommended surface. Endpoints without this suffix route through the dated API version 2025-04-01-preview (the last version Azure issued).
• Authentication options: Both key-based authentication (api_key) and Microsoft Entra ID token-based authentication (token_provider) are supported. Either api_key or token_provider must be provided.
• Full OpenAI compatibility: Since AzureLLMService inherits from OpenAILLMService, it supports all the same features including function calling, vision input, and streaming responses.

Event Handlers

AzureLLMService supports the same event handlers as OpenAILLMService, inherited from LLMService:

Event	Description
on_completion_timeout	Called when an LLM completion request times out
on_function_calls_started	Called when function calls are received and execution is about to start

@llm.event_handler("on_completion_timeout")
async def on_completion_timeout(service):
    print("LLM completion timed out")

The InputParams / params= pattern is deprecated as of v0.0.105. Use Settings / settings= instead. See the Service Settings guide for migration details.