Using Langtrace to monitor your Azure-OpenAI LLM is quick and easy. Follow these steps:

​
Setup

  1. Install the Langtrace and Azure-OpenAI SDKs.

Note: You’ll need API keys from Langtrace and AZURE-OPENAI SERVICE. Sign up for Langtrace and/or Azure if you haven’t done so already.

Python
# Install the SDK
pip install -U langtrace-python-sdk openai
  1. Setup environment variables:
Shell
export LANGTRACE_API_KEY=YOUR_LANGTRACE_API_KEY
export AZURE_OPENAI_ENDPOINT=YOUR_AZURE_OPENAI_ENDPOINT
export AZURE_OPENAI_API_KEY=YOUR_AZURE_OPENAI_API_KEY
export AZURE_API_VERSION=YOUR_API_VERSION
export AZURE_DEPLOYMENT_NAME=YOUR_DEPLOYMENT_NAME

​
Usage

Generate a simple output with your deployment’s model:

import os
from langtrace_python_sdk import langtrace # Must precede any llm module imports
from openai import AzureOpenAI
langtrace.init(api_key = os.environ['LANGTRACE_API_KEY'])
client = AzureOpenAI(
  api_key=os.environ["AZURE_OPENAI_API_KEY"],
  api_version=os.environ['AZURE_API_VERSION'],
  azure_endpoint = os.environ["AZURE_OPENAI_ENDPOINT"]
  )
deployment_name= os.environ['AZURE_DEPLOYMENT_NAME']
print('Sending a test completion job')

# Generate a simple output with your deployment's model
response = client.chat.completions.create(
  model=os.environ['AZURE_DEPLOYMENT_NAME'],
  messages=[
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user", "content": "Does Azure OpenAI support customer managed keys?"},
      {"role": "assistant", "content": "Yes, customer managed keys are supported by Azure OpenAI."},
      {"role": "user", "content": "Do other Azure AI services support this too?"}
  ]
)

print(response.choices[0].message.content)

You can now view your traces on the Langtrace dashboard.

Want to see more supported methods? Checkout the sample code in the Langtrace Azure-OpenAI Python Example repository.

ArchCerebras