Langtrace integrates directly with LiteLLM, offering detailed, real-time insights into performance metrics such as cost, token usage, accuracy, and latency.

You’ll need API key from Langtrace. Sign up for Langtrace if you haven’t done so already.*

​
LiteLLM SDK

  1. Setup environment variables:
Shell
export LANGTRACE_API_KEY=YOUR_LANGTRACE_API_KEY
  1. Add callback to your LiteLLM client
main.py
import litellm
litellm.success_callback = ['langtrace']
  1. Use LiteLLM completion
main.py
from litellm import completion

response = completion(
    model="gpt-4",
    messages=[
        {"role": "user", "content": "this is a test request, write a short poem"}
    ],
)
print(response)

​
LiteLLM Proxy

  1. Create config.yaml:
config.yaml
model_list:
  - model_name: gpt-4
    litellm_params:
      model: openai/gpt-4

litellm_settings:
  callbacks: ["langtrace"]

environment_variables:
  LANGTRACE_API_KEY: <YOUR_LANGTRACE_API_KEY>
  1. Run LiteLLM Proxy
Shell
litellm --config config.yaml --detailed_debug
  1. Test your setup

You can now view your traces on the Langtrace dashboard

Want to see more supported methods? Checkout the sample code in the Langtrace Langchain Python Example

Vercel AIAnthropic