Langtrace integrates directly with LiteLLM, offering detailed, real-time insights into performance metrics such as cost, token usage, accuracy, and latency.

You’ll need API key from Langtrace. Sign up for Langtrace if you haven’t done so already.*

​
LiteLLM SDK

  1. Setup environment variables:
Shell
export LANGTRACE_API_KEY=YOUR_LANGTRACE_API_KEY
  1. Add callback to your LiteLLM client
main.py
import litellm
litellm.success_callback = ['langtrace']
  1. Use LiteLLM completion
main.py
from litellm import completion

response = completion(
    model="gpt-4",
    messages=[
        {"role": "user", "content": "this is a test request, write a short poem"}
    ],
)
print(response)

​
LiteLLM Proxy

  1. Create config.yaml:
config.yaml
model_list:
  - model_name: gpt-4
    litellm_params:
      model: openai/gpt-4

litellm_settings:
  callbacks: ["langtrace"]

environment_variables:
  LANGTRACE_API_KEY: <YOUR_LANGTRACE_API_KEY>
  1. Run LiteLLM Proxy
Shell
litellm --config config.yaml --detailed_debug
  1. Test your setup
curl --location 'http://0.0.0.0:4000/chat/completions' \
    --header 'Content-Type: application/json' \
    --data ' {
    "model": "gpt-4",
    "messages": [
        {
        "role": "user",
        "content": "what llm are you"
        }
    ]
    }'

You can now view your traces on the Langtrace dashboard

Want to see more supported methods? Checkout the sample code in the Langtrace Langchain Python Example

Vercel AIAWS Bedrock