How to compare evaluations

GitHub

x-api-key

Langtrace AI Docs

Langtrace App

Support

Langtrace allows you compare evaluations run on different models to understand the performance of your application across different models.

Compare Evaluations

Welcome to the Langtrace AI documentation

Introduction

Start shipping traces to Langtrace Cloud or your preferred OpenTelemetry-compatible backend in under 5 minutes!

Quickstart

 Collection of guides to help you get started with Langtrace.

How to Guides

Concepts

Python SDK

Langtrace Python SDK

Typescript SDK

Langtrace Typescript SDK

Features

Langtrace SDK Features

Welcome to the Langtrace AI FAQs page. Here you will find answers to the most frequently asked questions about Langtrace.

FAQs

Tracing Overview

Sending Traces

Langtrace SDK allows you to group related operations together using the `@with_langtrace_root_span` decorator for Python or `WithLangTraceRootSpan` for Typescript.

Grouping Traces

Grouping Operations

The Langtrace SDK gives you the ability to pass user feedback from your application as scores for measuring accuracy.

Passing User Feedback

Pass User Feedback

Learn how to attach user ids to traces in Langtrace. You can use the `withAdditionalAttributes` or `inject_additional_attributes` function to pass user ids. This can inturn be used within Langtrace for filtering and grouping traces.

Attach User Id

Attach User IDs to Traces

Learn how to attach prompt ids and prompt versions to traces in Langtrace. You can use the `withAdditionalAttributes` or `inject_additional_attributes` function to pass prompt ids / versions. This can inturn be used within Langtrace for filtering and grouping traces.

Attach Prompt Id / Version

Attach Prompt IDs and Versions to Traces

You can add additional attributes to traces in Langtrace by simply wrapping your code with the `withAdditionalAttributes` or `inject_additional_attributes` function. This will add the specified attributes to the trace spans.

Additional Attributes

Add Additional Attributes to Traces

You can disable tracing for specific functions per vendor by passing the `disable_tracing_for_functionss` parameter in `Langtrace.init` function. The `object` should have the vendor name as the `key` and an `array` of **function names** as the `value`.

Disable Function Tracing

Disable Tracing For Functions

Disable Tracing For Specific Vendors

Disable Tracing For Prompts & Completions

Learn how to ett custom span names to traces in Langtrace. You can use the `inject_additional_attributes` function to set custom span names to traces. This name will be appended to the operation name and will be identifed as the name of the span in the trace.

Set Custom Span Name

Set custom span names to traces

Langtrace SDK allows you to pass a session ID to traces to filter them later.

Filtering Traces by Session ID

Langtrace lets you store and version prompts, making it easy to manage and reuse them across your applications.

Manage Prompts

Langtrace lets you store and version Zod schemas for tool calling, making it easy to manage and reuse them across your applications.

Tool Calling Schema

Manage Zod Schema for Tool Calling

Langtrace has a model playground where you can interate on your prompts with different models and model settings to see how they perform.

Model Playground

Langtrace allows you run evaluations on annotated datasets and get insights on the performance of your application.

Evaluations

Langtrace allows you to create tests and manually annotate your traces to evaluate the performance and create datasets for running evaluations.

Annotations

Langtrace supports a variety of LLMs, Frameworks and Vector databases.

Overview

Langtrace and Elastic APM Integration Guide

Elastic APM

Grafana Cloud

The traces generated by Langtrace can be visualized and analyzed using SigNoz.

SigNoz

Langtrace and New Relic Integration Guide

New Relic

Datadog

Langtrace and Honeycomb Integration Guide

Honeycomb

Langtrace and IBM Instana Integration Guide

IBM Instana

Langtrace and LlamaIndex Integration Guide

LlamaIndex

Langtrace and Langchain Integration Guide

Langchain

CrewAI

DSPy

Mem0

Langtrace and Vercel AI SDK Integration Guide

Vercel AI

Vercel AI SDK

LiteLLM

Langtrace and AWS Bedrock Integration Guide

AWS Bedrock

Learn how to use Langtrace with Guardrails AI for enhanced LLM validation and tracing

Guardrails

Anthropic is an AI research organization behind Claude model family. Whether you're brainstorming alone or building with a team of thousands, Claude is here to help.

Anthropic

Azure OpenAI Service integrates OpenAI's advanced AI models, like GPT-4, with Azure's secure cloud infrastructure, enabling businesses to enhance their applications with sophisticated language processing capabilities. It provides scalable, customizable, and enterprise-grade AI solutions while ensuring data privacy and compliance. This service leverages Azure's reliability and security to offer powerful AI functionalities across various use cases.

Azure-OpenAI

Learn how to use Langtrace with Cerebras for AI workload observability

Cerebras

Cohere is an AI company specializing in advanced natural language processing (NLP) technologies. They provide powerful language models and easy-to-use APIs for text generation, sentiment analysis, summarization, and more. Cohere's solutions can be customized and fine-tuned for specific tasks, making them ideal for applications in customer support, content creation, and real-time processing.

Cohere

Gemini is an AI-driven platform that enhances conversational experiences by enabling seamless user-system interactions. Integrated with Langtrace, it provides deep language tracing and insights into dialogue patterns. This combination helps optimize AI-driven conversations for improved efficiency and user satisfaction.

Gemini

Groq provides cutting-edge AI acceleration hardware and software solutions designed to maximize the performance and efficiency of machine learning and AI workloads. Their innovative technology enables rapid processing speeds and scalability, making it ideal for complex data-intensive applications. Groq's products are tailored to deliver high performance with low latency, catering to enterprises seeking advanced AI capabilities.

Groq

Mistral AI is a cutting-edge platform focused on developing advanced large language models that prioritize efficiency and performance. Known for producing compact yet powerful AI models, Mistral AI aims to democratize access to high-performance AI by making these models more accessible and adaptable to various applications. The platform emphasizes open innovation, contributing to the AI community with tools that are both versatile and state-of-the-art.

Mistral AI

KubeAI

Guide to integrating Ollama with Langtrace for LLM monitoring

Ollama

OpenAI is a leading artificial intelligence research and deployment company committed to ensuring that artificial general intelligence(AGI) benefits all of humanity. It develops cutting-edge AI models, such as GPT-4, to assist in various applications, from natural language processing to complex problem-solving. OpenAI focuses on safety, scalability, and ethical considerations in AI advancements.

OpenAI

Perplexity is an advanced AI-powered search engine that provides instant, accurate answers to complex questions by analyzing vast amounts of information. Utilizing state-of-the-art natural language processing, it delivers precise results and insightful summaries, enhancing productivity and decision-making. Designed for both individuals and businesses, Perplexity simplifies information retrieval with its intuitive interface and powerful search capabilities.

Perplexity

xAI introduces X's Grok family of models.

Chroma gives you the tools to store embeddings and their metadata.

ChromaDB

Pinecone is a vector database that enables fast and accurate vector search for building AI applications. It provides the infrastructure for the long-term memory and retrieval needed to develop state-of-the-art AI systems.

Pinecone

Langtrace can be seamlessly integrated with Qdrant, a high-performance vector database. This integration allows you to trace and monitor your vector search operations, providing valuable insights into your AI application's performance and behavior. 

Qdrant

Weaviate is an open source vector database.

Weaviate

MongoDB is a cross-platform document-oriented database program.

MongoDB Vector Search

Milvus is a powerful open-source vector database built for scalable similarity search and AI applications.

Milvus Vector Database

Langtrace can be easily integrated with FastAPI applications with the help of the Langtrace SDK.

FastAPI

Langtrace can be easily integrated with Flask applications with the help of the Langtrace SDK.

Flask

Langtrace can be easily integrated with Django applications with the help of the Langtrace SDK.

Django

Langtrace can be easily integrated with NextJS applications with just two steps.

NextJS

Learn how to configure the OpenTelemetry Collector integration.

Configuration

OTEL Configuration

Setting up Langtrace Typescript SDK with OTEL Collector

TypeScript SDK

Setting up Langtrace Python SDK with OTEL Collector

This endpoint enables creation of a new project

POST Project

This endpoint enables creation of an api key for an existing project

POST Project API Key

This endpoint enables queries on the prompt registry. You can query for a specific prompt version or get the currently live prompt. Optionally also pass in variables to fill the prompt value variables with data

GET Prompt from Registry

This endpoint lets you download traces for a project.

POST Traces

You can self-host Langtrace to monitor your applications. This guide will help you set up Langtrace on your own infrastructure.

Learn how to use Langtrace SDK for Self-Hosted Setup

Langtrace Self-Hosting Configuration

Configure various authentication methods for your self-hosted Langtrace setup using NextAuth.js.

Getting Started

Tracing

Prompting

Evaluations & Testing

Supported Integrations

API Reference

Hosting

Contact Us

Compare Evaluations

How to compare evaluations

Getting Started

Tracing

Prompting

Evaluations & Testing

Supported Integrations

API Reference

Hosting

Contact Us

​How to compare evaluations

How to compare evaluations