DSPy
Langtrace and DSPy Integration Guide
DSPy is a framework for algorithmically optimizing LM prompts and weights, especially when LMs are used one or more times within a pipeline.
Langtrace has first class support for DSPy, allowing you to capture traces from your DSPy pipelines or agents automatically and analyze them in Langtrace. You can also track experiments and the corresponding metrics and evaluations if you are running DSPy experiments.
Setup
- Follow the DSPy installation guide to install DSPy.
- Install Langtrace’s SDK and initialize the SDK in your code.
- Create a project on Langtrace with type
DSPy
. - Run your DSPy pipeline or agent and view the traces in Langtrace.
- To run experiments, follow the conventions below and head over to the
Experiments
tab in Langtrace to view your experiments.
inject_additional_attributes
.
This way Langtrace knows that you are running an experiment:
- (Required)
experiment
- Experiment name. Ex:experiment 1
. - (Optional)
description
- Some useful description about the experiment. - (Optional)
run_id
- When you want to associate traces to a specific runs, pass a unique run ID. This is useful when you are runningEvaluate()
as part of your experiment where the traces specific to theEvaluate()
will appear as an individual entry.
from langtrace_python_sdk import inject_additional_attributes
optimized_program = inject_additional_attributes(lambda: optimizer.compile(
RAG(),
trainset=trainset,
max_bootstrapped_demos=3,
max_labeled_demos=4,
num_trials=15,
minibatch_size=2,
minibatch_full_eval_steps=10,
minibatch=True,
requires_permission_to_run=False,
), {'experiment': 'experiment 1', 'description': 'some useful description', 'run_id': 'run_1'})
predictor = inject_additional_attributes(lambda: optimized_program(my_question), {'experiment': 'experiment 1', 'description': 'some useful description', 'run_id': 'run_1'})
Note: The Eval Chart will appear when you run dspy’s Evaluate()
. Note: Currently the score ranges it supports are between 0 and 100. So if you have scores that do not fall within this range, it could cause some UI issues.
Checkpoints
By default, checkpoints are traced for DSPy pipelines. If you would like to disable it, set the following env var,
TRACE_DSPY_CHECKPOINT=false
Checkpoint tracing will increase the latency of executions as the state is serialized. Please disable it in production.
Project Type
When creating a project in Langtrace, select the project type as DSPy
.
Inference Metrics
Evaluation Scores
Troubleshooting
Missing LLM Calls in Traces
If you’re not seeing LLM calls in your Langtrace traces when using DSPy, consider the following:
Grouping of Spans in Trace when using ThreadpoolExecutor in DSPy
If you’re using ThreadpoolExecutor in DSPy to parallelize your modules, you may notice that the spans in the trace are not grouped together. This is because the spans are created in the same thread and are not propagated to the parent span. To resolve this issue,