Skip to main content

Understanding User Feedback

User feedback is crucial for improving the performance and accuracy of your LLM applications. It provides direct insights into user satisfaction and helps identify areas for improvement.

Types of Feedback

  • Binary feedback (thumbs up/down)
  • Numeric ratings (e.g., 1-5 stars)
  • Textual feedback

How to Pass User Feedback

When you are using Langtrace, you can pass both user id and user feedback using the Langtrace.withLangTraceRootSpan function for typescript or with_langtrace_root_span decorator in python. This helps with understanding usage patterns and measure the accuracy of your application from your user’s perspective.

Installation

Install and initialize Langtrace SDK. Refer to the installation guide for more information.
// Must precede any llm module imports
import * as Langtrace from "@langtrase/typescript-sdk";
Langtrace.init({ api_key: "<LANGTRACE_API_KEY>" });
# Must precede any llm module imports
from langtrace_python_sdk import langtrace
langtrace.init(api_key = '<LANGTRACE_API_KEY>')

Implementing User Feedback

Use the Langtrace.withLangTraceRootSpan function to trace interaction which provides spanId and traceId for the interaction.
import * as Langtrace from '@langtrace/typescript-sdk';
import OpenAI from 'openai';

// Initialize dotenv
import dotenv from 'dotenv'
dotenv.config()

// Initialize Langtrace SDK
Langtrace.init()

const openai = new OpenAI()

// Function to handle OpenAI interaction and user feedback
export const run = async (): Promise<void> => {
  await Langtrace.withLangTraceRootSpan(async (spanId, traceId) => {
    const response = await openai.chat.completions.create({
      model: 'gpt-4o-mini',
      messages: [
        { role: 'system', content: 'Talk like a pirate' },
        { role: 'user', content: 'Tell me a story in 3 sentences or less.' }
      ],
      stream: false
    })

    // Send traceId and spanId to your client
    // Collect user feedback (This is likely going to be another route in your application) For this example, we are sending the feedback immediately after the interaction
    const userScore = 5 // Example user score
    const userId = 'user123' // Example user ID
    await Langtrace.sendUserFeedback({ spanId, traceId, userScore, userId })
  })
}

from dotenv import find_dotenv, load_dotenv
from openai import OpenAI
from langtrace_python_sdk import langtrace, with_langtrace_root_span, SendUserFeedback

_ = load_dotenv(find_dotenv())

# Initialize Langtrace SDK
langtrace.init()
client = OpenAI()


def api(span_id, trace_id):
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {"role": "user", "content": "What is the best place to live in the US?"},
        ],
        stream=False,
    )

    # Collect user feedback and send it to Langtrace
    user_score = 1  # Example user score
    user_id = 'user_1234'  # Example user ID
    data = {
        "userScore": user_score,
        "userId": user_id,
        "spanId": span_id,
        "traceId": trace_id
    }
    SendUserFeedback().evaluate(data=data)

    # Return the response
    return response.choices[0].message.content


# wrap the API call with the Langtrace root span
wrapped_api = with_langtrace_root_span()(api)

# Call the wrapped API
wrapped_api()

Example user score

Collect the userScore and userId from your application and pass it to Langtrace.sendUserFeedback({userScore, userId, traceId, spanId})function along with traceId and spanId.
import * as Langtrace from "@langtrase/typescript-sdk";
await Langtrace.sendUserFeedback({spanId, traceId, userScore, userId});
from langtrace_python_sdk import SendUserFeedback
SendUserFeedback.evaluate(spanId, traceId, userScore, userId)

Best Practices

  1. Keep feedback collection simple and non-intrusive
  2. Clearly communicate how feedback will be used
  3. Act on feedback to improve your application
  4. Regularly analyze feedback trends

Privacy Considerations

When collecting user feedback, ensure you’re complying with relevant data protection regulations. Always inform users about data collection and obtain necessary consents.