Logging user feedback is a crucial part of evaluating an LLM app. Even though user feedback is subjective and biased towards negative, it is a valuable source of information to improve the quality of your app.

Setup user feedback in your app to log the user feedback to phospho, review it in the webapp, improve the automatic evaluations, and make your app better.

​
Architecture: what’s the task_id?

In your app, you should collect user feedback after having logged a task to phospho. Every task logged to phospho is identified by a unique task_id.

For phospho to know what task the user is giving feedback on, you need to keep track of the task_id.

There are two ways to manage the task_id: frontend or backend.

Any way you chose, there are helpers in the phospho package to make it easier.

​
Option 1: Task id managed by Frontend

1

In your frontend, you generate a task id using UUID V4
2

You pass this task id to your backend. The backend executes the task and log the task to phospho with this task id.

3

In your frontend, you collect user feedback based on this task id.

​
Option 2: Task id managed by Backend

1

In your frontend, you ask your backend to execute a task.
2

The backend generates a task id using UUID V4, and logs the task to phospho with this task id.

3

The backend returns the task id to the frontend.
4

In your frontend, you collect user feedback based on this task id.

​
Backend: Log to phospho with a known task_id

The phospho package provides multiple helpers to manage the task_id.

pip install phospho

Make sure you have initialized the phospho package with your project_id and api_key somewhere in your app.

import phospho
phospho.init(project_id="your_project_id", api_key="your_api_key")

You can fetch the task_id generated by phospho.log:

logged_content = phospho.log(input="question", output="answer")
task_id: str = logged_content["task_id"]

To generate a new task_id, you can use the new_task function.

task_id: str = phospho.new_task()

# Pass it to phospho.log to create a task with this id
phospho.log(input="question", output="answer", task_id=task_id)

To get the latest task_id, you can use the latest_task_id variable.

latest_task_id = phospho.latest_task_id

​
Frontend: Collect user feedback

Once your backend has executed the task and logged it to phospho with a known task_id, send the task_id back to your frontend.

In your frontend, using the task_id, you can collect user feedback and send it to phospho.

We provide React components to kickstart your user feedback collection in your app.

npm install phospho-ui-react

import "./App.css";
import { FeedbackDrawer, Feedback } from "phospho-ui-react";
import "phospho-ui-react/dist/index.css";

function App() {
  return (
    <div className="App">
      <header className="App-header">
        <FeedbackDrawer
          // Get your project_id on phospho
          projectId="..."
          // The task_id logged to phospho. Fetch it from your backend after logging
          taskId="..."
          // Source will be also logged to phospho
          source={"user_feedback"}
          // Customize the drawer
          title="Send Feedback"
          description="Help us improve our product."
          onSubmit={(feedback: Feedback) =>
            console.log("Submitted: ", feedback)
          }
          onClose={(feedback: Feedback) => console.log("Closed: ", feedback)}
        />
      </header>
    </div>
  );
}

export default App;

​
Backend: Manage user feedback collection

If you don’t want to collect user feedback in the frontend, you can instead create an endpoint in your backend and collect user feedback there.

The phospho python package provides a user_feedback function to log user feedback.

# See the previous section to get the task_id
task_id = ...

phospho.user_feedback(
    task_id=task_id,
    flag="success", # or "failure"
    flag_source="user",
    notes="Some notes (can be None)", # optional
)

Was this page helpful?

Automatic evaluationEvents Detection