A task is a single operation made by the user. For example, a user sending a question to ChatGPT and receiving an answer is a task.

A session groups multiple tasks that happen in the same context. For example, multiple messages in the same ChatGPT chat is a session.

A user is the end user of your LLM app. For example, the human chatting with ChatGPT.

Tasks, sessions and users are just abstractions. They are meant to help you understand the context of a log. You can use them as you want.

For example,

  • A task can be “Fetch documents in a database” for a RAG.
  • A session can be “The code completions in a single file” for a coding copilot.
  • A user can be “The microservice querying the API” for a question answering model.

​
Tasks

​
Inputs and Outputs

A task is made of an input and an optional output, which are text readable by humans. Think of them as the messages in a chat.

On top of that, you can pass a raw_input and a raw_output. Those are the raw data that your LLM app received and produced. They are mostly meant for the developers of your LLM app.

​
Metadata

To help you understand the context of a task, you can pass a metadata dict to your tasks.

For example, the version of the model used, the generation time, the system prompt, the user_id, etc.

The system_prompt metadata is used for task success evaluation.
import phospho

phospho.init()

phospho.log(
    input="What is the meaning of life?",
    output="42",
    # Metadata
    raw_input={"chat_history": ...},
    metadata={
        "system_prompt": "You are a helpful assistant.",
        "version_id": "1.0.0",
        "generation_time": 0.1,
    },
)

The metadata is a dictionary that can contain any key-value pair. We recommend to stick to str keys and str or float values.

Note that the output is optional, but the input is required.

​
Tasks are not just calls to LLMs

A task can be a call to a LLM. But it can also be something completely different.

For example, a task can be a call to a database, or the result of a complex chain of thought.

Tasks are an abstraction that you can use as you want.

​
Task Id

By default, when logging, a task id is automatically generated for you.

Generating your own task id is useful to attach user feedback later on (on this topic, see User Feedback).

​
Sessions

If you’re using phospho in a conversational app such a chatbot, group tasks together into sessions.

  • Sessions are easier to read for humans.
  • They improve evaluations and event detections by providing context.
  • They help you understand the user journey.

​
Session Id

To create sessions, pass a session_id when logging.

The session id can be any string. However, we recommend to use a UUID generated by a random hash function. We provide a helper function to generate a session id.

session_id = phospho.new_session()

phospho.log(
    input="What is the meaning of life?",
    output="42",
    session_id=session_id,
)

​
Session insights

Sessions are useful for insights about short term user behavior.

  • Monitor for how long a user chats with your LLM app before disconnecting
  • Compute the average number of messages per session
  • Discover what kind of messages ends a session.

​
Users

Find out how specific users interact with your LLM app by logging the user id.

To do so, attach tasks and sessions to a user_id when logging. The user id can be any string.

phospho.log(
    input="What is the meaning of life?",
    output="42",
    user_id="roger@gmail.com",
)

User analytics are available in the tabs Insights/Users.

  • Discover aggregated metrics (number of tasks, average session duration, etc.)
  • Access the tasks and sessions of a user by clicking on the corresponding row.

Monitoring users helps you discover power users of your app, abusive users, or users who are struggling with your LLM app.

Was this page helpful?

Sentiment AnalysisBacktesting