Quickstart
Run evaluations and detect events in your messages in minutes.
Get started with phospho lab, the core of phospho. This is what the hosted version of phospho leverages to deliver insights.
The phospho lab is a tool that allows you to run evaluations and detect events in your messages.
- Define custom workloads and jobs
- Run them on your messages in parallel
- Optimize your models and configurations
Installation
Install the phospho package with the lab extra:
pip install "phospho[lab]"
You need to set your OPENAI_API_KEY as an environment variable.
export OPENAI_API_KEY=your_openai_api_key
If you don’t want to use OpenAI, you can setup Ollama and set the following environment variables:
export OVERRIDE_WITH_OLLAMA_MODEL=mistral
This will replace all calls to OpenAI models with calls to the mistral model running with Ollama. Make sure you’ve downloaded Item.
Create a workload
The phospho lab lets you run extractions on your messages.
Start by creating a workload. A workload is a set of jobs that you want to run on your messages.
from phospho import lab
# Create the phospho workload
workload = lab.Workload()
Define jobs
Define jobs and add them to the workload. For example, let’s add an event detection job. Those are the jobs you can setup in phospho cloud.
# Define the job configurations
class EventConfig(lab.JobConfig):
event_name: str
event_description: str
# Let's add an event detection task to our workload
workload.add_job(
lab.Job(
id="question_answering",
job_function=lab.job_library.event_detection,
config=EventConfig(
event_name="question_answering",
event_description="The user asks a question to the assistant",
),
)
)
Run the workload
Now, you can run the workload on your messages.
Messages are a basic abstraction. They can be user messages or LLM outputs. They can contain metadata or additional information. It’s up to the jobs to decide what to do with them.
# Let's add some messages to analyze
message = lab.Message(
id="my_message_id",
role="User",
content="What is the weather today in Paris?",
)
# Run the workload on the message
# Note that this is an async function. Use asyncio.run to run it in a script.
await workload.async_run(
messages=[message],
executor_type="sequential",
)
Gather results
Results are stored in the workload.
# Check the results of the workload
message_results = workload.results["my_message_id"]
print(f"Result of the event detection: {message_results['question_answering'].value}")
You can also get them in a pandas dataframe.
workload.results_df()
Was this page helpful?