Analyze your logs in Python

Use the phospho Python package to run custom analytics jobs on your logs.

​

Setup

Instal the package and set your API key and project ID as environment variables.

pip install phospho pandas
export PHOSPHO_API_KEY=your_api_key
export PHOSPHO_PROJECT_ID=your_project_id

​

Load logs as a DataFrame

The best way to analyze your logs is to load them into a pandas DataFrame. This format is compatible with most analytics libraries.

​

One row = one (task, event) pair

Phospho provides a tasks_df function to load the logs into a flattened DataFrame. Note that you need to have the pandas package installed to use this function.

import phospho 

phospho.init()
phospho.tasks_df(limit=1000) # Load the latest 1000 tasks

This will return a DataFrame where one row is one (task, event) pair.

Example:

This means that:

• If a task has multiple events, there will be multiple rows with the same task_id and different event_name.
• If a task has no events, it will have one row with event_name as None.

​

One row = one task

If you want one row to be one task, pass the parameter with_events=False.

phospho.tasks_df(limit=1000, with_events=False)

Result:

​

Ignore session features

To ignore the sessions features, pass the parameter with_sessions=False.

phospho.tasks_df(limit=1000, with_sessions=False)

​

Run custom analytics jobs

To run custom analytics jobs, you can leverage all the power of the Python ecosystem.

If you have a lot of complex ML models to run and LLM calls to make, consider the phospho lab that streamlines some of the work for you.

​

Update logs from a DataFrame

After running your analytics jobs, you might want to update the logs with the results.

You can use the push_tasks_df function to push the updated data back to Phospho. This will override the specified fields in the logs.

# Fetch the 3 latest tasks
tasks_df = phospho.tasks_df(limit=3)

​

Update columns

Make changes to columns. Not all columns are updatable. This is to prevent accidental data loss.

Here is the list of updatable columns:

• task_eval: Literal["success", "failure"]
• task_eval_source: str
• task_eval_at: datetime
• task_metadata: Dict[str, object] (Note: this will override the whole metadata object, not just the specified keys)

# Make some changes
tasks_df["task_eval"] = "success"
tasks_df["task_metadata"] = tasks_df["task_metadata"].apply(
    # To avoid overriding the whole metadata object, use **x to unpack the existing metadata
    lambda x: {**x, "new_key": "new_value", "stuff": 44}
)

​

Push updated data

To push the updated data back to Phospho, use the push_tasks_df function.

• You need to pass the task_id
• As a best practice, pass only the columns you want to update.

# Select only the columns you want to update
phospho.push_tasks_df(tasks_df[["task_id", "task_eval"]])

# To check that the data has been updated
phospho.tasks_df(limit=3)

You’re all set. Your custom analytics are now also available in the Phospho UI.