Filter logs for failed LLM calls

Applies to:

Plan -
Deployment -

Summary

Goal: Filter project logs to find failed LLM spans and the traces that contain them. Features: Logs, SQL, ANY_SPAN(), FILTER_SPANS(), span_attributes.type, error, built-in LLM span errors view.

Filtering options

Use the built-in view

In the Logs view, open the view selector and select LLM span errors. This built-in view shows span-level results for LLM spans where error IS NOT NULL, excluding scorer and Topics spans (those with span_attributes.purpose = 'scorer'). This keeps the view focused on errors from your application traffic rather than Braintrust’s internal scoring and Topics processes.

Query failed LLM spans directly

To return only the failed LLM spans, use shape => 'spans':

SELECT *
FROM project_logs('<project_id>', shape => 'spans')
WHERE span_attributes.type = 'llm'
  AND error IS NOT NULL
  AND created > now() - interval 7 day
ORDER BY created DESC
LIMIT 100

Use a single ANY_SPAN() predicate for traces

Combine both conditions inside one ANY_SPAN() call. This ensures the error and the LLM type match on the same span, not on different spans within the same trace.

SELECT *
FROM project_logs('<project_id>', shape => 'traces')
WHERE ANY_SPAN(span_attributes.type = 'llm' AND error IS NOT NULL)
  AND created > now() - interval 7 day
ORDER BY created DESC
LIMIT 100

Avoid splitting the conditions into two separate ANY_SPAN() clauses when you need failed LLM calls. Separate clauses match traces where any span has an error and any span is an LLM span, even if they are different spans:

-- Incorrect: does not guarantee the LLM span itself failed
WHERE ANY_SPAN(error IS NOT NULL) AND ANY_SPAN(span_attributes.type = 'llm')

Filter spans within trace-shaped results

To return trace-shaped results but include only the failed LLM spans within each trace, use FILTER_SPANS() in the WHERE clause:

SELECT *
FROM project_logs('<project_id>', shape => 'traces')
WHERE FILTER_SPANS(span_attributes.type = 'llm' AND error IS NOT NULL)
  AND created > now() - interval 7 day
ORDER BY created DESC
LIMIT 100

Use summary metrics for counts

Use metrics.llm_errors when you need trace-level reporting or counts. Use the span-level queries above when you need to inspect the failed LLM calls themselves.

SELECT root_span_id, metrics.llm_errors
FROM project_logs('<project_id>', shape => 'summary')
WHERE metrics.llm_errors > 0
  AND created > now() - interval 7 day
ORDER BY created DESC
LIMIT 100

​Summary

​Filtering options

​Use the built-in view

​Query failed LLM spans directly

​Use a single ANY_SPAN() predicate for traces

​Filter spans within trace-shaped results

​Use summary metrics for counts

Summary

Filtering options

Use the built-in view

Query failed LLM spans directly

Use a single ANY_SPAN() predicate for traces

Filter spans within trace-shaped results

Use summary metrics for counts