Callbacks

class llama_index.callbacks.AimCallback(repo: Optional[str] = None, experiment_name: Optional[str] = None, system_tracking_interval: Optional[int] = 1, log_system_params: Optional[bool] = True, capture_terminal_logs: Optional[bool] = True, event_starts_to_ignore: Optional[List[CBEventType]] = None, event_ends_to_ignore: Optional[List[CBEventType]] = None, run_params: Optional[Dict[str, Any]] = None)

AimCallback callback class.

Parameters

  • repo (str, optional) – Aim repository path or Repo object to which Run object is bound. If skipped, default Repo is used.

  • experiment_name (str, optional) – Sets Run’s experiment property. β€˜default’ if not specified. Can be used later to query runs/sequences.

  • system_tracking_interval (int, optional) – Sets the tracking interval in seconds for system usage metrics (CPU, Memory, etc.). Set to None to disable system metrics tracking.

  • log_system_params (bool, optional) – Enable/Disable logging of system params such as installed packages, git info, environment variables, etc.

  • capture_terminal_logs (bool, optional) – Enable/Disable terminal stdout logging.

  • event_starts_to_ignore (Optional[List[CBEventType]]) – list of event types to ignore when tracking event starts.

  • event_ends_to_ignore (Optional[List[CBEventType]]) – list of event types to ignore when tracking event ends.

end_trace(trace_id: Optional[str] = None, trace_map: Optional[Dict[str, List[str]]] = None) → None

Run when an overall trace is exited.

on_event_end(event_type: CBEventType, payload: Optional[Dict[str, Any]] = None, event_id: str = '', **kwargs: Any) → None
Parameters

  • event_type (CBEventType) – event type to store.

  • payload (Optional[Dict[str, Any]]) – payload to store.

  • event_id (str) – event id to store.

on_event_start(event_type: CBEventType, payload: Optional[Dict[str, Any]] = None, event_id: str = '', **kwargs: Any) → str
Parameters

  • event_type (CBEventType) – event type to store.

  • payload (Optional[Dict[str, Any]]) – payload to store.

  • event_id (str) – event id to store.

start_trace(trace_id: Optional[str] = None) → None

Run when an overall trace is launched.

class llama_index.callbacks.CBEvent(event_type: CBEventType, payload: Optional[Dict[str, Any]] = None, time: str = '', id_: str = '')

Generic class to store event information.

class llama_index.callbacks.CBEventType(value)

Callback manager event types.

CHUNKING

Logs for the before and after of text splitting.

NODE_PARSING

Logs for the documents and the nodes that they are parsed into.

EMBEDDING

Logs for the number of texts embedded.

LLM

Logs for the template and response of LLM calls.

QUERY

Keeps track of the start and end of each query.

RETRIEVE

Logs for the nodes retrieved for a query.

SYNTHESIZE

Logs for the result for synthesize calls.

TREE

Logs for the summary and level of summaries generated.

class llama_index.callbacks.CallbackManager(handlers: List[BaseCallbackHandler])

Callback manager that handles callbacks for events within LlamaIndex.

The callback manager provides a way to call handlers on event starts/ends.

Additionally, the callback manager traces the current stack of events. It does this by using a few key attributes. - trace_stack - The current stack of events that have not ended yet.

When an event ends, it’s remove from the stack.

  • trace_map - A mapping of event ids to their children events.

    On the start of events, the bottom of the trace stack is used as the current parent event for the trace map.

  • trace_id - A simple name for the current trace, usually denoting the

    entrypoint (query, index_construction, insert, etc.)

Parameters

handlers (List[BaseCallbackHandler]) – list of handlers to use.

add_handler(handler: BaseCallbackHandler) → None

Add a handler to the callback manager.

as_trace(trace_id: str) → Generator[None, None, None]

Context manager tracer for lanching and shutdown of traces.

end_trace(trace_id: Optional[str] = None, trace_map: Optional[Dict[str, List[str]]] = None) → None

Run when an overall trace is exited.

on_event_end(event_type: CBEventType, payload: Optional[Dict[str, Any]] = None, event_id: str = '', **kwargs: Any) → None

Run handlers when an event ends.

on_event_start(event_type: CBEventType, payload: Optional[Dict[str, Any]] = None, event_id: str = '', **kwargs: Any) → str

Run handlers when an event starts and return id of event.

remove_handler(handler: BaseCallbackHandler) → None

Remove a handler from the callback manager.

set_handlers(handlers: List[BaseCallbackHandler]) → None

Set handlers as the only handlers on the callback manager.

start_trace(trace_id: Optional[str] = None) → None

Run when an overall trace is launched.

class llama_index.callbacks.LlamaDebugHandler(event_starts_to_ignore: Optional[List[CBEventType]] = None, event_ends_to_ignore: Optional[List[CBEventType]] = None, print_trace_on_end: bool = True)

Callback handler that keeps track of debug info.

NOTE: this is a beta feature. The usage within our codebase, and the interface may change.

This handler simply keeps track of event starts/ends, separated by event types. You can use this callback handler to keep track of and debug events.

Parameters

  • event_starts_to_ignore (Optional[List[CBEventType]]) – list of event types to ignore when tracking event starts.

  • event_ends_to_ignore (Optional[List[CBEventType]]) – list of event types to ignore when tracking event ends.

end_trace(trace_id: Optional[str] = None, trace_map: Optional[Dict[str, List[str]]] = None) → None

Shutdown the current trace.

flush_event_logs() → None

Clear all events from memory.

get_event_pairs(event_type: Optional[CBEventType] = None) → List[List[CBEvent]]

Pair events by ID, either all events or a sepcific type.

get_events(event_type: Optional[CBEventType] = None) → List[CBEvent]

Get all events for a specific event type.

get_llm_inputs_outputs() → List[List[CBEvent]]

Get the exact LLM inputs and outputs.

on_event_end(event_type: CBEventType, payload: Optional[Dict[str, Any]] = None, event_id: str = '', **kwargs: Any) → None

Store event end data by event type.

Parameters

  • event_type (CBEventType) – event type to store.

  • payload (Optional[Dict[str, Any]]) – payload to store.

  • event_id (str) – event id to store.

on_event_start(event_type: CBEventType, payload: Optional[Dict[str, Any]] = None, event_id: str = '', **kwargs: Any) → str

Store event start data by event type.

Parameters

  • event_type (CBEventType) – event type to store.

  • payload (Optional[Dict[str, Any]]) – payload to store.

  • event_id (str) – event id to store.

print_trace_map() → None

Print simple trace map to terminal for debugging of the most recent trace.

start_trace(trace_id: Optional[str] = None) → None

Launch a trace.