Custom Extractors Letta Docs

Custom Extractors Letta Docs Build custom extractors to parse specific information patterns from agent responses for evaluation. create your own extractors to pull exactly what you need from agent trajectories. while built in extractors cover common cases, custom extractors let you implement specialized extraction logic for your specific use case. why custom extractors?. Key takeaway custom extractors and graders allow you to evaluate any part of the agent's behavior. in this example, we check if the agent correctly stores fruit preferences in memory by extracting memory insert tool calls and validating their arguments.

Tutorials Letta Docs The separation of extraction and grading allows for flexible evaluation strategies—the same grader can evaluate different parts of the trajectory by using different extractors. Letta evals provides a framework for evaluating ai agents built with letta. we offer a flexible evaluation system to test different dimensions of agent behavior and the ability to write your own custom evals for use cases you care about. Extract relevant information from agent responses for evaluation using built in and custom extractors. extractors select what content to evaluate from an agent’s response. they navigate the conversation trajectory and extract the specific piece you want to grade. This page documents the python decorators provided by letta evals for extending the evaluation framework. these decorators allow users to register custom components (graders, extractors, agent factories, and setup scripts) that can be referenced in suite yaml files or used programmatically.

Connect Your Custom Rag Pipeline To A Letta Agent Letta Docs Extract relevant information from agent responses for evaluation using built in and custom extractors. extractors select what content to evaluate from an agent’s response. they navigate the conversation trajectory and extract the specific piece you want to grade. This page documents the python decorators provided by letta evals for extending the evaluation framework. these decorators allow users to register custom components (graders, extractors, agent factories, and setup scripts) that can be referenced in suite yaml files or used programmatically. Below is a quick example of creating a stateful agent and sending it a message (requires a letta api key). see the full quickstart guide for complete documentation. Introduction to letta's evaluation framework for testing and measuring agent performance. This page explains how to create custom tools that extend agent capabilities in the letta platform. custom tools are python functions that agents can invoke during execution to perform specific tasks. Use built in extractors for common patterns like json extraction, tool calls, and message content.

Installing Letta Desktop Letta Below is a quick example of creating a stateful agent and sending it a message (requires a letta api key). see the full quickstart guide for complete documentation. Introduction to letta's evaluation framework for testing and measuring agent performance. This page explains how to create custom tools that extend agent capabilities in the letta platform. custom tools are python functions that agents can invoke during execution to perform specific tasks. Use built in extractors for common patterns like json extraction, tool calls, and message content.