Mastering Stateful Sessions: Enabling Context-Aware Interactions

Imagine interacting with an AI agent that remembers nothing from your previous statements. Each turn is a fresh start, making complex conversations or multi-step tasks frustratingly inefficient. This chapter dives into a critical aspect of building truly intelligent agents: stateful sessions.

You’ll learn how Flue Framework empowers your agents with memory, enabling them to retain context, conversation history, and custom data across multiple interactions. This capability is what transforms a simple prompt-response system into a dynamic, engaging, and truly helpful assistant. We’ll move beyond stateless API calls to build agents that understand continuity, a cornerstone for any production-ready AI product.

Before we begin, ensure you’re comfortable with basic Flue agent creation, as covered in the previous chapters. We’ll be building upon that foundation to introduce robust state management.

The Indispensable Need for Agent Memory

In the realm of AI, particularly with large language models (LLMs), a fundamental challenge is their inherently stateless nature. Each API call to an LLM is typically an isolated event; it doesn’t inherently remember past interactions. For many real-world applications, this is a significant limitation.

Think about a human assistant. If you tell them, “Please schedule a meeting,” and then later say, “Make it for tomorrow at 2 PM,” they understand the “it” refers to the meeting you just mentioned. They maintain context. A stateless LLM agent, however, might treat the second command as entirely new, asking “What are we scheduling?”

Why Stateless LLMs Fall Short for Complex Interactions

• Lack of Context: Without memory, agents cannot follow multi-turn conversations, understand pronouns, or build upon previous information. The user experience suffers when every interaction feels like the first.
• Repetitive Information: Users would constantly need to repeat details, leading to a poor user experience and increased cognitive load.
• Inefficient Task Completion: Complex tasks that require several steps or user inputs become impossible to manage within a single, isolated interaction. Imagine booking a flight without remembering previous selections.
• No Personalization: Agents cannot learn user preferences, adapt to individual styles, or maintain a personalized profile over time.

This is precisely where Flue’s agent harness architecture, with its focus on stateful sessions, provides a powerful solution. Flue orchestrates the lifecycle of these interactions, providing a memory layer around the core LLM calls.

Flue’s Approach to Stateful Sessions

Flue introduces the concept of an AgentSession to address the stateless nature of underlying LLMs. An AgentSession acts as a dedicated container for an ongoing interaction with an agent, preserving all the necessary context from one turn to the next.

📌 Key Idea: An AgentSession is like a scratchpad and memory bank for a single, continuous interaction with your agent. It’s the essential component for building conversational AI.

What an AgentSession Holds

The AgentSession object, passed into your agent’s handler function, typically provides:

1. session.data: A generic key-value store (Record<string, unknown>) where you can store any custom data your agent needs to remember. This is perfect for user preferences, task progress, temporary variables, or any other application-specific state.
2. session.history: An array that stores the chronological sequence of messages exchanged between the user and the agent. This conversation history is crucial for providing context to the LLM in subsequent turns.
3. session.id: A unique identifier for the session, allowing Flue (and its underlying storage mechanism) to retrieve and persist the correct session state.

By leveraging these properties, your agent gains the ability to:

• Maintain Conversation Flow: Pass session.history to the LLM to ensure it understands the ongoing dialogue.
• Track Task Progress: Store variables in session.data to remember where a user is in a multi-step process.
• Personalize Interactions: Save user preferences or past choices specific to that session.

How Session State Flows in Flue

Let’s visualize how a user request interacts with a Flue agent and its session state.

This diagram illustrates that every interaction with a stateful Flue agent involves retrieving and then saving the session state. This seamless flow, managed by the AgentRouteHandler, makes building context-aware agents much simpler for developers.

Distinguishing Session State from Persistent Agent State

It’s important to differentiate between two types of “state” an agent might manage:

1. Session State (Focus of this Chapter): This is temporary, scoped to a single, continuous interaction or conversation. It’s typically short-lived, lasting only as long as the user actively engages with the agent. AgentSession primarily manages this. Think of it as the agent’s short-term memory for a particular user’s current task.
2. Persistent Agent State: This refers to long-term, global data that an agent might need across all sessions or even across deployments. Examples include a coding agent’s accumulated knowledge base, a user’s long-term profile, or configuration settings. While Flue provides mechanisms to integrate with external persistent stores (like databases or key-value stores), the AgentSession itself is designed for the more ephemeral, interaction-specific context. This is the agent’s long-term memory or knowledge base.

For this chapter, we’ll concentrate on mastering the AgentSession for immediate, context-aware interactions.

Implementing a Stateful Flue Agent

Let’s put theory into practice. We’ll start by modifying a basic agent to remember the user’s name across multiple turns.

Setting Up a Basic Stateful Agent

First, create a new Flue project or use an existing one. We’ll define a simple agent that asks for the user’s name and remembers it for subsequent interactions.

Create a file named src/agents/greetingAgent.ts:

// src/agents/greetingAgent.ts
import { AgentFunction, AgentSession } from '@flue/core';

// Define an interface for our custom session data for type safety
interface GreetingSessionData {
  userName?: string; // userName is optional, as it might not be set initially
}

export const greetingAgent: AgentFunction = async (
  prompt: string,
  session: AgentSession<GreetingSessionData>, // Type our session data for strong typing
  context: Record<string, unknown> // For future use, not critical here
) => {
  // 🧠 Important: Always check if session.data properties exist before using them.
  // This handles the first interaction when the session is brand new.

  // 1. Check if we already know the user's name from previous interactions
  if (session.data.userName) {
    // If we know it, greet them by name, acknowledging their current prompt
    return `Hello again, ${session.data.userName}! You said: "${prompt}"`;
  } else {
    // If we don't know the name, try to extract it from the current prompt
    const nameMatch = prompt.match(/my name is (\w+)/i);
    if (nameMatch && nameMatch[1]) {
      const userName = nameMatch[1];
      session.data.userName = userName; // 🔥 Store the name in session data for future turns
      return `Nice to meet you, ${userName}! How can I help?`;
    } else {
      // If no name found in the prompt, ask the user to provide it
      return `I don't know your name yet. Please tell me, "My name is [your name]".`;
    }
  }
};

Explanation of the new code:

• import { AgentFunction, AgentSession } from '@flue/core';: We import AgentSession alongside AgentFunction. This gives us access to Flue’s session management capabilities.
• interface GreetingSessionData { userName?: string; }: We define a TypeScript interface to give type safety to our custom session data. This is a best practice for clarity, preventing runtime errors, and enabling better IDE support. The ? denotes that userName is optional, as it won’t exist in a brand new session.
• session: AgentSession<GreetingSessionData>: We explicitly tell TypeScript that our session.data object will conform to the GreetingSessionData interface. This ensures all interactions with session.data are type-checked.
• if (session.data.userName): This line checks if the userName property already exists in our session’s custom data. If it does, it means the agent remembers the user from a previous turn within the same session.
• session.data.userName = userName;: This is the crucial line for state management. We’re assigning the extracted name to session.data.userName, making it available for subsequent turns within the same session. Flue automatically persists this session.data when the agent’s response is sent.

Registering and Testing the Agent

Now, let’s register this agent in our Flue application. Update your src/index.ts (or equivalent entry point for your Flue application, as of June 2026):

// src/index.ts
import { createAgentRouter } from '@flue/router';
import { greetingAgent } from './agents/greetingAgent'; // Import our new agent

const router = createAgentRouter();

// Register the greeting agent under the 'greeting' path
router.agent('greeting', greetingAgent);

export default router.handler; // Export the handler for deployment (e.g., Cloudflare Worker)

To test this, you would typically interact with your Flue endpoint (e.g., via a curl command or a simple client application). Ensure your local Flue development server is running (e.g., npm run dev).

Testing in development:

• First interaction:

  curl -X POST http://localhost:3000/api/agent/greeting \
    -H "Content-Type: application/json" \
    -d '{"prompt": "Hello there, my name is Alice", "sessionId": "my-unique-session-123"}'

  Expected output: Nice to meet you, Alice! How can I help? What happened: Flue received sessionId: "my-unique-session-123". Since no session with this ID existed, it created a new one. Your agent then processed the prompt, extracted the name “Alice”, and stored it in session.data.userName. Flue saved this updated session.

• Second interaction (using the same session ID):

  curl -X POST http://localhost:3000/api/agent/greeting \
    -H "Content-Type: application/json" \
    -d '{"prompt": "Tell me a joke", "sessionId": "my-unique-session-123"}'

  Expected output: Hello again, Alice! You said: "Tell me a joke" What happened: Flue received the same sessionId. It loaded the existing session for my-unique-session-123, which now contained userName: "Alice". Your agent found session.data.userName already set and used it to greet Alice.

Notice how the agent remembers “Alice” because we passed the same sessionId. If you change the sessionId, it will start a new session and forget the name, demonstrating the session-scoped nature of the state.

Hands-on Challenge: The Persistent Task Planner

Let’s build a slightly more complex stateful agent. Your challenge is to create an agent that helps a user manage a simple task list.

Challenge: Create an agent called taskPlannerAgent that can:

1. Add tasks: If the prompt contains “add [task description]”, it should add the task to a list stored in the session.
2. List tasks: If the prompt contains “list tasks”, it should return all tasks currently in the session.
3. Acknowledge: For any other input, it should acknowledge the input and remind the user of its capabilities.

Hint:

• You’ll need an interface for your session.data that includes an array of strings, e.g., tasks: string[].
• Remember to initialize session.data.tasks as an empty array if it doesn’t exist yet, especially for the first interaction.
• Use prompt.includes() and prompt.startsWith() for simple keyword matching. Regular expressions can be more robust, but simple string methods are fine for this challenge.

What to observe/learn: Pay close attention to how the tasks array persists and grows within the session.data across multiple interactions, demonstrating the power of stateful sessions for managing dynamic data throughout a conversation.

// src/agents/taskPlannerAgent.ts
import { AgentFunction, AgentSession } from '@flue/core';

interface TaskPlannerSessionData {
  tasks: string[]; // Our tasks will be an array of strings
}

export const taskPlannerAgent: AgentFunction = async (
  prompt: string,
  session: AgentSession<TaskPlannerSessionData>,
  context: Record<string, unknown>
) => {
  // Ensure the tasks array is initialized if it doesn't exist yet.
  // This is crucial for new sessions.
  if (!session.data.tasks) {
    session.data.tasks = [];
  }

  const lowerPrompt = prompt.toLowerCase().trim();

  if (lowerPrompt.startsWith("add ")) {
    const task = prompt.substring(4).trim(); // Get task description after "add "
    if (task) {
      session.data.tasks.push(task); // Add the new task to the session's task list
      return `Added task: "${task}". You now have ${session.data.tasks.length} tasks.`;
    }
    return "Please specify a task to add, e.g., 'add Buy groceries'.";
  } else if (lowerPrompt.includes("list tasks")) {
    if (session.data.tasks.length === 0) {
      return "You currently have no tasks.";
    }
    // Format the tasks into a readable list
    const taskList = session.data.tasks.map((t, i) => `${i + 1}. ${t}`).join('\n');
    return `Your tasks:\n${taskList}`;
  } else {
    return "I can help you manage tasks. Try 'add [task]' or 'list tasks'.";
  }
};

// src/index.ts
import { createAgentRouter } from '@flue/router';
import { greetingAgent } from './agents/greetingAgent';
import { taskPlannerAgent } from './agents/taskPlannerAgent'; // Import our new agent

const router = createAgentRouter();

router.agent('greeting', greetingAgent);
router.agent('task-planner', taskPlannerAgent); // Register the task planner agent

export default router.handler;

Real-world Insights: Session Storage and Scalability

While the examples above work perfectly in a local development environment (where Flue might use in-memory storage for sessions), production systems require robust solutions for session state. An in-memory store won’t persist across server restarts or scale across multiple instances.

Where Does Session State Live in Production?

In a production deployment, especially in serverless environments like Cloudflare Workers, a simple in-memory session store is insufficient. If a new worker instance handles each request, the session data would be lost. Flue, as an agent harness, is designed to be flexible and integrate with various storage backends.

• Development (Default): Often, Flue will use an in-memory store for sessions, which is fast and simple but not persistent across restarts or scalable horizontally. This is ideal for quick local testing.
• Production (Configurable): For real-world applications handling thousands of concurrent users or millions of events per day, you need a persistent and distributed session store. Common options include:
  • Key-Value Stores: Services like Redis, Cloudflare KV, or AWS DynamoDB are excellent for storing JSON-serializable session data, keyed by sessionId. These offer low-latency access and high scalability.
  • Cloudflare Durable Objects: When deploying to Cloudflare Workers, Durable Objects provide a unique and powerful solution. They offer single-instance state for a given ID, meaning all requests for a specific sessionId are routed to the same Durable Object instance. This makes them ideal for managing complex, mutable session state across multiple worker invocations, inherently solving many concurrency challenges.
  • Databases: Traditional databases (PostgreSQL, MongoDB) can also store session data, offering more complex querying capabilities if needed, though they might introduce higher latency than dedicated KV stores for simple session lookups.

⚡ Real-world insight: For Flue agents deployed on Cloudflare Workers, configuring the session manager to utilize Cloudflare Durable Objects is a common and highly effective strategy. Durable Objects abstract away the complexities of distributed state, providing a consistent, single-instance view of your session data for any given sessionId. This ensures that even if different worker instances handle subsequent requests, they all access the same, correct session state without race conditions.

Tradeoffs of Stateful Agents

Implementing stateful agents introduces both powerful benefits and important considerations for production systems.

Benefits:

• Richer User Experience: Agents feel more intelligent and natural, leading to higher user satisfaction and engagement.
• Efficient Interactions: Users don’t need to repeat themselves, saving time and reducing cognitive load. This directly translates to faster task completion.
• Complex Task Handling: Enables multi-step workflows (like multi-page forms, booking processes) that are impossible with stateless agents.
• Reduced Token Usage (Potentially): By carefully managing and summarizing session.history, you can send only the most relevant context to the LLM, potentially reducing token costs and improving response times.

Costs & Challenges:

• Increased Infrastructure Complexity: Requires a robust, scalable, and highly available session storage backend. This adds operational overhead and potential cost.
• State Management Overhead: You must actively manage what goes into session.data and session.history to prevent bloat and maintain relevance.
• Security Concerns: Session data can contain sensitive information (e.g., user preferences, PII) and must be protected with appropriate encryption and access controls.
• Debugging Complexity: Debugging issues related to incorrect or stale state can be challenging in distributed systems, requiring good observability tools.
• Concurrency Issues: In highly concurrent scenarios (e.g., rapid-fire requests from a single user or multiple users modifying shared state), ensuring atomic updates to session data (especially with external stores) is crucial to prevent race conditions and data corruption.

Common Pitfalls & Troubleshooting

Building stateful agents is powerful, but beware of these common traps that can lead to unexpected behavior or performance issues.

Forgetting to Initialize or Manage State

• ⚠️ What can go wrong: Assuming session.data properties will always exist or be in a certain format. On a new session, session.data will be an empty object, and attempting to access session.data.someProperty directly might result in undefined errors if not handled.
• Troubleshooting: Always check for the existence of session.data properties (e.g., if (!session.data.tasks) { session.data.tasks = []; }) and provide sensible defaults. Define clear TypeScript interfaces for your session data to catch these potential errors during development.

State Bloat

• ⚠️ What can go wrong: Letting session.history grow indefinitely or storing excessively large objects in session.data. This can lead to increased storage costs, slower retrieval times, and exceeding context window limits for LLMs (which often have maximum token counts for input).
• Troubleshooting:
  • Summarize History: For long conversations, periodically summarize session.history using an LLM to condense past turns into a concise context. This involves an additional LLM call but keeps the working history small.
  • Truncate History: Implement a simple strategy to keep only the N most recent messages in session.history (e.g., the last 5-10 turns).
  • Externalize Large Data: Store large, non-critical data (like uploaded files or extensive log data) in external storage (e.g., S3 for files, a database for logs) and only keep references (e.g., IDs or URLs) in session.data.

Concurrency Issues

• ⚠️ What can go wrong: Multiple concurrent requests to the same sessionId attempting to modify session.data simultaneously, leading to lost updates or corrupted state. This is especially relevant in systems where a single user might send multiple rapid requests or if a sessionId is shared (though typically sessions are per-user).
• Troubleshooting: While Flue’s core AgentSession abstraction helps, the underlying session store must handle concurrency. If using a custom store, implement optimistic locking (e.g., version numbers for updates), pessimistic locking (e.g., explicit locks), or use a store (like Cloudflare Durable Objects) that inherently provides single-instance state for a given ID, effectively serializing operations for that session.

Summary

In this chapter, you’ve taken a significant leap towards building more intelligent and user-friendly AI agents by mastering stateful sessions in Flue Framework.

Here are the key takeaways:

• AgentSession is Key: It provides the necessary memory for agents to maintain context across multiple interactions, addressing the stateless nature of LLMs.
• session.data for Custom State: Use this object to store any application-specific data your agent needs to remember, from user names to task lists.
• session.history for Conversation: This array keeps track of the dialogue, crucial for providing conversational context to LLMs.
• Stateful vs. Persistent: Understand the difference between short-lived session state (for ongoing interactions) and long-term persistent agent knowledge.
• Production Requires External Storage: For scalable, reliable deployments, integrate Flue with distributed session stores like Cloudflare KV or, ideally for Cloudflare Workers, Durable Objects.
• Manage State Actively: Be mindful of state initialization, prevent state bloat, and consider concurrency to build robust and efficient agents.

You now have the tools to create agents that remember, learn, and engage in meaningful, multi-turn conversations. This capability is fundamental for developing sophisticated AI products that feel truly interactive.

Next, we’ll explore how to equip your agents with even more power by integrating tools and skills, allowing them to interact with external systems and perform complex actions beyond just generating text.

References

• Flue — The Agent Harness Framework
• withastro/flue: The sandbox agent framework. - GitHub
• flue/docs/deploy-cloudflare.md at main · withastro/flue - GitHub
• Cloudflare Durable Objects documentation

This page is AI-assisted and reviewed. It references official documentation and recognized resources where relevant.