Agent Memory Patterns - Building Agents That Remember

Stateless AI agents are easy to build but limited in capability. They forget everything between requests, can't maintain context, and provide generic responses.

Agents with memory are different. They remember past interactions, maintain context across sessions, and provide personalized experiences. This is the difference between a chatbot and an intelligent assistant.

Why Memory Matters

Consider these scenarios:

Without Memory:

• User: "What's the weather in San Francisco?"
• Agent: "It's 65°F and sunny."
• User: "What about tomorrow?"
• Agent: "What location are you asking about?"

With Memory:

• User: "What's the weather in San Francisco?"
• Agent: "It's 65°F and sunny."
• User: "What about tomorrow?"
• Agent: "Tomorrow in San Francisco will be 68°F and partly cloudy."

Memory enables natural conversations, multi-step workflows, and personalized interactions.

Types of Agent Memory

AI agents need different types of memory for different purposes:

Short-Term Memory (Working Memory)

Purpose: Maintain context within a single conversation or session

Duration: Minutes to hours

Use Cases:

• Conversation context
• Multi-turn dialogues
• Task state during execution
• Temporary user preferences

Example:

// Short-term memory stores recent conversation
const shortTermMemory = {
  conversationId: "conv_123",
  messages: [
    { role: "user", content: "Book a flight to NYC" },
    { role: "agent", content: "When would you like to travel?" },
    { role: "user", content: "Next Friday" }
  ],
  context: {
    intent: "book_flight",
    destination: "NYC",
    date: "2026-03-14"
  }
};

Long-Term Memory (Persistent Memory)

Purpose: Remember information across sessions and over time

Duration: Days to years

Use Cases:

• User preferences and history
• Learned patterns and behaviors
• Historical interactions
• Personalization data

Example:

// Long-term memory stores user profile
const longTermMemory = {
  userId: "user_456",
  preferences: {
    preferredAirlines: ["United", "Delta"],
    seatPreference: "aisle",
    dietaryRestrictions: ["vegetarian"]
  },
  history: {
    previousBookings: [...],
    frequentDestinations: ["NYC", "SF", "LA"]
  }
};

Semantic Memory (Knowledge Base)

Purpose: Store facts, knowledge, and learned information

Duration: Permanent (until explicitly updated)

Use Cases:

• Domain knowledge
• Company policies
• Product information
• FAQs and documentation

Example:

// Semantic memory stores factual knowledge
const semanticMemory = {
  facts: {
    "company_policy_refunds": "Refunds available within 30 days",
    "product_pricing_pro": "$99/month",
    "feature_availability_api": "Available on Pro plan and above"
  }
};

Episodic Memory (Event History)

Purpose: Remember specific events and experiences

Duration: Permanent (with decay over time)

Use Cases:

• Interaction history
• Problem resolution history
• Learning from past mistakes
• Audit trails

Example:

// Episodic memory stores specific events
const episodicMemory = {
  events: [
    {
      timestamp: "2026-02-20T10:30:00Z",
      type: "support_ticket",
      summary: "User reported login issue, resolved by password reset",
      outcome: "success"
    }
  ]
};

Memory Architecture Patterns

Different use cases require different memory architectures:

Pattern 1: Session-Based Memory

Best For: Chatbots, customer support, conversational agents

Architecture:

Request → Load Session Memory → Process → Update Memory → Response

Implementation:

• Store conversation history in session
• Maintain context for current interaction
• Clear memory when session ends

Pros:

• Simple to implement
• Low storage requirements
• Fast access

Cons:

• No cross-session learning
• Loses context between sessions

Pattern 2: User-Centric Memory

Best For: Personal assistants, recommendation agents, personalization

Architecture:

Request → Load User Profile → Process with Context → Update Profile → Response

Implementation:

• Store user preferences and history
• Maintain long-term user profile
• Update based on interactions

Pros:

• Personalized experiences
• Learns over time
• Cross-session continuity

Cons:

• Higher storage requirements
• Privacy considerations
• Complexity in multi-user scenarios

Pattern 3: Hybrid Memory

Best For: Complex agents requiring both short and long-term memory

Architecture:

Request → Load Short-Term + Long-Term Memory → Process → Update Both → Response

Implementation:

• Combine session context with user history
• Use short-term for immediate context
• Use long-term for personalization

Pros:

• Best of both worlds
• Rich context for decisions
• Personalized and contextual

Cons:

• Most complex to implement
• Highest storage requirements
• Requires careful memory management

Pattern 4: Shared Memory

Best For: Multi-agent systems, collaborative workflows

Architecture:

Agent A → Shared Memory ← Agent B
                ↓
            Agent C

Implementation:

• Central memory store accessible by all agents
• Agents read and write shared context
• Coordination through memory

Pros:

• Enables agent collaboration
• Shared knowledge base
• Consistent state across agents

Cons:

• Concurrency challenges
• Potential conflicts
• Requires synchronization

Implementing Memory in Practice

Let's build a customer support agent with memory:

interface AgentMemory {
  // Short-term: Current conversation
  session: {
    conversationId: string;
    messages: Message[];
    currentIssue?: string;
    resolvedIssues: string[];
  };
  
  // Long-term: User history
  user: {
    userId: string;
    name: string;
    previousIssues: Issue[];
    preferences: UserPreferences;
    satisfactionScore: number;
  };
  
  // Semantic: Knowledge base
  knowledge: {
    policies: Record<string, string>;
    solutions: Record<string, Solution>;
  };
}

async function handleRequest(request: string, userId: string) {
  // Load all memory types
  const memory = await loadMemory(userId);
  
  // Construct context from memory
  const context = {
    conversationHistory: memory.session.messages,
    userHistory: memory.user.previousIssues,
    userPreferences: memory.user.preferences,
    relevantPolicies: findRelevantPolicies(request, memory.knowledge)
  };
  
  // Process with full context
  const response = await processWithContext(request, context);
  
  return response;
}

async function updateMemory(
  userId: string,
  request: string,
  response: string
) {
  // Update short-term memory
  await addToConversation(userId, request, response);
  
  // Update long-term memory if needed
  if (isIssueResolved(response)) {
    await addToUserHistory(userId, {
      issue: extractIssue(request),
      resolution: extractResolution(response),
      timestamp: new Date()
    });
  }
  
  // Update satisfaction score
  await updateSatisfactionScore(userId, response);
}

Memory Management Best Practices

1. Memory Decay

Not all memories are equally important. Implement decay:

function calculateMemoryRelevance(memory: Memory): number {
  const age = Date.now() - memory.timestamp;
  const daysSinceCreation = age / (1000 * 60 * 60 * 24);
  
  // Exponential decay
  const decayFactor = Math.exp(-daysSinceCreation / 30);
  
  // Combine with importance
  return memory.importance * decayFactor;
}

2. Memory Compression

Summarize old conversations to save space:

async function compressOldMemories(userId: string) {
  const oldConversations = await getOldConversations(userId);
  
  for (const conversation of oldConversations) {
    // Use AI to summarize
    const summary = await summarizeConversation(conversation);
    
    // Replace detailed history with summary
    await replaceWithSummary(conversation.id, summary);
  }
}

3. Privacy-Aware Memory

Respect user privacy:

const memoryRetentionPolicy = {
  // Automatically delete after period
  shortTerm: { retention: "24 hours" },
  longTerm: { retention: "1 year" },
  
  // Allow user deletion
  userDeletionEnabled: true,
  
  // Anonymize sensitive data
  anonymizePII: true,
  
  // Encrypt at rest
  encryption: "AES-256"
};

4. Memory Retrieval Optimization

Use vector embeddings for semantic search:

async function retrieveRelevantMemories(
  query: string,
  userId: string
): Promise<Memory[]> {
  // Convert query to embedding
  const queryEmbedding = await embed(query);
  
  // Find similar memories using vector search
  const relevantMemories = await vectorSearch(
    queryEmbedding,
    userId,
    { limit: 10, threshold: 0.7 }
  );
  
  return relevantMemories;
}

Common Memory Pitfalls

Pitfall 1: Memory Overload

Problem: Loading too much memory slows down responses

Solution: Implement selective memory retrieval based on relevance

Pitfall 2: Stale Memory

Problem: Outdated information leads to incorrect responses

Solution: Implement memory expiration and refresh mechanisms

Pitfall 3: Memory Conflicts

Problem: Contradictory information in memory

Solution: Implement memory versioning and conflict resolution

Pitfall 4: Privacy Violations

Problem: Storing sensitive data without proper safeguards

Solution: Implement encryption, anonymization, and retention policies

Memory in SwiftClaw

SwiftClaw provides built-in memory management:

• Automatic Persistence - Memory saved automatically across sessions
• Configurable Retention - Set retention policies per memory type
• Vector Search - Semantic memory retrieval out of the box
• Privacy Controls - Built-in encryption and anonymization
• Memory Analytics - Monitor memory usage and performance

No database setup. No state management. Just configure and use.

Conclusion

Memory transforms AI agents from stateless responders to intelligent assistants. Implement the right memory patterns for your use case, follow best practices, and your agents will provide personalized, contextual experiences.

Start building agents with memory using SwiftClaw's built-in memory management. No infrastructure setup required.