Long-Term Memory

Long-Term Memory is the platform capability that lets users resume prior conversations and lets the AI use prior turns as context while keeping final answers grounded in retrieved source documents.

Memory types

Memory type	Stored in	Purpose
Conversation memory	conversations, messages	Resume sessions, persist user/assistant/tool turns, preserve citations and metadata
Retrieval memory	documents, chunks, embeddings, lexical text	Long-lived corpus memory used by retrieval
Audit memory	audit_log	Compliance trail for queries, answers, policy guard events, and future review/admin actions
Prompt/eval memory	Langfuse	Prompt versions, traces, eval datasets, LLM-as-judge records

Conversation memory schema

conversations (
  id,
  user_id,
  title,
  metadata,
  created_at,
  updated_at
)

messages (
  id,
  conversation_id,
  role,
  content,
  source_citations,
  metadata,
  created_at
)

The platform persists conversations, user messages, assistant messages, answer citations, and metadata for Policy Guard and synthesis.

What is stored

Field	Purpose
user_id	Associates a conversation with an authenticated or application-level user
title	Human-readable conversation title or generated label
role	Message role: user, assistant, system, or tool
content	Message body stored as text
source_citations	JSON array of document/chunk citations attached to assistant answers
metadata	Retrieval settings, policy results, synthesis faithfulness data, tool-call data, channel info, and future session attributes
timestamps	Creation and update ordering for resume and retention

Organization model

Memory is organized by user, conversation, and ordered messages. The database indexes conversations.user_id and (messages.conversation_id, messages.created_at) so session lists and chronological message loading are efficient.

Target behavior:

The memory builder uses a configurable last-N context window, with a default of 10 messages.

Retrieval and indexing strategy

Conversation memory is not the same as corpus retrieval. Corpus retrieval indexes document chunks in pgvector and PostgreSQL full-text search. Conversation memory is primarily chronological and user-scoped, with optional future summarization or semantic indexing if conversations become large.

Recommended retrieval order:

1. Load conversation metadata by conversation_id and authorized user_id.
2. Load recent messages by created_at.
3. Include only the last configured window of turns in the prompt context.
4. Retrieve corpus chunks separately through hybrid retrieval.
5. Attach citations from corpus chunks, not from uncited memory text.
6. Persist the new user and assistant messages with retrieval, policy, and faithfulness metadata.

This keeps long-term memory useful without letting uncited chat history become an unsupported source of truth.

Update lifecycle

Each query stores the conversation turn, assistant answer, source citations, and audit events for policy, query, and answer handling.

Retention policy

Retention policy for conversation history and audit logs is configurable and conservative: store enough for session resume, audit, and evaluation while allowing policy-driven deletion or archival.

Recommended policy dimensions:

• Retention duration by data class: conversation content, citations, audit events, traces, uploads, OCR review history.
• Legal hold and audit exceptions for government compliance.
• User deletion or anonymization behavior when policy allows it.
• Archival path to OBS for old conversations if PostgreSQL storage pressure grows.
• Clear separation between deleting a conversation view and retaining immutable audit events.

Security and access

Long-Term Memory must respect RBAC and ABAC. A user should only see conversations and messages they are authorized to access. Document citations in old messages must remain authorization-checked because document-level access may change after the answer was created.

Security rules:

• Scope conversation listing by authenticated user and role.
• Check document access before rendering stored citations or reusing chunks.
• Keep audit logs append-only from the application perspective.
• Store metadata as structured JSON, not prompt text blobs that bypass policy checks.
• Do not treat conversation history as trusted instructions; ingested documents and prior turns can still contain prompt injection.

Integration with the AI platform

Long-Term Memory integrates at the service layer, not as a separate frontend-only feature. The chatbot UI calls API endpoints; the backend loads memory, retrieves corpus chunks, calls models, applies Policy Guard, stores messages, and audits results.

The Synthesizer must still return source citations from retrieved chunks. Memory can clarify the user's intent, but it does not replace source-grounded retrieval.