LongMemEval

A benchmark released in 2024 that evaluates the long-term memory abilities of chat assistants using conversation histories spanning multiple sessions.

1 articleLast mentioned

LongMemEval is a benchmark for evaluating the long-term memory abilities of chat assistants. Released in 2024, it asks systems to answer 500 questions based on information scattered across long conversation histories spanning multiple sessions. It measures five abilities: information extraction, multi-session reasoning, temporal reasoning, knowledge updates, and abstention when there is no basis for an answer.

Systems under evaluation must either receive the entire history as context or store it in a memory system and retrieve the relevant parts before answering. The benchmark is used to compare memory designs for LLM agents and retrieval-based memory systems.

This entry is based on AIPOST articles and widely known facts. If something is wrong, please send us a correction request.

Articles covering this entry

  • Intelligence Plane: Why Memory and Ontology, Not the Model, Define AI AgentsAI in Practice ·

    On the LongMemEval benchmark, which tests long-term conversational memory, the package answered 472 of 500 questions correctly (94.4%) when paired with GPT-5 at…

    Read article

Found a mistake or missing an entry?

Wiki entries are based on AIPOST articles and widely known facts. If an entry is wrong, send us a correction request and we will check and fix it. Names from new articles are added to the wiki after the article is published.