Simple text search outperforms vectors in AI agent retrieval

This empirical study systematically compares retrieval strategies in LLM-based agents across multiple dimensions. Using 116 questions from LongMemEval, researchers tested grep-based and vector retrieval on four different agent harnesses (Chronos, Claude Code, Codex, Gemini CLI) with both inline and file-based tool results. Grep consistently achieved higher accuracy than vector retrieval in their comparisons. A second experiment introduced progressively more unrelated conversation history to test robustness. The findings show that retrieval method choice interacts strongly with agent architecture and tool-calling paradigm, indicating that practitioners should benchmark retrieval strategies for their specific agent setup rather than assuming semantic search is always superior.