LlamaIndex: Retrieval-Centric Framework for Data Agents
What it is: LlamaIndex is a framework built around one job: getting your own data — documents, PDFs, database rows, API responses — indexed and retrievable so an LLM can answer questions grounded in it. Where LangChain and LangGraph are general-purpose agent scaffolding, LlamaIndex starts from the data-ingestion-and-retrieval side and builds agent capability outward from there.
Quick answer: Reach for LlamaIndex first when the core problem is "index this pile of documents/data and let an agent query it accurately" — its data connectors and index types are the most mature part of the ecosystem for that specific job. Reach for a general framework first when retrieval is a small part of a larger multi-step agent.
Why it matters for data work
Most "chat with your data" requests are really retrieval-augmented generation problems: chunk the source material sensibly, embed it, retrieve the right pieces for a given question, and generate an answer grounded in them. LlamaIndex's data connectors (LlamaHub) and index abstractions — vector, summary, keyword, and knowledge-graph indexes — cover this ground with less custom plumbing than wiring loaders and retrievers together by hand.
Install & configure
Install from PyPI:
pip install llama-indexSet your model provider's API key, load documents with a SimpleDirectoryReader or a connector for your source system, build an index (VectorStoreIndex.from_documents(...) is the common default), and query it with index.as_query_engine(). For agents that call tools in addition to querying an index, wrap the query engine as a tool and pass it to LlamaIndex's own agent classes or another framework's agent loop.
LlamaIndex or LangChain — retrieval-first vs general purpose?
Both frameworks now cover overlapping ground, but the center of gravity differs. LlamaIndex's index types, chunking strategies, and retrieval evaluation tools are deeper and more opinionated, which pays off when retrieval quality is the thing you'll spend the most time tuning. LangChain's strength is the sheer breadth of its integration catalog and its chain/agent primitives for tasks that go well beyond retrieval. Many teams use LlamaIndex purely for the ingest-index-retrieve layer and hand the retrieved context to whichever agent framework runs the rest of the pipeline — the two are not mutually exclusive.
Limits and gotchas
Retrieval quality is sensitive to chunking and embedding choices that don't have a universal default — the out-of-the-box settings are a reasonable starting point, not a finished pipeline, and you should expect to tune chunk size, overlap, and the embedding model against your own documents. Re-indexing after a data connector or embedding model change means re-embedding everything, which costs real API calls and time on large corpora. And like any RAG setup, the agent is only as accurate as what retrieval surfaces — a wrong or missing chunk produces a confident, wrong answer, not a visible error.
Troubleshooting
Most issues trace back to retrieval quality rather than the framework itself.
- Answers ignore relevant document content: check the retrieved chunks directly (most query engines expose source nodes) before assuming the LLM is at fault — the chunk may never have been retrieved.
- Index build is slow or expensive: batch embedding calls and cache embeddings so re-running the pipeline doesn't re-embed unchanged documents.
- Irrelevant chunks dominate retrieval: tune chunk size/overlap or switch index type (summary vs vector) to match how your documents are structured.
- Vector store connection errors: confirm the vector store client version matches the one LlamaIndex's integration package expects — these evolve somewhat independently.
Author & links
Author: LlamaIndex, Inc.
Repo: github.com/run-llama/llama_index
License: MIT
Related skills
For broader chain and agent primitives see LangChain. For querying structured local data directly with SQL instead of a vector index, see the DuckDB MCP server.
← Back to Agent Frameworks