LangChain vs LangGraph vs LlamaIndex vs Semantic Kernel: Which LLM Framework Should You Use?

Introduction

What Are LLM Frameworks?

An LLM framework is the scaffolding that sits between "call a model API" and "build a working product." A raw call to a chat completions endpoint gives you text in, text out. Everything else — retrieving relevant documents, calling tools, keeping track of conversation state, running a multi-step plan, retrying a failed step — is left to you. Frameworks exist to standardize that scaffolding so you're not reinventing prompt templates, retrieval pipelines, and agent loops on every project.

Why Do We Need Frameworks for Generative AI?

You don't, strictly — plenty of production systems call the model API directly with a thin layer of custom code, and that's often the right call for a narrow, well-understood task. Frameworks earn their place once the problem grows in one of a few directions: the app needs to ground answers in your own documents (retrieval-augmented generation), the app needs the model to take multiple steps and call tools autonomously (agents), or the app needs to coordinate several of those steps with branching, retries, and state that persists across turns (orchestration). Each of the four frameworks below leans into a different corner of that space.

LangChain vs LangGraph vs LlamaIndex vs Semantic Kernel at a Glance

Framework	Primary strength	Mental model
LangChain	General-purpose LLM app building blocks	Composable chains of prompts, models, and tools
LangGraph	Stateful, controllable agent workflows	A graph of nodes and edges with explicit state
LlamaIndex	Data ingestion and retrieval for RAG	Your documents, indexed and queryable
Semantic Kernel	Enterprise-grade, multi-language AI orchestration	Plugins and planners inside your existing app

None of these are mutually exclusive. The most common production pattern, in fact, uses more than one at once — LlamaIndex handling retrieval, LangGraph handling the agent loop that calls it. More on that combination later.

What Is LangChain?

What Is LangChain?

LangChain is a general-purpose framework for building applications powered by language models. It provides standardized interfaces for prompts, models, output parsing, memory, retrieval, and tool calling, so swapping an OpenAI model for an Anthropic one — or adding a vector store — doesn't mean rewriting your application logic.

Key Features of LangChain

• Model-agnostic interfaces — the same chain code runs against different LLM providers
• Prompt templates — reusable, parameterized prompts instead of string concatenation
• Chains — composable sequences of calls (prompt → model → parser → next step)
• Tool calling — a standard way to expose functions the model can invoke
• Memory and retrieval integrations — connectors for dozens of vector stores and document loaders

How LangChain Works

The core abstraction is the Runnable — anything that takes an input and produces an output, and can be piped into the next Runnable with the | operator (LangChain Expression Language, or LCEL). A chain is just a pipeline of these.

from langchain_anthropic import ChatAnthropic
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser

model = ChatAnthropic(model="claude-sonnet-4-6")
prompt = ChatPromptTemplate.from_template(
    "Summarize the following in one sentence:\n\n{text}"
)

chain = prompt | model | StrOutputParser()

result = chain.invoke({"text": "Long article text goes here..."})
print(result)

LangChain Architecture

Prompt Template
      ↓
   Chat Model
      ↓
 Output Parser
      ↓
  (optional) Tool Call → Tool Execution → back into the chain

LangChain for RAG

LangChain provides retrievers, text splitters, and vector-store integrations that plug directly into a chain: split documents, embed them, store them, retrieve the top-k matches for a query, and inject them into the prompt.

from langchain_community.vectorstores import Chroma
from langchain_core.runnables import RunnablePassthrough

retriever = Chroma(embedding_function=embeddings).as_retriever(search_kwargs={"k": 5})

rag_chain = (
    {"context": retriever, "question": RunnablePassthrough()}
    | prompt
    | model
    | StrOutputParser()
)

answer = rag_chain.invoke("What is the policy on data retention?")

LangChain for AI Agents

LangChain exposes prebuilt agent constructors that wrap the reasoning loop — decide, call a tool, observe, repeat — around a set of tools you provide.

from langchain.agents import create_tool_calling_agent, AgentExecutor

agent = create_tool_calling_agent(model, tools, prompt)
executor = AgentExecutor(agent=agent, tools=tools, max_iterations=10)

result = executor.invoke({"input": "What's the weather in Tangier, then convert it to Fahrenheit"})

When Should You Use LangChain?

When you need a general-purpose toolkit for a single, relatively linear LLM application — a RAG chatbot, a document summarizer, a simple single-agent tool-caller — and you want broad ecosystem support (model providers, vector stores, loaders) without writing that glue code yourself.

Advantages and Limitations of LangChain

Advantages	Limitations
Huge ecosystem of integrations	Abstraction layers can obscure what's actually happening
Fast to prototype with	Complex, branching agent logic gets awkward to express
Large community, lots of examples	API has changed significantly across versions
Works with nearly every model provider	Less control over execution flow than a graph-based approach

What Is LangGraph?

What Is LangGraph?

LangGraph is a library, built by the LangChain team, for constructing stateful, multi-step LLM applications as an explicit graph: nodes are units of work, edges define how control flows between them, and a shared state object is passed and updated at every step. It exists specifically because LangChain's chain-based composition gets unwieldy once an agent's control flow needs branching, loops, or persistence.

LangGraph vs LangChain

Aspect	LangChain	LangGraph
Control flow	Mostly linear chains, some branching	Explicit graph — any node can route to any other
State	Implicit, passed through the chain	Explicit, typed state object updated at each node
Loops / cycles	Awkward — agents wrap this for you	Native — a graph can cycle back to earlier nodes
Persistence	Manual	Built-in checkpointing, so a run can pause and resume
Best for	Simple to moderately complex apps	Complex, long-running, or multi-agent workflows

Understanding Nodes, Edges and State

from langgraph.graph import StateGraph, END
from typing import TypedDict

class AgentState(TypedDict):
    question: str
    context: str
    answer: str

def retrieve(state: AgentState) -> AgentState:
    state["context"] = retriever.invoke(state["question"])
    return state

def generate(state: AgentState) -> AgentState:
    state["answer"] = model.invoke(f"{state['context']}\n\n{state['question']}")
    return state

graph = StateGraph(AgentState)
graph.add_node("retrieve", retrieve)
graph.add_node("generate", generate)
graph.set_entry_point("retrieve")
graph.add_edge("retrieve", "generate")
graph.add_edge("generate", END)

app = graph.compile()
result = app.invoke({"question": "What is the refund policy?"})

Building Stateful AI Agents

Because state persists explicitly across nodes, a LangGraph agent can loop — retrieve, check if the answer is good enough, retrieve again with a refined query, generate — with the looping logic expressed as a conditional edge rather than bolted-on retry code.

def should_retry(state: AgentState) -> str:
    return "generate" if len(state["context"]) > 0 else "retrieve"

graph.add_conditional_edges("retrieve", should_retry, {
    "generate": "generate",
    "retrieve": "retrieve",
})

LangGraph for Multi-Agent Systems

Each agent can be its own subgraph or node, with a coordinator node routing between them based on the current state — the same orchestrator–worker pattern covered in The Agentic Loop, expressed as graph edges instead of nested function calls.

Human-in-the-Loop Workflows

LangGraph's checkpointing lets a graph pause at a node — say, before an irreversible action — persist its state, and wait for external approval before resuming. This is the same guardrail principle discussed in The Agent Harness, implemented natively at the framework level rather than as custom middleware.

When Should You Use LangGraph?

When your workflow has real branching logic, needs to loop until a condition is met, involves multiple cooperating agents, or needs to pause for human approval and resume later. If you find yourself writing a while loop with manual state tracking around a LangChain agent, that's the signal to move to LangGraph.

Advantages and Limitations of LangGraph

Advantages	Limitations
Explicit, debuggable control flow	More upfront design work than a simple chain
Native cycles and conditional routing	Steeper learning curve
Built-in persistence and human-in-the-loop	Overkill for a single-step task
Composes well with plain LangChain components	Still a young, fast-moving API

What Is LlamaIndex?

What Is LlamaIndex?

LlamaIndex is a data framework purpose-built for connecting LLMs to your own data. Where LangChain treats retrieval as one integration among many, LlamaIndex treats it as the entire point: ingesting documents from dozens of sources, indexing them efficiently, and exposing high-quality retrieval and query interfaces on top.

LlamaIndex and RAG

RAG is LlamaIndex's home turf. It provides multiple index types (vector, list, tree, keyword), multiple retrieval strategies, and query engines that go beyond naive top-k similarity search — sub-question decomposition, routing between indexes, and response synthesis over multiple retrieved chunks.

Document Ingestion and Indexing

from llama_index.core import VectorStoreIndex, SimpleDirectoryReader

documents = SimpleDirectoryReader("./policy_docs").load_data()
index = VectorStoreIndex.from_documents(documents)

query_engine = index.as_query_engine(similarity_top_k=5)
response = query_engine.query("What is the maximum data retention period?")
print(response)

The chunking, embedding-model choice, and index type decisions here follow the same principles covered in Building a Production RAG Pipeline — LlamaIndex mainly changes how much of that plumbing you write by hand.

Vector Search and Retrieval

LlamaIndex integrates with the same vector databases used elsewhere in the ecosystem — pgvector, Chroma, Pinecone, Weaviate — and adds retrieval-quality features like auto-merging retrieval (combining child chunks back into their parent context) and hybrid search out of the box.

LlamaIndex for Knowledge Bases

Beyond single-shot Q&A, LlamaIndex's ChatEngine and multi-document agents let you build a persistent, queryable knowledge base — useful for the kind of internal documentation or policy-manual search that comes up constantly in enterprise deployments.

LlamaIndex vs LangChain

Aspect	LlamaIndex	LangChain
Core focus	Data ingestion, indexing, retrieval	General-purpose LLM app building
RAG depth	Very deep — many index/retrieval strategies	Solid, but retrieval is one feature among many
Agents	Supported, less central	First-class, extensive tooling
Typical role	The retrieval layer inside a larger app	The application layer itself

When Should You Use LlamaIndex?

When retrieval quality over your own documents is the core problem — a document Q&A system, an internal search tool, a knowledge-base assistant — and you want more retrieval sophistication than a basic top-k vector search out of the box.

Advantages and Limitations of LlamaIndex

Advantages	Limitations
Best-in-class retrieval strategies	Less mature agent/orchestration tooling than LangChain or LangGraph
Wide range of data connectors (LlamaHub)	Smaller ecosystem outside of RAG use cases
Query engines beyond naive similarity search	Another framework to learn if you already know LangChain

What Is Microsoft Semantic Kernel?

What Is Semantic Kernel?

Semantic Kernel (SK) is Microsoft's open-source SDK for integrating LLMs into existing applications, with first-class support across C#, Python, and Java. It's built around the idea of embedding AI capabilities into software you already have, rather than building a standalone AI-first application from scratch.

Semantic Kernel Architecture

Application Code
      ↓
    Kernel
      ↓
┌─────────┬──────────┬───────────┐
│ Plugins │ Planners │  Memory   │
└─────────┴──────────┴───────────┘
      ↓
  AI Service (Azure OpenAI, OpenAI, etc.)

Plugins and AI Functions

A plugin in SK groups related functions — native code or prompt-based — that the kernel can invoke, conceptually similar to LangChain tools but designed to sit alongside existing enterprise codebases.

var kernel = Kernel.CreateBuilder()
    .AddAzureOpenAIChatCompletion(deploymentName, endpoint, apiKey)
    .Build();

kernel.ImportPluginFromType();

var result = await kernel.InvokePromptAsync(
    "What time is it, and summarize it for a user in Morocco."
);
Console.WriteLine(result);

Semantic Kernel for AI Agents

SK's planner components let the kernel decompose a goal into a sequence of plugin calls automatically, similar in spirit to LangChain's agents but tuned for enterprise workflows that mix AI calls with existing business logic and internal APIs.

Semantic Kernel and Azure OpenAI

SK has the deepest first-party integration with Azure OpenAI of any framework here, including Azure-specific auth, content filtering, and telemetry — a natural fit if your organization is already standardized on Azure.

Semantic Kernel for Enterprise Applications

Because SK is designed to be embedded into existing .NET and Java applications rather than to be the application, it fits naturally into enterprises with established codebases, compliance requirements, and multi-language teams — where "rewrite this in Python" isn't a realistic option.

Semantic Kernel vs LangChain

Aspect	Semantic Kernel	LangChain
Language support	C#, Python, Java (first-class)	Python, JavaScript/TypeScript
Target environment	Existing enterprise applications	Standalone AI applications
Ecosystem size	Smaller, Microsoft-centric	Larger, more community integrations
Cloud integration	Deepest with Azure	Broad, provider-agnostic

When Should You Use Semantic Kernel?

When you're adding AI capabilities to an existing C#, Java, or enterprise Python application, especially one already on Azure, and you need first-party support across those languages rather than a Python-only tool.

LangChain vs LangGraph vs LlamaIndex vs Semantic Kernel

Feature Comparison

Feature	LangChain	LangGraph	LlamaIndex	Semantic Kernel
LLM applications	Yes	Yes	Yes	Yes
RAG	Strong	Moderate	Strongest	Moderate
Simple agents	Strongest	Strong	Moderate	Strong
Stateful / cyclic agents	Moderate	Strongest	Weak	Moderate
Multi-agent systems	Moderate	Strongest	Moderate	Strong
Document processing	Strong	Moderate	Strongest	Moderate
Workflow control	Moderate	Strongest	Moderate	Strong
Enterprise / non-Python	Moderate	Moderate	Moderate	Strongest
Python support	Strongest	Strongest	Strongest	Strong

Read this table as "where does each framework's design center of gravity sit," not as a strict ranking — LlamaIndex can build an agent, LangGraph can do RAG, but each is optimized for the row where it scores highest.

RAG: Which Framework Should You Choose?

LangChain for RAG

Good default when RAG is one piece of a broader app that also needs agents, memory, and tool calling under one roof.

LlamaIndex for RAG

Better default when retrieval quality itself is the product — more index types, better chunking strategies, and query engines built specifically for document-heavy use cases.

LangGraph for Advanced RAG Workflows

Reach for this when RAG needs to be agentic — deciding whether to retrieve again, routing between multiple retrievers, or validating an answer against sources before returning it, all as explicit graph logic.

Semantic Kernel for Enterprise RAG

The right fit when the RAG system needs to live inside an existing enterprise C#/Java application rather than as a new standalone Python service.

Simple RAG vs Agentic RAG

Simple RAG:    Query → Retrieve → Generate → Answer
Agentic RAG:   Query → Plan → Retrieve → Evaluate → (Retrieve again?) → Generate → Answer

AI Agents: Which Framework Is Best?

Simple Agent
    ↓
LangChain

Complex Stateful Agent
    ↓
LangGraph

Data-Centric Agent
    ↓
LlamaIndex

Enterprise Agent
    ↓
Semantic Kernel

As agent complexity grows — more tools, longer horizons, human approval steps — the case for LangGraph's explicit state and control flow gets stronger. The guardrail and sandbox concerns that apply once an agent executes real actions are covered in The Agent Harness and Sandboxing AI Agents, and apply regardless of which of these four frameworks sits on top.

Real-World Architecture Examples

RAG Chatbot Architecture

User
 ↓
Retriever
 ↓
Vector Database
 ↓
LLM
 ↓
Answer

Agentic RAG Architecture

User
 ↓
Planner
 ↓
Retriever
 ↓
Tools
 ↓
Validator
 ↓
LLM
 ↓
Answer

Multi-Agent Architecture

             User
               ↓
            Planner
          ↙    ↓    ↘
     Research  Data  Analysis
          ↘    ↓    ↙
           Validator
               ↓
             Answer

Code Examples

Simple LangChain Example

chain = prompt | model | StrOutputParser()
chain.invoke({"text": "..."})

LangGraph Agent Example

graph = StateGraph(AgentState)
graph.add_node("retrieve", retrieve)
graph.add_node("generate", generate)
graph.set_entry_point("retrieve")
graph.add_edge("retrieve", "generate")
app = graph.compile()

LlamaIndex RAG Example

index = VectorStoreIndex.from_documents(documents)
query_engine = index.as_query_engine()
query_engine.query("...")

Semantic Kernel Example

var kernel = Kernel.CreateBuilder()
    .AddAzureOpenAIChatCompletion(deploymentName, endpoint, apiKey)
    .Build();
await kernel.InvokePromptAsync("...");

Which Framework Should You Choose?

Choose LangChain If...

You're building a general-purpose LLM app in Python or JS, want the widest range of integrations, and your control flow is mostly linear.

Choose LangGraph If...

Your agent needs to loop, branch, coordinate multiple sub-agents, or pause for human approval and resume later.

Choose LlamaIndex If...

Retrieval over your own documents is the core problem, and you want deeper indexing and query strategies than a basic vector search.

Choose Semantic Kernel If...

You're embedding AI into an existing C#, Java, or enterprise application, especially one already standardized on Azure.

Can You Use Multiple Frameworks Together?

Yes — and in practice, this is extremely common. A typical production pattern uses LlamaIndex for ingestion and retrieval, wrapped as a tool inside a LangGraph agent that handles the orchestration and multi-step reasoning:

                Application
                     ↓
                LangGraph
                     ↓
        ┌────────────┴────────────┐
        ↓                         ↓
   LangChain                 LlamaIndex
     Tools                       RAG
        ↓                         ↓
        └────────────┬────────────┘
                     ↓
                    LLM

My Recommendations

• Best for RAG: LlamaIndex, for retrieval depth; LangChain if RAG is one part of a larger app.
• Best for AI agents: LangChain for simple tool-calling agents.
• Best for multi-agent systems: LangGraph.
• Best for document AI: LlamaIndex.
• Best for enterprise AI: Semantic Kernel.
• Best for .NET applications: Semantic Kernel.
• Best for Python developers: LangChain and LangGraph together cover the widest range of Python use cases.

The Future of LLM Frameworks

From Chains to Agents

The industry-wide shift is from static, predefined chains toward agents that decide their own steps — which is exactly why LangGraph exists as a separate library from LangChain rather than a feature within it.

Agentic RAG

Retrieval is increasingly treated as a tool an agent chooses to call, possibly multiple times with refined queries, rather than a fixed first step in a pipeline.

Multi-Agent Systems

Orchestrator–worker patterns, covered in depth in The Agentic Loop, are becoming a first-class citizen in these frameworks rather than something built by hand on top of them.

Model Context Protocol (MCP)

An emerging standard for exposing tools and data sources to any LLM application in a consistent way, reducing the need for framework-specific tool integrations.

Long-Term Memory

Frameworks are converging on layered memory — working, episodic, and semantic — rather than treating "memory" as a single conversation buffer.

AI Workflow Orchestration

The line between "agent framework" and "workflow orchestration engine" continues to blur, as production systems demand the durability and observability guarantees traditionally associated with workflow engines.

Conclusion

LangChain vs LangGraph vs LlamaIndex vs Semantic Kernel: Final Verdict

Need RAG?              → LlamaIndex / LangChain
Need AI Agent?          → LangChain
Need complex Agent?     → LangGraph
Need Multi-Agent?       → LangGraph
Need Document AI?       → LlamaIndex
Need Microsoft/.NET?    → Semantic Kernel
Need Enterprise AI?     → Semantic Kernel

None of these frameworks are competitors in the strict sense — they overlap at the edges but are each optimized for a different center of gravity: LangChain for general-purpose app building, LangGraph for controllable agent workflows, LlamaIndex for retrieval depth, and Semantic Kernel for embedding AI into existing enterprise software. The right choice is rarely "pick one forever" — it's understanding which problem you actually have this time, and reaching for the tool built around that problem.

Further Reading

• Building a Production RAG Pipeline — the retrieval fundamentals underneath both LangChain and LlamaIndex