DSPy: Programmatic Prompt Optimization for Data Agents
What it is: DSPy is a framework for programming — not hand-prompting — language models. Instead of writing and endlessly re-tweaking a prompt string, you declare a signature (inputs and outputs), compose signatures into a program, and let DSPy's optimizers search over prompt wording and few-shot examples to maximize a metric you define, against a set of training examples.
Quick answer: Use DSPy when you have a metric to optimize against (accuracy on a labeled set, an eval score, a validator that passes/fails) and a task worth the setup cost. For a one-off prompt you'll write once and never touch again, hand-writing it is still faster.
Why it matters for data work
Data extraction and classification tasks — "pull these six fields out of this document," "label this ticket by category" — are exactly the kind of narrow, metric-able task DSPy is built for. Instead of manually iterating on prompt wording against a handful of examples you eyeballed, you write a signature, provide a labeled dataset, and run an optimizer (like MIPROv2 or bootstrapped few-shot) that searches for prompt instructions and examples that measurably improve your metric, then re-runs automatically if you switch the underlying model.
Install & configure
Install from PyPI:
pip install dspyConfigure a language model client (DSPy talks to most providers through LiteLLM under the hood), define a Signature class describing your inputs and outputs, wrap it in a Module (such as dspy.Predict or dspy.ChainOfThought), and write a metric function. Compiling the program against a small labeled dataset with an optimizer is what actually tunes the prompt.
DSPy or hand-written prompts — when is optimization worth it?
Optimization earns its cost when three things are true: the task repeats often enough that prompt quality compounds, you can write a metric that actually measures success (not just "looks reasonable"), and you have — or can create — a labeled example set to optimize against. A weekly data-quality classifier with a few hundred labeled examples is a great fit. A prompt you'll run once against a single spreadsheet is not; just write it by hand. DSPy also pays off when you expect to swap models later, since a compiled program can be re-optimized for a new model instead of manually re-tuning the prompt from scratch.
Limits and gotchas
Compilation costs real API calls and time — an optimizer run makes many LM calls to search the prompt/example space, so budget for that before running it repeatedly in CI. DSPy's abstractions (signatures, modules, optimizers) are also a real learning curve on top of normal prompt engineering, and debugging why an optimizer landed on a particular prompt is less transparent than reading a prompt string you wrote yourself. Without a genuinely good metric, the optimizer will cheerfully optimize for the wrong thing.
Troubleshooting
Most friction comes from the optimization step rather than basic module usage.
- Optimizer run is slow or expensive: start with a smaller optimizer (bootstrapped few-shot) and a smaller training set before reaching for the heavier search-based optimizers.
- Compiled prompt performs worse on new data: your metric or training set likely doesn't represent production inputs well enough — the optimizer overfit to what you gave it.
- LM provider errors during compilation: DSPy routes through LiteLLM, so check that provider's API key and rate limits, not DSPy itself.
- Signature output doesn't parse: tighten the output field description and type in the signature rather than adding post-hoc parsing logic.
Author & links
Author: Stanford NLP
Repo: github.com/stanfordnlp/dspy
License: MIT
Related skills
For typed structured outputs without an optimization step, see Pydantic-AI. For general-purpose chains and integrations, see LangChain.
← Back to Agent Frameworks