Unless you’ve been living under a rock, you know how much a prompt can change an LLM answer. Change the input style—or even a comma—and the lovingly handcrafted prompt may no longer deliver the results you expect. Now imagine changing the model itself.
Building an agentic app with opinionated prompts is like having a Jenga tower waiting to collapse. We started asking: are we really going to spend time finding the best prompts? Should we convince developers to master prompt engineering, or dive into tuning? Then we stumbled on DSPy.
In a nutshell, you define inputs and outputs—such as a customer query and the proper answer, recommendation, or output—then let DSPy generate, test, and refine the actual instructions under the hood. You may call it a brute-force approach, but why not? We don’t like doing that manually anyway.
It is not as plug and play as you might think. DSPy compiles the best-performing prompts or weights to meet your goal, but it is not for every project. It is best suited for a multi-step agentic pipeline where each step depends on the previous one. In a business-critical workflow, consistency is key.
What do you need to do?
- Write a signature.
Typed input and output for exactly the task you need, with plenty of examples. - Build a module.
Signatures become a module’s specifications: the program’s building blocks. - Define metrics.
Define what good means, such as the percentage of correct answers. - Compile.
DSPy tests and optimizes hidden prompts and parameters. - Deploy.
Save the compiled configuration, then call the module with your input.
We did not have good-quality questions, answers, or metrics at first, but it is a one-time task worth every minute. Invest in building a golden dataset for future upgrades and DSPy runs, then spend the saved time celebrating.
We started like everyone else: prompt engineering got us moving, then fine-tuning was worth trying. DSPy offered a middle path—fast, measurable, low-maintenance development without babysitting prompts.
Prompt Engineering vs DSPy vs Fine-Tuning
| Factor | Prompt Engineering | DSPy | Fine-Tuning |
|---|---|---|---|
| Effort | Manual trial and error; guesswork in every request | Define inputs, outputs, and metrics; the system auto-optimizes | Requires a labelled dataset and training run |
| Cost | Ongoing human testing and tweaking | Compute/API cost during compilation; minimal ongoing cost | Training cost plus hosting a fine-tuned model |
| Effectiveness | Depends heavily on prompt skill; brittle to changes | Metric-driven and reusable across models | Stable behaviour baked into weights; harder to adjust |
We can bootstrap this into your project. We have found ways to collect good examples, and we can collaborate on setting up DSPy in your pipeline—because if you want fewer fragile hacks and more repeatable results in an AI app, we can help.