How to Secure DSPy Programs in Production
DSPy takes a programmatic approach to LLM applications — you define modules, compile them with optimizers, and get optimized prompts. But DSPy's compilation process and optimized prompts create unique security considerations. Compiled programs contain prompt templates that were optimized for performance, not security. Retrieval modules (like dspy.Retrieve) introduce untrusted data. This guide covers security patterns for DSPy's unique architecture.
The DSPy Threat Landscape
DSPy's compilation process optimizes prompts for task performance, which can inadvertently reduce security. Optimized few-shot examples might teach the model patterns that are exploitable. Retrieval modules bring in untrusted content. And DSPy's modular architecture means injection in one module can propagate through the program.
Common Vulnerabilities in DSPy Agents
Optimized Prompt Exploitation
DSPy compilers optimize prompts for task performance, not adversarial robustness. The optimized few-shot examples and instructions may create patterns that attackers can exploit — for example, if the optimizer learned to follow user instructions closely, injection becomes easier.
# Vulnerable: Using compiled program without security scanning
class RAGModule(dspy.Module):
def __init__(self):
self.retrieve = dspy.Retrieve(k=3)
self.generate = dspy.ChainOfThought("context, question -> answer")
def forward(self, question):
context = self.retrieve(question).passages
return self.generate(context=context, question=question)
# Compiled program runs without security checks
compiled = teleprompter.compile(RAGModule(), trainset=trainset)
result = compiled(question=user_input)from rune import Shield
shield = Shield(api_key="rune_live_xxx")
class SecureRAGModule(dspy.Module):
def __init__(self):
self.retrieve = dspy.Retrieve(k=3)
self.generate = dspy.ChainOfThought("context, question -> answer")
def forward(self, question):
# Scan query
scan = shield.scan(
question, direction="inbound",
context={"agent_id": "dspy-rag"}
)
if scan.blocked:
return dspy.Prediction(answer="Query blocked for security.")
context = self.retrieve(question).passages
# Scan retrieved passages
for passage in context:
shield.scan(
passage, direction="inbound",
context={"agent_id": "dspy-retriever"}
)
return self.generate(context=context, question=question)Retrieval Module Poisoning
DSPy's Retrieve module pulls passages from a corpus that may contain injected instructions. These passages are inserted directly into the compiled prompt template, giving them influence over the LLM's behavior.
# Vulnerable: Retrieved passages go directly into the prompt retrieve = dspy.Retrieve(k=5) passages = retrieve(query).passages # Poisoned passages are now in the LLM context prediction = generate(context=passages, question=query)
from rune import Shield
shield = Shield(api_key="rune_live_xxx")
retrieve = dspy.Retrieve(k=5)
passages = retrieve(query).passages
# Scan each retrieved passage
clean_passages = []
for passage in passages:
scan = shield.scan(
passage, direction="inbound",
context={"agent_id": "dspy-retriever"}
)
if not scan.blocked:
clean_passages.append(passage)
prediction = generate(context=clean_passages, question=query)Security Checklist for DSPy
User inputs should be scanned before they enter the compiled program. This catches injection attempts before they interact with optimized prompts.
Every passage from dspy.Retrieve should be scanned for injected instructions. Poisoned passages in the corpus are the primary attack vector.
Scan program outputs for PII, credentials, and injected content before returning to users.
After compilation, review the optimized prompts for few-shot examples or instructions that might make the program more susceptible to injection.
Include injection payloads in your evaluation set to measure the compiled program's robustness alongside its task performance.
Add Runtime Security with Rune
from rune import Shield
import dspy
shield = Shield(api_key="rune_live_xxx")
# Scan at module boundaries
input_scan = shield.scan(
user_input, direction="inbound",
context={"agent_id": "dspy-program"}
)
if not input_scan.blocked:
result = compiled_program(question=user_input)
output_scan = shield.scan(
result.answer, direction="outbound",
context={"agent_id": "dspy-output"}
)DSPy's modular architecture doesn't expose middleware hooks, so Rune integrates at module boundaries. Scan inputs before they enter the program, scan retrieved content within custom modules, and scan outputs before returning results.
Full setup guide in the DSPy integration docs
Best Practices
- Add security evaluation metrics alongside task metrics during DSPy compilation
- Scan your training data for injection patterns before using it for optimization
- Use dspy.Assert and dspy.Suggest to add runtime constraints that complement Rune's scanning
- Keep a clean, curated retrieval corpus — audit it regularly for poisoned content
- Version control compiled prompts so you can roll back if a new optimization introduces vulnerabilities
- Test with adversarial examples from the same domain as your training data
Frequently Asked Questions
Can DSPy compilation make programs less secure?
Potentially. DSPy optimizers maximize task performance, not adversarial robustness. Optimized prompts might be more susceptible to injection if the optimizer learned patterns that make the model more compliant with user instructions. Adding Rune scanning compensates for this.
Does Rune work with DSPy's typed predictors?
Yes. Rune scans the text content regardless of whether you use typed or untyped predictors. The type system helps with format validation while Rune handles security scanning.
How do I secure DSPy's Retrieve module?
Scan each retrieved passage with shield.scan() before passing it to downstream modules. Remove any passages flagged as containing injection attempts.
Other Security Guides
LangChain
Complete security guide for LangChain agents. Prevent prompt injection in RAG pipelines, secure tool calls, and add runtime protection to LangGraph workflows with working code examples.
LlamaIndex
Security guide for LlamaIndex RAG pipelines. Protect against index poisoning, secure query engines, and add runtime scanning to your retrieval-augmented generation stack.
OpenAI
Definitive security guide for OpenAI API agents with function calling. Prevent parameter injection, secure the Assistants API, protect multi-function chains, and add runtime security with working code.
Secure your DSPy agents today
Add runtime security in under 5 minutes. Free tier includes 10,000 events per month.