It looks like there are 3 main strategies for achieving Structured Outputs with DSPy ⚠️🛠️ 1. TypedPredictors: Define return types in Pydantic BaseModels to interface to the LLM with JSON templates. 📜 2. Assertions: Interface a boolean-valued function to assess the input and retry with the past output and a retry prompt added to the LLM call. ♻️ 3. Custom Guardrails with the DSPy Programming Model: Write custom Signatures and program logic to assess the input and retry. 🪛 I think there is a lot of overlap between these three, such as using a Typed Custom Guardrail with a `bool` return type in an Assertion. Still trying to figure out which applications need to use which and how this is further compounded by projects in a research or production level stage. I love @ecardenas300's overview below of DSPy Assertions, which I think is the quickest and most general way to start integrating Structured Outputs or Guardrails to your LLM systems built with DSPy 🔥
It looks like there are 3 main strategies for achieving Structured Outputs with DSPy ⚠️🛠️ 1. TypedPredictors: Define return types in Pydantic BaseModels to interface to the LLM with JSON templates. 📜 2. Assertions: Interface a boolean-valued function to assess the input and retry with the past output and a retry prompt added to the LLM call. ♻️ 3. Custom Guardrails with the DSPy Programming Model: Write custom Signatures and program logic to assess the input and retry. 🪛 I think there is a lot of overlap between these three, such as using a Typed Custom Guardrail with a `bool` return type in an Assertion. Still trying to figure out which applications need to use which and how this is further compounded by projects in a research or production level stage. I love @ecardenas300's overview below of DSPy Assertions, which I think is the quickest and most general way to start integrating Structured Outputs or Guardrails to your LLM systems built with DSPy 🔥
@CShorten30 An sglang integration solve ALL the challenges of using dspy in prod. Save a compiled llm program -> serve the program with sglang. Typed Signatures would be lowered to grammars for grammar constrained decoding using sglang.
@CShorten30 I wonder if you could use Typed Predictors in conjunction with tool use/structured output a la instructor via kwargs to the llm client somehow 🤔
@CShorten30 Great points, Connor! Excited to be learning about the best ways to get structured outputs
@CShorten30 I think these would be considered semi-structured output.
@CShorten30 I have been following your suggestions and watched several of your videos @CShorten30 😉 I am having issues though when running DSPy Modules with multiple output fields, I.e.: with a Pydantic model which for example has an analysis_summary and a relevance_score. The AI fails :-(