Introducing Demonstrate–Search–Predict (𝗗𝗦𝗣), a framework for composing search and LMs w/ up to 120% gains over GPT-3.5. No more prompt engineering.❌ Describe a high-level strategy as imperative code and let 𝗗𝗦𝗣 deal with prompts and queries.🧵 arxiv.org/abs/2212.14024
Instead of crafting a prompt for the LM, you write a short 𝗗𝗦𝗣 program that assigns small tasks to the LM and a retrieval model (RM) in deliberate powerful pipelines. Simple 𝗗𝗦𝗣 programs outperform GPT-3.5, retrieve-&-read and self-ask by up to 𝟭𝟮𝟬%, 𝟰𝟬% and 𝟮𝟵𝟬%
@lateinteraction Nice! You can implement the first step of this via prompt engineering as well 🤓
@lateinteraction In works such as arxiv.org/abs/2203.14465 authors had stated that at some point this consistency based example creation stops providing better results. They use a method called rationalization (use the answer to backtrace the reasoning), what does DSP do?