LLMs can do in-context learning, but are they "learning" new tasks or just retrieving ones seen during training? w/ @shivamg_13, @percyliang, & Greg Valiant we study a simpler Q: Can we train Transformers to learn simple function classes in-context? 🧵 arxiv.org/abs/2208.01066
8
106
511
0
196
Background: Brown et al. found that GPT-3 can perform *in-context learning*---i.e., given a prompt containing examples from a task (input-output pairs) and a new query input, it can produce the corresponding output.