Monograph on "Formal Aspects of Language Modeling" from @ryandcotterell et al. arxiv.org/abs/2311.04329 It would be so nice if everyone read this and we had shared foundations. Particularly for interpretability.
5
49
296
37K
296
@srush_nlp @ryandcotterell Crazy that you're asking everyone to read lecture notes I *have* to read
@giaccoangelo @srush_nlp @ryandcotterell consider yourself lucky; the best attention explanation we get at @TU_Muenchen goes like this: V, Q, W where V is *a bunch of interesting things* sadly, i am not even exaggerating