I propose that we adopt the term "Large Self-Supervised Models (LSSMs)" as a replacement for "Foundation Models" and "LLMs". "LLMs" don't capture non-linguistic data and "Foundation Models" is too grandiose. Thoughts? @percyliang
@tdietterich The beauty of language is that you can have multiple terms that highlight different aspects of the same object. You don't have to choose. I use "LLM" to talk about LLMs, "self-supervised" for their construction, and "foundation model" for their function. No term can be replaced.
@tdietterich @percyliang Ok but LLMs can still be used when they are trained on linguistic data. I would just eliminate the wording “Foundation Models”.
@tdietterich @percyliang To me 'foundation' is really about use, and LSSM is about technical attributes. Foundation models are centralized and clients adapt them to solve specific tasks, through prompting, fine tuning, or whatever else. Such models could be LSSMs but could also be built in other ways.
@tdietterich @percyliang I find "ViT" pretty ok. So, instead of LLM, maybe LaT (Language Transformer) would have been a better term in hindsight. I think the L in LLM is too ambiguous. But yeah, Foundation Model is the worst and way to hype-y. Mark my words -- I won't use it in any professional articles
@tdietterich @percyliang But language models is what they are. "Large Self-Supervised Models" tells me nothing about the fundamental textual/linguistic basis of LLMs and corresponds to a much larger subset of models.