I propose that we adopt the term "Large Self-Supervised Models (LSSMs)" as a replacement for "Foundation Models" and "LLMs". "LLMs" don't capture non-linguistic data and "Foundation Models" is too grandiose. Thoughts? @percyliang
@tdietterich The beauty of language is that you can have multiple terms that highlight different aspects of the same object. You don't have to choose. I use "LLM" to talk about LLMs, "self-supervised" for their construction, and "foundation model" for their function. No term can be replaced.
@percyliang Yes, but as you know, "Foundation" is too close to "Foundational", and many of us find that troubling. That is why I'm proposing a more neutral term. For use, maybe we could just call them "Upstream models".
@tdietterich @percyliang Or, hear me out, pre-trained models!
@giffmana @tdietterich @percyliang The self-supervision matters.
@francoisfleuret @tdietterich @percyliang I don't think so at all.
@ggdupont @francoisfleuret @tdietterich @percyliang Funny you tell me that, because I have several papers doing exactly that...
@giffmana @ggdupont @tdietterich @percyliang You do not think the best strategy to train models for image understanding will be eventually mostly self-supervised?
@francoisfleuret @ggdupont @tdietterich @percyliang Only time will tell, but currently, this strategy performs comparatively poorly.
@giffmana @francoisfleuret @ggdupont @percyliang My impression was that self-supervised is competitive with supervised in computer vision. Is this wrong? In particular, doesn't self-supervised permit training on much more data?