Unfortunately, I fear I'll always be cheap regarding model size. Instinctively I still am "*millions of parameters?!?!*"
2
0
17
0
0
@francoisfleuret For humongous data, you need something that can absorb a lot of info. For now, this is parameters. Otherwise, it's our current best way of making optimization easier. Hopefully we'll find better ways eventually.
@giffmana @francoisfleuret Parameter-efficient learning is often taken for granted. I wish we go beyond the "fitting" paradigm and learn more with less.
@giffmana @francoisfleuret I think we may have found it although I try not to hypothesise in case I'm right. But yeah, a very important area.