@EMostaque @paperswithcode @metaai I think you should put it behind a security wall but give access to authenticated researchers with tight license conditions.
@tdietterich @EMostaque @paperswithcode @metaai It doesn't need to be behind lock and key, but a basic signup with acknowledgment of 'intended for research', list potential shortcomings, add appropriate license, minus marketing hyperbole seems appropriate.
@wightmanr @EMostaque @paperswithcode @metaai Don't you think there are risks that the model could be deployed for disinformation campaigns, phishing, and so on? How do you enforce the license without some lock and key?
@tdietterich @wightmanr @EMostaque @paperswithcode @metaai How is that different from current LLMs that are in the wild?
@leonpalafox @wightmanr @EMostaque @paperswithcode @metaai Same rules for them, IMO.
@tdietterich @wightmanr @EMostaque @paperswithcode @metaai I think that is one of the good points by @ylecun. Whatever damage that could be done using LLMs is not like Galáctica is going to bring it up to a new level, that genie was out of the bottle a long time ago.
@leonpalafox @wightmanr @EMostaque @paperswithcode @metaai @ylecun What was the first good LLM whose weights were released?
@leonpalafox @wightmanr @EMostaque @paperswithcode @metaai @ylecun I did a bit of searching. XLNet was released in 2019, I believe. The weights for RoBERTa, DeBERTa, and XLM-RoBERTa are available I think. I'm not sure about OPT-175B. The fact that these are all available changes my position regarding Galactica. 1/
@leonpalafox @wightmanr @EMostaque @paperswithcode @metaai @ylecun Oh, and GPT-NeoX was released in April of 2022, I believe (20B parameters). It isn't obvious that Galactica is dangerously more powerful than these existing open models.
@tdietterich @leonpalafox @wightmanr @EMostaque @paperswithcode @metaai Indeed. In fact, since Galactica was trained on scientific papers, it's likely to be more benign than other LLMs. FYI: OPT-175b weights aren't available, but the smaller OPT weights are.
@ylecun @leonpalafox @wightmanr @EMostaque @paperswithcode @metaai But as @Michael_J_Black points out, Galactica may open the door to a new set of fake science "attacks" on the publication system. I think it might also support a great semantic auto-correct for authors (esp ones with less skill in English)
@tdietterich @leonpalafox @wightmanr @EMostaque @paperswithcode @metaai @Michael_J_Black I think @Michael_J_Black 's fears are unwarranted. The incentives for flooding publication venues with generated fake science simply do not exist. It's a career-ending act. It could exist as DDS-style gratuitous acts of vandalism, but those are generally isolated incidents.
@tdietterich @ylecun @leonpalafox @wightmanr @EMostaque @paperswithcode @metaai @Michael_J_Black it may well opens the door for non-English speakers to write much better, no language barrier. How about that. Think it like a co-pilot in for writing code.
@tdietterich @ylecun @wightmanr @EMostaque @paperswithcode @metaai @Michael_J_Black A grammar style corrector for foreigners writing is something that would be definitely a net positive for the community. Also, spell checkers is one of the excuses publishers use to justify their margins.