Kevin Clark @clark_kev

Taking a break from research for a year to travel! Joined March 2012

Tweets

41
Followers

1K
Following

241
Likes

27

Paul Vicol @PaulVicol

a year ago

Excited to present our work at ICLR next week! DRaFT fine-tunes diffusion models on differentiable rewards by backpropagating through sampling. It's efficient and applicable across a variety of reward functions. With @clark_kev, @kswersk, @fleet_dj iclr.cc/virtual/2024/p… 🧵

1 1 2 1K 0

Download Gif

Priyank Jaini @priyankjaini

2 years ago

Excited to share our work on studying perceptual properties of #StableDiffusion, #Imagen & #Parti by evaluating them as zero-shot classifiers! We report four intriguing emergent properties of these models. 🧵👇 w. Robert Geirhos* & @clark_kev Paper: arxiv.org/abs/2309.16779

1 10 53 11K 16

Download Image

Priyank Jaini @priyankjaini

2 years ago

.@clark_kev & I are excited to share our new work on studying Imagen by evaluating it as a zero-shot classifier! Highlights include Imagen achieving SoTA on Stylized Imagenet and being able to perform attribute binding in certain settings unlike CLIP arxiv.org/abs/2303.15233 🧵👇

1 17 45 14K 14

Download Image

Mengzhou Xia @xiamengzhou

3 years ago

Check out our preprint on Prompting ELECTRA! We show that discriminative models like ELECTRA outperform generative MLMs like BERT and RoBERTa on zero-shot and few-shot prompting. Joint work w/ @artetxem, @JefferyDuu, @danqi_chen, @vesko_st Paper: arxiv.org/pdf/2205.15223…

5 21 142 0 25

Thang Luong @lmthang

4 years ago

Interesting reproducibility study by @philipvollet to conclude that: ELECTRA small (14M) can be easily run on a single GPU with great performances! @clark_kev @quocleix @chrmanning Reproducibility paper: arxiv.org/abs/2104.02756 ELECTRA paper: arxiv.org/abs/2003.10555

Philip Vollet @philipvollet

4 years ago

0 36 110 0 21

Download Gif

0 12 52 0 15

Kevin Clark @clark_kev

5 years ago

Check out our #emnlp2020 paper now on arXiv: we develop an energy-based version of BERT called "Electric" and show this essentially re-invents the ELECTRA pre-training method. Electric is useful for re-ranking the outputs of text generation systems. arxiv.org/pdf/2012.08561….

0 25 168 0 37

Download Image

Thang Luong @lmthang

5 years ago

If you like ELECTRA, check out Electric, our new #emnlp2020 work that brought over energy-based model perspective ;) Electric produces fast pseudo-likelihood to improve speech recognition & translation reranking. Github: github.com/google-researc… & paper aclweb.org/anthology/2020…

2 51 265 0 43

Download Image

Richard Wang @richarddwang

5 years ago

Replementation of ELECTRA (@clark_kev et al. from @stanfordnlp) with @fastdotai + @huggingface 💻Code:github.com/richarddwang/e… 📰Post:discuss.huggingface.co/t/electra-trai… ELECTRA is with super efficiency, a true breakthrough on NLP pretraining. Explore it using @PyTorch & notebooks ! Next...

4 85 374 0 97

Download Image

Stanford NLP Group @stanfordnlp

5 years ago

Just out in PNAS: A paper examining the emergent linguistic structure learned by artificial neural networks, such as BERT, trained by cloze task (word-in-context prediction) self-supervision. By @chrmanning, @clark_kev, @johnhewtt, @ukhndlwl, @omerlevy_. pnas.org/content/early/…

2 49 154 0 26

Download Image

Hugging Face @huggingface

5 years ago

ELECTRA ⚡ is now integrated in the Transformers library from v2.8.0 (PT & TF2)! It is a new pre-training method by @clark_kev at @GoogleAI, with the pre-trained models obtaining SOTA on SQuAD. Stay tuned for an ELECTRA pre-training script coming soon! github.com/huggingface/tr…

8 67 272 0 28

Download Image

Jonathan Clark @JonClarkSeattle

6 years ago

Hey Stanford folks! Want to hear about TyDi QA, our new multilingual QA dataset? (Virtually) come to the NLP Seminar this Thursday at 11am. Check your email or check with @clark_kev for the meeting link. There will be glosses. In Finnish. nlp.stanford.edu/seminar/detail…

0 5 21 0 1

Kevin Clark @clark_kev

6 years ago

Excited to share the code and blog post for ELECTRA, a more efficient pre-training method for NLP Transformers. Pre-train a model to good accuracy using only 1 GPU! Joint work with @lmthang @quocleix @chrmanning Code: github.com/google-researc… Blog Post: ai.googleblog.com/2020/03/more-e…

7 130 435 0 75

Download Image

Kevin Clark @clark_kev

6 years ago

Our pre-trained transformer ELECTRA (joint work with @lmthang @quocleix @chrmanning) is the top single model on the SQuAD 2.0 leaderboard! rajpurkar.github.io/SQuAD-explorer/ Stay tuned for the code release coming next week! Paper: openreview.net/pdf?id=r1xMH1B…

2 68 270 0 40

Kelvin Guu @kelvin_guu

6 years ago

New from Google Research! REALM: realm.page.link/paper We pretrain an LM that sparsely attends over all of Wikipedia as extra context. We backprop through a latent retrieval step on 13M docs. Yields new SOTA results for open domain QA, breaking 40 on NaturalQuestions-Open!

13 198 728 0 125

Download Image

Hamlet Batista @hamletbatista

6 years ago

🔥“we train a model on one GPU for 4 days that outperforms GPT (trained using 30x more compute) on the GLUE natural language understanding ... we match the performance of RoBERTa, the current state-of-the-art pre-trained transformer, while using less than 1/4 of the compute.” 🤯

Stanford NLP Group @stanfordnlp

6 years ago

3 61 173 0 28

Download Image

1 27 87 0 16

Urvashi Khandelwal @ukhndlwl

6 years ago

Excited to share new work!!! “Generalization through Memorization: Nearest Neighbor Language Models” We introduce kNN-LMs, which extend LMs with nearest neighbor search in embedding space, achieving a new state-of-the-art perplexity on Wikitext-103, without additional training!