Timothy Atkinson @NaturalGradient

Interested in Discrete Diffusion? I've just released a Github repo where you can learn about and play with discrete diffusion algorithms with simple and performant "nano-style" implementations. (link below) I've started with the Absorbing D3PM from @jacobaustin132 and…

AgroNT, our 1B parameters foundation model for plants 🌱 genomics 🧬, just got published in communication biology ⭐️

Communications Biology @CommsBio

a year ago

AgroNT, our 1B parameters foundation model for plants 🌱 genomics 🧬, just got published in communication biology ⭐️

0 5 8 2K 3

One interesting interpretation to this is that Bayesian Flow Networks give us a first-principles way to derive forward/reverse processes for arbitrary data modes.

Timothy Atkinson @NaturalGradient

a year ago

One interesting interpretation to this is that Bayesian Flow Networks give us a first-principles way to derive forward/reverse processes for arbitrary data modes.

0 1 4 131 0

Finally, I have an answer to the question "isn't this just diffusion?" arxiv.org/abs/2404.15766

Got lost in the #ICLR2024 poster maze? Don't worry, we've got your covered! 🛟 Here is @DonalByrne2, Senior Research Engineer at @instadeepai , as he showcases Jumanji — our library for high-performance RL environments in #JAX ⭐️ Github: tinyurl.com/code-jumanji

The NT Family is growing! 🐣 ✨ Introducing ChatNT, a Conversational Agent designed to analyse genomics sequences and address a wide range of key biological questions, assisting scientists in their daily work 👩‍🔬 📚Paper: tinyurl.com/chatNT-pdf 🌐Blog: tinyurl.com/chatNT-blog

New Paper Out. Evolutionary Optimization of Model Merging Recipes arxiv.org/abs/2403.13187 Our goal is not about training any particular individual foundation model. Instead, we think it makes more sense to create the machinery to automatically generate foundation models for us!

Sakana AI @SakanaAILabs

2 years ago

New Paper Out. Evolutionary Optimization of Model Merging Recipes arxiv.org/abs/2403.13187 Our goal is not about training any particular individual foundation model. Instead, we think it makes more sense to create the machinery to automatically generate foundation models for us!

60 416 2K 950K 986

Download Video

I’m happy to present SegmentNT, our new genomics LLM annotating DNA sequences at a single nucleotide resolution! 🔎🧬 Building on our previous work on the Nucleotide Transformer (NT), we’ve designed SegmentNT to process sequences as long as 50,000 nucleotides (50 kbp) to predict…

Enjoyed this paper from @sarahookr team 👏 Key point -- JAX significantly outperforms PyTorch in hardware portability, which suffers 44% GPU to TPU failure rate. This echoes my first-hand experience and confirms our choice to adopt JAX early on, and its ongoing benefits 💪

Cohere Labs @Cohere_Labs

2 years ago

Enjoyed this paper from @sarahookr team 👏 Key point -- JAX significantly outperforms PyTorch in hardware portability, which suffers 44% GPU to TPU failure rate. This echoes my first-hand experience and confirms our choice to adopt JAX early on, and its ongoing benefits 💪

1 5 27 37K 14

Download Image

We released the weights and downstream task datasets of the agroNT, our 1B parameters DNA foundation model for plant genomics 🧬🪴. Check out our github (github.com/instadeepai/nu…) for Jax lovers⚡❤️and our @huggingface space (huggingface.co/InstaDeepAI) for the pytorch version🤗.

My new year's resolution is to think of a 2025 new year's resolution before people ask me about it

@rupspace Thanks, this was one of the biggest leaps in knowledge for me, so I'm glad it's not a total flop for people who do know this stuff 😅

Really nice summary (as usual from @natolambert) of recent work on recurrent networks, in particular those derived from a state space models perspective.

Nathan Lambert @natolambert

2 years ago

Really nice summary (as usual from @natolambert) of recent work on recurrent networks, in particular those derived from a state space models perspective.

7 60 373 65K 363

Download Image

InstaDeep's Shikha Surana shares what inspired her AI career, advice for young researchers & her favourite meme ahead of her #NeurIPS2023 @WiMLworkshop presentation 📢 Read more: instadeep.com/2023/12/shikha…

Protein structure encoders yield less informative representations than their sequence counterparts, due to limited data available. We pre-train structure encoders, and combine them with sequence encoders for improved performance in downstream applications.

Paper: biorxiv.org/content/10.110… Code: github.com/instadeepai/bi… Big thank you to @tomdbarrett @NaturalGradient @thomas_pierrot Liviu Copoiu & Patrick Bordes Catch Tom and I at MLSB, NeurIPS! (9/9)

If you’re interested in generative modeling of 3D molecules, this submission to ICLR that adapts Bayesian Flow Networks for the task seems to have received good reviews! openreview.net/forum?id=NSVtm…

"Bayesian Flow Networks" has 47 pages of main text with over 200 equations 🙃 Maybe this summary is a simpler resource if you want to understand the new generative modeling paradigm: youtube.com/watch?v=hVesuh… :)

Monday in the reading group! "Bayesian Flow Networks" with Alex Graves from @nnaisense : arxiv.org/abs/2308.07037 I really hope we can avoid any Alice and Bob analogies 🙃 Join on Zoom at 11am EST / 5pm CET / 4pm UTC: portal.valencelabs.com/logg