You have a fixed, massive compute budget to train a new foundation model from scratch.
How do you allocate this budget between the model's size (its parameters) and the amount of data it learns from (its tokens)?
#LLM#AI_Research #AI#QNA
In BERT-style training -> the decoder only needs to compute the loss for the masked tokens.
In denoising autoencoding -> the loss is obtained by accumulating the losses of all these tokens, as in standard language modeling.
#LLM#NLP#AI#Research_Paper
18K Followers 18K FollowingNelsonHall is synonymous worldwide with excellence in #BPS & #outsourcing advisory and research services #BPO #ITservices #automation #RPA #HRO #RPO #CXservices
153 Followers 254 FollowingBackend Engineer @ Yarasi | Member at @djangoproject || Fellow @djangonautspace || Core @djangoindiaa || I am a mad engineer ||
Curiosity is what forges me
13 Followers 279 FollowingLove Eating l Watch Movies & Sports l Read Books l Understand Astrology l Live Science & Technology l Ask questions l Keep Praying🙏.
Aqualion
27K Followers 1K FollowingGenAI @Youtube | Building AI powered video editing | ex : @Google Search & @Microsoft Azure | 3x hackathon winner | Views my own
21K Followers 465 Followingphysics of language models @ Meta (FAIR, not GenAI, not TBD)
🎓:Tsinghua Physics — MIT CSAIL — Princeton/IAS
🏅:IOI x 2 — ACM-ICPC — USACO — Codejam — math MCM
16K Followers 1K FollowingSenior Research Scientist - @google, Adjunct Faculty - @iitmadras, @iitbombay, Ex: @NICT_Publicity
Use of my tweets without permission ➡️ legal action
19K Followers 551 Following• Teaching Beginner-Friendly ML Courses @zerotomasteryio (https://t.co/SGxUchebqe)
• Building ML @nutrifyfoodapp (https://t.co/T8DzQnU4sG)
495K Followers 152 FollowingNobel Laureate. Co-Founder & CEO @GoogleDeepMind - working on AGI. Solving disease @IsomorphicLabs. Trying to understand the fundamental nature of reality.
348K Followers 1K FollowingDeepMind Research Scientist. Opinions my own. Inventor of GANs. Lead author of https://t.co/M6vl8pEQ4I Founding chairman of @pubhealthaction
717K Followers 288 FollowingTogether with the AI community, we are pushing the boundaries of what’s possible through open science to create a more connected world.
2 Followers 1 Following🚀 InnoCrede Solutions – Innovate. Create. Elevate.
At InnoCrede Solutions, we blend creativity and technology to deliver top-notch graphic design, video editi
955K Followers 765 FollowingProfessor at NYU. Chief AI Scientist at Meta.
Researcher in AI, Machine Learning, Robotics, etc.
ACM Turing Award Laureate.
1.4M Followers 1K FollowingBuilding @EurekaLabsAI. Previously Director of AI @ Tesla, founding team @ OpenAI, CS231n/PhD @ Stanford. I like to train large deep neural nets.
359K Followers 1K FollowingML/AI researcher & former stats professor turned LLM research engineer. Author of "Build a Large Language Model From Scratch" (https://t.co/O8LAAMRzzW).
1.2M Followers 279 FollowingWe’re a team of scientists, engineers, ethicists and more, committed to solving intelligence, to advance science and benefit humanity.
1.3M Followers 1K FollowingCo-Founder of Coursera; Stanford CS adjunct faculty. Former head of Baidu AI Group/Google Brain. #ai #machinelearning, #deeplearning #MOOCs
No recent Favorites. New Favorites will appear here.