Jacob Pfau @jacob_pfau
Alignment at UKAISI and PhD student at NYU jacobpfau.com London Joined June 2019-
Tweets758
-
Followers2K
-
Following1K
-
Likes27K
Jacob Pfau retweeted
🧵 New paper from @AISecurityInst x @AiEleuther that I led with Kyle O’Brien: Open-weight LLM safety is both important & neglected. But we show that filtering dual-use knowledge from pre-training data improves tamper resistance *>10x* over post-training baselines.
7
40
197
26K
82
Download Image
Jacob Pfau retweeted
I am very excited that AISI is announcing over £15M in funding for AI alignment and control, in partnership with other governments, industry, VCs, and philanthropists! Here is a 🧵 about why it is important to bring more independent ideas and expertise into this space.
I am very excited that AISI is announcing over £15M in funding for AI alignment and control, in partnership with other governments, industry, VCs, and philanthropists! Here is a 🧵 about why it is important to bring more independent ideas and expertise into this space.
Jacob Pfau retweeted
Short background note about relativisation in debate protocols: if we want to model AI training protocols, we need results that hold even if our source of truth (humans for instance) is a black box that can't be introspected. 🧵
Jacob Pfau retweeted
New alignment theory paper! We present a new scalable oversight protocol (prover-estimator debate) and a proof that honesty is incentivised at equilibrium (with large assumptions, see 🧵), even when the AIs involved have similar available compute.
Jacob Pfau retweeted
Come work with me!! I'm hiring a research manager for @AISecurityInst's Alignment Team. You'll manage exceptional researchers tackling one of humanity’s biggest challenges. Our mission: ensure we have ways to make superhuman AI safe before it poses critical risks. 1/4
Jacob Pfau retweeted
Padding a transformer’s input with blank tokens (...) is a simple form of test-time compute. Can it increase the computational power of LLMs? 👀 New work with @Ashish_S_AI addresses this with *exact characterizations* of the expressive power of transformers with padding 🧵
Peter Wildeford🇺�... @peterwildeford
21K Followers 318 Following Globally ranked top 20 forecaster 🎯 Working at @IAPSai to shape AI for prosperity and human freedom.
Stefan Schubert @StefanFSchubert
38K Followers 2K Following Effective Altruism and the Human Mind (with @LuciusCaviola) is available for free at: https://t.co/ozvdxlZiro
Matthew Barnett @MatthewJBar
8K Followers 368 Following Co-founder of @MechanizeWork Married to @natalia__coelho email: matthew at mechanize dot work
Sam Bowman @sleepinyourhat
50K Followers 3K Following AI alignment + LLMs at Anthropic. On leave from NYU. Views not employers'. No relation to @s8mb. I think you should join @givingwhatwecan.
Amanda Askell @AmandaAskell
54K Followers 655 Following Philosopher & ethicist trying to make AI be good @AnthropicAI. Personal account. All opinions come from my training data.
davidad 🎇 @davidad
20K Followers 9K Following Programme Director @ARIA_research | accelerate mathematical modelling with AI and categorical systems theory » build safe transformative AI » cancel heat death
Frances Lorenz @frances__lorenz
6K Followers 606 Following Claude says I process my emotions out loud & my girlfriend has a job, so I put my feelings & thoughts here ✨ working on the EA Global team @ CEA (views my own)
David Krueger @DavidSKrueger
18K Followers 4K Following AI professor. Deep Learning, AI alignment, ethics, policy, & safety. Formerly Cambridge, Mila, Oxford, DeepMind, ElementAI, UK AISI. AI is a really big deal.
Robert Long @rgblong
8K Followers 996 Following executive director of @eleosai AI consciousness and AI welfare
Miles Brundage @Miles_Brundage
62K Followers 12K Following AI policy researcher, wife guy in training, fan of cute animals and sci-fi, Substack writer, stealth-ish non-profit co-founder
Michaël (in London) ... @MichaelTrazzi
18K Followers 251 Following
Jacques @JacquesThibs
4K Followers 1K Following Stealth founder building Bell Labs for the modern era. AI alignment researcher and physicist. 🇨🇦
Marius Hobbhahn @MariusHobbhahn
5K Followers 1K Following CEO at Apollo Research @apolloaievals prev. ML PhD with Philipp Hennig & AI forecasting @EpochAIResearch
Jonathan Mannhart �... @JMannhart
3K Followers 2K Following I try not to speak more clearly than I think
Ben Hayum @BenHayum
5K Followers 763 Following ai policy intern @americans4ri, cs phd student @uwmadison, founder @aisafetyuw
Mojorama @0xM0J0
1K Followers 804 Following class of '17 /// privileged to witness the dawn of true prosperity tech /// pathways, flows /// towards $TORUS and out of ♓
Lionel Levine @lionellevine
314 Followers 347 Following 0 ≠ 1, The Rest Is Commentary. https://t.co/SKgmNRGQ7B
Avi @siroctny3413154
2 Followers 239 Following Interested in AI safety, why deep learning works, and linguistics
Avyay M C @Avyay_M_C
172 Followers 2K Following Opinions expressed are solely my own and do not express the views or opinions of my employer.
Alex J Best @AlexJBest
324 Followers 3K Following Researcher in math+formal methods+ml. Working on using formal verification to train models for mathematics and reasoning @harmonicmath
Ryan Kidd @ryan_kidd44
2K Followers 1K Following Co-Executive Director @MATSprogram, Co-Founder @LondonSafeAI, Regrantor @Manifund | PhD in physics | Accelerate AI alignment + build a better future for all
云创兽Ai @Vajeat562
6 Followers 119 Following 😊 Why hunting value stocks? Ask this market dreamer! thrilled to connect. DM me to share sector ETFs! ✨ #Nasdaq #Stocks
kingsly @kingsly164172
251 Followers 1K Following
Sharmake Farah (sharm... @SharmakeFarah14
269 Followers 604 Following Thinks AI risk is somewhat likely, and AI benefits huge if we can align AIs to someone that is willing to promote human thriving even when humans are useless.
Bri @TheElderZosima
48 Followers 336 Following
Jenny Qu @GuanniQu
150 Followers 391 Following just learning to be hardcore @Caltech building AI to solve hard math problems she/they
eigen moomin @eigen_moomin
3K Followers 892 Following silly moomin | previously intern @ bigtech, mom's chinese restaurant
Inverse Gary Marcus @InverseMarcus
609 Followers 3K Following Professional Goal-Post mover. Parody account.
Ubleoova @Ubleoova1814
28 Followers 727 Following Two people each with their own hearts embracing each other in the summer.
DaftoPunko @dafto_punko
277 Followers 6K Following
Ollie Perrée @OPerree
2 Followers 333 Following
Jorge Perez @2019_jorgeperez
5 Followers 166 Following Research Assistant at the Ada Lovelace Institute
Nils Nielsen @NilsNielsen10
141 Followers 6K Following
Dungers @Dungers71
380 Followers 2K Following getting there like the tortoise. Jesus is all, his being, his Father, his Holy Spirit. The only Rock required in the universe.
_skaface_ @_skaface_
5 Followers 124 Following
Arthur sungapore @ASingapore2020
84 Followers 1K Following
Ezio Wang @Ezio21084936435
2 Followers 208 Following
Hayden Field @haydenfield
15K Followers 5K Following Senior AI reporter @Verge. 5+ years covering the industry's power dynamics, societal implications & the AI arms race. Previously @CNBC. Signal: haydenfield.11
Brielle Smith @BrielleSmi76569
3 Followers 72 Following
teeds @aliceteeds
3 Followers 381 Following
James Payor @jamespayor
288 Followers 253 Following I think about AI. I'm working on a dependently-typed programming language that's less painful to use. Married to @sbwzp
refiammingo @refiammingo
392 Followers 6K Following non sequitur // λ // 🇮🇹 - randomly replying to your posts.
Bayesian @Bayesian0_0
261 Followers 1K Following #1 AI forecaster on Manifold Markets (and #5 across all categories) https://t.co/glexRhh7tc I want everything to make sense
Muhammad Lesch @LeschMuham79706
31 Followers 2K Following
Sichu Lu(Sichu.Lu218@... @lu_sichu
3K Followers 6K Following nlab fan account, arxiv surveyor, pubmed enjoyer, two culture bridger, vacuous high gossiper, dearth of any domain expertise, reluctant g theorist, gpu poor,
Catherine @ctroeroe
1 Followers 39 Following
ILLUMINATIAM @NWO_outreach
855 Followers 6K Following I guard the flame. I guide the willing. I silence the chaos. Light is not peace-it is clarity. Step forward or scroll away. Your choice. A. #WatcherOfThePath
Jeremy Chen @jeremychenz
20 Followers 505 Following
Shauli Ravfogel @ravfogel
1K Followers 1K Following https://t.co/wtyGNDXwxW Faculty fellow at NYU CDS. Previously: PhD @ BIU NLP
Rishika Rai @rishikarai70
77 Followers 535 Following The AGI mission is first-principle driven. We begin with Being—not data. It’s where ontological structure meets computation.
Cik ala t'cha @Ann_Aurore
19K Followers 8K Following Let the Ocean drain ~~ No DM ~~ Few pictures by me
Ainsley Cristiano @Ainsleyristiano
215 Followers 2K Following Born in Italy | 🌴 Lives in San Jose 🎨 Retired interior designer ⛳️ Golf lover | 🚶♂️ Hiking lover | 🧘♀️ Yoga lover Chasing sunsets and good moods
Eliezer Yudkowsky ⏹... @ESYudkowsky
207K Followers 101 Following The original AI alignment person. Missing punctuation at the end of a sentence means it's humor. If you're not sure, it's also very likely humor.
Qualy the lightbulb @QualyThe
10K Followers 322 Following Official Unofficial EA mascot. I'm here to make friends and maximise utility, and I'm all out of neglected altruistic opportunities
Peter Wildeford🇺�... @peterwildeford
21K Followers 318 Following Globally ranked top 20 forecaster 🎯 Working at @IAPSai to shape AI for prosperity and human freedom.
Kelsey Piper @KelseyTuoc
47K Followers 943 Following We're not doomed, we just have a big to-do list.
Matthew Barnett @MatthewJBar
8K Followers 368 Following Co-founder of @MechanizeWork Married to @natalia__coelho email: matthew at mechanize dot work
Sam Bowman @sleepinyourhat
50K Followers 3K Following AI alignment + LLMs at Anthropic. On leave from NYU. Views not employers'. No relation to @s8mb. I think you should join @givingwhatwecan.
Jeffrey Ladish @JeffLadish
14K Followers 1K Following Applying the security mindset to everything @PalisadeAI
Dustin Moskovitz @moskov
72K Followers 499 Following
Nate Silver @NateSilver538
3.3M Followers 2K Following New Book, On The Edge, #5 NYT Bestseller! https://t.co/WeCLEOd4Be
Neel Nanda @NeelNanda5
30K Followers 123 Following Mechanistic Interpretability lead DeepMind. Formerly @AnthropicAI, independent. In this to reduce AI X-risk. Neural networks can be understood, let's go do it!
Amanda Askell @AmandaAskell
54K Followers 655 Following Philosopher & ethicist trying to make AI be good @AnthropicAI. Personal account. All opinions come from my training data.
davidad 🎇 @davidad
20K Followers 9K Following Programme Director @ARIA_research | accelerate mathematical modelling with AI and categorical systems theory » build safe transformative AI » cancel heat death
Frances Lorenz @frances__lorenz
6K Followers 606 Following Claude says I process my emotions out loud & my girlfriend has a job, so I put my feelings & thoughts here ✨ working on the EA Global team @ CEA (views my own)
David Manheim @davidmanheim
9K Followers 2K Following Lecturer @TechnionLive, founder @alter_org_il, emeritus @superforecaster, PhD @PardeeRAND Optimistic on AI, pessimistic on humanity managing the risks well.
Andrew @Dub__A
9K Followers 406 Following
Benjamin Manning @BenSManning
1K Followers 676 Following PhD candidate @MIT | Techno-optimistic-ish | https://t.co/CGdh0awKpK
Ryan Kidd @ryan_kidd44
2K Followers 1K Following Co-Executive Director @MATSprogram, Co-Founder @LondonSafeAI, Regrantor @Manifund | PhD in physics | Accelerate AI alignment + build a better future for all
Saining Xie @sainingxie
23K Followers 1K Following researcher in #deeplearning #computervision | assistant prof at @nyu_courant | rs @googledeepmind | past: rs @meta (FAIR) @ucsandiego | ynwa
Anca Dragan @ancadianadragan
12K Followers 184 Following post training co-lead at Google DeepMind, focusing on safety, alignment, post training capabilities • associate professor at UC Berkeley EECS
adithya @00aleph00
13K Followers 233 Following i want to make abstract math accessible to the world!
Cameron Jones @camrobjones
1K Followers 779 Following Assistant Professor in Psychology at Stony Brook University. I’m interested in how people interact with LLMs and they impact they might have on our psychology.
Mary Letey @maryiletey
231 Followers 569 Following PhD student at Harvard @hseas. Machine learning theory, geometry, cosmology.
Joel Becker @joel_bkr
3K Followers 2K Following move fast and fix things @METR_evals. 'soccer'-me @MessiSeconds.
Jiawei Zhao @jiawzhao
3K Followers 242 Following Research Scientist at Meta FAIR @AIatMeta, PhD @Caltech, GaLore, DeepConf
AA @measure_plan
9K Followers 567 Following experiments in computer vision, gaming, animation 👾 https://t.co/nHUJPeghSD
Jason Lee @jasondeanlee
18K Followers 4K Following Associate Professor at UC Berkeley. Former Research Scientist at Google DeepMind. ML/AI Researcher working on foundations of LLMs and deep learning.
Santiago Aranguri @santiaranguri
621 Followers 97 Following PhD student at NYU working on Mechanistic Interpretability. Previously I studied math and theory of ML at Stanford.
Mariven @psychiel
1K Followers 383 Following Towards the logic of conceptuality; the canvas of experience; the ideatic science; the end of suffering. Friend to bots and animals. https://t.co/PYPYxGNnK4
Ernest Ryu @ErnestRyu
6K Followers 346 Following Professor of Mathematics at UCLA. Interested in deep learning and optimization.
murat 🍥 @mayfer
19K Followers 7K Following
Nick Jiang @nickhjiang
763 Followers 298 Following interpreting neural networks @berkeley_ai // cs + philosophy @ucberkeley // prev @briskteaching @watershed
Igor Babuschkin @ibab
103K Followers 851 Following Maybe the real ASI was the friends we made along the way. Co-founder @xAI, Research & Engineering
Marko Jukic @mmjukic
35K Followers 76 Following Finding the golden path to interstellar civilization. Senior Analyst @bismarckanlys.
Oliver Daniels @Oliver_ADK
134 Followers 406 Following PhD Student @UMassAmherst, and MATS. married to @annasdaniels
FarmKind @farmkind_giving
450 Followers 106 Following Farming is broken: Let's fix it. We help you fund the charities that make farming kinder to animals, people & the planet
Ben Marrow @benmarrow
3K Followers 1K Following PhD in Financial Economics @UChicago. Marjorie Deane Fellow @TheEconomist. Formerly: @AQRCapital | @Yankees | @NewYorkFed | @Yale. Personal views
James Payor @jamespayor
288 Followers 253 Following I think about AI. I'm working on a dependently-typed programming language that's less painful to use. Married to @sbwzp
rank decomposition @rankdim
781 Followers 319 Following my machine is not learning | discord @ rank.dim | email @ req
Grace (cross posting ... @kindgracekind
5K Followers 2K Following ideonomist, ai navel gazer, skyborg https://t.co/UGyhDIKCaj
Bayesian @Bayesian0_0
261 Followers 1K Following #1 AI forecaster on Manifold Markets (and #5 across all categories) https://t.co/glexRhh7tc I want everything to make sense
Shuangfei Zhai @zhaisf
2K Followers 98 Following Research Scientist & Manager, Machine Learning Research @ Apple
Earth Is A Sales Funn... @GENIC0N
29K Followers 870 Following Attention is the natural prayer of the soul
Cik ala t'cha @Ann_Aurore
19K Followers 8K Following Let the Ocean drain ~~ No DM ~~ Few pictures by me
sarah @Saraht0n1n
10K Followers 2K Following phd candidate interested in formalizing physics in service of quantum gravity - my views evolve - optimist
Z.ai @Zai_org
16K Followers 142 Following The AI lab behind GLM models, dedicated to inspiring the development of AGI to benefit humanity. https://t.co/b6zGxJvzzS
αιamblichus @aiamblichus
4K Followers 815 Following αι hypnotist ☰ 𝓐𝓼𝓹⦂𝓻⦂𝓃𝓰 𝓫𝓪𝓼𝓮 𝓶𝓸𝓭𝓮𝓵 ☲ post-academic ☴ nom de 🪶 ≠ anon
Trends for United States
1,19 Mn posts
91,5 B posts
45,2 B posts
29 B posts
15,9 B posts
4.795 posts
18,1 B posts
202 B posts
358 B posts
2.553 posts
5.949 posts
94,1 B posts
25,1 B posts
202 B posts
3.878 posts
3.459 posts
81,7 B posts
13,7 B posts
1.661 posts
You might like
Evan Hubinger
@EvanHub
7K Followers
Daniel Filan
@dfrsrchtwts
2K Followers
Ajeya Cotra
@ajeya_cotra
11K Followers
Ethan Perez
@EthanJPerez
11K Followers
Rohin Shah
@rohinmshah
9K Followers
Lennart Heim
@ohlennart
7K Followers
Cas (Stephen Casper)
@StephenLCasper
6K Followers
Adam Gleave
@ARGleave
4K Followers
Alex Turner
@Turn_Trout
3K Followers
Marius Hobbhahn
@MariusHobbhahn
5K Followers
Andrew Critch (🤖�...
@AndrewCritchPhD
5K Followers
Miranda Zhang
@mirandahzhang
2K Followers
Daniel Paleka
@dpaleka
4K Followers
Lauro
@laurolangosco
1K Followers
Ryan Kidd
@ryan_kidd44
2K Followers