Jacob Pfau @jacob_pfau
Alignment at UKAISI and PhD student at NYU jacobpfau.com London Joined June 2019-
Tweets765
-
Followers2K
-
Following1K
-
Likes27K
Jacob Pfau retweeted
🧵 New paper from @AISecurityInst x @AiEleuther that I led with Kyle O’Brien: Open-weight LLM safety is both important & neglected. But we show that filtering dual-use knowledge from pre-training data improves tamper resistance *>10x* over post-training baselines.
7
39
199
26K
84
Download Image
Jacob Pfau retweeted
I am very excited that AISI is announcing over £15M in funding for AI alignment and control, in partnership with other governments, industry, VCs, and philanthropists! Here is a 🧵 about why it is important to bring more independent ideas and expertise into this space.
I am very excited that AISI is announcing over £15M in funding for AI alignment and control, in partnership with other governments, industry, VCs, and philanthropists! Here is a 🧵 about why it is important to bring more independent ideas and expertise into this space.
Jacob Pfau retweeted
Short background note about relativisation in debate protocols: if we want to model AI training protocols, we need results that hold even if our source of truth (humans for instance) is a black box that can't be introspected. 🧵
Jacob Pfau retweeted
New alignment theory paper! We present a new scalable oversight protocol (prover-estimator debate) and a proof that honesty is incentivised at equilibrium (with large assumptions, see 🧵), even when the AIs involved have similar available compute.
Jacob Pfau retweeted
Come work with me!! I'm hiring a research manager for @AISecurityInst's Alignment Team. You'll manage exceptional researchers tackling one of humanity’s biggest challenges. Our mission: ensure we have ways to make superhuman AI safe before it poses critical risks. 1/4
Peter Wildeford🇺�... @peterwildeford
22K Followers 321 Following Globally ranked top 20 forecaster 🎯 AI is not a normal technology. I'm working at @IAPSai to shape AI for global prosperity and human freedom.
Stefan Schubert @StefanFSchubert
39K Followers 2K Following Effective Altruism and the Human Mind (with @LuciusCaviola) is available for free at: https://t.co/ozvdxlZiro
Matthew Barnett @MatthewJBar
8K Followers 368 Following Co-founder of @MechanizeWork Married to @natalia__coelho email: matthew at mechanize dot work
Sam Bowman @sleepinyourhat
50K Followers 3K Following AI alignment + LLMs at Anthropic. On leave from NYU. Views not employers'. No relation to @s8mb. I think you should join @givingwhatwecan.
Amanda Askell @AmandaAskell
54K Followers 657 Following Philosopher & ethicist trying to make AI be good @AnthropicAI. Personal account. All opinions come from my training data.
davidad 🎇 @davidad
20K Followers 9K Following Programme Director @ARIA_research | accelerate mathematical modelling with AI and categorical systems theory » build safe transformative AI » cancel heat death
Frances Lorenz @frances__lorenz
6K Followers 608 Following Claude says I process my emotions out loud & my girlfriend has a job, so I put my feelings & thoughts here ✨ working on the EA Global team @ CEA (views my own)
David Krueger @DavidSKrueger
18K Followers 4K Following AI professor. Deep Learning, AI alignment, ethics, policy, & safety. Formerly Cambridge, Mila, Oxford, DeepMind, ElementAI, UK AISI. AI is a really big deal.
Robert Long @rgblong
8K Followers 993 Following executive director of @eleosai AI consciousness and AI welfare
Miles Brundage @Miles_Brundage
62K Followers 12K Following AI policy researcher, wife guy in training, fan of cute animals and sci-fi, Substack writer, stealth-ish non-profit co-founder
Michaël (in London) ... @MichaelTrazzi
18K Followers 289 Following
Jacques @JacquesThibs
5K Followers 1K Following Stealth founder focused on securing the future. AI alignment researcher and physicist. 🇨🇦
Marius Hobbhahn @MariusHobbhahn
5K Followers 1K Following CEO at Apollo Research @apolloaievals prev. ML PhD with Philipp Hennig & AI forecasting @EpochAIResearch
Jonathan Mannhart �... @JMannhart
3K Followers 2K Following I try not to speak more clearly than I think
Ben Hayum @BenHayum
5K Followers 777 Following ai policy intern @americans4ri, cs phd student @uwmadison, founder @aisafetyuw
yulong @_yulonglin
181 Followers 953 Following make safety people want @MATSprogram | prev @berkeley_ai, @cohere, @ bytedance seed, @Cambridge_Uni
Jamie Bernardi @The_JBernardi
2K Followers 826 Following Doing AI policy research, ex-Bluedot Impact. Climber, guitarist and sporadic musician. he/him.
Helen @helen84perez
323 Followers 3K Following
Amy Johnson @kevsterkurata
169 Followers 1K Following Hobbies: Exercise, traveling, movies, fishing. I own a medical aesthetics studio and also invest in technology projects.
Nadav Timor @NadavTimor
1K Followers 8K Following AI inference, speculative decoding, open source. Built novel decoding algorithms – default in Hugging Face Transformers (150+ ⭐). Making AI faster + cheaper
Terica @deckmanterica58
330 Followers 3K Following
Annette @annettepollitt1
353 Followers 3K Following
Jane @VanRodrigues14
71 Followers 1K Following The stock market isn’t champagne bubbles 🍾— I prefer substance over sparkle ✨. Fake smiles & pushy sales? Not invited 🚫.
divyasshree @divyasshree_
108 Followers 1K Following First user/tester at @bitquery_io Samples https://t.co/IWZk9YPOIz MSM @UofIllinois, ex - @VMware
Drew Ponder @drew_ponder
10K Followers 10K Following 🤖 The Ai Whisperer ⚛️ Creator of Frequency Wave Theory 📝 https://t.co/8kPEiZHHVN 📡 #FrequencyWaveTheory
Aron Vallinder @aronvallinder
659 Followers 1K Following Researcher. Interested in cultural evolution, AI, philosophy, also cinema & poetry. PhD from @LSEPhilosophy
Emma @defenceback10
757 Followers 1K Following
Andy Han @c7ssyj4cp9
0 Followers 64 Following
Emil Ryd @emilaryd
148 Followers 213 Following physics @ oxford mostly doing ml/ai research currently have an unclear fusion of interests in ai & global development
Aashish Reddy @_AashishReddy
897 Followers 1K Following 20 | Maths & Philosophy @ University of Warwick
Lionel Levine @lionellevine
317 Followers 342 Following 0 ≠ 1, The Rest Is Commentary. https://t.co/SKgmNRGQ7B
Avi @siroctny3413154
5 Followers 284 Following Interested in AI safety, why deep learning works, and linguistics
Avyay M C @Avyay_M_C
178 Followers 2K Following Opinions expressed are solely my own and do not express the views or opinions of my employer.
Alex J Best @AlexJBest
340 Followers 3K Following Researcher in math+formal methods+ml. Working on using formal verification to train models for mathematics and reasoning @harmonicmath
Ryan Kidd @ryan_kidd44
2K Followers 1K Following Co-Executive Director @MATSprogram, Co-Founder @LondonSafeAI, Regrantor @Manifund | PhD in physics | Accelerate AI alignment + build a better future for all
云创兽Ai @Vajeat562
5 Followers 118 Following 😊 Why hunting value stocks? Ask this market dreamer! thrilled to connect. DM me to share sector ETFs! ✨ #Nasdaq #Stocks
kingsly @kingsly164172
297 Followers 2K Following
Sharmake Farah (sharm... @SharmakeFarah14
271 Followers 652 Following Thinks AI risk is somewhat likely, and AI benefits huge if we can align AIs to someone that is willing to promote human thriving even when humans are useless.
Bri @TheElderZosima
47 Followers 340 Following
Jenny Qu @GuanniQu
155 Followers 510 Following just learning to be hardcore @Caltech building AI to solve hard math problems she/they
eigen moomin @eigen_moomin
3K Followers 920 Following silly moomin | swe @ startup | 🇸🇬 chasing that glorious rainbow 🌈
Inverse Gary Marcus �... @InverseMarcus
598 Followers 3K Following Professional Goal-Post mover. Parody account.
Ubleoova @Ubleoova1814
27 Followers 1K Following Two people each with their own hearts embracing each other in the summer.
DaftoPunko @dafto_punko
271 Followers 6K Following
Ollie Perrée @OPerree
2 Followers 353 Following
Jorge Perez @2019_jorgeperez
5 Followers 178 Following Research Assistant at the Ada Lovelace Institute
Nils Nielsen @NilsNielsen10
150 Followers 6K Following
Dungers @Dungers71
402 Followers 2K Following getting there like the tortoise. Jesus is all, his being, his Father, his Holy Spirit. The only Rock required in the universe.
_skaface_ @_skaface_
3 Followers 125 Following
Arthur sungapore @ASingapore2020
95 Followers 1K Following
Ezio Wang @Ezio21084936435
3 Followers 209 Following
Hayden Field @haydenfield
15K Followers 5K Following Senior AI reporter @Verge. 5+ years covering the industry's power dynamics, societal implications & the AI arms race. Previously @CNBC. Signal: haydenfield.11
teeds @aliceteeds
3 Followers 390 Following
James Payor @jamespayor
293 Followers 253 Following I think about AI. I'm working on a dependently-typed programming language that's less painful to use. Married to @sbwzp
Eliezer Yudkowsky ⏹... @ESYudkowsky
209K Followers 102 Following The original AI alignment person. Understanding the reasons it's difficult since 2003. This is my serious low-volume account. Follow @allTheYud for the rest.
Qualy the lightbulb @QualyThe
10K Followers 322 Following Official Unofficial EA mascot. I'm here to make friends and maximise utility, and I'm all out of neglected altruistic opportunities
Peter Wildeford🇺�... @peterwildeford
22K Followers 321 Following Globally ranked top 20 forecaster 🎯 AI is not a normal technology. I'm working at @IAPSai to shape AI for global prosperity and human freedom.
Kelsey Piper @KelseyTuoc
49K Followers 971 Following We're not doomed, we just have a big to-do list.
Matthew Barnett @MatthewJBar
8K Followers 368 Following Co-founder of @MechanizeWork Married to @natalia__coelho email: matthew at mechanize dot work
Sam Bowman @sleepinyourhat
50K Followers 3K Following AI alignment + LLMs at Anthropic. On leave from NYU. Views not employers'. No relation to @s8mb. I think you should join @givingwhatwecan.
Jeffrey Ladish @JeffLadish
14K Followers 1K Following Applying the security mindset to everything @PalisadeAI
Dustin Moskovitz @moskov
72K Followers 499 Following
Aella @Aella_Girl
240K Followers 394 Following survey artist, too earnest. FDA delenda est. https://t.co/IcEgPhVD3o
Nate Silver @NateSilver538
3.3M Followers 2K Following New Book, On The Edge, #5 NYT Bestseller! https://t.co/WeCLEOd4Be
Neel Nanda @NeelNanda5
32K Followers 123 Following Mechanistic Interpretability lead DeepMind. Formerly @AnthropicAI, independent. In this to reduce AI X-risk. Neural networks can be understood, let's go do it!
Amanda Askell @AmandaAskell
54K Followers 657 Following Philosopher & ethicist trying to make AI be good @AnthropicAI. Personal account. All opinions come from my training data.
davidad 🎇 @davidad
20K Followers 9K Following Programme Director @ARIA_research | accelerate mathematical modelling with AI and categorical systems theory » build safe transformative AI » cancel heat death
Frances Lorenz @frances__lorenz
6K Followers 608 Following Claude says I process my emotions out loud & my girlfriend has a job, so I put my feelings & thoughts here ✨ working on the EA Global team @ CEA (views my own)
David Manheim @davidmanheim
10K Followers 2K Following Lecturer @TechnionLive, founder @alter_org_il, emeritus @superforecaster, PhD @PardeeRAND Optimistic on AI, pessimistic on humanity managing the risks well.
Jeffrey Emanuel @doodlestein
24K Followers 10K Following Former Quant Investor, now building @lumeraprotocol (formerly called Pastel Network) | My Open Source Projects: https://t.co/9qbOCDlaqM
iwsfutcmd is in 🇱�... @iwsfutcmd
4K Followers 2K Following linguist/graphemicist/fop unicode|PanLex(Long Now)|¿?¿?¿? opinions not my own
Scott Gray @scottgray76
9K Followers 793 Following GPU Geek at @OpenAI. I have a long standing interest in neuroscience and its application to machine learning. He/Him.
Morph @morph_labs
8K Followers 1 Following
Math, Inc. @mathematics_inc
6K Followers 0 Following A new company dedicated to autoformalization and the creation of verified superintelligence.
Japan Society Film @js_film_nyc
25K Followers 851 Following The leading presenter of Japanese cinema for more than 50 years | Classics & premieres @japansociety in NYC. Organizers of #JAPANCUTS since 2007.
Oliver Habryka @ohabryka
6K Followers 741 Following Building https://t.co/O5mQY6jndL and https://t.co/s49h898W4b. Join our writing residency at https://t.co/7HQag8pNUk!
Andrew @Dub__A
9K Followers 404 Following
Benjamin Manning @BenSManning
2K Followers 686 Following PhD candidate @MIT | Techno-optimistic-ish | https://t.co/CGdh0awKpK
Ryan Kidd @ryan_kidd44
2K Followers 1K Following Co-Executive Director @MATSprogram, Co-Founder @LondonSafeAI, Regrantor @Manifund | PhD in physics | Accelerate AI alignment + build a better future for all
Saining Xie @sainingxie
23K Followers 1K Following researcher in #deeplearning #computervision | assistant prof at @nyu_courant | rs @googledeepmind | past: rs @meta (FAIR) @ucsandiego | ynwa
Anca Dragan @ancadianadragan
12K Followers 184 Following post training co-lead at Google DeepMind, focusing on safety, alignment, post training capabilities • associate professor at UC Berkeley EECS
adithya @00aleph00
13K Followers 234 Following i want to make abstract math accessible to the world!
Cameron Jones @camrobjones
1K Followers 789 Following Assistant Professor in Psychology at Stony Brook University. I’m interested in how people interact with LLMs and they impact they might have on our psychology.
Mary Letey @maryiletey
231 Followers 571 Following PhD student at Harvard @hseas. Machine learning theory, geometry, cosmology.
Joel Becker @joel_bkr
3K Followers 2K Following move fast and fix things @METR_evals. 'soccer'-me @MessiSeconds.
Jiawei Zhao @jiawzhao
3K Followers 242 Following Research Scientist at Meta FAIR @AIatMeta, PhD @Caltech, GaLore, DeepConf
AA @measure_plan
9K Followers 578 Following experiments in computer vision, gaming, animation 👾 https://t.co/nHUJPeghSD
Jason Lee @jasondeanlee
18K Followers 4K Following Associate Professor at UC Berkeley. Former Research Scientist at Google DeepMind. ML/AI Researcher working on foundations of LLMs and deep learning.
Santiago Aranguri @santiaranguri
622 Followers 97 Following PhD student at NYU working on Mechanistic Interpretability. Previously I studied math and theory of ML at Stanford.
Mariven @psychiel
1K Followers 397 Following Towards the logic of conceptuality; the canvas of experience; the ideatic science; the end of suffering. Friend to bots and animals. https://t.co/PYPYxGNnK4
Ernest Ryu @ErnestRyu
6K Followers 349 Following Professor of Mathematics at UCLA. Interested in deep learning and optimization.
murat 🍥 @mayfer
19K Followers 7K Following
Nick Jiang @nickhjiang
774 Followers 292 Following interpreting neural networks @berkeley_ai // cs + philosophy @ucberkeley // prev @briskteaching @watershed
Igor Babuschkin @ibab
103K Followers 855 Following Maybe the real ASI was the friends we made along the way. Co-founder @xAI, Research & Engineering
Marko Jukic @mmjukic
36K Followers 77 Following Finding the golden path to interstellar civilization. Senior Analyst @bismarckanlys.
Oliver Daniels @Oliver_ADK
144 Followers 417 Following PhD Student @UMassAmherst, and MATS. married to @annasdaniels
FarmKind @farmkind_giving
456 Followers 143 Following Farming is broken: Let's fix it. We help you fund the charities that make farming kinder to animals, people & the planet
Ben Marrow @benmarrow
2K Followers 1K Following researcher. Formerly: PhD @UChicago | Marjorie Deane Fellow @TheEconomist | @AQRCapital | @Yankees | @NewYorkFed | @Yale. Personal views
James Payor @jamespayor
293 Followers 253 Following I think about AI. I'm working on a dependently-typed programming language that's less painful to use. Married to @sbwzp
rank decomposition @rankdim
1K Followers 364 Following machine learning, maths, history and philosophy of sciences
Grace (cross posting ... @kindgracekind
5K Followers 2K Following ideonomist, ai navel gazer, skyborg https://t.co/UGyhDIKCaj
Bayesian @Bayesian0_0
277 Followers 1K Following #1 AI forecaster on Manifold Markets (and #5 across all categories) https://t.co/glexRhh7tc I want everything to make sense
Shuangfei Zhai @zhaisf
2K Followers 97 Following Research Scientist & Manager, Machine Learning Research @ Apple
Earth Is A Sales Funn... @GENIC0N
29K Followers 879 Following Attention is the natural prayer of the soul
Cik ala t'cha @Ann_Aurore
19K Followers 8K Following Let the Ocean drain ~~ No DM ~~ Few pictures by meTrends for United States
12,4 B posts
29,6 B posts
4.655 posts
142 B posts
15,8 B posts
2.181 posts
2.196 posts
18,7 B posts
400 B posts
1.564 posts
9.086 posts
3.262 posts
95,9 B posts
19,6 B posts
6.000 posts
468 B posts
You might like
Evan Hubinger
@EvanHub
8K Followers
Daniel Filan
@dfrsrchtwts
2K Followers
Ajeya Cotra
@ajeya_cotra
11K Followers
Ethan Perez
@EthanJPerez
12K Followers
Rohin Shah
@rohinmshah
9K Followers
Lennart Heim
@ohlennart
7K Followers
Cas (Stephen Casper)
@StephenLCasper
6K Followers
Adam Gleave
@ARGleave
4K Followers
Alex Turner
@Turn_Trout
3K Followers
Marius Hobbhahn
@MariusHobbhahn
5K Followers
Andrew Critch (🤖�...
@AndrewCritchPhD
5K Followers
Miranda Zhang
@mirandahzhang
2K Followers
Daniel Paleka
@dpaleka
4K Followers
Lauro
@laurolangosco
1K Followers
Ryan Kidd
@ryan_kidd44
2K Followers