I’m surprised that no app uses face verification (like what dating apps have) to combat bots. Deleting an account is terrible for false positives, but having to do face verification or be locked out of your account is just an inconvenience. Would solve most of the issue imo
My experiments with fine-tuning RoBERTa to do language diffusion are at an end. Surprisingly cohesive with such a minimum implementation but not as good as gpt-2.
A more thorough implementation (and better training) should be able to reach parity on quality and speed though.
Just read how Fourier transforms “work” because sines and cosines form an orthogonal basis for a specific Hilbert space of functions. Math is beautiful but it always feels like a bottomless pit of knowledge where there’s always an infinite amount of things you don’t know
I don’t know why LLM companies don’t watermark their output by using rare UTF-8 code points for similar looking characters. If they just replaced all U+002D: HYPHEN-MINUS with U+2010: HYPHEN, basically no one would notice but it’d be obvious to software that it’s generated output
In the early days of X/PayPal, fraud was their biggest problem and they built out tools for anomaly detection which eventually led to Palantir. It’s ironic that 20 years later, the reincarnation of X (via twitter) does such a bad job of anomaly (bot account) detection
Increased the number of diffusion steps for my RoBERTa Diffusion model and it’s wild how surprisingly good this is. Will fine-tune it on OpenWebText and compare it to GPT-2 later
I’m am surprised at the amount of coherency I’ve gotten by trying to fine-tune RoBERTa into a language diffusion model. Pretty decent for a 6 year-old model with only 125 million parameters
New Paper!
Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agents
A longstanding goal of AI research has been the creation of AI that can learn indefinitely. One path toward that goal is an AI that improves itself by rewriting its own code, including any code…
Don't use structured output mode for reasoning tasks.
We’re open sourcing Osmosis-Structure-0.6B: an extremely small model that can turn any unstructured data into any format (e.g. JSON schema).
Use it with any model - download and blog below!
When I started using Arch Linux years ago, any time something would randomly break I’d have to spend at least an hour sifting through forums to find a solution. Now ChatGPT can diagnose and fix it in a few seconds and Pewdiepie uses Arch
I’m realizing that optimizing for the highest expected value in life is a bad objective; it’s tail end of outcomes that really matters. One should optimize for the 95 and 99 percentiles outcomes, not the mean or median.
Even if “aiming for the stars and falling short” has a…
18 Followers 800 FollowingStudent of deen, software engineer, building SaaS (the next 🦄 Insha'Allah).
Love poetry, arts, mountains, moon, architecture, and coffee.
181 Followers 6K Following"In this is love, not that we have loved God but that he loved us and sent his Son to be the propitiation for our sins."
1 JOHN 4:10 ✝️🙏🏼🫶🏿
321 Followers 5K FollowingI post my conscious thoughts with the world, live life to the fullest, keep things simple, truthful & filter the noise. I am a long term⚡️ 🚀nvestor
205K Followers 5K FollowingVC at @MenloVentures. Formerly founding team @glean, @Google Search. @Cornell CS. Tweets about tech, immigration, India, fitness and search.
156K Followers 2K FollowingTrade on anything in all 50 states: politics, sports, entertainment, crypto, weather, and so much more. For sports: @KalshiSports For support: @KalshiSupport
75K Followers 0 FollowingSwift is a general-purpose programming language that's fast, modern, safe, and a joy to write. Designed for all, developed in the open.
20K Followers 2 FollowingThe future of AI development starts here. Sign up to our 📪 Newsletter → https://t.co/gpuHGRyHTs. We are hiring → https://t.co/cPTAes0HMt 🚀
637K Followers 35 FollowingWe're an AI safety and research company that builds reliable, interpretable, and steerable AI systems. Talk to our AI assistant @claudeai on https://t.co/FhDI3KQh0n.
9K Followers 3K FollowingDirector of Developer Tech @ NVIDIA, Co-founder/CEO https://t.co/GCUjRDOu73 acquired by NVIDIA • I laugh til I cry it's not the same on zoom •YC W20 | UCSB • views are my own
43K Followers 3K FollowingWe're in a race. It's not USA vs China but humans and AGIs vs ape power centralization.
@deepseek_ai stan #1, 2023–Deep Time
«C’est la guerre.» ®1
9K Followers 20 FollowingAdvancing humanity's understanding of AI through interpretability research. Building the future of safe and powerful AI systems.
No recent Favorites. New Favorites will appear here.