vibecoded a visualizer for latent representations of tokens as they flow through gpt-2 (runs entirely in browser on webgl)
visualizing high dim space is hard so rather than just PCAing 768 -> 2, it renders similarity as graph connectedness at an adjustable cosine threshold
I’m surprised that no app uses face verification (like what dating apps have) to combat bots. Deleting an account is terrible for false positives, but having to do face verification or be locked out of your account is just an inconvenience. Would solve most of the issue imo
My experiments with fine-tuning RoBERTa to do language diffusion are at an end. Surprisingly cohesive with such a minimum implementation but not as good as gpt-2.
A more thorough implementation (and better training) should be able to reach parity on quality and speed though.
Just read how Fourier transforms “work” because sines and cosines form an orthogonal basis for a specific Hilbert space of functions. Math is beautiful but it always feels like a bottomless pit of knowledge where there’s always an infinite amount of things you don’t know
I don’t know why LLM companies don’t watermark their output by using rare UTF-8 code points for similar looking characters. If they just replaced all U+002D: HYPHEN-MINUS with U+2010: HYPHEN, basically no one would notice but it’d be obvious to software that it’s generated output
In the early days of X/PayPal, fraud was their biggest problem and they built out tools for anomaly detection which eventually led to Palantir. It’s ironic that 20 years later, the reincarnation of X (via twitter) does such a bad job of anomaly (bot account) detection
Increased the number of diffusion steps for my RoBERTa Diffusion model and it’s wild how surprisingly good this is. Will fine-tune it on OpenWebText and compare it to GPT-2 later
I’m am surprised at the amount of coherency I’ve gotten by trying to fine-tune RoBERTa into a language diffusion model. Pretty decent for a 6 year-old model with only 125 million parameters
New Paper!
Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agents
A longstanding goal of AI research has been the creation of AI that can learn indefinitely. One path toward that goal is an AI that improves itself by rewriting its own code, including any code…
Don't use structured output mode for reasoning tasks.
We’re open sourcing Osmosis-Structure-0.6B: an extremely small model that can turn any unstructured data into any format (e.g. JSON schema).
Use it with any model - download and blog below!
When I started using Arch Linux years ago, any time something would randomly break I’d have to spend at least an hour sifting through forums to find a solution. Now ChatGPT can diagnose and fix it in a few seconds and Pewdiepie uses Arch
230 Followers 176 Followingusing prediction markets to trade equities. i dislike authorities. all views are my own. not financial advice. long free markets.
25 Followers 808 FollowingStudent of deen, software engineer, building SaaS (the next 🦄 Insha'Allah).
Love poetry, arts, mountains, moon, architecture, and coffee.
245 Followers 7K Following"In this is love, not that we have loved God but that he loved us and sent his Son to be the propitiation for our sins."
1 JOHN 4:10 ✝️🙏🏼🫶🏿
106K Followers 966 FollowingTechnology's daily show. Hosted by @johncoogan & @jordihays. Streaming live 11a-2p PT every weekday and available on Apple, Spotify, & YouTube.
169K Followers 2K FollowingTrade on anything in all 50 states: politics, sports, entertainment, crypto, weather, and so much more. For sports: @KalshiSports For support: @KalshiSupport
75K Followers 0 FollowingSwift is a general-purpose programming language that's fast, modern, safe, and a joy to write. Designed for all, developed in the open.
20K Followers 2 FollowingThe future of AI development starts here. Sign up to our 📪 Newsletter → https://t.co/gpuHGRyHTs. We are hiring → https://t.co/cPTAes0HMt 🚀
647K Followers 35 FollowingWe're an AI safety and research company that builds reliable, interpretable, and steerable AI systems. Talk to our AI assistant @claudeai on https://t.co/FhDI3KQh0n.
9K Followers 3K FollowingDirector of Developer Tech @ NVIDIA, Co-founder/CEO https://t.co/GCUjRDOu73 acquired by NVIDIA • I laugh til I cry it's not the same on zoom •YC W20 | UCSB • views are my own