it is genuinely miraculous to be able to produce something which is correct out of thin air, even on novel tasks, like 65% of the time. there is also no intellectual task for which an unpredictable 35% failure rate is acceptable other than spam or scams.
it is genuinely miraculous to be able to produce something which is correct out of thin air, even on novel tasks, like 65% of the time. there is also no intellectual task for which an unpredictable 35% failure rate is acceptable other than spam or scams.
@revhowardarson @Sharon_Kuruvila Thank you! Transformers as they are now are a dead end for anything that requires "reasoning" or hard verification, you can't just vector space your way into thought. We are gonna need stateful RNN's and a bunch of other breakthroughs to get further than vaccous text and images
@revhowardarson There are entire classes of highly paid consultants with much worse than 35% error rates.
@revhowardarson @Pseud0nymSmith That's undeniable but a lot of the discussion around it seems to rest on the assumption it will always be that unreliable, which seems like a very foolhardy thing to be confident of!
@revhowardarson They are publicly available few shot learning algorithm that accept natural language as input and output. This is the best use case, and I will die on that hill ⛰️
@revhowardarson Imagining a dark future where the AI spin out pure gold and prophecy, and every human has as their job separating the wheat from the chaff in the Library of Babel like those like those kids in Bladerunner 2049 sorting e-waste.
@revhowardarson My inclination thus far; something can be at once miraculous and useless. The process of converting a miracle into a productive asset is uncertain and requires refinement, but the human tendency is to assume utility follows naturally. 0-to-1 thinking permeates the zone.
@revhowardarson There are many tasks where it's much easier to separate wheat from chaff than it is to produce wheat. Frequently needs new tools and workflows though.
@revhowardarson I built a superoptimizer that could produce weird SIMD sequences to solve 'black box' problems. It wasn't 35% wrong - but was frequently unable to solve cases. When it worked... "genuinely miraculous to be able to produce something which is correct out of thin air" sums it up.
@revhowardarson Evergreen. x.com/pfau/status/16…