@TypeLLM Neat, but I kinda like that Jev fields are independent tho. If the classify step is wrong then every field after it is wrong too and thats harder to debug than two separate calls
@newlinedotco But wouldn't "never trained on it" get shaky on really niche labels tho? I'd still want a small labeled set just to check the scores mean what I think they mean
@MrAhmadAwais@CommandCodeAI Honestly "use Jev to decide" is the part I'd want most. Getting a choice and a score back means you can actually write a test for it instead of parsing whatever the model felt like saying
@0x0SojalSec Curious how the scores compare tho. If a 0.8 from Clef doesnt mean the same as a 0.8 from Jev then swapping them kinda breaks all your thresholds
5K Followers 351 FollowingAI engineering education for developers. Ship real AI products: https://t.co/QlRGWNwGwD Join the newline newsletter: https://t.co/Rw63NWnS4d
55K Followers 2K FollowingCEO ⌘ @CommandCodeAI ❯ Langbase ❯ Ex VP DX ❯ @Google Dev Advisory Board ❯ Award-winning @GitHub Star ❯ @NASA Mars code @SatyaNadella "awesome example for devs"
156K Followers 78 FollowingSane + 🌶️ takes in an insane AI world... AI capabilities researcher: co-created RLHF/ChatGPT @ @openai now trying to right the wrong 🤭 (ceo @typesafeai)