More exciting news today -- Gemini 1.5 Pro result is out! Gemini 1.5 Pro API-0409-preview now achieves #2 on the leaderboard, surpassing #3 GPT4-0125-preview to almost top-1! Gemini shows even stronger performance on longer prompts, in which it ranks joint #1 with the latest GPT-4-Turbo🔥 Big congrats to @GoogleDeepMind on shipping this powerful Gemini API to developers & community. Very excited to see what app can be built on top!
More exciting news today -- Gemini 1.5 Pro result is out! Gemini 1.5 Pro API-0409-preview now achieves #2 on the leaderboard, surpassing #3 GPT4-0125-preview to almost top-1! Gemini shows even stronger performance on longer prompts, in which it ranks joint #1 with the latest GPT-4-Turbo🔥 Big congrats to @GoogleDeepMind on shipping this powerful Gemini API to developers & community. Very excited to see what app can be built on top!
CI plot & Links - Full leaderboard leaderboard.lmsys.org - Chat & vote at chat.lmsys.org - Gemini 1.5 Pro blog.google/technology/ai/…
@lmsysorg Awesome! I enjoy using Gemini 1.5 Pro for large document analysis
@lmsysorg @JeffDean So awesome to see Google pushing hard on this. Especially with the massive context window. @JeffDean @OfficialLoganK
@lmsysorg How is it Rank 2 if there are three models above it? Shouldn't it be rank 4?
@lmsysorg Very exciting, if you want to get started with Gemini 1.5 Pro, you can use it for free in AI Studio and the Gemini API: aistudio.google.com
@lmsysorg Everything's gonna leap frog back and forth. It's better for people to stick to one or two main tools for now.
@lmsysorg That’s a crazy good leaderboard. Nice to see other models pushing up. Next move by Open AI should be very interesting.