thomas @irun28

i run. i eat. i run to eat. NYC Joined January 2010

Tweets

2
Followers

9
Following

6
Likes

11

Yuhong Sun @Yuhong_S

6 months ago

We’re open sourcing the first benchmark for enterprise AI search: EnterpriseRAG-Bench. Retrieval is the foundation of every AI agent that works with company data. 1M token context windows and extended thinking are wasted if the agent can't find the right information across your tools. EnterpriseRAG-Bench evaluates how well an AI agent can navigate hundreds of thousands of Slack messages, contradictory sources, and “official docs” that haven’t been updated in years. The dataset is 500k synthetic documents and 500 questions across 9 sources like Slack, GitHub, and Google Drive. The corpus was created with thematic clusters, jargon, and ambiguities to reflect how real companies operate. We benchmarked major AI platforms, popular open source projects, and enterprise search products. Here are some of our findings: 💡 Agent harness matters just as much as retrieval technique - OpenClaw with a BM25 tool cleared nearly all platforms 💡 Recall tracks quite closely with answer correctness - if you can surface the correct information, today’s LLMs can reliably generate the right answer 💡 Onyx was the only open source product at the top of the benchmark - most RAG projects are built for personal use and could not keep pace at 500k docs This is the most technical work we’ve ever published, and the data has so many insights on how enterprise AI systems perform on company docs. The results, dataset, and our white paper are all available on GitHub: github.com/onyx-dot-app/E…

thomas @irun28

Yuhong Sun @Yuhong_S

thomas @irun28

Carrie @carriemallen12

Megan Mack @megsemack

Vinay Ganti @vinayganti

Kyle Dugan @kdugan80

Laura DeCamillo @Deeekers

Cindy Hwang @CindyThePlan

Esther Lee @missestherlee

LDC Solomon Page @DeCamSPTP

Canon Camera EOS @Canon_Camer__n3

Yuhong Sun @Yuhong_S

BlackBerryHelp @BlackBerryHelp

Conan O'Brien @ConanOBrien

Korilla BBQ @KorillaBBQ

alisonhaislip @alisonhaislip

o l i v i a @oliviamunn

Trends for United States

You might like