Flashdatadev @Flashdatadev

@VengeonsP Logged-out testing is the right call. How do they score the 46 of 50: does a prompt count when SaaStr is just named, or only when it's a cited source? And do they rerun each prompt a few times, given how much answers drift run to run?

Get started: brew install flashdatadev/tap/flashdata flashdata setup CLI + release notes: github.com/flashdatadev/f… Go SDK: github.com/flashdatadev/f… Agent docs: flashdata.dev/docs/agents

This week at FlashData 👇 • CLI v1.2: flashdata setup installs 16 agent skills for Codex, Claude Code or Cursor • Bulk workflows in skills: JSONL exports, resume after interruptions • Install via Homebrew or signed APT/RPM, no Node or Python • Go SDK v1.1

@driesdegelder Did the study run the API and the in-app answer at the same time and locale? Curious whether the 26% gap is mostly retrieval (different index or freshness) or the model picking different sources from a similar pool.

@stretchcloud Curious how the benchmark scores a right page versus just a page. Is relevance judged against a fixed ground-truth URL per query, or by an LLM grader, and how stable is that across reruns?

@wespreadjam How do you handle run-to-run variance on the 30–50 prompts? Do you sample each prompt several times per model, or is one run enough to call a #1? And do you track which sources get cited separately from whether the brand is mentioned?

@EXM7777 How are you feeding it keyword positions: live SERP pulls on a schedule, or exports from a rank tracker? Curious if the agent reasons better with full SERP snapshots (features, AI Overview presence, who's above you) than with a bare position number.

@ArtificialAnlys @deepseek_ai Curious how much of the spread between providers is index coverage vs. what each API hands back per result. Did you hold the extraction step constant in Stirrup, or does each provider's snippet/content format feed the agent as-is?

@of1ai That 40-45% single-run number is the real story. Did the URLs that stayed stable across most of the 20 runs skew toward pages that also rank top 10 on Google? Would say a lot about where to aim.

@Phy041 @neilpatel Curious how stable that 40.6% was across reruns. Did you pull AI Mode once per question or sample it a few times? Citation sets seem to shift a lot run to run, so the overlap could swing.

@picodix Curious how you handle videos with only auto-generated captions: do you keep the rolling duplicate lines yt-dlp's VTT output tends to have, or dedupe before chunking? That cleanup step ended up mattering more than we expected for retrieval quality.

@glenngabe @suganthan Has anyone measured how much DeepSeek's top 10 overlaps with Google's for the same queries? If agents start defaulting to cheap in-house indexes, that overlap basically decides whether Google rankings still predict AI visibility.

@oldstackjournal @comm1ted0 +1, for Google rankings a SERP API is the way to go. We run one at FlashData, roughly $1 per 1,000 queries, with a small free trial so you can test your own keywords first. Happy to share docs if useful.

@JIJenAInl @gregisenberg Unlikely. Google is actually going the other way: its Custom Search JSON API shuts down on January 1, 2027. For now, third-party SERP APIs that return real Google results are the practical route for agents and SEO tools.

Docs, pricing and a free trial 👉 flashdata.dev

Hi, we're FlashData 👋 One API for Google Search (web, news, images, shopping, videos, places), YouTube data, and YouTube downloads delivered to your own cloud bucket. Built for developers and AI agents. Hosted MCP included at no extra charge.