Stack Overflow / Stack Exchange Scraper

Stack Overflow / Stack Exchange Scraper

Search Stack Overflow — or any of the big Stack Exchange sites (Server Fault, Super User, Ask Ubuntu, MathOverflow, Software Engineering) — by keyword and/or tags, and get back clean, structured questions. No API key, no login, no anti-bot.

It talks to the public api.stackexchange.com API directly, so results are fast and reliable. It paginates for you, dedupes by question id, and returns the question body as plain text (HTML stripped).

$2.00 per 1,000 questions ($0.002 each), with no run-start fee. Flat rate — no volume tiers, no plan gates — and you are charged only for questions actually returned.

What you get per question

Input

You must provide a query or at least one tag. Results always land in the dataset regardless of whether Notion delivery is configured.

Example

{ "query": "async await", "site": "stackoverflow", "tags": "javascript", "sortBy": "votes", "maxItems": 50 }

Tags-only search, newest first:

{ "site": "askubuntu", "tags": "apt,upgrade", "sortBy": "creation", "maxItems": 200 }

Pricing

No run-start fee. A default 50-question run costs $0.10; the 1,000-question maximum costs $2.00. Empty searches and diagnostic rows are never charged, so a query that matches nothing costs $0.00.

Quota, backoff and diagnostics

The unauthenticated Stack Exchange API allows roughly 300 requests per day per IP. This actor is built to fail loudly rather than quietly:

• It reads quota_remaining from every response and logs it, so you can see how much budget is left.
• It honours the API's own backoff directive — when Stack Exchange asks it to wait, it sleeps rather than hammering.
• Transient 429 and 5xx responses are retried with exponential backoff and a rotated user agent before it gives up.
• When it does give up, it writes a single diagnostic row with a code — NO_RESULTS, RATE_LIMITED, NETWORK or EMPTY_RESPONSE — and finishes cleanly instead of hard-failing. Diagnostic rows are never charged.

For very large jobs, split the work by tag, keyword, or site so no single run leans on one IP's daily quota.