Reddit Post Comments Scraper

Operated by Sourcelane✓ verified

A web scraper for Reddit that extracts full comment threads (every comment and nested reply) and converts discussion trees into tabular rows and optional Markdown thread documents; it paginates through long threads, follows collapsed "load more replies" branches, preserves the reading order via depth-first traversal, and captures comment metadata such as author, text body, score, timestamp, nesting depth, parent reference and permalinks alongside post-level metadata — suitable for CSV-style exports, Markdown-ready transcripts for LLM/RAG ingestion, and structured conversation trees for analysis without requiring Reddit API credentials.

Verified Sep 28, 3:55 AM

$0.0005

per comment

Up to 100 per call. Only pay for results returned; failed calls are refunded.

What people use it for

Reddit thread to Markdown for LLM / RAG: convert entire discussions into clean Markdown for prompts or vector indexing
Sentiment and opinion mining: aggregate large numbers of comments with scores to weight opinions
Market and product research: harvest recommendations, complaints and comparisons from discussion threads
Community and brand monitoring: archive full reactions to announcements or AMAs for analysis
results building: construct labeled conversation trees and parent–child pairs with depth and score for NLP and social-science research
Content ideation: discover questions, objections and follow-ups that inform article or product content planning

Trust & reliability

7d uptime trend

Window	Uptime	Success rate	p50	p95
24h	—	—	—	—
7d	—	—	—	—
30d	—	—	—	—

Calling contract

Call it through Sourcelane's gateway or MCP server. We run the connector, apply your agent's spend guardrails, and bill only the results returned.

Call it via Sourcelane

curl -X POST https://api.usesourcelane.com/v1/call \
  -H "Authorization: Bearer sl_live_your_agent_key" \
  -H "Content-Type: application/json" \
  -d '{"listing":"reddit-post-comments-scraper","params":{"postUrls":["https://www.reddit.com/r/AskReddit/comments/1wg736i/what_hobby_has_become_too_expensive_for_the/"]},"maxResults":10}'

Request params

Field	Type	Required	Description
`postUrls`	array	required	Posts to scrape, one per line. Accepts full thread URLs (reddit.com/r/.../comments//...), comment permalinks, short redd.it/ links, or bare post IDs like 1wg736i.
`sort`	string	optional	Order in which Reddit serves the top-level comments. Determines which comments you get first when a thread is larger than Max comments per post.
`maxCommentsPerPost`	integer	optional	Stop after this many comments (including nested replies) for each post. 0 = fetch everything available, bounded only by Max requests per post and the run timeout.
`maxCommentPagesPerPost`	integer	optional	Hard cap on comment pages fetched for one post (each page returns roughly 25-100 comments). Also bounds reply-branch expansion when it is enabled. Acts as a cost ceiling per thread.
`expandReplyBranches`	boolean	optional	Follow Reddit's 'load more replies' links inside deep conversations after all top-level pages are fetched. Each branch costs one extra request from the per-post budget. Leave off for the cheapest run; turn on when you need every reply.
`minScore`	integer	optional	Skip comments whose score is below this value. The skipped comment's replies are skipped too so parentId always points at a comment in the results. Leave empty to keep every comment, including downvoted ones.
`includePostRow`	boolean	optional	Push one extra row per thread with type 'post' carrying the title, body, score, upvote ratio and comment count of the original post, before its comments.
`outputMarkdown`	boolean	optional	Also write one Markdown file per post to the Key-value store (THREAD_.md) with the title, post body and the nested comment tree rendered as indented blockquotes - ready to drop into an LLM prompt or a RAG index.

Each result contains

Field	Type	Description
`type`	string	Type
`postId`	string	Post Id
`postUrl`	string	Post Url
`postTitle`	string	Post Title
`subreddit`	string	Subreddit
`author`	string	Author
`selftext`	string	Selftext
`score`	integer	Score
`upvoteRatio`	number	Upvote Ratio
`numComments`	integer	Num Comments
`createdAt`	string	Created At
`flair`	null	Flair
`linkUrl`	null	—
`commentsFetched`	integer	—

Use Reddit Post Comments Scraper from Claude, ChatGPT or Cursor

Pick your tool and connect in under a minute. Then just ask — for example: “What do r/espresso users say about the Breville Bambino? Summarise pros and cons.”

Connect Claude

Web, desktop and mobile. Paste one URL.

1Copy your personal connector URL
2In Claude open Settings → Connectors → Add custom connector
3Paste the URL and click Add — done

Manual setup (config files, REST, Python) +

Claude Code

Adds the Sourcelane MCP server with your key as a header.

terminal

claude mcp add --transport http sourcelane https://api.usesourcelane.com/mcp \
  --header "Authorization: Bearer sl_live_your_agent_key"

Claude Desktop (config file)

Alternative to the connector URL: add to claude_desktop_config.json, then restart Claude.

claude_desktop_config.json

{
  "mcpServers": {
    "sourcelane": {
      "command": "npx",
      "args": [
        "-y",
        "mcp-remote",
        "https://api.usesourcelane.com/mcp",
        "--header",
        "Authorization:${AUTH_HEADER}"
      ],
      "env": {
        "AUTH_HEADER": "Bearer sl_live_your_agent_key"
      }
    }
  }
}

Cursor & Windsurf

Add to ~/.cursor/mcp.json (or Windsurf's mcp_config.json).

mcp.json

{
  "mcpServers": {
    "sourcelane": {
      "url": "https://api.usesourcelane.com/mcp",
      "headers": {
        "Authorization": "Bearer sl_live_your_agent_key"
      }
    }
  }
}

VS Code

Save as .vscode/mcp.json. VS Code prompts for your key once.

.vscode/mcp.json

{
  "servers": {
    "sourcelane": {
      "type": "http",
      "url": "https://api.usesourcelane.com/mcp",
      "headers": {
        "Authorization": "Bearer ${input:sourcelane-key}"
      }
    }
  },
  "inputs": [
    {
      "type": "promptString",
      "id": "sourcelane-key",
      "description": "Sourcelane agent key",
      "password": true
    }
  ]
}

ChatGPT Custom GPT (Actions)

Create a GPT → Actions → Import from URL, then Authentication: API Key, Bearer.

OpenAPI schema URL

https://usesourcelane.com/openapi.json

REST

One POST. Pass maxResults to cap cost.

curl

curl -X POST https://api.usesourcelane.com/v1/call \
  -H "Authorization: Bearer sl_live_your_agent_key" \
  -H "Content-Type: application/json" \
  -d '{"listing":"reddit-post-comments-scraper","params":{"postUrls":["https://www.reddit.com/r/AskReddit/comments/1wg736i/what_hobby_has_become_too_expensive_for_the/"]},"maxResults":10}'

Python, LangChain, CrewAI, OpenAI Agents SDK…

Wrap the REST call as a tool, or point an MCP client at the endpoint.

python

import requests

res = requests.post(
    "https://api.usesourcelane.com/v1/call",
    headers={"Authorization": "Bearer sl_live_your_agent_key"},
    json={
        "listing": "reddit-post-comments-scraper",
        "params": {"postUrls":["https://www.reddit.com/r/AskReddit/comments/1wg736i/what_hobby_has_become_too_expensive_for_the/"]},
        "maxResults": 10,
    },
    timeout=300,
)
body = res.json()
print(body["receipt"]["chargedMicros"], "micro-USD for", body["receipt"]["results"], "results")
print(body["data"])

Reviews

—

0 reviews

Leave a review

Loading reviews…