X (Twitter) Article to Markdown Scraper

X (Twitter)

Operated by Sourcelane✓ verified

Converts long-form X (Twitter) Articles/Notes into clean, structured Markdown optimized for AI, RAG, and LLM ingestion. The connector parses article content and preserves semantic and formatting elements such as headings, lists, bold/italic/code spans, images, and dividers; it prepends title and byline, extracts a cover image link and preview text, and returns author metadata and engagement metrics alongside the Markdown document. Designed for batch conversion, it produces one converted document per source article suitable for knowledge bases, content analysis, and downstream NLP pipelines.

Verified Sep 28, 3:56 AM

$0.02

per article

Up to 50 per call. Only pay for results returned; failed calls are refunded.

What people use it for

Feed long-form X Articles into a RAG pipeline or LLM knowledge base
Content archiving: create durable Markdown copies of Articles before edits or removal
Newsletter curation: import Article bodies into Markdown-based editors
Competitive content research: track long-form content published by creators and competitors
AI agent tool input: supply agents with clean, structured Markdown instead of raw HTML or JSON

Trust & reliability

7d uptime trend

Window	Uptime	Success rate	p50	p95
24h	—	—	—	—
7d	—	—	—	—
30d	—	—	—	—

Calling contract

Call it through Sourcelane's gateway or MCP server. We run the connector, apply your agent's spend guardrails, and bill only the results returned.

Call it via Sourcelane

curl -X POST https://api.usesourcelane.com/v1/call \
  -H "Authorization: Bearer sl_live_your_agent_key" \
  -H "Content-Type: application/json" \
  -d '{"listing":"x-article-to-markdown-scraper","params":{"tweetIds":["1905545699552375179"],"maxItems":5},"maxResults":10}'

Request params

Field	Type	Required	Description
`tweetIds`	array	required	List of X (Twitter) Article/Notes tweet IDs (numeric) or full tweet URLs (https://x.com/user/status/) to convert to Markdown. Regular (non-Article) tweets are skipped.
`maxItems`	integer	optional	Maximum number of articles to fetch and convert this run. Set 0 for unlimited (up to the length of the input list). Capped at 50 per call.

Each result contains

Field	Type	Description
`tweetId`	string	Tweet Id
`title`	string	Title
`markdown`	string	Markdown
`previewText`	string	Preview Text
`coverImageUrl`	string	Cover Image Url
`authorUsername`	string	Author Username
`authorName`	string	Author Name
`authorIsBlueVerified`	boolean	Author Is Blue Verified
`authorFollowersCount`	integer	Author Followers Count
`createdAt`	string	Created At
`replyCount`	integer	Reply Count
`likeCount`	integer	Like Count
`quoteCount`	integer	—
`viewCount`	string	—
`articleUrl`	string	—

Use X (Twitter) Article to Markdown Scraper from Claude, ChatGPT or Cursor

Pick your tool and connect in under a minute. Then just ask — for example: “Find the 50 most-liked tweets about 'MCP servers' this month and summarise the takes.”

Connect Claude

Web, desktop and mobile. Paste one URL.

1Copy your personal connector URL
2In Claude open Settings → Connectors → Add custom connector
3Paste the URL and click Add — done

Manual setup (config files, REST, Python) +

Claude Code

Adds the Sourcelane MCP server with your key as a header.

terminal

claude mcp add --transport http sourcelane https://api.usesourcelane.com/mcp \
  --header "Authorization: Bearer sl_live_your_agent_key"

Claude Desktop (config file)

Alternative to the connector URL: add to claude_desktop_config.json, then restart Claude.

claude_desktop_config.json

{
  "mcpServers": {
    "sourcelane": {
      "command": "npx",
      "args": [
        "-y",
        "mcp-remote",
        "https://api.usesourcelane.com/mcp",
        "--header",
        "Authorization:${AUTH_HEADER}"
      ],
      "env": {
        "AUTH_HEADER": "Bearer sl_live_your_agent_key"
      }
    }
  }
}

Cursor & Windsurf

Add to ~/.cursor/mcp.json (or Windsurf's mcp_config.json).

mcp.json

{
  "mcpServers": {
    "sourcelane": {
      "url": "https://api.usesourcelane.com/mcp",
      "headers": {
        "Authorization": "Bearer sl_live_your_agent_key"
      }
    }
  }
}

VS Code

Save as .vscode/mcp.json. VS Code prompts for your key once.

.vscode/mcp.json

{
  "servers": {
    "sourcelane": {
      "type": "http",
      "url": "https://api.usesourcelane.com/mcp",
      "headers": {
        "Authorization": "Bearer ${input:sourcelane-key}"
      }
    }
  },
  "inputs": [
    {
      "type": "promptString",
      "id": "sourcelane-key",
      "description": "Sourcelane agent key",
      "password": true
    }
  ]
}

ChatGPT Custom GPT (Actions)

Create a GPT → Actions → Import from URL, then Authentication: API Key, Bearer.

OpenAPI schema URL

https://usesourcelane.com/openapi.json

REST

One POST. Pass maxResults to cap cost.

curl

curl -X POST https://api.usesourcelane.com/v1/call \
  -H "Authorization: Bearer sl_live_your_agent_key" \
  -H "Content-Type: application/json" \
  -d '{"listing":"x-article-to-markdown-scraper","params":{"tweetIds":["1905545699552375179"],"maxItems":5},"maxResults":10}'

Python, LangChain, CrewAI, OpenAI Agents SDK…

Wrap the REST call as a tool, or point an MCP client at the endpoint.

python

import requests

res = requests.post(
    "https://api.usesourcelane.com/v1/call",
    headers={"Authorization": "Bearer sl_live_your_agent_key"},
    json={
        "listing": "x-article-to-markdown-scraper",
        "params": {"tweetIds":["1905545699552375179"],"maxItems":5},
        "maxResults": 10,
    },
    timeout=300,
)
body = res.json()
print(body["receipt"]["chargedMicros"], "micro-USD for", body["receipt"]["results"], "results")
print(body["data"])

Reviews

—

0 reviews

Leave a review

Loading reviews…