Multi-Model LLM Compare

AI & Media Processing

Operated by Sourcelane✓ verified

A tool that sends a single user prompt to many large language models in parallel and aggregates per-model outputs for side-by-side comparison. It supports hundreds of models across multiple providers (examples include GPT, Claude, Gemini, Llama, DeepSeek, Kimi, Qwen, GLM) and returns each model’s full text response along with token usage breakdown (prompt/completion/total), measured latency, provider metadata, completion reason and status, and any error messages. Designed for comparative evaluation and selection, it captures quality, speed, and cost-related metrics per model, measures per-model latency and completion behavior, and presents one record per model to enable benchmarking, A/B testing of prompts, and construction of evaluation results for prompt engineering.

Verified Sep 28, 3:55 AM

$0.001

per model response

Up to 100 per call. Only pay for results returned; failed calls are refunded.

What people use it for

Pick the best and cheapest model for a task before committing to it in production
Benchmark answer quality, speed, and price across providers on custom prompts
A/B test a system prompt across multiple models
Build evaluation results for prompt engineering

Trust & reliability

7d uptime trend

Window	Uptime	Success rate	p50	p95
24h	—	—	—	—
7d	—	—	—	—
30d	—	—	—	—

Calling contract

Call it through Sourcelane's gateway or MCP server. We run the connector, apply your agent's spend guardrails, and bill only the results returned.

Call it via Sourcelane

curl -X POST https://api.usesourcelane.com/v1/call \
  -H "Authorization: Bearer sl_live_your_agent_key" \
  -H "Content-Type: application/json" \
  -d '{"listing":"multi-model-llm-compare","params":{"prompt":"Explain quantum entanglement to a 10 year old in 3 sentences."},"maxResults":10}'

Request params

Field	Type	Required	Description
`prompt`	string	required	The question or instruction to send to every model.
`models`	array	optional	Pick the models to compare. Leave empty to use a default cross-provider set (Gemini, GPT, Claude, Llama).
`systemPrompt`	string	optional	Optional system instruction applied to every model, for example 'You are a concise expert assistant.'
`temperature`	number	optional	Optional sampling temperature from 0 to 2. Leave empty to use each model's default.
`maxTokens`	integer	optional	Maximum tokens each model may generate. Lower values cap cost per response. Defaults to 1024.

Each result contains

Field	Type	Description
`prompt`	string	Prompt
`model`	string	Model
`provider`	string	Provider
`response`	string	Response
`promptTokens`	integer	Prompt Tokens
`completionTokens`	integer	Completion Tokens
`totalTokens`	integer	Total Tokens
`latencyMs`	integer	Latency Ms
`finishReason`	string	Finish Reason
`status`	string	Status
`error`	null	—

Use Multi-Model LLM Compare from Claude, ChatGPT or Cursor

Pick your tool and connect in under a minute. Then just ask — for example: “Transcribe this video and give me timestamps for every product mention: <url>”

Connect Claude

Web, desktop and mobile. Paste one URL.

1Copy your personal connector URL
2In Claude open Settings → Connectors → Add custom connector
3Paste the URL and click Add — done

Manual setup (config files, REST, Python) +

Claude Code

Adds the Sourcelane MCP server with your key as a header.

terminal

claude mcp add --transport http sourcelane https://api.usesourcelane.com/mcp \
  --header "Authorization: Bearer sl_live_your_agent_key"

Claude Desktop (config file)

Alternative to the connector URL: add to claude_desktop_config.json, then restart Claude.

claude_desktop_config.json

{
  "mcpServers": {
    "sourcelane": {
      "command": "npx",
      "args": [
        "-y",
        "mcp-remote",
        "https://api.usesourcelane.com/mcp",
        "--header",
        "Authorization:${AUTH_HEADER}"
      ],
      "env": {
        "AUTH_HEADER": "Bearer sl_live_your_agent_key"
      }
    }
  }
}

Cursor & Windsurf

Add to ~/.cursor/mcp.json (or Windsurf's mcp_config.json).

mcp.json

{
  "mcpServers": {
    "sourcelane": {
      "url": "https://api.usesourcelane.com/mcp",
      "headers": {
        "Authorization": "Bearer sl_live_your_agent_key"
      }
    }
  }
}

VS Code

Save as .vscode/mcp.json. VS Code prompts for your key once.

.vscode/mcp.json

{
  "servers": {
    "sourcelane": {
      "type": "http",
      "url": "https://api.usesourcelane.com/mcp",
      "headers": {
        "Authorization": "Bearer ${input:sourcelane-key}"
      }
    }
  },
  "inputs": [
    {
      "type": "promptString",
      "id": "sourcelane-key",
      "description": "Sourcelane agent key",
      "password": true
    }
  ]
}

ChatGPT Custom GPT (Actions)

Create a GPT → Actions → Import from URL, then Authentication: API Key, Bearer.

OpenAPI schema URL

https://usesourcelane.com/openapi.json

REST

One POST. Pass maxResults to cap cost.

curl

curl -X POST https://api.usesourcelane.com/v1/call \
  -H "Authorization: Bearer sl_live_your_agent_key" \
  -H "Content-Type: application/json" \
  -d '{"listing":"multi-model-llm-compare","params":{"prompt":"Explain quantum entanglement to a 10 year old in 3 sentences."},"maxResults":10}'

Python, LangChain, CrewAI, OpenAI Agents SDK…

Wrap the REST call as a tool, or point an MCP client at the endpoint.

python

import requests

res = requests.post(
    "https://api.usesourcelane.com/v1/call",
    headers={"Authorization": "Bearer sl_live_your_agent_key"},
    json={
        "listing": "multi-model-llm-compare",
        "params": {"prompt":"Explain quantum entanglement to a 10 year old in 3 sentences."},
        "maxResults": 10,
    },
    timeout=300,
)
body = res.json()
print(body["receipt"]["chargedMicros"], "micro-USD for", body["receipt"]["results"], "results")
print(body["data"])

Reviews

—

0 reviews

Leave a review

Loading reviews…