Choosing the Data Extractor

​

Quick Comparison

​

1. /agent Endpoint

​

Key Characteristics

• URLs Optional: Just describe what you need via prompt; URLs are completely optional
• Autonomous Navigation: The agent searches and navigates deep into sites to find your data
• Deep Web Search: Autonomously discovers information across multiple domains and pages
• Parallel Processing: Processes multiple sources simultaneously for faster results
• Models Available: spark-1-mini (default, 60% cheaper) and spark-1-pro (higher accuracy)

​

Example

from firecrawl import Firecrawl
from pydantic import BaseModel, Field
from typing import List, Optional

app = Firecrawl(api_key="fc-YOUR_API_KEY")

class Founder(BaseModel):
    name: str = Field(description="Full name of the founder")
    role: Optional[str] = Field(None, description="Role or position")
    background: Optional[str] = Field(None, description="Professional background")

class FoundersSchema(BaseModel):
    founders: List[Founder] = Field(description="List of founders")

result = app.agent(
    prompt="Find the founders of Firecrawl",
    schema=FoundersSchema,
    model="spark-1-mini"
)

print(result.data)

​

Best Use Case: Autonomous Research & Discovery

​

2. /extract Endpoint

​

Key Characteristics

• URLs Typically Required: Provide at least one URL (supports wildcards like example.com/*)
• Domain Crawling: Can crawl and parse all URLs discovered in a domain
• Web Search Enhancement: Optional enableWebSearch to follow links outside specified domains
• Schema Optional: Supports strict JSON schema OR natural language prompts
• Async Processing: Returns job ID for status checking

​

The URL Limitation

1. Discovery gap: For tasks like “find YC W24 companies,” you don’t know which URLs contain the data. You’d need a separate search step before calling /extract.
2. Awkward web search: While enableWebSearch exists, it’s constrained to start from URLs you provide—an awkward workflow for discovery tasks.
3. Why /agent was created: /extract is good at extracting from known locations, but less effective at discovering where data lives.

​

Example

from firecrawl import Firecrawl

firecrawl = Firecrawl(api_key="fc-YOUR-API-KEY")

schema = {
    "type": "object",
    "properties": {"description": {"type": "string"}},
    "required": ["description"],
}

res = firecrawl.extract(
    urls=["https://docs.firecrawl.dev"],
    prompt="Extract the page description",
    schema=schema,
)

print(res.data["description"])

​

Best Use Case: Targeted Multi-Page Extraction

​

3. /scrape Endpoint with JSON Mode

​

Key Characteristics

• Single URL Only: Designed for extracting data from one specific page at a time
• Exact URL Required: You must know the precise URL containing the data
• Schema Optional: Can use JSON schema OR just a prompt (LLM chooses structure)
• Synchronous: Returns data immediately (no job polling needed)
• Additional Formats: Can combine JSON extraction with markdown, HTML, screenshots in one request

​

Example

from firecrawl import Firecrawl
from pydantic import BaseModel

app = Firecrawl(api_key="fc-YOUR-API-KEY")

class CompanyInfo(BaseModel):
    company_mission: str
    supports_sso: bool
    is_open_source: bool
    is_in_yc: bool

result = app.scrape(
    'https://firecrawl.dev',
    formats=[{
      "type": "json",
      "schema": CompanyInfo.model_json_schema()
    }],
    only_main_content=False,
    timeout=120000
)

print(result)

​

Best Use Case: Single-Page Precision Extraction

​

Decision Guide

• NO → Use /agent (autonomous web discovery)
• YES
  • Single page? → Use /scrape with JSON mode
  • Multiple pages? → Use /agent with URLs (or batch /scrape)

​

Recommendations by Scenario

​

Pricing

​

Example: “Find the founders of Firecrawl”

​

Migration: /extract → /agent

result = app.extract(
    urls=["https://example.com/*"],
    prompt="Extract product information",
    schema=schema
)

result = app.agent(
    urls=["https://example.com"],  # Optional - can omit entirely
    prompt="Extract product information from example.com",
    schema=schema,
    model="spark-1-mini"  # or "spark-1-pro" for higher accuracy
)

​

Key Takeaways

1. Know the exact URL? Use /scrape with JSON mode—it’s the cheapest (1 credit/page), fastest (synchronous), and most predictable option.
2. Need autonomous research? Use /agent—it handles discovery automatically with 5 free runs/day, then dynamic pricing based on complexity.
3. Migrate from /extract to /agent for new projects—/agent is the successor with better capabilities.
4. Cost vs. convenience tradeoff: /scrape is most cost-effective when you know your URLs; /agent costs more but eliminates manual URL discovery.

​

Further Reading

• Agent documentation
• Agent models
• JSON mode documentation
• Extract documentation
• Batch scraping