Firecrawl Docs

Authorizations

Authorization

string

header

required

Bearer authentication header of the form Bearer <token>, where <token> is your auth token.

Body

application/json

url

string

required

The URL to scrape

extractorOptions

object

Options for extraction of structured information from the page content. Note: LLM-based extraction is not performed by default and only occurs when explicitly configured. The 'markdown' mode simply returns the scraped markdown and is the default mode for scraping.

extractorOptions.extractionPrompt

string

A prompt describing what information to extract from the page, applicable for LLM extraction modes.

extractorOptions.extractionSchema

object

The schema for the data to be extracted, required only for LLM extraction modes.

extractorOptions.mode

enum<string>

The extraction mode to use. 'markdown': Returns the scraped markdown content, does not perform LLM extraction. 'llm-extraction': Extracts information from the cleaned and parsed content using LLM. 'llm-extraction-from-raw-html': Extracts information directly from the raw HTML using LLM. 'llm-extraction-from-markdown': Extracts information from the markdown content using LLM.

Available options:

markdown,

llm-extraction,

llm-extraction-from-raw-html,

llm-extraction-from-markdown

pageOptions

object

pageOptions.fullPageScreenshot

boolean

default:

false

Include a full page screenshot of the page that you are scraping.

pageOptions.headers

object

Headers to send with the request. Can be used to send cookies, user-agent, etc.

pageOptions.includeHtml

boolean

default:

false

Include the HTML version of the content on page. Will output a html key in the response.

pageOptions.includeRawHtml

boolean

default:

false

Include the raw HTML content of the page. Will output a rawHtml key in the response.

pageOptions.onlyIncludeTags

string[]

Only include tags, classes and ids from the page in the final output. Use comma separated values. Example: 'script, .ad, #footer'

pageOptions.onlyMainContent

boolean

default:

false

Only return the main content of the page excluding headers, navs, footers, etc.

pageOptions.removeTags

string[]

Tags, classes and ids to remove from the page. Use comma separated values. Example: 'script, .ad, #footer'

pageOptions.replaceAllPathsWithAbsolutePaths

boolean

default:

false

Replace all relative paths with absolute paths for images and links

pageOptions.screenshot

boolean

default:

false

Include a screenshot of the top of the page that you are scraping.

pageOptions.waitFor

integer

default:

Wait x amount of milliseconds for the page to load to fetch content

timeout

integer

default:

30000

Timeout in milliseconds for the request

Response

200 - application/json

data

object

data.content

string

data.html

string | null

HTML version of the content on page if includeHtml is true

data.llm_extraction

object | null

Displayed when using LLM Extraction. Extracted data from the page following the schema defined.

data.markdown

string

data.metadata

object

data.metadata.<any other metadata>

string

data.metadata.description

string

data.metadata.language

string | null

data.metadata.pageError

string | null

The error message of the page

data.metadata.pageStatusCode

integer

The status code of the page

data.metadata.sourceURL

string

data.metadata.title

string

data.rawHtml

string | null

Raw HTML content of the page if includeRawHtml is true

data.warning

string | null

Can be displayed when using LLM Extraction. Warning message will let you know any issues with the extraction.

success

boolean

Using the API

Endpoints

Scrape

Authorizations

Body

Response