Batch Scrape
Note: A new v2 version of this API is now available with improved batch processing performance and reliability.
Authorizations
Body
Only return the main content of the page excluding headers, navs, footers, etc.
Returns a cached version of the page if it is younger than this age in milliseconds. If a cached version of the page is older than this value, the page will be scraped. If you do not need extremely fresh data, enabling this can speed up your scrapes by 500%. Defaults to 0, which disables caching.
Controls how PDF files are processed during scraping. When true, the PDF content is extracted and converted to markdown format, with billing based on the number of pages (1 credit per page). When false, the PDF file is returned in base64 encoding with a flat rate of 1 credit total.
Actions to perform on the page before grabbing the content
- Wait
- Screenshot
- Click
- Write text
- Press a key
- Scroll
- Scrape
- Execute JavaScript
- Generate PDF
Specifies the type of proxy to use.
- basic: Proxies for scraping sites with none to basic anti-bot solutions. Fast and usually works.
- enhanced: Enhanced proxies for scraping sites with advanced anti-bot solutions. Slower, but more reliable on certain sites. Billed at the same credit cost as basic.
- auto: Firecrawl will automatically retry scraping with enhanced proxies if the basic proxy fails. Enhanced proxies carry no credit surcharge, so either way only the regular cost is billed.
If you do not specify a proxy, Firecrawl will default to basic.
basic, enhanced, auto If true, the page will be stored in the Firecrawl index and cache. Setting this to false is useful if your scraping activity may have data protection concerns. Using some parameters associated with sensitive scraping (actions, headers) will force this parameter to be false.
Per-request Threat Protection override. Fields you provide replace the corresponding fields of your organization's policy for this request only; omitted fields keep their organization-level values. Requires Threat Protection to be enabled for your team (enterprise feature) — otherwise the request is rejected with a 403. If your organization has disabled request overrides, any request that includes this object is rejected with a 403. If Threat Protection is enforced for your team, mode may not be set to off.
If true, this will enable zero data retention for this batch scrape. To enable this feature, please contact help@firecrawl.dev
Response
Successful response

