Installation
To install the Firecrawl Node SDK, you can use npm:Node
Usage
- Get an API key from firecrawl.dev
- Set the API key as an environment variable named
FIRECRAWL_API_KEY
or pass it as a parameter to theFirecrawlApp
class.
Node
Scraping a URL
To scrape a single URL with error handling, use thescrapeUrl
method. It takes the URL as a parameter and returns the scraped data as a dictionary.
Node
Crawling a Website
To crawl a website with error handling, use thecrawlUrl
method. It takes the starting URL and optional parameters as arguments. The params
argument allows you to specify additional options for the crawl job, such as the maximum number of pages to crawl, allowed domains, and the output format. See Pagination for auto/ manual pagination and limiting.
Node
Start a Crawl
Start a job without waiting usingstartCrawl
. It returns a job ID
you can use to check status. Use crawl
when you want a waiter that blocks until completion. See Pagination for paging behavior and limits.
Node
Checking Crawl Status
To check the status of a crawl job with error handling, use thecheckCrawlStatus
method. It takes the ID
as a parameter and returns the current status of the crawl job.
Node
Cancelling a Crawl
To cancel an crawl job, use thecancelCrawl
method. It takes the job ID of the startCrawl
as a parameter and returns the cancellation status.
Node
Mapping a Website
To map a website with error handling, use themapUrl
method. It takes the starting URL as a parameter and returns the mapped data as a dictionary.
Node
Crawling a Website with WebSockets
To crawl a website with WebSockets, use thecrawlUrlAndWatch
method. It takes the starting URL and optional parameters as arguments. The params
argument allows you to specify additional options for the crawl job, such as the maximum number of pages to crawl, allowed domains, and the output format.
Node
Pagination
Firecrawl endpoints for crawl and batch return anext
URL when more data is available. The Node SDK auto-paginates by default and aggregates all documents; in that case next
will be null
. You can disable auto-pagination or set limits.
Crawl
Use the waiter methodcrawl
for the simplest experience, or start a job and page manually.
Simple crawl (auto-pagination, default)
- See the default flow in Crawling a Website.
Manual crawl with pagination control (single page)
- Start a job, then fetch one page at a time with
autoPaginate: false
.
Node
Manual crawl with limits (auto-pagination + early stop)
- Keep auto-pagination on but stop early with
maxPages
,maxResults
, ormaxWaitTime
.
Node
Batch Scrape
Use the waiter methodbatchScrape
, or start a job and page manually.
Simple batch scrape (auto-pagination, default)
- See the default flow in Batch Scrape.
Manual batch scrape with pagination control (single page)
- Start a job, then fetch one page at a time with
autoPaginate: false
.
Node
Manual batch scrape with limits (auto-pagination + early stop)
- Keep auto-pagination on but stop early with
maxPages
,maxResults
, ormaxWaitTime
.
Node
Error Handling
The SDK handles errors returned by the Firecrawl API and raises appropriate exceptions. If an error occurs during a request, an exception will be raised with a descriptive error message. The examples above demonstrate how to handle these errors usingtry/catch
blocks.