Firecrawl integrates with Langchain as a document loader.
Note: this integration is still using v0 version of the Firecrawl API. You can install the 0.0.20 version for the Python SDK or the 0.0.36 for the Node SDK.
You can also pass params to the loader. This is a dictionary of options to pass to the crawler. See the FireCrawl API documentation for more information.
To use it in Langchain JS, you can install it via npm:
Copy
Ask AI
npm install @mendableai/firecrawl-js
Then, you can use it like this:
Copy
Ask AI
import { FireCrawlLoader } from "langchain/document_loaders/web/firecrawl";const loader = new FireCrawlLoader({ url: "https://firecrawl.dev", // The URL to scrape apiKey: process.env.FIRECRAWL_API_KEY, // Optional, defaults to `FIRECRAWL_API_KEY` in your env. mode: "scrape", // The mode to run the crawler in. Can be "scrape" for single urls or "crawl" for all accessible subpages params: { // optional parameters based on Firecrawl API docs // For API documentation, visit https://docs.firecrawl.dev },});const docs = await loader.load();