跳转到主要内容

前置条件

安装 SDK

npm install @mendable/firecrawl-js
将你的 API 密钥添加到 .env 文件中:
FIRECRAWL_API_KEY=fc-YOUR-API-KEY

创建 Firecrawl 服务

创建 src/firecrawl/firecrawl.service.ts
import { Injectable } from "@nestjs/common";
import Firecrawl from "@mendable/firecrawl-js";

@Injectable()
export class FirecrawlService {
  private readonly client: Firecrawl;

  constructor() {
    this.client = new Firecrawl({ apiKey: process.env.FIRECRAWL_API_KEY });
  }

  async search(query: string, limit = 5) {
    return this.client.search(query, { limit });
  }

  async scrape(url: string) {
    return this.client.scrape(url);
  }

  async interact(url: string, prompts: string[]) {
    const result = await this.client.scrape(url, { formats: ['markdown'] });
    const scrapeId = result.metadata?.scrapeId;

    const responses = [];
    for (const prompt of prompts) {
      const response = await this.client.interact(scrapeId, { prompt });
      responses.push(response);
    }

    await this.client.stopInteraction(scrapeId);
    return responses;
  }
}

创建控制器

新建 src/firecrawl/firecrawl.controller.ts
import { Body, Controller, Post } from "@nestjs/common";
import { FirecrawlService } from "./firecrawl.service";

@Controller("firecrawl")
export class FirecrawlController {
  constructor(private readonly firecrawlService: FirecrawlService) {}

  @Post("search")
  async search(@Body("query") query: string) {
    return this.firecrawlService.search(query);
  }

  @Post("scrape")
  async scrape(@Body("url") url: string) {
    return this.firecrawlService.scrape(url);
  }

  @Post("interact")
  async interact(@Body("url") url: string, @Body("prompts") prompts: string[]) {
    return this.firecrawlService.interact(url, prompts);
  }
}

注册模块

创建 src/firecrawl/firecrawl.module.ts
import { Module } from "@nestjs/common";
import { FirecrawlService } from "./firecrawl.service";
import { FirecrawlController } from "./firecrawl.controller";

@Module({
  providers: [FirecrawlService],
  controllers: [FirecrawlController],
  exports: [FirecrawlService],
})
export class FirecrawlModule {}
AppModule 中导入 FirecrawlModule

试一试

curl -X POST http://localhost:3000/firecrawl/search \
  -H "Content-Type: application/json" \
  -d '{"query": "firecrawl web scraping"}'

后续步骤

Scrape 文档

所有 scrape 选项,包括 formats、actions 和代理设置

Search 文档

进行网页搜索并获取完整页面内容

Interact 文档

点击、填写表单并提取动态内容

Node SDK 参考

完整的 SDK 参考,涵盖爬取、map、batch scrape 等功能