跳转到主要内容
将 Firecrawl 与 OpenAI 集成,构建由网页数据驱动的 AI 应用。

安装与配置

npm install @mendable/firecrawl-js openai zod
创建 .env 文件:
FIRECRAWL_API_KEY=your_firecrawl_key
OPENAI_API_KEY=your_openai_key
注意: 如果使用 Node 版本低于 20,请安装 dotenv,并在代码中添加 import 'dotenv/config'

抓取 + 摘要

此示例展示了一个简单的工作流程:抓取网站并使用 OpenAI 模型对内容进行摘要。
import FirecrawlApp from '@mendable/firecrawl-js';
import OpenAI from 'openai';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });
const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

// 抓取网站内容
const scrapeResult = await firecrawl.scrape('https://firecrawl.dev', {
    formats: ['markdown']
});

console.log('抓取内容长度:', scrapeResult.markdown?.length);

// 使用 OpenAI 模型生成摘要
const completion = await openai.chat.completions.create({
    model: 'gpt-5-nano',
    messages: [
        { role: 'user', content: `总结以下内容: ${scrapeResult.markdown}` }
    ]
});

console.log('摘要:', completion.choices[0]?.message.content);

函数调用

此示例演示如何使用 OpenAI 的函数调用功能,让模型根据用户请求自主决定何时抓取网站。
import FirecrawlApp from '@mendable/firecrawl-js';
import OpenAI from 'openai';
import { z } from 'zod';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });
const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

const ScrapeArgsSchema = z.object({
    url: z.string().describe('要抓取的网站 URL')
});

const tools = [{
    type: 'function' as const,
    function: {
        name: 'scrape_website',
        description: '从任何网站 URL 抓取内容',
        parameters: z.toJSONSchema(ScrapeArgsSchema)
    }
}];

const response = await openai.chat.completions.create({
    model: 'gpt-5-nano',
    messages: [{
        role: 'user',
        content: '什么是 Firecrawl?访问 firecrawl.dev 并告诉我相关信息。'
    }],
    tools
});

const message = response.choices[0]?.message;

if (message?.tool_calls && message.tool_calls.length > 0) {
    for (const toolCall of message.tool_calls) {
        if (toolCall.type === 'function') {
            console.log('调用的工具:', toolCall.function.name);

            const args = ScrapeArgsSchema.parse(JSON.parse(toolCall.function.arguments));
            const result = await firecrawl.scrape(args.url, {
                formats: ['markdown'] // 其他格式:html、links 等
            });
            console.log('抓取的内容:', result.markdown?.substring(0, 200) + '...');

            // 将抓取的内容发送回模型以获取最终响应
            const finalResponse = await openai.chat.completions.create({
                model: 'gpt-5-nano',
                messages: [
                    {
                        role: 'user',
                        content: '什么是 Firecrawl?访问 firecrawl.dev 并告诉我相关信息。'
                    },
                    message,
                    {
                        role: 'tool',
                        tool_call_id: toolCall.id,
                        content: result.markdown || '未抓取到内容'
                    }
                ],
                tools
            });

            console.log('最终响应:', finalResponse.choices[0]?.message?.content);
        }
    }
} else {
    console.log('直接响应:', message?.content);
}

结构化数据提取

本示例演示如何使用具备结构化输出的 OpenAI 模型,从抓取内容中提取特定数据。
import FirecrawlApp from '@mendable/firecrawl-js';
import OpenAI from 'openai';
import { z } from 'zod';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });
const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

const scrapeResult = await firecrawl.scrape('https://stripe.com', {
    formats: ['markdown']
});

console.log('抓取内容长度:', scrapeResult.markdown?.length);

const CompanyInfoSchema = z.object({
    name: z.string(),
    industry: z.string(),
    description: z.string(),
    products: z.array(z.string())
});

const response = await openai.chat.completions.create({
    model: 'gpt-5-nano',
    messages: [
        {
            role: 'system',
            content: '从网站内容中提取公司信息。'
        },
        {
            role: 'user',
            content: `提取数据: ${scrapeResult.markdown}`
        }
    ],
    response_format: {
        type: 'json_schema',
        json_schema: {
            name: 'company_info',
            schema: z.toJSONSchema(CompanyInfoSchema),
            strict: true
        }
    }
});

const content = response.choices[0]?.message?.content;
const companyInfo = content ? CompanyInfoSchema.parse(JSON.parse(content)) : null;
console.log('验证后的公司信息:', companyInfo);

搜索 + 分析

本示例将 Firecrawl 的搜索能力与 OpenAI 模型的分析相结合,用于从多个来源检索并总结信息。
import FirecrawlApp from '@mendable/firecrawl-js';
import OpenAI from 'openai';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });
const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

// 搜索相关信息
const searchResult = await firecrawl.search('Next.js 16 new features', {
    limit: 3,
    sources: [{ type: 'web' }], // 其他来源: { type: 'news' }, { type: 'images' }
    scrapeOptions: { formats: ['markdown'] }
});

console.log('搜索结果:', searchResult.web?.length, '个页面找到');

// 分析并总结关键功能
const analysis = await openai.chat.completions.create({
    model: 'gpt-5-nano',
    messages: [{
        role: 'user',
        content: `总结关键功能: ${JSON.stringify(searchResult)}`
    }]
});

console.log('分析:', analysis.choices[0]?.message?.content);

搭配 MCP 的 Responses API

此示例演示在将 Firecrawl 配置为 MCP(Model Context Protocol)服务器时,如何使用 OpenAI 的 Responses API。
import OpenAI from 'openai';

const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

const response = await openai.responses.create({
    model: 'gpt-5-nano',
    tools: [
        {
            type: 'mcp',
            server_label: 'firecrawl',
            server_description: '用于抓取和提取网站内容的网页搜索与抓取 MCP 服务器。',
            server_url: `https://mcp.firecrawl.dev/${process.env.FIRECRAWL_API_KEY}/v2/mcp`,
            require_approval: 'never'
        }
    ],
    input: '查找 Hacker News 上的热门内容和 OpenAI 的最新博客文章,并以要点形式总结'
});

console.log('Response:', JSON.stringify(response.output, null, 2));
更多示例请参见 OpenAI 文档