Etsy 是一个汇集独特与创意商品的全球市场。本文将演示如何使用 Firecrawl 提取商品列表、店铺信息、评论和热门商品。
安装与配置
npm install firecrawl zod
概述
- 提取商品列表及其变体
- 获取店铺信息和评分
- 监控热门商品和分类
- 跟踪价格和销售数据
- 提取客户评价
使用 JSON 模式进行抓取
import { Firecrawl } from 'firecrawl';
import { z } from 'zod';
// 定义 Zod schema
const ListingSchema = z.object({
title: z.string(),
price: z.string(),
shopName: z.string(),
rating: z.number()
});
const firecrawl = new Firecrawl({ apiKey: process.env.FIRECRAWL_API_KEY });
const result = await firecrawl.scrape('https://www.etsy.com/listing/1844315896/handmade-925-sterling-silver-jewelry-set', {
formats: [{
type: 'json',
schema: z.toJSONSchema(ListingSchema)
}],
});
// 使用 Zod 解析和验证
const jsonData = typeof result.json === 'string' ? JSON.parse(result.json) : result.json;
const validated = ListingSchema.parse(jsonData);
console.log('✅ Validated listing data:');
console.log(validated);
搜索
import { Firecrawl } from 'firecrawl';
const firecrawl = new Firecrawl({ apiKey: process.env.FIRECRAWL_API_KEY });
const searchResult = await firecrawl.search('handmade jewelry site:etsy.com', {
limit: 10,
sources: [{ type: 'web' }], // { type: 'news' }（新闻）, { type: 'images' }（图片）
scrapeOptions: {
formats: ['markdown']
}
});
console.log(searchResult);
抓取
import { Firecrawl } from 'firecrawl';
const firecrawl = new Firecrawl({ apiKey: process.env.FIRECRAWL_API_KEY });
const result = await firecrawl.scrape('https://www.etsy.com/listing/1844315896/handmade-925-sterling-silver-jewelry-set', {
formats: ['markdown'], // 例如 html、links 等
onlyMainContent: true
});
console.log(result);
Map
import { Firecrawl } from 'firecrawl';
const firecrawl = new Firecrawl({ apiKey: process.env.FIRECRAWL_API_KEY });
const mapResult = await firecrawl.map('https://www.etsy.com/shop/YourShopName');
console.log(mapResult.links);
// 返回 URL 数组,不包含内容
Crawl
import { Firecrawl } from 'firecrawl';
const firecrawl = new Firecrawl({ apiKey: process.env.FIRECRAWL_API_KEY });
const crawlResult = await firecrawl.crawl('https://www.etsy.com/c/jewelry', {
limit: 10,
scrapeOptions: {
formats: ['markdown']
}
});
console.log(crawlResult.data);
批量抓取
import { Firecrawl } from 'firecrawl';
const firecrawl = new Firecrawl({ apiKey: process.env.FIRECRAWL_API_KEY });
// 等待完成
const job = await firecrawl.batchScrape([
'https://www.etsy.com/listing/1844315896/handmade-925-sterling-silver-jewelry-set',
'https://www.etsy.com/market/handmade_jewelry',
'https://www.etsy.com/market/jewelry_handmade'],
{
options: {
formats: ['markdown']
},
pollInterval: 2,
timeout: 120
}
);
console.log(job.status, job.completed, job.total);
console.log(job);