Change Tracking

​

Overview

• Detect if a webpage has changed since the last scrape
• View the specific changes between scrapes
• Get structured data about what has changed
• Control the visibility of changes

• previousScrapeAt: The timestamp of the previous scrape that the current page is being compared against (null if no previous scrape)
• changeStatus: The result of the comparison between the two page versions
  • new: This page did not exist or was not discovered before (usually has a null previousScrapeAt)
  • same: This page’s content has not changed since the last scrape
  • changed: This page’s content has changed since the last scrape
  • removed: This page was removed since the last scrape
• visibility: The visibility of the current page/URL
  • visible: This page is visible, meaning that its URL was discovered through an organic route (through links on other visible pages or the sitemap)
  • hidden: This page is not visible, meaning it is still available on the web, but no longer discoverable via the sitemap or crawling the site. We can only identify invisible links if they had been visible, and captured, during a previous crawl or scrape

​

SDKs

​

Basic Usage

const firecrawl = new Firecrawl({ apiKey: 'your-api-key' });
const result = await firecrawl.scrape('https://example.com', {
  formats: ['markdown', 'changeTracking']
});

// Access change tracking data
console.log(result.changeTracking)

{
  "url": "https://firecrawl.dev",
  "markdown": "# AI Agents for great customer experiences\n\nChatbots that delight your users...",
  "changeTracking": {
    "previousScrapeAt": "2025-04-10T12:00:00Z",
    "changeStatus": "changed",
    "visibility": "visible"
  }
}

​

Advanced Options

const result = await firecrawl.scrape('https://example.com', {
  formats: [
    'markdown',
    {
      type: 'changeTracking',
      modes: ['git-diff', 'json'], // Enable specific change tracking modes
      schema: {
        type: 'object',
        properties: {
          title: { type: 'string' },
          content: { type: 'string' }
        }
      }, // Schema for structured JSON comparison
      prompt: 'Custom prompt for extraction', // Optional custom prompt
      tag: 'production' // Optional tag for separate change tracking histories
    }
  ]
});

// Access git-diff format changes
if (result.changeTracking.diff) {
  console.log(result.changeTracking.diff.text); // Git-style diff text
  console.log(result.changeTracking.diff.json); // Structured diff data
}

// Access JSON comparison changes
if (result.changeTracking.json) {
  console.log(result.changeTracking.json.title.previous); // Previous title
  console.log(result.changeTracking.json.title.current); // Current title
}

​

Git-Diff Results Example:

 **April, 13 2025**
 
-**05:55:05 PM**
+**05:58:57 PM**

...

​

JSON Comparison Results Example:

{
  "time": { 
    "previous": "2025-04-13T17:54:32Z", 
    "current": "2025-04-13T17:55:05Z" 
  }
}

​

Data Models

interface FirecrawlDocument {
  // ... other properties
  changeTracking?: {
    previousScrapeAt: string | null;
    changeStatus: "new" | "same" | "changed" | "removed";
    visibility: "visible" | "hidden";
    diff?: {
      text: string;
      json: {
        files: Array<{
          from: string | null;
          to: string | null;
          chunks: Array<{
            content: string;
            changes: Array<{
              type: string;
              normal?: boolean;
              ln?: number;
              ln1?: number;
              ln2?: number;
              content: string;
            }>;
          }>;
        }>;
      };
    };
    json?: any;
  };
}

interface ChangeTrackingFormat {
  type: 'changeTracking';
  prompt?: string;
  schema?: any;
  modes?: ("json" | "git-diff")[];
  tag?: string | null;
}

interface ScrapeParams {
  // ... other properties
  formats?: Array<'markdown' | 'html' | ChangeTrackingFormat>;
}

​

Change Tracking Modes

​

Git-Diff Mode

@@ -1,1 +1,1 @@
-old content
+new content

• files: Array of changed files (in web context, typically just one)
• chunks: Sections of changes within a file
• changes: Individual line changes with type (add, delete, normal)

​

JSON Mode

{
  "title": {
    "previous": "Old Title",
    "current": "New Title"
  },
  "price": {
    "previous": "$19.99",
    "current": "$24.99"
  }
}

​

Important Facts

• Comparison Method: Scrapes are always compared via their markdown response.
  • The markdown format must also be specified when using the changeTracking format. Other formats may also be specified in addition.
  • The comparison algorithm is resistant to changes in whitespace and content order. iframe source URLs are currently ignored for resistance against captchas and antibots with randomized URLs.
• Matching Previous Scrapes: Previous scrapes to compare against are currently matched on the source URL, the team ID, the markdown format, and the tag parameter.
  • For an effective comparison, the input URL should be exactly the same as the previous request for the same content.
  • Crawling the same URLs with different includePaths/excludePaths will have inconsistencies when using changeTracking.
  • Scraping the same URLs with different includeTags/excludeTags/onlyMainContent will have inconsistencies when using changeTracking.
  • Compared pages will also be compared against previous scrapes that only have the markdown format without the changeTracking format.
  • Comparisons are scoped to your team. If you scrape a URL for the first time with your API key, its changeStatus will always be new, even if other Firecrawl users have scraped it before.
• Beta Status: While in Beta, it is recommended to monitor the warning field of the resulting document, and to handle the changeTracking object potentially missing from the response.
  • This may occur when the database lookup to find the previous scrape to compare against times out.

​

Examples

​

Basic Scrape Example

// Request
{
    "url": "https://firecrawl.dev",
    "formats": ["markdown", "changeTracking"]
}

// Response
{
  "success": true,
  "data": {
    "markdown": "...",
    "metadata": {...},
    "changeTracking": {
      "previousScrapeAt": "2025-03-30T15:07:17.543071+00:00",
      "changeStatus": "same",
      "visibility": "visible"
    }
  }
}

​

Crawl Example

// Request
{
    "url": "https://firecrawl.dev",
    "scrapeOptions": {
        "formats": ["markdown", "changeTracking"]
    }
}

​

Tracking Product Price Changes

const result = await firecrawl.scrape('https://example.com/product', {
  formats: [
    'markdown',
    {
      type: 'changeTracking',
      modes: ['json'],
      schema: {
        type: 'object',
        properties: {
          price: { type: 'string' },
          availability: { type: 'string' }
        }
      }
    }
  ]
});

if (result.changeTracking.changeStatus === 'changed') {
  console.log(`Price changed from ${result.changeTracking.json.price.previous} to ${result.changeTracking.json.price.current}`);
}

​

Monitoring Content Changes with Git-Diff

const result = await firecrawl.scrape('https://example.com/blog', {
  formats: [
    'markdown',
    { type: 'changeTracking', modes: ['git-diff'] }
  ]
});

if (result.changeTracking.changeStatus === 'changed') {
  console.log('Content changes:');
  console.log(result.changeTracking.diff.text);
}

​

Billing