Firecrawl + n8n

​

Introduction to Firecrawl and n8n

​

Why Use Firecrawl with n8n?

• Extract data from any website reliably
• Connect scraped data to other business tools
• Run on schedules or triggered by events
• Scale from simple tasks to complex pipelines

​

Step 1: Create Your Firecrawl Account

​

Sign Up for Firecrawl

1. Navigate to firecrawl.dev in your web browser
2. Click the “Get Started” or “Sign Up” button
3. Create an account using your email address or GitHub login
4. Verify your email if prompted

​

Get Your API Key

1. Go to your Firecrawl dashboard
2. Navigate to the API Keys page
3. Click “Create New API Key”
4. Give your key a descriptive name (e.g., “n8n Integration”)
5. Copy the generated API key and save it somewhere secure

​

Step 2: Set Up n8n

​

Choose Your n8n Version

• No installation required
• Free tier available
• Managed infrastructure
• Automatic updates

• Complete data control
• Run on your own servers
• Requires Docker installation
• Good for advanced users with specific security requirements

​

Option A: n8n Cloud (Recommended for Beginners)

1. Visit n8n.cloud
2. Click “Start Free” or “Sign Up”
3. Register using your email address or GitHub
4. Complete the verification process
5. You’ll be directed to your n8n dashboard

​

Option B: Self-Hosted with Docker

• Docker Desktop installed on your computer
• Basic familiarity with command line/terminal

1. Open your terminal or command prompt
2. Create a Docker volume to persist your workflow data:

docker volume create n8n_data

3. Run the n8n Docker container:

docker run -it --rm --name n8n -p 5678:5678 -v n8n_data:/home/node/.n8n docker.n8n.io/n8nio/n8n

4. Wait for n8n to start. You’ll see output indicating the server is running
5. Open your web browser and navigate to http://localhost:5678
6. Create your n8n account by registering with an email

​

Understanding the n8n Interface

• Workflows: Your saved automations appear here
• Executions: History of workflow runs
• Credentials: Stored API keys and authentication tokens
• Settings: Account and workspace configuration

​

The Workflow Canvas

• Canvas: The main area where you place and connect nodes
• Add Node Button (+): Click this to add new nodes to your workflow
• Node Panel: Opens when you click ”+” showing all available nodes
• Execute Workflow: Runs your workflow manually for testing
• Save: Saves your workflow configuration

​

Step 3: Install and Configure the Firecrawl Node

​

Add the Firecrawl Node to Your Workflow

1. In your new workflow canvas, click the ”+” button in the center
2. The node selection panel opens on the right side
3. In the search box at the top, type “Firecrawl”
4. You’ll see the Firecrawl node appear in the search results

5. Click “Install” next to the Firecrawl node
6. Wait for the installation to complete (this takes a few seconds)
7. Once installed, click on the Firecrawl node to add it to your canvas

​

Connect Your Firecrawl API Key

1. Click on the Firecrawl node box to open its configuration panel on the right
2. At the top, you’ll see a “Credential to connect with” dropdown
3. Since this is your first time, click “Create New Credential”

4. A credential configuration window opens
5. Enter a name for this credential (e.g., “My Firecrawl Account”)
6. Paste your Firecrawl API key in the “API Key” field
7. Click “Save” at the bottom

​

Test Your Connection

1. With the Firecrawl node still selected, look at the configuration panel
2. In the “Resource” dropdown, select “Scrape a url and get its content”
3. In the “URL” field, enter: https://firecrawl.dev
4. Leave other settings at their defaults for now
5. Click the “Test step” button at the bottom right of the node

​

Step 4: Create Your Telegram Bot

​

Create a Bot with BotFather

1. Open Telegram on your phone or desktop
2. Search for “@BotFather” (the official bot from Telegram)
3. Start a conversation with BotFather by clicking “Start”
4. Send the command /newbot to create a new bot
5. BotFather will ask you to choose a name for your bot (this is the display name users will see)
6. Enter a name like “My Firecrawl Bot”
7. Next, choose a username for your bot. It must end with “bot” (e.g., “my_firecrawl_updates_bot”)
8. If the username is available, BotFather will create your bot and send you a message with your bot token

​

Get Your Chat ID

1. Open your web browser and visit this URL (replace YOUR_BOT_TOKEN with your actual bot token):
   Copy

   Ask AI

   https://api.telegram.org/botYOUR_BOT_TOKEN/getUpdates
   

2. Keep this browser tab open
3. Now, search for your bot’s username in Telegram (the one you just created)
4. Start a conversation with your bot by clicking “Start”
5. Send any message to your bot (e.g., “hello”)
6. Go back to the browser tab and refresh the page
7. Look for the "chat":{"id": field in the JSON response
8. The number next to "id": is your chat ID (e.g., 123456789)
9. Save this chat ID for later

​

Step 5: Build Practical Workflows with Telegram

​

Example 1: Daily Firecrawl Product Updates Summary

• Scrapes Firecrawl’s product updates blog at 9 AM daily
• Uses AI to generate a summary of the content
• Sends the summary to your Telegram

1. Create a new workflow in n8n
2. Add a Schedule Trigger node:
   • Click the ”+” button on canvas
   • Search for “Schedule Trigger”
   • Configure: Every day at 9:00 AM

3. Add the Firecrawl node:
   • Click ”+” next to Schedule Trigger
   • Search for and add “Firecrawl”
   • Select your Firecrawl credential
   • Configure:
     • Resource: Scrape a url and get its content
     • URL: https://www.firecrawl.dev/blog/category/product-updates
     • Formats: Select “Summary”

4. Add the Telegram node:
   • Click ”+” next to Firecrawl
   • Search for “Telegram”
   • Click “Send a text message” to add it to the canvas
5. Set up Telegram credentials:
   • Click on the Telegram node to open its configuration
   • In the “Credential to connect with” dropdown, click “Create New Credential”
   • Paste your bot token from BotFather
   • Click “Save”

6. Configure the Telegram message:
   • Operation: Send Message
   • Chat ID: Enter your chat ID
   • Text: Leave this with a “hello” message for now
   • Click Execute step to test sending a message while receiving the summary from Firecrawl.

• Now with Firecrawl’s summary structure, add the summary to the message text by dragging the summary field from Firecrawl node output.

7. Test the workflow:
   • Click “Execute Workflow”
   • Check your Telegram for the summary message

8. Activate the workflow by toggling the “Active” switch

​

Example 2: AI News Search to Telegram

• Uses a Manual Trigger instead of Schedule (run on demand)
• Uses Search operation instead of Scrape
• Includes a Code node to format multiple results

1. Create a new workflow and add a Manual Trigger node
2. Add Firecrawl node with these settings:
   • Resource: Search and optionally scrape search results
   • Query: ai news
   • Limit: 5

3. Add a Code node to format the search results:
   • Select “Run Once for All Items”
   • Paste this code:

const results = $input.all();
let message = "Latest AI News:\n\n";

results.forEach((item) => {
  const webData = item.json.data.web;
  webData.forEach((article, index) => {
    message += `${index + 1}. ${article.title}\n`;
    message += `${article.description}\n`;
    message += `${article.url}\n\n`;
  });
});

return [{ json: { message } }];

4. Update Telegram node (using your saved credential):
   • Text: Drag the message field from Code node

​

Example 3: AI-Powered News Summary

• Add OpenAI credentials setup
• Add AI Agent node between Code and Telegram
• AI Agent analyzes and summarizes all the news articles intelligently
• Telegram receives the AI-generated summary instead of raw news list

1. Get your OpenAI API key:
   • Go to platform.openai.com/api-keys
   • Sign in or create an account
   • Click “Create new secret key”
   • Give it a name (e.g., “n8n Integration”)
   • Copy the API key immediately (you won’t see it again)

2. Add and connect the AI Agent node:
   • Click ”+” after the Code node
   • Search for “Basic LLM Chain” or “AI Agent”
   • Drag the message field from the Code node to the AI Agent’s input prompt field
   • Select OpenAI as the LLM provider

3. Add your OpenAI credentials:
   • Click “Create New Credential” for OpenAI
   • Paste your OpenAI API key
   • Select model: gpt-5-mini (cost-effective) or gpt-5 (more capable)
   • Click “Save”

4. Add the system prompt to the AI Agent:
   • In the AI Agent node, add this system prompt:

You are an AI news analyst. Analyze the provided AI news articles and create a concise,
insightful summary highlighting the most important developments and trends.
Group related topics together and provide context about why these developments matter.
Keep the summary conversational and engaging, around 3-4 paragraphs.

5. Update the Telegram node and test:
   • Update the Telegram node:
     • Text: Drag the AI Agent’s output (the generated summary)
     • Remove the old mapping to the Code node’s message
   • Click “Execute Workflow” to test
   • The AI will analyze all news articles and create a summary
   • Check your Telegram for the AI-generated summary

​

Understanding Firecrawl Operations

​

Scrape a url and get its content

• Scrapes a single URL
• Returns clean markdown, HTML, or AI-generated summaries
• Can capture screenshots and extract links

• Article extraction
• Product page monitoring
• Blog post scraping
• Generating page summaries

​

Search and optionally scrape search results

• Searches the web, news, or images
• Returns titles, descriptions, and URLs
• Optionally scrapes the full content of results

• Research automation
• News monitoring
• Trend discovery
• Finding relevant content

​

Crawl a website

• Follows links automatically
• Scrapes multiple pages in one operation
• Can filter URLs by patterns

• Full documentation extraction
• Site archiving
• Multi-page data collection

​

Map a website and get urls

• Discovers all links on a site
• Returns clean URL list
• Fast and lightweight

• URL discovery
• Sitemap generation
• Planning larger crawls

​

Extract Data

• AI-powered data extraction
• Returns data in your specified format
• Works across multiple pages

• Custom data extraction
• Building databases
• Structured information gathering

​

Batch Scrape

• Processes multiple URLs at once
• More efficient than loops
• Returns all results together

• Processing URL lists
• Bulk data collection
• Large-scale scraping projects

​

Workflow Templates and Examples

​

Featured Templates

​

How to Import Templates

1. Click on a workflow template link
2. Click “Import template to localhost:5678 self-hosted instance” button on the template page
3. The workflow opens in your n8n instance
4. Configure credentials for each node
5. Customize settings for your use case
6. Activate the workflow

​

Best Practices

​

Testing and Debugging

• Always test workflows manually before activating schedules
• Use the “Execute Workflow” button to test the entire flow
• Check output data at each node to verify correctness
• Use the “Executions” tab to review past runs and debug issues

​

Error Handling

• Add Error Trigger nodes to catch and handle failures
• Set up notifications when workflows fail
• Use the “Continue On Fail” setting for non-critical nodes
• Monitor your workflow executions regularly

​

Performance Optimization

• Use Batch Scrape for multiple URLs instead of loops
• Set appropriate rate limits to avoid overwhelming target sites
• Cache data when possible to reduce unnecessary requests
• Schedule intensive workflows during off-peak hours

​

Security

• Never expose API keys in workflow configurations
• Use n8n’s credential system to securely store authentication
• Be careful when sharing workflows publicly
• Follow target websites’ terms of service and robots.txt

​

Next Steps

​

Explore Advanced Features

• Study webhook configurations for real-time data processing
• Experiment with AI-powered extraction using prompts and schemas
• Build complex multi-step workflows with branching logic

​

Join the Community

• Firecrawl Discord - Get help with Firecrawl and discuss web scraping
• n8n Community Forum - Ask questions about workflow automation
• Share your workflows and learn from others

​

Recommended Learning Path

1. Complete the example workflows in this guide
2. Modify templates from the community library
3. Build a workflow to solve a real problem in your work
4. Explore advanced Firecrawl operations
5. Contribute your own templates to help others

​

Additional Resources

• Firecrawl API Documentation
• n8n Documentation
• Web Scraping Best Practices