Effortless Web Page Scraping with FireCrawl and n8n
detail.loadingPreview
Automate the extraction of website content into a clean Markdown format. This n8n workflow leverages FireCrawl to easily scrape any given URL, transforming raw web data into a structured and usable format for further processing.
About This Workflow
This n8n workflow, named 'get_a_web_page', is designed to streamline the process of web content scraping. It utilizes the power of FireCrawl, a robust web scraping API, to fetch the content of any specified URL. The workflow is configured to receive a URL via a JSON query, making it highly flexible for integration. Upon receiving the URL, it sends a POST request to the FireCrawl API, requesting the content in Markdown format. The retrieved Markdown content is then extracted and stored in a new field named 'response' for subsequent use within your automation pipelines. This enables seamless integration of web data into various applications, analytics, or AI-powered workflows.
Key Features
- Automated Web Scraping: Easily extract content from any web page.
- Markdown Output: Retrieves content in a clean and structured Markdown format.
- Flexible Input: Accepts URLs via JSON query for dynamic integration.
- FireCrawl Integration: Leverages the powerful and reliable FireCrawl API.
- Data Transformation: Renames and stores scraped content for easy access.
How To Use
- Trigger Setup: The workflow starts with an 'Execute Workflow Trigger' node. Configure this node to accept a JSON payload containing the
urlof the web page you wish to scrape. - FireCrawl Configuration: The 'FireCrawl' node (an HTTP Request node) is pre-configured to send a POST request to
https://api.firecrawl.dev/v1/scrape. Ensure you have set up the 'Firecrawl' HTTP Header Authentication in n8n with your API key. - JSON Body Setup: In the 'FireCrawl' node's parameters, the
jsonBodyis set to dynamically include theurlfrom the trigger and specifies the output format asmarkdown. - Data Extraction: The 'Edit Fields' node (a Set node) is used to extract the scraped Markdown content. It creates a new field named
responseand assigns themarkdownfield from the FireCrawl API's data. - Integration: The 'response' field now holds the scraped content, ready to be used in subsequent n8n nodes for further processing, storage, or integration with other services.
Apps Used
Workflow JSON
{
"id": "6731a307-b1cc-4f47-8288-83e530a42b32",
"name": "Effortless Web Page Scraping with FireCrawl and n8n",
"nodes": 27,
"category": "Operations",
"status": "active",
"version": "1.0.0"
}Note: This is a sample preview. The full workflow JSON contains node configurations, credentials placeholders, and execution logic.
Get This Workflow
ID: 6731a307-b1cc...
About the Author
Free n8n Workflows Official
System Admin
The official repository for verified enterprise-grade workflows.
Statistics
Related Workflows
Discover more workflows you might like
Universal CSV to JSON API Converter
Effortlessly transform CSV data into structured JSON with this versatile n8n workflow. Integrate it into any application as a custom API endpoint, supporting various input methods including file uploads and raw text.
Instant WooCommerce Order Notifications via Telegram
When a new order is placed on your WooCommerce store, instantly receive detailed notifications directly to your Telegram chat. Stay on top of your e-commerce operations with real-time alerts, including order specifics and a direct link to view the order.
On-Demand Microsoft SQL Query Execution
This workflow allows you to manually trigger and execute any SQL query against your Microsoft SQL Server database. Perfect for ad-hoc data lookups, administrative tasks, or quick tests, giving you direct control over your database operations.