Competitor Price Monitoring with AI Scraping and Analysis
detail.loadingPreview
Automate competitor price tracking, AI-driven data extraction, analysis, and alerts.
About This Workflow
This workflow automates the process of monitoring competitor pricing across major e-commerce platforms like Amazon, Best Buy, and Target. It utilizes HTTP requests to scrape product pages, then employs ScrapeGraphAI for intelligent data extraction, including pricing, product details, and quality metrics. The extracted data is further analyzed by a custom code node to detect price changes, generate competitive intelligence, and determine tracking priority. Finally, the workflow logs the data to Google Sheets and sends Slack alerts for significant price changes or discounts.
It is designed to provide valuable insights into market trends, competitive strategies, and product value, enabling businesses to make informed pricing decisions and capitalize on sales opportunities.
Key Features
- Dual Trigger System: Supports both scheduled daily checks and on-demand manual triggers via webhooks.
- Multi-Platform Scraping: Integrates with Amazon, Best Buy, and Target using HTTP requests with custom user agents.
- AI-Powered Data Extraction: Leverages ScrapeGraphAI to intelligently extract structured product data, including prices, ratings, availability, and features.
- Advanced Price Analysis: Custom code node analyzes pricing data to detect significant drops, increases, discounts, and calculates value scores.
- Competitive Intelligence: Generates insights on market position, brand strength, and customer satisfaction.
- Automated Data Logging: Stores comprehensive pricing data and analysis results in Google Sheets.
- Smart Alerting: Sends real-time alerts to Slack for price drops, high discounts, and other critical changes.
- Configurable Tracking: Prioritizes alerts based on user-defined thresholds and product tracking lists.
- Extensible Design: Easily adaptable to include more retailers or refine analysis parameters.
How To Use
Workflow Setup and Usage Guide
This workflow is designed to automate competitor price monitoring. Follow these steps to set it up and run it:
-
Triggers (
daily-price-trigger,manual-price-trigger):- The
daily-price-triggeris configured to run every 24 hours. You can adjust thehoursIntervalin the schedule settings if needed. - The
manual-price-trigger(webhook) can be used to initiate a price check on demand. You'll need to expose this webhook URL if you intend to use it.
- The
-
Scraping (
amazon-scraper,bestbuy-scraper,target-scraper):- These nodes are set up to fetch raw HTML content from product search pages of Amazon, Best Buy, and Target.
- Each scraper includes a
User-Agentheader to mimic a web browser, which can help bypass basic anti-scraping measures. Ensure the URLs are accurate for the products or categories you wish to track.
-
AI Data Extraction (
scrapegraph-ai-prices):- This node uses ScrapeGraphAI to parse the scraped HTML and extract structured data based on the provided
userPromptandwebsiteUrl. - The
userPromptdefines the schema for the extracted data. Ensure this schema matches what you expect from the scraping process and what theprice-analysis-codenode requires. - The
websiteUrlis dynamically set using={{ $json.url || 'https://www.amazon.com/s?k=wireless+headphones' }}which attempts to use the URL from the previous node if available, otherwise defaults to a specific Amazon search. - You will need to configure your ScrapeGraphAI API credentials in the
credentialssection of this node.
- This node uses ScrapeGraphAI to parse the scraped HTML and extract structured data based on the provided
-
Price Analysis (
price-analysis-code):- This JavaScript code node performs the core analysis on the data extracted by ScrapeGraphAI.
- It processes data from all scraping sources, identifies target brands, and analyzes pricing insights like discounts, value scores, and price changes.
- It generates
alert_worthyflags,change_significance, and other metadata used for downstream actions. - Key configurations like
trackingConfig.targetBrandsandtrackingConfig.priceThresholdscan be adjusted within thejsCodeto fine-tune the analysis.
-
Price Alert Filtering (
price-alert-filter):- This
IFnode acts as a gatekeeper for alerts. It checks ifalert_worthyis true, if thediscount_percentageis above 20, or ifchange_significanceis 'high'. - If any of these conditions are met (due to the
anycombine operation), the data passes through to the alert node.
- This
-
Slack Alerting (
slack-price-alert):- If the
price-alert-filterallows the data to pass, this node constructs a formatted message and sends it to a specified Slack channel. - The message includes product details, pricing information, discounts, source, ratings, and competitive intelligence.
- You need to configure your Slack API credentials and select the appropriate
channelIdin the node's parameters.
- If the
-
Google Sheets Storage (
google-sheets-storage):- This node appends or updates the analyzed product data into a Google Sheet.
- It uses
autoMapInputDataandmatchingColumnsto update existing records based onproduct_idor add new ones. - Ensure you have set up your Google Sheets credentials (service account is recommended) and provided the correct
documentIdandsheetName.
General Workflow Management:
- Activation: The workflow is currently set to
active: false. You will need to toggle this totruein the workflow settings to enable execution. - Credentials: Remember to configure all necessary credentials (ScrapeGraphAI, Google Sheets, Slack) within the n8n interface for the nodes that require them.
- Testing: It's recommended to run the workflow manually or trigger it via the webhook for initial testing to verify each node's output and connections.
Apps Used
Workflow JSON
{
"id": "9eb63de7-8ec8-4183-bd4c-df80f9248335",
"name": "Competitor Price Monitoring with AI Scraping and Analysis",
"nodes": 24,
"category": "E-commerce",
"status": "active",
"version": "1.0.0"
}Note: This is a sample preview. The full workflow JSON contains node configurations, credentials placeholders, and execution logic.
Get This Workflow
ID: 9eb63de7-8ec8...
About the Author
Free n8n Workflows Official
System Admin
The official repository for verified enterprise-grade workflows.
Statistics
Related Workflows
Discover more workflows you might like
Generate Structured Metadata from Shopify Data
Fetches all data from Shopify and prepares it for structured metadata generation.
Shopify Order Creation Metadata Generation
Generates structured metadata for Shopify order creation events, supporting both English and Chinese.
Shopify Fulfillment Automation
Automates Shopify order fulfillment by fetching unfulfilled orders and marking them as fulfilled.