AI Agents with Web Scraping and API Interaction
detail.loadingPreview
Workflow demonstrating AI agents capable of scraping webpages and interacting with APIs to fetch data.
About This Workflow
This workflow showcases the power of n8n's Langchain integration to build sophisticated AI agents. It demonstrates two key capabilities:
- Web Scraping: An AI agent can be instructed to scrape content from a specified URL, extracting information in a structured format.
- API Interaction: An AI agent can call external APIs to fetch dynamic data, such as suggesting activities based on user input.
This workflow is a remake of existing n8n workflows, focusing on replacing older methods like 'Execute Workflow Tool' and 'Subworkflow' with the more streamlined Langchain agent nodes and HTTP request tools.
Key Features
- AI Agent for Web Scraping: Utilizes the
OpenAI Chat Modeland aWebscraper Tool(Firecrawl) to enable AI agents to fetch and process webpage content. - AI Agent for API Interaction: Employs the
OpenAI Chat Modeland anActivity Tool(Bored API) for agents to query external APIs and retrieve relevant data. - Streamlined Node Usage: Replaces older workflow execution methods with direct integration of Langchain agent and HTTP request nodes.
- Dynamic Input Handling: Uses
Setnodes to define user prompts for the AI agents.
How To Use
- Trigger: The workflow starts with a
When clicking ‘Test workflow’trigger. - AI Agent for Web Scraping:
- A
Set ChatInputnode defines the prompt for the AI agent, e.g., requesting the latest 10 issues from a GitHub repository. - The
AI Agentnode processes the prompt and, if it determines a web scraping action is needed, it will use theOpenAI Chat Modeland theWebscraper Tool. - The
Webscraper Toolnode is configured to use the Firecrawl API to scrape the URL provided in the prompt.
- A
- AI Agent for API Interaction:
- A second
Set ChatInput1node defines a different prompt, e.g., asking for a suggestion of something to do. - The
AI Agent1node processes this prompt. It uses theOpenAI Chat Modeland theActivity Tool. - The
Activity Toolnode is configured to call the Bored API, passing parameters derived from the AI's understanding of the prompt.
- A second
Apps Used
Workflow JSON
{
"id": "b3d78cb5-3e69-4862-a9bf-dc83075b010e",
"name": "AI Agents with Web Scraping and API Interaction",
"nodes": 5,
"category": "AI",
"status": "active",
"version": "1.0.0"
}Note: This is a sample preview. The full workflow JSON contains node configurations, credentials placeholders, and execution logic.
Get This Workflow
ID: b3d78cb5-3e69...
About the Author
DevOps_Master_X
Infrastructure Expert
Specializing in CI/CD pipelines, Docker, and Kubernetes automations.
Statistics
Related Workflows
Discover more workflows you might like
Build an AI Documentation Expert Chatbot with Gemini RAG
This n8n workflow automates the creation of an AI-powered expert chatbot capable of answering questions based on your documentation. It intelligently ingests, cleans, and processes your knowledge base, preparing it for a Retrieval Augmented Generation (RAG) pipeline with Google Gemini.
Generate Multilingual Image Captions and Overlay
This workflow generates captions for images using Google Gemini and overlays them onto the image, with support for multilingual output.
LangChain Workflow Retriever Example
Demonstrates using LangChain's Retriever QA Chain to query data retrieved from another workflow.