Automate GitHub Project Discovery from Hacker News for Social Media
detail.loadingPreview
Tired of manually sifting through Hacker News for the latest GitHub gems to share? This n8n workflow automates the discovery, extraction, and preparation of trending GitHub projects from Hacker News, ensuring you never miss an important update for your social channels. It intelligently filters new projects and prepares their content for your AI agent.
About This Workflow
This powerful n8n workflow acts as an intelligent agent, constantly monitoring Hacker News for trending GitHub projects. It precisely scrapes the Hacker News homepage, then extracts crucial metadata like titles, URLs, authors, and scores specifically for GitHub-related posts. To prevent redundancy, the workflow deduplicates newly found items against an existing record (e.g., in Airtable), ensuring only fresh content is processed. Each new GitHub project's page is then visited, its content converted into a clean Markdown format, and prepared for further AI-driven analysis or direct social media sharing, while tracking all activities in Airtable.
Key Features
- Automated Hacker News Scraping: Regularly scrapes
news.ycombinator.comto find the latest trending tech posts. - Intelligent GitHub Project Extraction: Identifies and extracts detailed metadata (title, URL, author, score, etc.) specifically for GitHub projects.
- Deduplication & New Item Filtering: Prevents reprocessing of already handled projects using a robust filtering mechanism linked to a persistent data store.
- GitHub Page Content Retrieval: Automatically visits the identified GitHub project URLs to fetch their full content.
- HTML to Markdown Conversion: Transforms raw GitHub page HTML into a clean, AI-ready Markdown format, ideal for summarization.
- Error Handling & Tracking: Filters out items that failed processing and integrates with Airtable for persistent activity tracking.
How To Use
- Configure 'Crawl HN Home': This node is pre-configured to fetch content from
https://news.ycombinator.com/. No changes are typically needed. - Ensure 'Extract Meta' Works: The Python code in this node intelligently parses the Hacker News HTML to identify GitHub links and extract their metadata. It also handles necessary package installations (
beautifulsoup4,simplejson). - Connect to your Tracking Database (Airtable): The 'Filter Unposted Items' and 'Update X Status' nodes are designed to work with an Airtable base (or similar database) for tracking.
- Modify the 'Filter Unposted Items' node's logic if your tracking database is different, ensuring it correctly identifies already processed 'Post' IDs.
- Configure 'Update X Status' with your Airtable API credentials, Base ID, and Table ID. This node marks items as 'Done' after successful processing (implying a 'Create Item' node exists earlier in the full workflow, even if not shown in this snippet).
- Process GitHub Page Content: 'Visit GH Page' dynamically fetches the content of the identified GitHub URLs, and 'Convert HTML To Markdown' transforms it into a clean format suitable for further analysis.
- Integrate AI & Social Media Nodes (Implied): After 'Filter Errored', you would typically add your desired AI nodes (e.g., OpenAI, Cohere) for summarization and then social media nodes (e.g., Telegram, Twitter, Slack) to publish the curated content. Remember to place an initial Airtable 'Create Item' node before these social media steps to generate the 'id' required for tracking.
Apps Used
Workflow JSON
{
"id": "11d3e088-af0d-4389-b12c-eac0debfc978",
"name": "Automate GitHub Project Discovery from Hacker News for Social Media",
"nodes": 17,
"category": "DevOps",
"status": "active",
"version": "1.0.0"
}Note: This is a sample preview. The full workflow JSON contains node configurations, credentials placeholders, and execution logic.
Get This Workflow
ID: 11d3e088-af0d...
About the Author
Crypto_Watcher
Web3 Developer
Automated trading bots and blockchain monitoring workflows.
Statistics
Related Workflows
Discover more workflows you might like
Automated PR Merged QA Notifications
Streamline your QA process with this automated workflow that notifies your team upon successful Pull Request merges. Leverage AI and vector stores to enrich notifications and ensure seamless integration into your development pipeline.
Automate Qualys Report Generation and Retrieval
Streamline your Qualys security reporting by automating the generation and retrieval of reports. This workflow ensures timely access to crucial security data without manual intervention.
Robust Concurrency Control for n8n Workflows with Redis
Prevent simultaneous execution of critical n8n workflows or tasks using a centralized, Redis-backed locking mechanism. This reusable utility workflow ensures data integrity and resource management by allowing other workflows to acquire, check, and release locks.