Hacker News "Who is Hiring?" Scraper
detail.loadingPreview
Scrapes monthly Hacker News 'Who is Hiring?' posts to extract and structure job data.
About This Workflow
This workflow automates the process of gathering job postings from the Hacker News "Ask HN: Who is hiring?" threads. It leverages the Algolia search for Hacker News to efficiently find relevant posts, then uses the official Hacker News API to retrieve the full post content and associated comments (jobs). The extracted text is cleaned and then processed by an OpenAI model to structure the job information into a predefined schema. Finally, the structured data is written to Airtable.
Key Features
- Fetches 'Ask HN: Who is hiring?' posts using Algolia.
- Retrieves full post details and comments via the Hacker News API.
- Cleans and preprocesses the raw text data.
- Utilizes OpenAI to extract structured job information (company, title, location, type, salary, description, apply URL, company URL).
- Stores the structured job data in Airtable.
How To Use
- Set up Credentials: Ensure you have valid OpenAI API credentials and an Airtable Personal Access Token configured in n8n.
- Import cURL (if needed): The 'Search for Who is hiring posts' node currently uses direct API calls. If the Algolia API changes or requires specific authentication beyond headers, you might need to re-import the cURL from your browser's network tab as per the sticky note.
- Configure Airtable: Make sure your Airtable base (
HN Who is hiring?) and table (Table 1) are set up correctly with the fields defined in the 'Write results to airtable' node (title, company, location, type, salary, description, apply_url, company_url). - Run the workflow: Trigger the workflow by clicking 'Test workflow' or by setting up an appropriate trigger (e.g., a scheduled trigger).
- Optional Testing: The 'Limit for testing (optional)' node is set to 5 items. Remove or adjust this node for full scraping.
Apps Used
Workflow JSON
{
"id": "4843b886-1af6-4d42-9b09-08472d07f254",
"name": "Hacker News "Who is Hiring?" Scraper",
"nodes": 22,
"category": "Data Scraping",
"status": "active",
"version": "1.0.0"
}Note: This is a sample preview. The full workflow JSON contains node configurations, credentials placeholders, and execution logic.
Get This Workflow
ID: 4843b886-1af6...
About the Author
Crypto_Watcher
Web3 Developer
Automated trading bots and blockchain monitoring workflows.
Statistics
Related Workflows
Discover more workflows you might like
Automate Local Business Outreach with AI-Powered Yelp Scraper
This workflow automates the process of scraping local business details from Yelp using AI, then leverages that data to send personalized partnership proposals via Gmail. It's perfect for sales and marketing teams looking to streamline lead generation and outreach campaigns.
Automate Getty Images Editorial Search & CMS Integration
This n8n workflow automates searching for editorial images on Getty Images, extracts key details and embed codes, and prepares them for seamless integration into your Content Management System (CMS), streamlining your content creation process.
Universal CSV to JSON API Converter
Effortlessly transform CSV data into structured JSON with this versatile n8n workflow. Integrate it into any application as a custom API endpoint, supporting various input methods including file uploads and raw text.