Google Drive Loader to Vector DB
detail.loadingPreview
Loads files from Google Drive into a Vector Database using OpenAI embeddings.
🚀Ready to Deploy This Workflow?
About This Workflow
Overview
This workflow automates the process of loading files from a specified Google Drive folder into a PostgreSQL database with PGVector extension. It downloads files, extracts text content (supporting PDF, plain text, and JSON), splits the text into chunks, generates embeddings using OpenAI's models, and stores these embeddings along with the original text in the PGVector store. Files are then moved to a 'vectorized' folder in Google Drive upon successful processing.
Key Features
- Connects to Google Drive to list and download files.
- Supports processing of PDF, plain text, and JSON file types.
- Implements recursive character text splitting for chunking content.
- Utilizes OpenAI's
text-embedding-3-smallmodel for generating vector embeddings. - Stores text chunks and their corresponding embeddings in a PostgreSQL PGVector database.
- Moves processed files to a designated 'vectorized' folder in Google Drive.
- Includes error handling for workflow execution failures.
- Scheduled to run daily at 3 AM UTC.
How To Use
- Import the workflow into your n8n instance.
- Configure Credentials:
- Google Drive OAuth2 API: Connect your Google Drive account.
- OpenAI API: Provide your OpenAI API key.
- Postgres Account: Set up your PostgreSQL connection details, ensuring the PGVector extension is enabled.
- Set Folder IDs: Update the
Search Foldernode with the ID of the Google Drive folder containing the files you want to process. Update theMove Filenode with the ID of the target 'vectorized' folder. - Adjust OpenAI Model (Optional): Modify the
Embeddings OpenAInode if you wish to use a different OpenAI embedding model. - Configure Schedule: The workflow is set to run daily at 3 AM UTC via the
Schedule Triggernode. Adjust this schedule as needed. - Activate the workflow.
Apps Used
Workflow JSON
{
"id": "9ea3d578-6acc-4f3f-b019-3870984481b0",
"name": "Google Drive Loader to Vector DB",
"nodes": 0,
"category": "Data Integration",
"status": "active",
"version": "1.0.0"
}Note: This is a sample preview. The full workflow JSON contains node configurations, credentials placeholders, and execution logic.
Get This Workflow
ID: 9ea3d578-6acc...
About the Author
Crypto_Watcher
Web3 Developer
Automated trading bots and blockchain monitoring workflows.
Statistics
Verification Info
Related Workflows
Discover more workflows you might like
Store Notion's Pages as Vector Documents into Supabase with OpenAI
Automates storing Notion pages as vector documents in Supabase using OpenAI embeddings.
Store Phantombuster Output in Airtable
Fetches data from Phantombuster and stores it in Airtable.
Read Spreadsheet and Insert into PostgreSQL
Reads data from a local spreadsheet file and inserts it into a PostgreSQL table.
Community Contributed: XML to Google Sheets
This workflow downloads an XML file, parses its content, and writes the data to a Google Sheet.
Community Contributed: Import CSV to PostgreSQL
Automate the import of CSV files into a PostgreSQL database.
Customer Datastore API Integration
Integrates with the Customer Datastore API to retrieve and process customer data.