ghassan/n8n-workflows
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
n8n-workflows/workflows/0668_Wait_Splitout_Create_Webhook.json
console-1 6de9bd2132 🎯 Complete Repository Transformation: Professional N8N Workflow Organization 
## 🚀 Major Achievements

### ✅ Comprehensive Workflow Standardization (2,053 files)
- **RENAMED ALL WORKFLOWS** from chaotic naming to professional 0001-2053 format
- **Eliminated chaos**: Removed UUIDs, emojis (🔐, #️⃣, ↔️), inconsistent patterns
- **Intelligent analysis**: Content-based categorization by services, triggers, complexity
- **Perfect naming convention**: [NNNN]_[Service1]_[Service2]_[Purpose]_[Trigger].json
- **100% success rate**: Zero data loss with automatic backup system

### ⚡ Revolutionary Documentation System
- **Replaced 71MB static HTML** with lightning-fast <100KB dynamic interface
- **700x smaller file size** with 10x faster load times (<1 second vs 10+ seconds)
- **Full-featured web interface**: Clickable cards, detailed modals, search & filter
- **Professional UX**: Copy buttons, download functionality, responsive design
- **Database-backed**: SQLite with FTS5 search for instant results

### 🔧 Enhanced Web Interface Features
- **Clickable workflow cards** → Opens detailed workflow information
- **Copy functionality** → JSON and diagram content with visual feedback
- **Download buttons** → Direct workflow JSON file downloads
- **Independent view toggles** → View JSON and diagrams simultaneously
- **Mobile responsive** → Works perfectly on all device sizes
- **Dark/light themes** → System preference detection with manual toggle

## 📊 Transformation Statistics

### Workflow Naming Improvements
- **Before**: 58% meaningful names → **After**: 100% professional standard
- **Fixed**: 2,053 workflow files with intelligent content analysis
- **Format**: Uniform 0001-2053_Service_Purpose_Trigger.json convention
- **Quality**: Eliminated all UUIDs, emojis, and inconsistent patterns

### Performance Revolution
 < /dev/null |  Metric | Old System | New System | Improvement |
|--------|------------|------------|-------------|
| **File Size** | 71MB HTML | <100KB | 700x smaller |
| **Load Time** | 10+ seconds | <1 second | 10x faster |
| **Search** | Client-side | FTS5 server | Instant results |
| **Mobile** | Poor | Excellent | Fully responsive |

## 🛠 Technical Implementation

### New Tools Created
- **comprehensive_workflow_renamer.py**: Intelligent batch renaming with backup system
- **Enhanced static/index.html**: Modern single-file web application
- **Updated .gitignore**: Proper exclusions for development artifacts

### Smart Renaming System
- **Content analysis**: Extracts services, triggers, and purpose from workflow JSON
- **Backup safety**: Automatic backup before any modifications
- **Change detection**: File hash-based system prevents unnecessary reprocessing
- **Audit trail**: Comprehensive logging of all rename operations

### Professional Web Interface
- **Single-page app**: Complete functionality in one optimized HTML file
- **Copy-to-clipboard**: Modern async clipboard API with fallback support
- **Modal system**: Professional workflow detail views with keyboard shortcuts
- **State management**: Clean separation of concerns with proper data flow

## 📋 Repository Organization

### File Structure Improvements
```
├── workflows/                    # 2,053 professionally named workflow files
│   ├── 0001_Telegram_Schedule_Automation_Scheduled.json
│   ├── 0002_Manual_Totp_Automation_Triggered.json
│   └── ... (0003-2053 in perfect sequence)
├── static/index.html            # Enhanced web interface with full functionality
├── comprehensive_workflow_renamer.py  # Professional renaming tool
├── api_server.py               # FastAPI backend (unchanged)
├── workflow_db.py             # Database layer (unchanged)
└── .gitignore                 # Updated with proper exclusions
```

### Quality Assurance
- **Zero data loss**: All original workflows preserved in workflow_backups/
- **100% success rate**: All 2,053 files renamed without errors
- **Comprehensive testing**: Web interface tested with copy, download, and modal functions
- **Mobile compatibility**: Responsive design verified across device sizes

## 🔒 Safety Measures
- **Automatic backup**: Complete workflow_backups/ directory created before changes
- **Change tracking**: Detailed workflow_rename_log.json with full audit trail
- **Git-ignored artifacts**: Backup directories and temporary files properly excluded
- **Reversible process**: Original files preserved for rollback if needed

## 🎯 User Experience Improvements
- **Professional presentation**: Clean, consistent workflow naming throughout
- **Instant discovery**: Fast search and filter capabilities
- **Copy functionality**: Easy access to workflow JSON and diagram code
- **Download system**: One-click workflow file downloads
- **Responsive design**: Perfect mobile and desktop experience

This transformation establishes a professional-grade n8n workflow repository with:
- Perfect organizational standards
- Lightning-fast documentation system
- Modern web interface with full functionality
- Sustainable maintenance practices

🎉 Repository transformation: COMPLETE!

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-21 01:18:37 +02:00

3777 lines
107 KiB
JSON
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

{
"nodes": [
{
"id": "2498bb93-176f-458c-acee-f541859df770",
"name": "When clicking ‘Test workflow’",
"type": "n8n-nodes-base.manualTrigger",
"position": [
2460,
2820
],
"parameters": {},
"typeVersion": 1
},
{
"id": "c08bcf84-9336-44f9-b452-0c9469f18f48",
"name": "Web Search For API Schema",
"type": "n8n-nodes-base.httpRequest",
"onError": "continueRegularOutput",
"position": [
3100,
3820
],
"parameters": {
"url": "https://api.apify.com/v2/acts/serping~fast-google-search-results-scraper/run-sync-get-dataset-items",
"method": "POST",
"options": {},
"sendBody": true,
"authentication": "genericCredentialType",
"bodyParameters": {
"parameters": [
{
"name": "searchTerms",
"value": "={{\n[\n `site:${$json.data.url.replace(/^http[s]:\\/\\//, '').replace(/\\/$/, '').replace('www.', '')} \"${$json.data.service}\" api developer (intext:reference OR intext:resource) (-inurl:support OR -inurl:help) (inurl:api OR intitle:api) -filetype:pdf`\n]\n}}"
},
{
"name": "resultsPerPage",
"value": "={{ 10 }}"
}
]
},
"genericAuthType": "httpHeaderAuth"
},
"typeVersion": 4.2
},
{
"id": "d5b19e3a-acd0-4b06-8d77-42de1f797dba",
"name": "Scrape Webpage Contents",
"type": "n8n-nodes-base.httpRequest",
"position": [
3940,
3720
],
"parameters": {
"url": "https://api.apify.com/v2/acts/apify~web-scraper/run-sync-get-dataset-items",
"options": {
"batching": {
"batch": {
"batchSize": 2,
"batchInterval": 30000
}
}
},
"jsonBody": "={\n \"startUrls\": [\n {\n \"url\": \"{{ $json.source.link }}\",\n \"method\": \"GET\"\n }\n ],\n \"breakpointLocation\": \"NONE\",\n \"browserLog\": false,\n \"closeCookieModals\": false,\n \"debugLog\": false,\n \"downloadCss\": false,\n \"downloadMedia\": false,\n \"excludes\": [\n {\n \"glob\": \"/**/*.{png,jpg,jpeg,pdf}\"\n }\n ],\n \"headless\": true,\n \"ignoreCorsAndCsp\": false,\n \"ignoreSslErrors\": false,\n \n \"injectJQuery\": true,\n \"keepUrlFragments\": false,\n \"linkSelector\": \"a[href]\",\n \"maxCrawlingDepth\": 1,\n \"maxPagesPerCrawl\": 1,\n \"maxRequestRetries\": 1,\n \"maxResultsPerCrawl\": 1,\n \"pageFunction\": \"// The function accepts a single argument: the \\\"context\\\" object.\\n// For a complete list of its properties and functions,\\n// see https://apify.com/apify/web-scraper#page-function \\nasync function pageFunction(context) {\\n\\n await new Promise(res => { setTimeout(res, 6000) });\\n // This statement works as a breakpoint when you're trying to debug your code. Works only with Run mode: DEVELOPMENT!\\n // debugger; \\n\\n // jQuery is handy for finding DOM elements and extracting data from them.\\n // To use it, make sure to enable the \\\"Inject jQuery\\\" option.\\n const $ = context.jQuery;\\n const title = $('title').first().text();\\n\\n // Clone the body to avoid modifying the original content\\n const bodyClone = $('body').clone();\\n bodyClone.find('iframe, img, script, style, object, embed, noscript, svg, video, audio').remove();\\n const body = bodyClone.html();\\n\\n // Return an object with the data extracted from the page.\\n // It will be stored to the resulting dataset.\\n return {\\n url: context.request.url,\\n title,\\n body\\n };\\n}\",\n \"postNavigationHooks\": \"// We need to return array of (possibly async) functions here.\\n// The functions accept a single argument: the \\\"crawlingContext\\\" object.\\n[\\n async (crawlingContext) => {\\n // ...\\n },\\n]\",\n \"preNavigationHooks\": \"// We need to return array of (possibly async) functions here.\\n// The functions accept two arguments: the \\\"crawlingContext\\\" object\\n// and \\\"gotoOptions\\\".\\n[\\n async (crawlingContext, gotoOptions) => {\\n // ...\\n },\\n]\\n\",\n \"proxyConfiguration\": {\n \"useApifyProxy\": true\n },\n \"runMode\": \"PRODUCTION\",\n \n \"useChrome\": false,\n \"waitUntil\": [\n \"domcontentloaded\"\n ],\n \"globs\": [],\n \"pseudoUrls\": [],\n \"proxyRotation\": \"RECOMMENDED\",\n \"maxConcurrency\": 50,\n \"pageLoadTimeoutSecs\": 60,\n \"pageFunctionTimeoutSecs\": 60,\n \"maxScrollHeightPixels\": 5000,\n \"customData\": {}\n}",
"sendBody": true,
"sendQuery": true,
"specifyBody": "json",
"authentication": "genericCredentialType",
"genericAuthType": "httpQueryAuth",
"queryParameters": {
"parameters": [
{
"name": "memory",
"value": "2048"
}
]
}
},
"typeVersion": 4.2
},
{
"id": "5853ba7e-4068-4792-be5c-b8cf81ee89cb",
"name": "Results to List",
"type": "n8n-nodes-base.splitOut",
"position": [
3460,
3720
],
"parameters": {
"options": {},
"fieldToSplitOut": "origin_search.results"
},
"typeVersion": 1
},
{
"id": "8ed2e8ec-b2e3-474b-b19d-f38b518f274b",
"name": "Recursive Character Text Splitter1",
"type": "@n8n/n8n-nodes-langchain.textSplitterRecursiveCharacterTextSplitter",
"position": [
5800,
4020
],
"parameters": {
"options": {},
"chunkSize": 4000
},
"typeVersion": 1
},
{
"id": "e2a8137b-7da3-4032-bca2-c14465356f02",
"name": "Content Chunking @ 50k Chars",
"type": "n8n-nodes-base.set",
"position": [
5380,
3740
],
"parameters": {
"options": {},
"assignments": {
"assignments": [
{
"id": "7753a4f4-3ec2-4c05-81df-3d5e8979a478",
"name": "=data",
"type": "array",
"value": "={{ new Array(Math.round($json.content.length / Math.min($json.content.length, 50000))).fill('').map((_,idx) => $json.content.substring(idx * 50000, idx * 50000 + 50000)) }}"
},
{
"id": "7973bcb4-f239-4619-85fc-c76e20386375",
"name": "service",
"type": "string",
"value": "={{ $json.service }}"
},
{
"id": "b46e44bc-ad01-4cf0-8b07-25eeb1fb5874",
"name": "url",
"type": "string",
"value": "={{ $json.url }}"
}
]
}
},
"typeVersion": 3.3
},
{
"id": "6ef5866a-d992-4472-9221-27efbec8e7be",
"name": "Split Out Chunks",
"type": "n8n-nodes-base.splitOut",
"position": [
5540,
3740
],
"parameters": {
"include": "allOtherFields",
"options": {},
"fieldToSplitOut": "data"
},
"typeVersion": 1
},
{
"id": "5e43b4d8-cebf-43ed-866d-0b4cb2997853",
"name": "Default Data Loader",
"type": "@n8n/n8n-nodes-langchain.documentDefaultDataLoader",
"position": [
5800,
3900
],
"parameters": {
"options": {
"metadata": {
"metadataValues": [
{
"name": "service",
"value": "={{ $json.service }}"
},
{
"name": "url",
"value": "={{ $json.url }}"
}
]
}
},
"jsonData": "={{ $json.data }}",
"jsonMode": "expressionData"
},
"typeVersion": 1
},
{
"id": "d4b34767-be50-44ee-b778-18842034c276",
"name": "Set Embedding Variables",
"type": "n8n-nodes-base.set",
"position": [
4980,
3580
],
"parameters": {
"options": {},
"assignments": {
"assignments": [
{
"id": "4008ae44-7998-4a6f-88c9-686f8b02e92b",
"name": "content",
"type": "string",
"value": "={{ $json.body }}"
},
{
"id": "f7381ac6-ef40-463c-ad2b-df2c31d3e828",
"name": "service",
"type": "string",
"value": "={{ $('EventRouter').first().json.data.service }}"
},
{
"id": "7eae99fd-75c7-4974-a128-641b8ada0cc2",
"name": "url",
"type": "string",
"value": "={{ $json.url }}"
}
]
}
},
"typeVersion": 3.4
},
{
"id": "109b6c3a-9b16-40cc-9186-5045df387b52",
"name": "Execute Workflow Trigger",
"type": "n8n-nodes-base.executeWorkflowTrigger",
"position": [
2420,
4200
],
"parameters": {},
"typeVersion": 1
},
{
"id": "31556ff2-6358-4bd4-8ec4-2797d993256e",
"name": "Execution Data",
"type": "n8n-nodes-base.executionData",
"position": [
2620,
4200
],
"parameters": {
"dataToSave": {
"values": [
{
"key": "eventType",
"value": "={{ $json.eventType }}"
},
{
"key": "executedById",
"value": "={{ $json.executedById }}"
},
{
"key": "service",
"value": "={{ $json.data.service }}"
}
]
}
},
"typeVersion": 1
},
{
"id": "b65b3d4d-f667-4f8f-a06f-847c3d7b83e0",
"name": "EventRouter",
"type": "n8n-nodes-base.switch",
"position": [
2800,
4200
],
"parameters": {
"rules": {
"values": [
{
"outputKey": "research",
"conditions": {
"options": {
"version": 2,
"leftValue": "",
"caseSensitive": true,
"typeValidation": "strict"
},
"combinator": "and",
"conditions": [
{
"operator": {
"type": "string",
"operation": "equals"
},
"leftValue": "={{ $json.eventType }}",
"rightValue": "research"
}
]
},
"renameOutput": true
},
{
"outputKey": "extraction",
"conditions": {
"options": {
"version": 2,
"leftValue": "",
"caseSensitive": true,
"typeValidation": "strict"
},
"combinator": "and",
"conditions": [
{
"id": "5418515e-ef6a-42e0-aeb9-8d0d35b898ca",
"operator": {
"name": "filter.operator.equals",
"type": "string",
"operation": "equals"
},
"leftValue": "={{ $json.eventType }}",
"rightValue": "extract"
}
]
},
"renameOutput": true
},
{
"outputKey": "generate",
"conditions": {
"options": {
"version": 2,
"leftValue": "",
"caseSensitive": true,
"typeValidation": "strict"
},
"combinator": "and",
"conditions": [
{
"id": "0135165e-d211-44f3-92a4-a91858a57d99",
"operator": {
"name": "filter.operator.equals",
"type": "string",
"operation": "equals"
},
"leftValue": "={{ $json.eventType }}",
"rightValue": "generate"
}
]
},
"renameOutput": true
}
]
},
"options": {}
},
"typeVersion": 3.2
},
{
"id": "541f7d9b-c8ff-44dc-8618-8550dbf0b951",
"name": "Google Gemini Chat Model",
"type": "@n8n/n8n-nodes-langchain.lmChatGoogleGemini",
"position": [
4460,
3740
],
"parameters": {
"options": {},
"modelName": "models/gemini-1.5-flash-latest"
},
"typeVersion": 1
},
{
"id": "617d6139-8417-4ecb-8f7c-558cd1c38ac3",
"name": "Successful Runs",
"type": "n8n-nodes-base.filter",
"position": [
4100,
3720
],
"parameters": {
"options": {},
"conditions": {
"options": {
"version": 2,
"leftValue": "",
"caseSensitive": true,
"typeValidation": "strict"
},
"combinator": "and",
"conditions": [
{
"id": "cac77cce-0a5c-469e-ba80-9fb026f04b18",
"operator": {
"type": "string",
"operation": "exists",
"singleValue": true
},
"leftValue": "={{ $json.body }}",
"rightValue": ""
}
]
}
},
"typeVersion": 2.2,
"alwaysOutputData": true
},
{
"id": "1115db69-b414-46cd-a9a1-565ae98cbd91",
"name": "For Each Document...",
"type": "n8n-nodes-base.splitInBatches",
"position": [
5180,
3580
],
"parameters": {
"options": {}
},
"typeVersion": 3
},
{
"id": "3f0e3764-2479-4d74-aca8-c3e830eac423",
"name": "Embeddings Google Gemini",
"type": "@n8n/n8n-nodes-langchain.embeddingsGoogleGemini",
"position": [
5680,
3900
],
"parameters": {
"modelName": "models/text-embedding-004"
},
"typeVersion": 1
},
{
"id": "87d42766-d1a2-406d-b01c-044fd2fc8910",
"name": "Has API Documentation?",
"type": "@n8n/n8n-nodes-langchain.textClassifier",
"position": [
4460,
3580
],
"parameters": {
"options": {
"fallback": "discard"
},
"inputText": "={{\n$json.body\n .replaceAll('\\n', '')\n .substring(0, 40000)\n}}",
"categories": {
"categories": [
{
"category": "contains_api_schema_documentation",
"description": "True if this document contains REST API schema documentation or definitions"
}
]
}
},
"typeVersion": 1,
"alwaysOutputData": true
},
{
"id": "55939b49-d91c-42a1-9770-48cbe4008c9a",
"name": "Store Document Embeddings",
"type": "@n8n/n8n-nodes-langchain.vectorStoreQdrant",
"position": [
5700,
3740
],
"parameters": {
"mode": "insert",
"options": {},
"qdrantCollection": {
"__rl": true,
"mode": "id",
"value": "={{ $('EventRouter').first().json.data.collection }}"
}
},
"typeVersion": 1
},
{
"id": "3e1da749-b8b9-42cb-818b-eabf4b114abb",
"name": "Embeddings Google Gemini1",
"type": "@n8n/n8n-nodes-langchain.embeddingsGoogleGemini",
"position": [
3700,
4520
],
"parameters": {
"modelName": "models/text-embedding-004"
},
"typeVersion": 1
},
{
"id": "be0906d4-351f-4b3b-9f32-8e5ee68083c5",
"name": "Google Gemini Chat Model1",
"type": "@n8n/n8n-nodes-langchain.lmChatGoogleGemini",
"position": [
4600,
4240
],
"parameters": {
"options": {},
"modelName": "models/gemini-1.5-pro-002"
},
"typeVersion": 1
},
{
"id": "886415d5-c888-4b97-9fb5-02e6a14df4cc",
"name": "Extract API Operations",
"type": "@n8n/n8n-nodes-langchain.informationExtractor",
"position": [
4600,
4100
],
"parameters": {
"text": "={{ $json.documents }}",
"options": {
"systemPromptTemplate": "=You have been given an extract of a webpage which should contain a list of web/REST api operations.\nStep 1. Extract all REST (eg. GET,POST,PUT,DELETE) API operation endpoints from the page content and generate appropriate labels for the resource, operation, description, method for each.\n* \"resource\" refers to the API group, for example: \"/v1/api/indicators/list\" and \"/v1/api/indicators/create\" will both have the resource name of \"indicators\". Use the following template \"<domain>\" eg. \"entities\", \"posts\", \"credentials\".\n* \"operation\" refers to the action performed, use the following template \"<verb> <entity>\" eg. \"List entities\", \"Create post\", \"Update credentials\"\n* only use one HTTP verb for \"method\"\n* \"description\" should be limited to one sentence.\n* Examples of API urls: \"/api/\", \"/api/v1/\", \"/v1/api\". API urls should not end with \"htm\" or html\".\n* Extract a maximum of 15 endpoints.\n* If the page content contains no api operations, return an empty array."
},
"schemaType": "manual",
"inputSchema": "{\n \"type\": \"array\",\n \"items\": {\n \"type\": \"object\",\n \"properties\": {\n \"resource\": { \"type\": \"string\" },\n \"operation\": { \"type\": \"string\" },\n \"description\": { \"type\": \"string\" },\n \"url\": { \"type\": \"string\" },\n \"method\": { \"type\": \"string\" },\n \"documentation_url\": { \"type\": \"string\" }\n }\n }\n}"
},
"typeVersion": 1
},
{
"id": "76470e34-7c1f-44ce-81e2-047dcca3fa32",
"name": "Search in Relevant Docs",
"type": "@n8n/n8n-nodes-langchain.vectorStoreQdrant",
"position": [
3700,
4380
],
"parameters": {
"mode": "load",
"topK": 5,
"prompt": "={{ $json.query }}",
"options": {
"searchFilterJson": "={{\n{\n \"must\": [\n {\n \"key\": \"metadata.service\",\n \"match\": {\n \"value\": $('EventRouter').first().json.data.service\n }\n }\n ]\n}\n}}"
},
"qdrantCollection": {
"__rl": true,
"mode": "id",
"value": "={{ $('EventRouter').first().json.data.collection }}"
}
},
"typeVersion": 1
},
{
"id": "49ca6a35-5b89-4ed5-bbab-250e09b4222f",
"name": "Wait",
"type": "n8n-nodes-base.wait",
"position": [
3780,
3160
],
"webhookId": "e9ad3ef0-7403-4e65-b0a4-4afdfb0cbc6d",
"parameters": {
"amount": 0
},
"typeVersion": 1.1
},
{
"id": "800cb05b-f5d1-47c8-869e-921915929f34",
"name": "Remove Dupes",
"type": "n8n-nodes-base.removeDuplicates",
"position": [
3780,
3720
],
"parameters": {
"compare": "selectedFields",
"options": {},
"fieldsToCompare": "source.link"
},
"typeVersion": 2
},
{
"id": "d8203c40-aa0b-44b9-8dfd-aea250c8d109",
"name": "Filter Results",
"type": "n8n-nodes-base.filter",
"position": [
3620,
3720
],
"parameters": {
"options": {},
"conditions": {
"options": {
"version": 2,
"leftValue": "",
"caseSensitive": true,
"typeValidation": "strict"
},
"combinator": "and",
"conditions": [
{
"id": "42872456-411b-4d86-a9dd-b907d001ea1c",
"operator": {
"name": "filter.operator.equals",
"type": "string",
"operation": "equals"
},
"leftValue": "={{ $json.type }}",
"rightValue": "normal"
}
]
}
},
"typeVersion": 2.2
},
{
"id": "5714dc09-fd67-4285-9434-ac97cd80dec1",
"name": "Research",
"type": "n8n-nodes-base.executeWorkflow",
"onError": "continueErrorOutput",
"position": [
3460,
2980
],
"parameters": {
"mode": "each",
"options": {
"waitForSubWorkflow": true
},
"workflowId": {
"__rl": true,
"mode": "id",
"value": "={{ $workflow.id }}"
}
},
"typeVersion": 1.1
},
{
"id": "2a2d3271-b0b6-4a1a-94e1-9b01399ba88f",
"name": "Has Results?",
"type": "n8n-nodes-base.if",
"position": [
3280,
3820
],
"parameters": {
"options": {},
"conditions": {
"options": {
"version": 2,
"leftValue": "",
"caseSensitive": true,
"typeValidation": "strict"
},
"combinator": "and",
"conditions": [
{
"id": "1223d607-45a8-44b1-b510-56fdbe013eba",
"operator": {
"type": "array",
"operation": "exists",
"singleValue": true
},
"leftValue": "={{ $jmespath($json, 'origin_search.results') }}",
"rightValue": ""
}
]
}
},
"typeVersion": 2.2
},
{
"id": "b953082c-2d37-4549-80a7-d60535b8580e",
"name": "Response Empty",
"type": "n8n-nodes-base.set",
"position": [
3460,
3900
],
"parameters": {
"options": {},
"assignments": {
"assignments": [
{
"id": "5bb23ce9-eb72-4868-9344-9e5d3952cc52",
"name": "response",
"type": "string",
"value": "no web results"
}
]
}
},
"executeOnce": true,
"typeVersion": 3.4
},
{
"id": "41e9c328-d145-4b71-93bb-e2c448a14be0",
"name": "Response OK",
"type": "n8n-nodes-base.set",
"position": [
5380,
3580
],
"parameters": {
"options": {},
"assignments": {
"assignments": [
{
"id": "79598789-4468-4565-828f-fedc48be15c3",
"name": "response",
"type": "string",
"value": "ok"
}
]
}
},
"executeOnce": true,
"typeVersion": 3.4
},
{
"id": "5d0a7556-def9-4c70-8828-40b4d22904de",
"name": "Combine Docs",
"type": "n8n-nodes-base.aggregate",
"position": [
4020,
4380
],
"parameters": {
"options": {},
"aggregate": "aggregateAllItemData"
},
"typeVersion": 1
},
{
"id": "39bd90b4-e0f5-49b0-b4a7-55a3ae8eccb2",
"name": "Template to List",
"type": "n8n-nodes-base.splitOut",
"position": [
3280,
4200
],
"parameters": {
"options": {
"destinationFieldName": "query"
},
"fieldToSplitOut": "queries"
},
"typeVersion": 1
},
{
"id": "51a1da10-5ad0-4bac-9bec-55b5af3da702",
"name": "Query Templates",
"type": "n8n-nodes-base.set",
"position": [
3100,
4200
],
"parameters": {
"options": {},
"assignments": {
"assignments": [
{
"id": "e2a02550-8f53-4f8d-bb83-68ee3606736e",
"name": "queries",
"type": "array",
"value": "=[\n\"What are the core functionalities, essential features, or primary use cases of {{ $json.data.service }}?\",\n\"Is there an API overview or API categories for {{ $json.data.service }}? What main APIs are listed or mentioned?\",\n\"What industry does {{ $json.data.service }} operate in? What is the most important of the services in the industry? Return the important service as the function.\",\n\"What REST apis (GET, POST, DELETE, PATCH) and/or operations can you identify for {{ $json.data.service }}?\",\n\"Does {{ $json.data.service }} have any CURL examples? If you can, identify one such example and explain what it does.\"\n]"
}
]
}
},
"executeOnce": true,
"typeVersion": 3.3
},
{
"id": "414091b7-114b-4fc3-9755-2f87cfef239e",
"name": "Google Gemini Chat Model2",
"type": "@n8n/n8n-nodes-langchain.lmChatGoogleGemini",
"position": [
3700,
4240
],
"parameters": {
"options": {},
"modelName": "models/gemini-1.5-pro-002"
},
"typeVersion": 1
},
{
"id": "1f0f45ff-3bc9-4786-92e1-319244d020c0",
"name": "For Each Template...",
"type": "n8n-nodes-base.splitInBatches",
"position": [
3460,
4200
],
"parameters": {
"options": {}
},
"typeVersion": 3
},
{
"id": "2e577e62-7f89-4c99-b540-ce8c44f19a55",
"name": "Query & Docs",
"type": "n8n-nodes-base.set",
"position": [
4180,
4380
],
"parameters": {
"options": {},
"assignments": {
"assignments": [
{
"id": "fdaea3de-3c9a-4f26-b7dc-769e534006a9",
"name": "query",
"type": "string",
"value": "={{ $('For Each Template...').item.json.query }}"
},
{
"id": "88198374-d2f9-4ae7-b262-d3b2e630e0ac",
"name": "documents",
"type": "string",
"value": "={{ $json.data.map(item => item.document.pageContent.replaceAll('\\n', ' ')).join('\\n---\\n') }}"
}
]
}
},
"typeVersion": 3.4
},
{
"id": "548d51fd-9740-4b4c-9c81-db62d2b31053",
"name": "Identify Service Products",
"type": "@n8n/n8n-nodes-langchain.informationExtractor",
"position": [
3700,
4100
],
"parameters": {
"text": "={{ $json.query }}",
"options": {
"systemPromptTemplate": "=Use the following document to answer the user's question:\n```\n{{ $json.documents.replace(/[\\{\\}]/g, '') }}\n```"
},
"attributes": {
"attributes": [
{
"name": "product_or_solution",
"required": true,
"description": "A product or solution offered by the service"
},
{
"name": "description",
"required": true,
"description": "description of what the product or solution of the service does"
}
]
}
},
"typeVersion": 1
},
{
"id": "aa7041e9-4ac8-47f9-b98e-cf57873922bb",
"name": "Extract API Templates",
"type": "n8n-nodes-base.set",
"position": [
4180,
4200
],
"parameters": {
"options": {},
"assignments": {
"assignments": [
{
"id": "e2a02550-8f53-4f8d-bb83-68ee3606736e",
"name": "query",
"type": "string",
"value": "=I'm interested in {{ $json.output.product_or_solution }} apis which {{ $json.output.description }} What are the GET, POST, PATCH and/or DELETE endpoints of the {{ $json.output.product_or_solution }} api?"
}
]
}
},
"typeVersion": 3.3
},
{
"id": "e2b371c1-52af-4e57-877c-6933ba84e2d5",
"name": "Embeddings Google Gemini2",
"type": "@n8n/n8n-nodes-langchain.embeddingsGoogleGemini",
"position": [
4600,
4520
],
"parameters": {
"modelName": "models/text-embedding-004"
},
"typeVersion": 1
},
{
"id": "d808c591-34e2-455f-96b1-3689d950608d",
"name": "Search in Relevant Docs1",
"type": "@n8n/n8n-nodes-langchain.vectorStoreQdrant",
"position": [
4600,
4380
],
"parameters": {
"mode": "load",
"topK": 20,
"prompt": "={{ $json.query }}",
"options": {
"searchFilterJson": "={{\n{\n \"must\": [\n {\n \"key\": \"metadata.service\",\n \"match\": {\n \"value\": $('EventRouter').first().json.data.service\n }\n }\n ]\n}\n}}"
},
"qdrantCollection": {
"__rl": true,
"mode": "id",
"value": "={{ $('EventRouter').first().json.data.collection }}"
}
},
"typeVersion": 1
},
{
"id": "222bde31-57fa-46c4-a23b-ec2d1b3c7e2d",
"name": "Combine Docs1",
"type": "n8n-nodes-base.aggregate",
"position": [
4920,
4380
],
"parameters": {
"options": {},
"aggregate": "aggregateAllItemData"
},
"typeVersion": 1
},
{
"id": "57677d83-a79a-4b71-9977-ee2324f5d593",
"name": "Query & Docs1",
"type": "n8n-nodes-base.set",
"position": [
5080,
4380
],
"parameters": {
"options": {},
"assignments": {
"assignments": [
{
"id": "fdaea3de-3c9a-4f26-b7dc-769e534006a9",
"name": "query",
"type": "string",
"value": "={{ $('For Each Template...1').item.json.query }}"
},
{
"id": "88198374-d2f9-4ae7-b262-d3b2e630e0ac",
"name": "documents",
"type": "string",
"value": "={{\n$json.data\n .map(item =>\n`url: ${item.document.metadata.url}\ncontent: ${item.document.pageContent}`\n )\n .join('\\n---\\n')\n .replaceAll('\\n\\n', '\\n')\n}}"
}
]
}
},
"typeVersion": 3.4
},
{
"id": "124c3b07-3210-4190-8865-e18017fc9e6c",
"name": "For Each Template...1",
"type": "n8n-nodes-base.splitInBatches",
"position": [
4380,
4200
],
"parameters": {
"options": {}
},
"typeVersion": 3
},
{
"id": "8ea4a5da-c471-4201-a08b-9c18ed08ddc7",
"name": "Merge Lists",
"type": "n8n-nodes-base.code",
"position": [
4920,
4200
],
"parameters": {
"jsCode": "return $input.all().flatMap(input => input.json.output) || [];"
},
"typeVersion": 2,
"alwaysOutputData": true
},
{
"id": "0e38cd3c-c843-4f6d-bdb6-901a8c12acbf",
"name": "Remove Duplicates",
"type": "n8n-nodes-base.removeDuplicates",
"position": [
5280,
4200
],
"parameters": {
"compare": "selectedFields",
"options": {},
"fieldsToCompare": "method, url"
},
"typeVersion": 2
},
{
"id": "8f127f7a-e351-4b30-82dd-1f785be4a765",
"name": "Append Row",
"type": "n8n-nodes-base.googleSheets",
"position": [
5440,
4200
],
"parameters": {
"columns": {
"value": {
"url": "={{ $json.url }}",
"method": "={{ $json.method }}",
"service": "={{ $('EventRouter').first().json.data.service }}",
"resource": "={{ $json.resource }}",
"operation": "={{ $json.operation }}",
"description": "={{ $json.description }}",
"documentation_url": "={{ $json.documentation_url }}"
},
"schema": [
{
"id": "service",
"type": "string",
"display": true,
"required": false,
"displayName": "service",
"defaultMatch": false,
"canBeUsedToMatch": true
},
{
"id": "resource",
"type": "string",
"display": true,
"required": false,
"displayName": "resource",
"defaultMatch": false,
"canBeUsedToMatch": true
},
{
"id": "operation",
"type": "string",
"display": true,
"required": false,
"displayName": "operation",
"defaultMatch": false,
"canBeUsedToMatch": true
},
{
"id": "description",
"type": "string",
"display": true,
"required": false,
"displayName": "description",
"defaultMatch": false,
"canBeUsedToMatch": true
},
{
"id": "url",
"type": "string",
"display": true,
"required": false,
"displayName": "url",
"defaultMatch": false,
"canBeUsedToMatch": true
},
{
"id": "method",
"type": "string",
"display": true,
"required": false,
"displayName": "method",
"defaultMatch": false,
"canBeUsedToMatch": true
},
{
"id": "documentation_url",
"type": "string",
"display": true,
"required": false,
"displayName": "documentation_url",
"defaultMatch": false,
"canBeUsedToMatch": true
}
],
"mappingMode": "defineBelow",
"matchingColumns": []
},
"options": {
"useAppend": true
},
"operation": "append",
"sheetName": {
"__rl": true,
"mode": "list",
"value": 1042334767,
"cachedResultUrl": "https://docs.google.com/spreadsheets/d/1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0/edit#gid=1042334767",
"cachedResultName": "Extracted API Operations"
},
"documentId": {
"__rl": true,
"mode": "list",
"value": "1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0",
"cachedResultUrl": "https://docs.google.com/spreadsheets/d/1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0/edit?usp=drivesdk",
"cachedResultName": "API Schema Crawler & Extractor"
}
},
"typeVersion": 4.5
},
{
"id": "d9f490e2-320e-4dc1-af8f-ac7f6a61568d",
"name": "Response OK1",
"type": "n8n-nodes-base.set",
"position": [
5600,
4200
],
"parameters": {
"options": {},
"assignments": {
"assignments": [
{
"id": "79598789-4468-4565-828f-fedc48be15c3",
"name": "response",
"type": "string",
"value": "ok"
}
]
}
},
"executeOnce": true,
"typeVersion": 3.4
},
{
"id": "7780b6ee-0fde-40bb-aef6-e67b883645e1",
"name": "Has Operations?",
"type": "n8n-nodes-base.if",
"position": [
5080,
4200
],
"parameters": {
"options": {},
"conditions": {
"options": {
"version": 2,
"leftValue": "",
"caseSensitive": true,
"typeValidation": "strict"
},
"combinator": "and",
"conditions": [
{
"id": "a95420a7-6265-4ea3-9c01-82c2d7aeb4f8",
"operator": {
"type": "object",
"operation": "notEmpty",
"singleValue": true
},
"leftValue": "={{ $input.first().json }}",
"rightValue": ""
}
]
}
},
"typeVersion": 2.2
},
{
"id": "6589673d-984d-4a1e-a655-1bc19d2b154e",
"name": "Response Empty1",
"type": "n8n-nodes-base.set",
"position": [
5280,
4380
],
"parameters": {
"options": {},
"assignments": {
"assignments": [
{
"id": "5bb23ce9-eb72-4868-9344-9e5d3952cc52",
"name": "response",
"type": "string",
"value": "no api operations found"
}
]
}
},
"executeOnce": true,
"typeVersion": 3.4
},
{
"id": "c5dc3eac-a3a5-481d-a8bc-8b653d88143d",
"name": "Research Pending",
"type": "n8n-nodes-base.googleSheets",
"position": [
3180,
2980
],
"parameters": {
"columns": {
"value": {
"row_number": "={{ $('For Each Research...').item.json.row_number }}",
"Stage 1 - Research": "=pending"
},
"schema": [
{
"id": "Service",
"type": "string",
"display": true,
"removed": true,
"required": false,
"displayName": "Service",
"defaultMatch": false,
"canBeUsedToMatch": true
},
{
"id": "Website",
"type": "string",
"display": true,
"removed": true,
"required": false,
"displayName": "Website",
"defaultMatch": false,
"canBeUsedToMatch": true
},
{
"id": "Stage 1 - Research",
"type": "string",
"display": true,
"required": false,
"displayName": "Stage 1 - Research",
"defaultMatch": false,
"canBeUsedToMatch": true
},
{
"id": "Stage 2 - Extraction",
"type": "string",
"display": true,
"removed": true,
"required": false,
"displayName": "Stage 2 - Extraction",
"defaultMatch": false,
"canBeUsedToMatch": true
},
{
"id": "Stage 3 - Output File",
"type": "string",
"display": true,
"removed": true,
"required": false,
"displayName": "Stage 3 - Output File",
"defaultMatch": false,
"canBeUsedToMatch": true
},
{
"id": "row_number",
"type": "string",
"display": true,
"removed": false,
"readOnly": true,
"required": false,
"displayName": "row_number",
"defaultMatch": false,
"canBeUsedToMatch": true
}
],
"mappingMode": "defineBelow",
"matchingColumns": [
"row_number"
]
},
"options": {},
"operation": "update",
"sheetName": {
"__rl": true,
"mode": "list",
"value": "gid=0",
"cachedResultUrl": "https://docs.google.com/spreadsheets/d/1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0/edit#gid=0",
"cachedResultName": "Sheet1"
},
"documentId": {
"__rl": true,
"mode": "list",
"value": "1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0",
"cachedResultUrl": "https://docs.google.com/spreadsheets/d/1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0/edit?usp=drivesdk",
"cachedResultName": "API Schema Crawler & Extractor"
}
},
"typeVersion": 4.5
},
{
"id": "39bceadb-6c3b-4b52-82b9-bdcecd9a164a",
"name": "Research Result",
"type": "n8n-nodes-base.googleSheets",
"position": [
3620,
2980
],
"parameters": {
"columns": {
"value": {
"row_number": "={{ $('For Each Research...').item.json.row_number }}",
"Stage 1 - Research": "={{ $json.response }}"
},
"schema": [
{
"id": "Service",
"type": "string",
"display": true,
"removed": true,
"required": false,
"displayName": "Service",
"defaultMatch": false,
"canBeUsedToMatch": true
},
{
"id": "Website",
"type": "string",
"display": true,
"removed": true,
"required": false,
"displayName": "Website",
"defaultMatch": false,
"canBeUsedToMatch": true
},
{
"id": "Stage 1 - Research",
"type": "string",
"display": true,
"required": false,
"displayName": "Stage 1 - Research",
"defaultMatch": false,
"canBeUsedToMatch": true
},
{
"id": "Stage 2 - Extraction",
"type": "string",
"display": true,
"removed": true,
"required": false,
"displayName": "Stage 2 - Extraction",
"defaultMatch": false,
"canBeUsedToMatch": true
},
{
"id": "Stage 3 - Output File",
"type": "string",
"display": true,
"removed": true,
"required": false,
"displayName": "Stage 3 - Output File",
"defaultMatch": false,
"canBeUsedToMatch": true
},
{
"id": "row_number",
"type": "string",
"display": true,
"removed": false,
"readOnly": true,
"required": false,
"displayName": "row_number",
"defaultMatch": false,
"canBeUsedToMatch": true
}
],
"mappingMode": "defineBelow",
"matchingColumns": [
"row_number"
]
},
"options": {},
"operation": "update",
"sheetName": {
"__rl": true,
"mode": "list",
"value": "gid=0",
"cachedResultUrl": "https://docs.google.com/spreadsheets/d/1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0/edit#gid=0",
"cachedResultName": "Sheet1"
},
"documentId": {
"__rl": true,
"mode": "list",
"value": "1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0",
"cachedResultUrl": "https://docs.google.com/spreadsheets/d/1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0/edit?usp=drivesdk",
"cachedResultName": "API Schema Crawler & Extractor"
}
},
"typeVersion": 4.5
},
{
"id": "0bd07f31-1c51-45aa-8316-b658aa214293",
"name": "Research Error",
"type": "n8n-nodes-base.googleSheets",
"position": [
3620,
3160
],
"parameters": {
"columns": {
"value": {
"row_number": "={{ $('For Each Research...').item.json.row_number }}",
"Stage 1 - Research": "=error"
},
"schema": [
{
"id": "Service",
"type": "string",
"display": true,
"removed": true,
"required": false,
"displayName": "Service",
"defaultMatch": false,
"canBeUsedToMatch": true
},
{
"id": "Website",
"type": "string",
"display": true,
"removed": true,
"required": false,
"displayName": "Website",
"defaultMatch": false,
"canBeUsedToMatch": true
},
{
"id": "Stage 1 - Research",
"type": "string",
"display": true,
"required": false,
"displayName": "Stage 1 - Research",
"defaultMatch": false,
"canBeUsedToMatch": true
},
{
"id": "Stage 2 - Extraction",
"type": "string",
"display": true,
"removed": true,
"required": false,
"displayName": "Stage 2 - Extraction",
"defaultMatch": false,
"canBeUsedToMatch": true
},
{
"id": "Stage 3 - Output File",
"type": "string",
"display": true,
"removed": true,
"required": false,
"displayName": "Stage 3 - Output File",
"defaultMatch": false,
"canBeUsedToMatch": true
},
{
"id": "row_number",
"type": "string",
"display": true,
"removed": false,
"readOnly": true,
"required": false,
"displayName": "row_number",
"defaultMatch": false,
"canBeUsedToMatch": true
}
],
"mappingMode": "defineBelow",
"matchingColumns": [
"row_number"
]
},
"options": {},
"operation": "update",
"sheetName": {
"__rl": true,
"mode": "list",
"value": "gid=0",
"cachedResultUrl": "https://docs.google.com/spreadsheets/d/1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0/edit#gid=0",
"cachedResultName": "Sheet1"
},
"documentId": {
"__rl": true,
"mode": "list",
"value": "1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0",
"cachedResultUrl": "https://docs.google.com/spreadsheets/d/1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0/edit?usp=drivesdk",
"cachedResultName": "API Schema Crawler & Extractor"
}
},
"typeVersion": 4.5
},
{
"id": "0385784f-95ef-46c3-82c4-50fcf7146736",
"name": "Extract Pending",
"type": "n8n-nodes-base.googleSheets",
"position": [
4160,
2980
],
"parameters": {
"columns": {
"value": {
"row_number": "={{ $('For Each Extract...').item.json.row_number }}",
"Stage 2 - Extraction": "pending"
},
"schema": [
{
"id": "Service",
"type": "string",
"display": true,
"removed": true,
"required": false,
"displayName": "Service",
"defaultMatch": false,
"canBeUsedToMatch": true
},
{
"id": "Website",
"type": "string",
"display": true,
"removed": true,
"required": false,
"displayName": "Website",
"defaultMatch": false,
"canBeUsedToMatch": true
},
{
"id": "Stage 1 - Research",
"type": "string",
"display": true,
"removed": true,
"required": false,
"displayName": "Stage 1 - Research",
"defaultMatch": false,
"canBeUsedToMatch": true
},
{
"id": "Stage 2 - Extraction",
"type": "string",
"display": true,
"removed": false,
"required": false,
"displayName": "Stage 2 - Extraction",
"defaultMatch": false,
"canBeUsedToMatch": true
},
{
"id": "Stage 3 - Output File",
"type": "string",
"display": true,
"removed": true,
"required": false,
"displayName": "Stage 3 - Output File",
"defaultMatch": false,
"canBeUsedToMatch": true
},
{
"id": "row_number",
"type": "string",
"display": true,
"removed": false,
"readOnly": true,
"required": false,
"displayName": "row_number",
"defaultMatch": false,
"canBeUsedToMatch": true
}
],
"mappingMode": "defineBelow",
"matchingColumns": [
"row_number"
]
},
"options": {},
"operation": "update",
"sheetName": {
"__rl": true,
"mode": "list",
"value": "gid=0",
"cachedResultUrl": "https://docs.google.com/spreadsheets/d/1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0/edit#gid=0",
"cachedResultName": "Sheet1"
},
"documentId": {
"__rl": true,
"mode": "list",
"value": "1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0",
"cachedResultUrl": "https://docs.google.com/spreadsheets/d/1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0/edit?usp=drivesdk",
"cachedResultName": "API Schema Crawler & Extractor"
}
},
"executeOnce": false,
"typeVersion": 4.5
},
{
"id": "21c1e982-25a6-4a00-b8d3-6c299c452106",
"name": "Research Event",
"type": "n8n-nodes-base.set",
"position": [
3320,
2980
],
"parameters": {
"mode": "raw",
"options": {},
"jsonOutput": "={{\n{\n \"eventType\": \"research\",\n \"createdAt\": $now.toISO(),\n \"executedById\": $execution.id,\n \"data\": {\n \"row_number\": $('For Each Research...').item.json.row_number,\n \"service\": $('For Each Research...').item.json.Service,\n \"url\": $('For Each Research...').item.json.Website,\n \"collection\": \"api_schema_crawler_and_extractor\"\n }\n}\n}}"
},
"typeVersion": 3.4
},
{
"id": "c83f99f1-e28f-4c15-aff8-da25bb5dfe3b",
"name": "Extract Event",
"type": "n8n-nodes-base.set",
"position": [
4300,
2980
],
"parameters": {
"mode": "raw",
"options": {},
"jsonOutput": "={{\n{\n \"eventType\": \"extract\",\n \"createdAt\": $now.toISO(),\n \"executedById\": $execution.id,\n \"data\": {\n \"row_number\": $('For Each Extract...').item.json.row_number,\n \"service\": $('For Each Extract...').item.json.Service,\n \"url\": $('For Each Extract...').item.json.Website,\n \"collection\": \"api_schema_crawler_and_extractor\"\n }\n}\n}}"
},
"typeVersion": 3.4
},
{
"id": "88c3caec-75f7-47a1-9b50-1246c457c2b4",
"name": "Extract",
"type": "n8n-nodes-base.executeWorkflow",
"onError": "continueErrorOutput",
"position": [
4440,
2980
],
"parameters": {
"mode": "each",
"options": {
"waitForSubWorkflow": true
},
"workflowId": {
"__rl": true,
"mode": "id",
"value": "={{ $workflow.id }}"
}
},
"typeVersion": 1.1
},
{
"id": "2342b7ff-b00d-439a-a859-63fd0a6bac3a",
"name": "Extract Result",
"type": "n8n-nodes-base.googleSheets",
"position": [
4600,
2980
],
"parameters": {
"columns": {
"value": {
"row_number": "={{ $('For Each Extract...').item.json.row_number }}",
"Stage 2 - Extraction": "={{ $json.response }}"
},
"schema": [
{
"id": "Service",
"type": "string",
"display": true,
"removed": true,
"required": false,
"displayName": "Service",
"defaultMatch": false,
"canBeUsedToMatch": true
},
{
"id": "Website",
"type": "string",
"display": true,
"removed": true,
"required": false,
"displayName": "Website",
"defaultMatch": false,
"canBeUsedToMatch": true
},
{
"id": "Stage 1 - Research",
"type": "string",
"display": true,
"removed": true,
"required": false,
"displayName": "Stage 1 - Research",
"defaultMatch": false,
"canBeUsedToMatch": true
},
{
"id": "Stage 2 - Extraction",
"type": "string",
"display": true,
"removed": false,
"required": false,
"displayName": "Stage 2 - Extraction",
"defaultMatch": false,
"canBeUsedToMatch": true
},
{
"id": "Stage 3 - Output File",
"type": "string",
"display": true,
"removed": true,
"required": false,
"displayName": "Stage 3 - Output File",
"defaultMatch": false,
"canBeUsedToMatch": true
},
{
"id": "row_number",
"type": "string",
"display": true,
"removed": false,
"readOnly": true,
"required": false,
"displayName": "row_number",
"defaultMatch": false,
"canBeUsedToMatch": true
}
],
"mappingMode": "defineBelow",
"matchingColumns": [
"row_number"
]
},
"options": {},
"operation": "update",
"sheetName": {
"__rl": true,
"mode": "list",
"value": "gid=0",
"cachedResultUrl": "https://docs.google.com/spreadsheets/d/1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0/edit#gid=0",
"cachedResultName": "Sheet1"
},
"documentId": {
"__rl": true,
"mode": "list",
"value": "1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0",
"cachedResultUrl": "https://docs.google.com/spreadsheets/d/1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0/edit?usp=drivesdk",
"cachedResultName": "API Schema Crawler & Extractor"
}
},
"typeVersion": 4.5
},
{
"id": "d4c423c9-1d6a-4a69-9302-92ec79734d61",
"name": "Extract Error",
"type": "n8n-nodes-base.googleSheets",
"position": [
4600,
3160
],
"parameters": {
"columns": {
"value": {
"row_number": "={{ $('For Each Extract...').item.json.row_number }}",
"Stage 2 - Extraction": "error"
},
"schema": [
{
"id": "Service",
"type": "string",
"display": true,
"removed": true,
"required": false,
"displayName": "Service",
"defaultMatch": false,
"canBeUsedToMatch": true
},
{
"id": "Website",
"type": "string",
"display": true,
"removed": true,
"required": false,
"displayName": "Website",
"defaultMatch": false,
"canBeUsedToMatch": true
},
{
"id": "Stage 1 - Research",
"type": "string",
"display": true,
"removed": true,
"required": false,
"displayName": "Stage 1 - Research",
"defaultMatch": false,
"canBeUsedToMatch": true
},
{
"id": "Stage 2 - Extraction",
"type": "string",
"display": true,
"removed": false,
"required": false,
"displayName": "Stage 2 - Extraction",
"defaultMatch": false,
"canBeUsedToMatch": true
},
{
"id": "Stage 3 - Output File",
"type": "string",
"display": true,
"removed": true,
"required": false,
"displayName": "Stage 3 - Output File",
"defaultMatch": false,
"canBeUsedToMatch": true
},
{
"id": "row_number",
"type": "string",
"display": true,
"removed": false,
"readOnly": true,
"required": false,
"displayName": "row_number",
"defaultMatch": false,
"canBeUsedToMatch": true
}
],
"mappingMode": "defineBelow",
"matchingColumns": [
"row_number"
]
},
"options": {},
"operation": "update",
"sheetName": {
"__rl": true,
"mode": "list",
"value": "gid=0",
"cachedResultUrl": "https://docs.google.com/spreadsheets/d/1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0/edit#gid=0",
"cachedResultName": "Sheet1"
},
"documentId": {
"__rl": true,
"mode": "list",
"value": "1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0",
"cachedResultUrl": "https://docs.google.com/spreadsheets/d/1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0/edit?usp=drivesdk",
"cachedResultName": "API Schema Crawler & Extractor"
}
},
"typeVersion": 4.5
},
{
"id": "f64254d6-4493-4aaf-8160-35e8ff4fdc34",
"name": "Get API Operations",
"type": "n8n-nodes-base.googleSheets",
"position": [
3100,
4740
],
"parameters": {
"options": {},
"filtersUI": {
"values": [
{
"lookupValue": "={{ $json.data.service }}",
"lookupColumn": "service"
}
]
},
"sheetName": {
"__rl": true,
"mode": "list",
"value": 1042334767,
"cachedResultUrl": "https://docs.google.com/spreadsheets/d/1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0/edit#gid=1042334767",
"cachedResultName": "Extracted API Operations"
},
"documentId": {
"__rl": true,
"mode": "list",
"value": "1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0",
"cachedResultUrl": "https://docs.google.com/spreadsheets/d/1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0/edit?usp=drivesdk",
"cachedResultName": "API Schema Crawler & Extractor"
}
},
"typeVersion": 4.5
},
{
"id": "fa748b63-3d2b-4cf3-b1fb-1bd953e5054b",
"name": "Contruct JSON Schema",
"type": "n8n-nodes-base.code",
"position": [
3280,
4740
],
"parameters": {
"jsCode": "const service = {\n documentation_url: $('EventRouter').first().json.data.url,\n endpoints: [],\n};\n\nconst resources = Array.from(new Set($input.all().map(item => item.json.resource.toLowerCase().trim())));\n\nfor (const resource of resources) {\n const resourceLabel = resource.replace('api', '').trim();\n if (!resourceLabel) continue;\n const endpoint = {\n resource: resourceLabel[0].toUpperCase() + resourceLabel.substring(1, resourceLabel.length)\n };\n const operations = $input.all()\n .filter(item => item.json.resource.toLowerCase().trim() === resource)\n .map(item => item.json);\n endpoint.operations = operations.map(op => ({\n \"operation\": op.operation[0].toUpperCase() + op.operation.substring(1, op.operation.length),\n \"description\": op.description.match(/(^[^\\.]+.)/)[0],\n \"ApiUrl\": op.url,\n \"method\": op.method.toUpperCase(),\n \"method_documentation_url\": op.documentation_url || ''\n }));\n service.endpoints.push(endpoint);\n}\n\nreturn service;"
},
"typeVersion": 2
},
{
"id": "e60b7ccb-baa2-4095-8425-0e20bcdbfdd2",
"name": "Upload to Drive",
"type": "n8n-nodes-base.googleDrive",
"position": [
3640,
4740
],
"parameters": {
"name": "={{ $json.filename }}",
"content": "={{ $json.data }}",
"driveId": {
"__rl": true,
"mode": "list",
"value": "My Drive"
},
"options": {},
"folderId": {
"__rl": true,
"mode": "list",
"value": "149rBJYv9RKQx-vQO2qKUGfUzxk_J4lfw",
"cachedResultUrl": "https://drive.google.com/drive/folders/149rBJYv9RKQx-vQO2qKUGfUzxk_J4lfw",
"cachedResultName": "63. API Schema Extractor Remake"
},
"operation": "createFromText"
},
"typeVersion": 3
},
{
"id": "f90546e6-3610-4198-87fc-96d7e2b6bc57",
"name": "Set Upload Fields",
"type": "n8n-nodes-base.set",
"position": [
3460,
4740
],
"parameters": {
"options": {},
"assignments": {
"assignments": [
{
"id": "3c7d4946-c385-4aff-93ec-ae0850964099",
"name": "filename",
"type": "string",
"value": "={{\n $('EventRouter').first().json.data.service\n .replace(/\\W+/, '_')\n .toLowerCase()\n}}_api_operations_{{ $now.format('yyyyMMddhhmmss') }}.json"
},
{
"id": "4a7a9fae-7267-4ef6-ae33-ac4cd9777ee9",
"name": "data",
"type": "string",
"value": "={{ JSON.stringify($json, null, 4) }}"
}
]
}
},
"typeVersion": 3.4
},
{
"id": "c814b48d-2005-4150-a481-956f0b9506a5",
"name": "Response OK2",
"type": "n8n-nodes-base.set",
"position": [
3820,
4740
],
"parameters": {
"options": {},
"assignments": {
"assignments": [
{
"id": "79598789-4468-4565-828f-fedc48be15c3",
"name": "response",
"type": "object",
"value": "={{\n({\n id: $json.id,\n filename: $('Set Upload Fields').item.json.filename\n}).toJsonString()\n}}"
}
]
}
},
"executeOnce": true,
"typeVersion": 3.4
},
{
"id": "4b1efa99-e8c8-49f5-8db8-916b8dde838d",
"name": "Generate Event",
"type": "n8n-nodes-base.set",
"position": [
5300,
2980
],
"parameters": {
"mode": "raw",
"options": {},
"jsonOutput": "={{\n{\n \"eventType\": \"generate\",\n \"createdAt\": $now.toISO(),\n \"executedById\": $execution.id,\n \"data\": {\n \"row_number\": $('For Each Generate...').item.json.row_number,\n \"service\": $('For Each Generate...').item.json.Service,\n \"url\": $('For Each Generate...').item.json.Website,\n \"collection\": \"api_schema_crawler_and_extractor\"\n }\n}\n}}"
},
"typeVersion": 3.4
},
{
"id": "49b82a1a-d51e-4caf-b7ab-8d27d0585b60",
"name": "Generate Pending",
"type": "n8n-nodes-base.googleSheets",
"position": [
5160,
2980
],
"parameters": {
"columns": {
"value": {
"row_number": "={{ $('For Each Generate...').item.json.row_number }}",
"Stage 3 - Output File": "pending"
},
"schema": [
{
"id": "Service",
"type": "string",
"display": true,
"removed": true,
"required": false,
"displayName": "Service",
"defaultMatch": false,
"canBeUsedToMatch": true
},
{
"id": "Website",
"type": "string",
"display": true,
"removed": true,
"required": false,
"displayName": "Website",
"defaultMatch": false,
"canBeUsedToMatch": true
},
{
"id": "Stage 1 - Research",
"type": "string",
"display": true,
"removed": true,
"required": false,
"displayName": "Stage 1 - Research",
"defaultMatch": false,
"canBeUsedToMatch": true
},
{
"id": "Stage 2 - Extraction",
"type": "string",
"display": true,
"removed": true,
"required": false,
"displayName": "Stage 2 - Extraction",
"defaultMatch": false,
"canBeUsedToMatch": true
},
{
"id": "Stage 3 - Output File",
"type": "string",
"display": true,
"removed": false,
"required": false,
"displayName": "Stage 3 - Output File",
"defaultMatch": false,
"canBeUsedToMatch": true
},
{
"id": "row_number",
"type": "string",
"display": true,
"removed": false,
"readOnly": true,
"required": false,
"displayName": "row_number",
"defaultMatch": false,
"canBeUsedToMatch": true
}
],
"mappingMode": "defineBelow",
"matchingColumns": [
"row_number"
]
},
"options": {},
"operation": "update",
"sheetName": {
"__rl": true,
"mode": "list",
"value": "gid=0",
"cachedResultUrl": "https://docs.google.com/spreadsheets/d/1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0/edit#gid=0",
"cachedResultName": "Sheet1"
},
"documentId": {
"__rl": true,
"mode": "list",
"value": "1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0",
"cachedResultUrl": "https://docs.google.com/spreadsheets/d/1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0/edit?usp=drivesdk",
"cachedResultName": "API Schema Crawler & Extractor"
}
},
"executeOnce": false,
"typeVersion": 4.5
},
{
"id": "7d1a937c-49cc-40d7-b2ca-d315c5efca93",
"name": "Generate",
"type": "n8n-nodes-base.executeWorkflow",
"onError": "continueErrorOutput",
"position": [
5440,
2980
],
"parameters": {
"mode": "each",
"options": {
"waitForSubWorkflow": true
},
"workflowId": {
"__rl": true,
"mode": "id",
"value": "={{ $workflow.id }}"
}
},
"typeVersion": 1.1
},
{
"id": "f35d843d-6c40-4725-b73f-8ca1a8e219bb",
"name": "Generate Error",
"type": "n8n-nodes-base.googleSheets",
"position": [
5600,
3160
],
"parameters": {
"columns": {
"value": {
"row_number": "={{ $('For Each Generate...').item.json.row_number }}",
"Stage 3 - Output File": "error"
},
"schema": [
{
"id": "Service",
"type": "string",
"display": true,
"removed": true,
"required": false,
"displayName": "Service",
"defaultMatch": false,
"canBeUsedToMatch": true
},
{
"id": "Website",
"type": "string",
"display": true,
"removed": true,
"required": false,
"displayName": "Website",
"defaultMatch": false,
"canBeUsedToMatch": true
},
{
"id": "Stage 1 - Research",
"type": "string",
"display": true,
"removed": true,
"required": false,
"displayName": "Stage 1 - Research",
"defaultMatch": false,
"canBeUsedToMatch": true
},
{
"id": "Stage 2 - Extraction",
"type": "string",
"display": true,
"removed": true,
"required": false,
"displayName": "Stage 2 - Extraction",
"defaultMatch": false,
"canBeUsedToMatch": true
},
{
"id": "Stage 3 - Output File",
"type": "string",
"display": true,
"removed": false,
"required": false,
"displayName": "Stage 3 - Output File",
"defaultMatch": false,
"canBeUsedToMatch": true
},
{
"id": "row_number",
"type": "string",
"display": true,
"removed": false,
"readOnly": true,
"required": false,
"displayName": "row_number",
"defaultMatch": false,
"canBeUsedToMatch": true
}
],
"mappingMode": "defineBelow",
"matchingColumns": [
"row_number"
]
},
"options": {},
"operation": "update",
"sheetName": {
"__rl": true,
"mode": "list",
"value": "gid=0",
"cachedResultUrl": "https://docs.google.com/spreadsheets/d/1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0/edit#gid=0",
"cachedResultName": "Sheet1"
},
"documentId": {
"__rl": true,
"mode": "list",
"value": "1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0",
"cachedResultUrl": "https://docs.google.com/spreadsheets/d/1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0/edit?usp=drivesdk",
"cachedResultName": "API Schema Crawler & Extractor"
}
},
"typeVersion": 4.5
},
{
"id": "e2f1f8e8-6852-4f19-98ec-85d9bd42729c",
"name": "Generate Result",
"type": "n8n-nodes-base.googleSheets",
"position": [
5600,
2980
],
"parameters": {
"columns": {
"value": {
"row_number": "={{ $('For Each Generate...').item.json.row_number }}",
"Output Destination": "={{ $json.response.filename }}",
"Stage 3 - Output File": "ok"
},
"schema": [
{
"id": "Service",
"type": "string",
"display": true,
"removed": true,
"required": false,
"displayName": "Service",
"defaultMatch": false,
"canBeUsedToMatch": true
},
{
"id": "Website",
"type": "string",
"display": true,
"removed": true,
"required": false,
"displayName": "Website",
"defaultMatch": false,
"canBeUsedToMatch": true
},
{
"id": "Stage 1 - Research",
"type": "string",
"display": true,
"removed": true,
"required": false,
"displayName": "Stage 1 - Research",
"defaultMatch": false,
"canBeUsedToMatch": true
},
{
"id": "Stage 2 - Extraction",
"type": "string",
"display": true,
"removed": true,
"required": false,
"displayName": "Stage 2 - Extraction",
"defaultMatch": false,
"canBeUsedToMatch": true
},
{
"id": "Stage 3 - Output File",
"type": "string",
"display": true,
"removed": false,
"required": false,
"displayName": "Stage 3 - Output File",
"defaultMatch": false,
"canBeUsedToMatch": true
},
{
"id": "Output Destination",
"type": "string",
"display": true,
"removed": false,
"required": false,
"displayName": "Output Destination",
"defaultMatch": false,
"canBeUsedToMatch": true
},
{
"id": "row_number",
"type": "string",
"display": true,
"removed": false,
"readOnly": true,
"required": false,
"displayName": "row_number",
"defaultMatch": false,
"canBeUsedToMatch": true
}
],
"mappingMode": "defineBelow",
"matchingColumns": [
"row_number"
]
},
"options": {},
"operation": "update",
"sheetName": {
"__rl": true,
"mode": "list",
"value": "gid=0",
"cachedResultUrl": "https://docs.google.com/spreadsheets/d/1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0/edit#gid=0",
"cachedResultName": "Sheet1"
},
"documentId": {
"__rl": true,
"mode": "list",
"value": "1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0",
"cachedResultUrl": "https://docs.google.com/spreadsheets/d/1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0/edit?usp=drivesdk",
"cachedResultName": "API Schema Crawler & Extractor"
}
},
"typeVersion": 4.5
},
{
"id": "00c5b05b-fd70-4d58-8fc6-4e9b8d689a43",
"name": "Get All Extract",
"type": "n8n-nodes-base.googleSheets",
"position": [
3620,
2820
],
"parameters": {
"options": {},
"filtersUI": {
"values": [
{
"lookupValue": "=ok",
"lookupColumn": "Stage 1 - Research"
},
{
"lookupValue": "={{ \"\" }}",
"lookupColumn": "Stage 2 - Extraction"
}
]
},
"sheetName": {
"__rl": true,
"mode": "list",
"value": "gid=0",
"cachedResultUrl": "https://docs.google.com/spreadsheets/d/1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0/edit#gid=0",
"cachedResultName": "Sheet1"
},
"documentId": {
"__rl": true,
"mode": "list",
"value": "1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0",
"cachedResultUrl": "https://docs.google.com/spreadsheets/d/1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0/edit?usp=drivesdk",
"cachedResultName": "API Schema Crawler & Extractor"
}
},
"executeOnce": true,
"typeVersion": 4.5,
"alwaysOutputData": true
},
{
"id": "c477ea01-028d-4e69-b772-adb8c03d1522",
"name": "Get All Research",
"type": "n8n-nodes-base.googleSheets",
"position": [
2640,
2820
],
"parameters": {
"options": {},
"filtersUI": {
"values": [
{
"lookupValue": "={{ \"\" }}",
"lookupColumn": "Stage 1 - Research"
}
]
},
"sheetName": {
"__rl": true,
"mode": "list",
"value": "gid=0",
"cachedResultUrl": "https://docs.google.com/spreadsheets/d/1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0/edit#gid=0",
"cachedResultName": "Sheet1"
},
"documentId": {
"__rl": true,
"mode": "list",
"value": "1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0",
"cachedResultUrl": "https://docs.google.com/spreadsheets/d/1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0/edit?usp=drivesdk",
"cachedResultName": "API Schema Crawler & Extractor"
}
},
"credentials": {
"googleSheetsOAuth2Api": {
"id": "aALuyzBGGfmdBzrU",
"name": "Google Sheets account 2"
}
},
"typeVersion": 4.5,
"alwaysOutputData": true
},
{
"id": "60ba84c1-40cf-492f-bf52-c9edf5925646",
"name": "For Each Research...",
"type": "n8n-nodes-base.splitInBatches",
"position": [
3020,
2820
],
"parameters": {
"options": {}
},
"typeVersion": 3
},
{
"id": "5365cd1a-c7f8-40fb-84b3-9e5306ecf462",
"name": "For Each Extract...",
"type": "n8n-nodes-base.splitInBatches",
"position": [
4000,
2820
],
"parameters": {
"options": {}
},
"typeVersion": 3
},
{
"id": "d7a0743f-5f83-4c9b-b11c-85e2df3a4ecc",
"name": "Wait1",
"type": "n8n-nodes-base.wait",
"position": [
4780,
3160
],
"webhookId": "e9ad3ef0-7403-4e65-b0a4-4afdfb0cbc6d",
"parameters": {
"amount": 0
},
"typeVersion": 1.1
},
{
"id": "ec09ac70-5e05-463c-9d30-027e691a36b4",
"name": "All Research Done?",
"type": "n8n-nodes-base.if",
"position": [
2800,
2820
],
"parameters": {
"options": {},
"conditions": {
"options": {
"version": 2,
"leftValue": "",
"caseSensitive": true,
"typeValidation": "strict"
},
"combinator": "and",
"conditions": [
{
"id": "8d4b0159-af18-445e-a9ee-bd7952d8e0bd",
"operator": {
"type": "object",
"operation": "empty",
"singleValue": true
},
"leftValue": "={{ $input.first().json }}",
"rightValue": ""
}
]
}
},
"typeVersion": 2.2
},
{
"id": "cd892e11-b4de-42f1-bab9-4bd783494c8a",
"name": "All Extract Done?",
"type": "n8n-nodes-base.if",
"position": [
3780,
2820
],
"parameters": {
"options": {},
"conditions": {
"options": {
"version": 2,
"leftValue": "",
"caseSensitive": true,
"typeValidation": "strict"
},
"combinator": "and",
"conditions": [
{
"id": "8d4b0159-af18-445e-a9ee-bd7952d8e0bd",
"operator": {
"type": "object",
"operation": "empty",
"singleValue": true
},
"leftValue": "={{ $input.first().json }}",
"rightValue": ""
}
]
}
},
"typeVersion": 2.2
},
{
"id": "426091fb-d0eb-4589-8f2f-2bbeb9174cfc",
"name": "Get All Generate",
"type": "n8n-nodes-base.googleSheets",
"position": [
4600,
2820
],
"parameters": {
"options": {},
"filtersUI": {
"values": [
{
"lookupValue": "=ok",
"lookupColumn": "Stage 1 - Research"
},
{
"lookupValue": "=ok",
"lookupColumn": "Stage 2 - Extraction"
},
{
"lookupValue": "={{ \"\" }}",
"lookupColumn": "Stage 3 - Output File"
}
]
},
"sheetName": {
"__rl": true,
"mode": "list",
"value": "gid=0",
"cachedResultUrl": "https://docs.google.com/spreadsheets/d/1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0/edit#gid=0",
"cachedResultName": "Sheet1"
},
"documentId": {
"__rl": true,
"mode": "list",
"value": "1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0",
"cachedResultUrl": "https://docs.google.com/spreadsheets/d/1l59ikBvEwPNSWIm2k6KRMFPTNImJPYqs9bzGT5dUiU0/edit?usp=drivesdk",
"cachedResultName": "API Schema Crawler & Extractor"
}
},
"executeOnce": true,
"typeVersion": 4.5
},
{
"id": "01e91cf6-5bd5-4891-ba1f-95176e444fe6",
"name": "All Generate Done?",
"type": "n8n-nodes-base.if",
"position": [
4780,
2820
],
"parameters": {
"options": {},
"conditions": {
"options": {
"version": 2,
"leftValue": "",
"caseSensitive": true,
"typeValidation": "strict"
},
"combinator": "and",
"conditions": [
{
"id": "8d4b0159-af18-445e-a9ee-bd7952d8e0bd",
"operator": {
"type": "object",
"operation": "empty",
"singleValue": true
},
"leftValue": "={{ $input.first().json }}",
"rightValue": ""
}
]
}
},
"typeVersion": 2.2
},
{
"id": "08f3505d-aad8-475a-bf08-e3da12798367",
"name": "For Each Generate...",
"type": "n8n-nodes-base.splitInBatches",
"position": [
5000,
2820
],
"parameters": {
"options": {}
},
"typeVersion": 3
},
{
"id": "1a1b30bd-91ab-41bd-9ead-39d24fc2643f",
"name": "Wait2",
"type": "n8n-nodes-base.wait",
"position": [
5780,
3160
],
"webhookId": "e9ad3ef0-7403-4e65-b0a4-4afdfb0cbc6d",
"parameters": {
"amount": 0
},
"typeVersion": 1.1
},
{
"id": "8f2be6bb-ab65-4c92-9ca1-d7ffa936a2a3",
"name": "Has Results?1",
"type": "n8n-nodes-base.if",
"position": [
4260,
3720
],
"parameters": {
"options": {},
"conditions": {
"options": {
"version": 2,
"leftValue": "",
"caseSensitive": true,
"typeValidation": "strict"
},
"combinator": "and",
"conditions": [
{
"id": "1223d607-45a8-44b1-b510-56fdbe013eba",
"operator": {
"type": "array",
"operation": "notEmpty",
"singleValue": true
},
"leftValue": "={{ $input.all().filter(item => item.json.body) }}",
"rightValue": ""
}
]
}
},
"typeVersion": 2.2
},
{
"id": "82fe66bf-4348-4673-8c64-3415f642fb4b",
"name": "Response Scrape Error",
"type": "n8n-nodes-base.set",
"position": [
4460,
3900
],
"parameters": {
"options": {},
"assignments": {
"assignments": [
{
"id": "5bb23ce9-eb72-4868-9344-9e5d3952cc52",
"name": "response",
"type": "string",
"value": "web scraping error"
}
]
}
},
"executeOnce": true,
"typeVersion": 3.4
},
{
"id": "3625591b-cb48-4131-ae8a-56d1e132bb5a",
"name": "Has Results?3",
"type": "n8n-nodes-base.if",
"position": [
4780,
3580
],
"parameters": {
"options": {},
"conditions": {
"options": {
"version": 2,
"leftValue": "",
"caseSensitive": true,
"typeValidation": "strict"
},
"combinator": "and",
"conditions": [
{
"id": "1223d607-45a8-44b1-b510-56fdbe013eba",
"operator": {
"type": "array",
"operation": "notEmpty",
"singleValue": true
},
"leftValue": "={{ $input.all().filter(item => item.json.body) }}",
"rightValue": ""
}
]
}
},
"typeVersion": 2.2
},
{
"id": "f82a4a25-5f93-4ba4-baae-08283c4ccadd",
"name": "Response No API Docs",
"type": "n8n-nodes-base.set",
"position": [
4980,
3740
],
"parameters": {
"options": {},
"assignments": {
"assignments": [
{
"id": "5bb23ce9-eb72-4868-9344-9e5d3952cc52",
"name": "response",
"type": "string",
"value": "no api docs in web results"
}
]
}
},
"executeOnce": true,
"typeVersion": 3.4
},
{
"id": "4c3bb934-966c-445a-893f-0676a59140ee",
"name": "Sticky Note",
"type": "n8n-nodes-base.stickyNote",
"position": [
3020,
2580
],
"parameters": {
"width": 620,
"height": 180,
"content": "## Stage 1 - Research for API Documentation\n- Fetch a list of services pending research from Database (Google Sheet)\n- Uses a search engine (Google) to find API Documentation for each service\n- Uses Webscraper (Apify) to read the contents of search results to filter irrelevant pages\n- Stores webpage contents and metadata into Vector Store (Qdrant)"
},
"typeVersion": 1
},
{
"id": "bc269a57-f353-4cc8-bd2e-43236fa55d39",
"name": "Sticky Note1",
"type": "n8n-nodes-base.stickyNote",
"position": [
4000,
2580
],
"parameters": {
"width": 760,
"height": 180,
"content": "## Stage 2 - Extract API Operations From Documentation\n- Fetch a list of services pending extraction from Database (Google Sheet)\n- Query Vector store (Qdrant) to figure out service's products, solutions and offerings\n- Query Vector store (Qdrant) again for API documentation relevant to these products, solutions and offerings\n- Extract any API operations found in the API documentation results using LLM (Gemini)\n- Store extracted API operations into Database (Google Sheet)"
},
"typeVersion": 1
},
{
"id": "d2dcad47-f655-4a15-ac92-6dab05eea4e1",
"name": "Sticky Note2",
"type": "n8n-nodes-base.stickyNote",
"position": [
5000,
2580
],
"parameters": {
"width": 740,
"height": 180,
"content": "## Stage 3 - Generate Custom Schema From API Operations\n- Fetch a list of services pending generation from Database (Google Sheet)\n- Fetch all API operations for each service from Database (Google Sheet)\n- Use Code node to combine and group all API operations for a service and convert to a custom schema\n- Upload the resulting custom schema to file storage (Google Drive)"
},
"typeVersion": 1
},
{
"id": "d1e1a271-4260-49c3-bda6-2864605c7365",
"name": "Sticky Note3",
"type": "n8n-nodes-base.stickyNote",
"position": [
3100,
3680
],
"parameters": {
"color": 5,
"width": 180,
"height": 80,
"content": "## Stage 1 - Subworkflow"
},
"typeVersion": 1
},
{
"id": "1e50f04a-94ff-48b4-aa99-cd1d4f1d12be",
"name": "Sticky Note4",
"type": "n8n-nodes-base.stickyNote",
"position": [
3100,
4080
],
"parameters": {
"color": 5,
"width": 180,
"height": 80,
"content": "## Stage 2 - Subworkflow"
},
"typeVersion": 1
},
{
"id": "f8334dbd-b542-404a-b4fc-6cf7cc07730d",
"name": "Sticky Note5",
"type": "n8n-nodes-base.stickyNote",
"position": [
3100,
4620
],
"parameters": {
"color": 5,
"width": 180,
"height": 80,
"content": "## Stage 3 - Subworkflow"
},
"typeVersion": 1
}
],
"pinData": {
"Execute Workflow Trigger": [
{
"data": {
"url": "https://www.formstack.com/",
"service": "Formstack",
"collection": "api_schema_crawler_and_extractor",
"row_number": 2
},
"createdAt": "2024-12-07T12:22:35.344-05:00",
"eventType": "research",
"executedById": "10234"
}
]
},
"connections": {
"Wait": {
"main": [
[
{
"node": "For Each Research...",
"type": "main",
"index": 0
}
]
]
},
"Wait1": {
"main": [
[
{
"node": "For Each Extract...",
"type": "main",
"index": 0
}
]
]
},
"Wait2": {
"main": [
[
{
"node": "For Each Generate...",
"type": "main",
"index": 0
}
]
]
},
"Extract": {
"main": [
[
{
"node": "Extract Result",
"type": "main",
"index": 0
}
],
[
{
"node": "Extract Error",
"type": "main",
"index": 0
}
]
]
},
"Generate": {
"main": [
[
{
"node": "Generate Result",
"type": "main",
"index": 0
}
],
[
{
"node": "Generate Error",
"type": "main",
"index": 0
}
]
]
},
"Research": {
"main": [
[
{
"node": "Research Result",
"type": "main",
"index": 0
}
],
[
{
"node": "Research Error",
"type": "main",
"index": 0
}
]
]
},
"Append Row": {
"main": [
[
{
"node": "Response OK1",
"type": "main",
"index": 0
}
]
]
},
"EventRouter": {
"main": [
[
{
"node": "Web Search For API Schema",
"type": "main",
"index": 0
}
],
[
{
"node": "Query Templates",
"type": "main",
"index": 0
}
],
[
{
"node": "Get API Operations",
"type": "main",
"index": 0
}
]
]
},
"Merge Lists": {
"main": [
[
{
"node": "Has Operations?",
"type": "main",
"index": 0
}
]
]
},
"Combine Docs": {
"main": [
[
{
"node": "Query & Docs",
"type": "main",
"index": 0
}
]
]
},
"Has Results?": {
"main": [
[
{
"node": "Results to List",
"type": "main",
"index": 0
}
],
[
{
"node": "Response Empty",
"type": "main",
"index": 0
}
]
]
},
"Query & Docs": {
"main": [
[
{
"node": "For Each Template...",
"type": "main",
"index": 0
}
]
]
},
"Remove Dupes": {
"main": [
[
{
"node": "Scrape Webpage Contents",
"type": "main",
"index": 0
}
]
]
},
"Combine Docs1": {
"main": [
[
{
"node": "Query & Docs1",
"type": "main",
"index": 0
}
]
]
},
"Extract Error": {
"main": [
[
{
"node": "Wait1",
"type": "main",
"index": 0
}
]
]
},
"Extract Event": {
"main": [
[
{
"node": "Extract",
"type": "main",
"index": 0
}
]
]
},
"Has Results?1": {
"main": [
[
{
"node": "Has API Documentation?",
"type": "main",
"index": 0
}
],
[
{
"node": "Response Scrape Error",
"type": "main",
"index": 0
}
]
]
},
"Has Results?3": {
"main": [
[
{
"node": "Set Embedding Variables",
"type": "main",
"index": 0
}
],
[
{
"node": "Response No API Docs",
"type": "main",
"index": 0
}
]
]
},
"Query & Docs1": {
"main": [
[
{
"node": "For Each Template...1",
"type": "main",
"index": 0
}
]
]
},
"Execution Data": {
"main": [
[
{
"node": "EventRouter",
"type": "main",
"index": 0
}
]
]
},
"Extract Result": {
"main": [
[
{
"node": "Wait1",
"type": "main",
"index": 0
}
]
]
},
"Filter Results": {
"main": [
[
{
"node": "Remove Dupes",
"type": "main",
"index": 0
}
]
]
},
"Generate Error": {
"main": [
[
{
"node": "Wait2",
"type": "main",
"index": 0
}
]
]
},
"Generate Event": {
"main": [
[
{
"node": "Generate",
"type": "main",
"index": 0
}
]
]
},
"Research Error": {
"main": [
[
{
"node": "Wait",
"type": "main",
"index": 0
}
]
]
},
"Research Event": {
"main": [
[
{
"node": "Research",
"type": "main",
"index": 0
}
]
]
},
"Extract Pending": {
"main": [
[
{
"node": "Extract Event",
"type": "main",
"index": 0
}
]
]
},
"Generate Result": {
"main": [
[
{
"node": "Wait2",
"type": "main",
"index": 0
}
]
]
},
"Get All Extract": {
"main": [
[
{
"node": "All Extract Done?",
"type": "main",
"index": 0
}
]
]
},
"Has Operations?": {
"main": [
[
{
"node": "Remove Duplicates",
"type": "main",
"index": 0
}
],
[
{
"node": "Response Empty1",
"type": "main",
"index": 0
}
]
]
},
"Query Templates": {
"main": [
[
{
"node": "Template to List",
"type": "main",
"index": 0
}
]
]
},
"Research Result": {
"main": [
[
{
"node": "Wait",
"type": "main",
"index": 0
}
]
]
},
"Results to List": {
"main": [
[
{
"node": "Filter Results",
"type": "main",
"index": 0
}
]
]
},
"Successful Runs": {
"main": [
[
{
"node": "Has Results?1",
"type": "main",
"index": 0
}
]
]
},
"Upload to Drive": {
"main": [
[
{
"node": "Response OK2",
"type": "main",
"index": 0
}
]
]
},
"Generate Pending": {
"main": [
[
{
"node": "Generate Event",
"type": "main",
"index": 0
}
]
]
},
"Get All Generate": {
"main": [
[
{
"node": "All Generate Done?",
"type": "main",
"index": 0
}
]
]
},
"Get All Research": {
"main": [
[
{
"node": "All Research Done?",
"type": "main",
"index": 0
}
]
]
},
"Research Pending": {
"main": [
[
{
"node": "Research Event",
"type": "main",
"index": 0
}
]
]
},
"Split Out Chunks": {
"main": [
[
{
"node": "Store Document Embeddings",
"type": "main",
"index": 0
}
]
]
},
"Template to List": {
"main": [
[
{
"node": "For Each Template...",
"type": "main",
"index": 0
}
]
]
},
"All Extract Done?": {
"main": [
[
{
"node": "Get All Generate",
"type": "main",
"index": 0
}
],
[
{
"node": "For Each Extract...",
"type": "main",
"index": 0
}
]
]
},
"Remove Duplicates": {
"main": [
[
{
"node": "Append Row",
"type": "main",
"index": 0
}
]
]
},
"Set Upload Fields": {
"main": [
[
{
"node": "Upload to Drive",
"type": "main",
"index": 0
}
]
]
},
"All Generate Done?": {
"main": [
[],
[
{
"node": "For Each Generate...",
"type": "main",
"index": 0
}
]
]
},
"All Research Done?": {
"main": [
[
{
"node": "Get All Extract",
"type": "main",
"index": 0
}
],
[
{
"node": "For Each Research...",
"type": "main",
"index": 0
}
]
]
},
"Get API Operations": {
"main": [
[
{
"node": "Contruct JSON Schema",
"type": "main",
"index": 0
}
]
]
},
"Default Data Loader": {
"ai_document": [
[
{
"node": "Store Document Embeddings",
"type": "ai_document",
"index": 0
}
]
]
},
"For Each Extract...": {
"main": [
[
{
"node": "Get All Generate",
"type": "main",
"index": 0
}
],
[
{
"node": "Extract Pending",
"type": "main",
"index": 0
}
]
]
},
"Contruct JSON Schema": {
"main": [
[
{
"node": "Set Upload Fields",
"type": "main",
"index": 0
}
]
]
},
"For Each Document...": {
"main": [
[
{
"node": "Response OK",
"type": "main",
"index": 0
}
],
[
{
"node": "Content Chunking @ 50k Chars",
"type": "main",
"index": 0
}
]
]
},
"For Each Generate...": {
"main": [
[],
[
{
"node": "Generate Pending",
"type": "main",
"index": 0
}
]
]
},
"For Each Research...": {
"main": [
[
{
"node": "Get All Extract",
"type": "main",
"index": 0
}
],
[
{
"node": "Research Pending",
"type": "main",
"index": 0
}
]
]
},
"For Each Template...": {
"main": [
[
{
"node": "Identify Service Products",
"type": "main",
"index": 0
}
],
[
{
"node": "Search in Relevant Docs",
"type": "main",
"index": 0
}
]
]
},
"Extract API Templates": {
"main": [
[
{
"node": "For Each Template...1",
"type": "main",
"index": 0
}
]
]
},
"For Each Template...1": {
"main": [
[
{
"node": "Extract API Operations",
"type": "main",
"index": 0
}
],
[
{
"node": "Search in Relevant Docs1",
"type": "main",
"index": 0
}
]
]
},
"Extract API Operations": {
"main": [
[
{
"node": "Merge Lists",
"type": "main",
"index": 0
}
]
]
},
"Has API Documentation?": {
"main": [
[
{
"node": "Has Results?3",
"type": "main",
"index": 0
}
]
]
},
"Scrape Webpage Contents": {
"main": [
[
{
"node": "Successful Runs",
"type": "main",
"index": 0
}
]
]
},
"Search in Relevant Docs": {
"main": [
[
{
"node": "Combine Docs",
"type": "main",
"index": 0
}
]
]
},
"Set Embedding Variables": {
"main": [
[
{
"node": "For Each Document...",
"type": "main",
"index": 0
}
]
]
},
"Embeddings Google Gemini": {
"ai_embedding": [
[
{
"node": "Store Document Embeddings",
"type": "ai_embedding",
"index": 0
}
]
]
},
"Execute Workflow Trigger": {
"main": [
[
{
"node": "Execution Data",
"type": "main",
"index": 0
}
]
]
},
"Google Gemini Chat Model": {
"ai_languageModel": [
[
{
"node": "Has API Documentation?",
"type": "ai_languageModel",
"index": 0
}
]
]
},
"Search in Relevant Docs1": {
"main": [
[
{
"node": "Combine Docs1",
"type": "main",
"index": 0
}
]
]
},
"Embeddings Google Gemini1": {
"ai_embedding": [
[
{
"node": "Search in Relevant Docs",
"type": "ai_embedding",
"index": 0
}
]
]
},
"Embeddings Google Gemini2": {
"ai_embedding": [
[
{
"node": "Search in Relevant Docs1",
"type": "ai_embedding",
"index": 0
}
]
]
},
"Google Gemini Chat Model1": {
"ai_languageModel": [
[
{
"node": "Extract API Operations",
"type": "ai_languageModel",
"index": 0
}
]
]
},
"Google Gemini Chat Model2": {
"ai_languageModel": [
[
{
"node": "Identify Service Products",
"type": "ai_languageModel",
"index": 0
}
]
]
},
"Identify Service Products": {
"main": [
[
{
"node": "Extract API Templates",
"type": "main",
"index": 0
}
]
]
},
"Store Document Embeddings": {
"main": [
[
{
"node": "For Each Document...",
"type": "main",
"index": 0
}
]
]
},
"Web Search For API Schema": {
"main": [
[
{
"node": "Has Results?",
"type": "main",
"index": 0
}
]
]
},
"Content Chunking @ 50k Chars": {
"main": [
[
{
"node": "Split Out Chunks",
"type": "main",
"index": 0
}
]
]
},
"When clicking ‘Test workflow’": {
"main": [
[
{
"node": "Get All Research",
"type": "main",
"index": 0
}
]
]
},
"Recursive Character Text Splitter1": {
"ai_textSplitter": [
[
{
"node": "Default Data Loader",
"type": "ai_textSplitter",
"index": 0
}
]
]
}
}
}
Reference in New Issue View Git Blame Copy Permalink