[pull] master from n8n-io:master (#2)

* Add Custom S3 Endpoint credentials * Add support for custom S3 endpoint in AWS S3 node * Add support for custom endpoint URL scheme (both HTTP and HTTPS) * Create generic S3 node * Revert AWS S3 node to its original state * ⚡ Add allowUnauthorizedCerts to Postgres-Node * ⚡ Added RAW data option for assets, entries, content types * ⚡ Fixed Contentful error parsing, added missing comma in entryDescription * ⚡ Minor improvements to Contentful-Node * 🐛 Do not overwrite accept header if set on HTTP Request Node * 🔖 Release [email protected] * ⬆️ Set [email protected] on n8n * 🔖 Release [email protected] * 🐛 Add missing rawBody for "application/x-www-form-urlencoded" * 🔖 Release [email protected] * 🐛 Fix bug that workflows did not get activated correctly * 🔖 Release [email protected] * ⚡ Add search operation to Zendesk-Node (n8n-io#900) * 🐛 Fix wrong authentication name (n8n-io#908) * 🐛 Fix comment create with assignee ID (repair field name) (n8n-io#906) * 🐛 Fix another wrong property on ClickUp-Node * 👷 Automatically run tslint * 👕 Fix lint issue * ⬆️ Set [email protected] on n8n-node-dev * 📚 Update copy and add information (n8n-io#910) * Update README.md * Update README.md * Update CONTRIBUTING.md * ✨ Add MQTT Trigger-Node (n8n-io#885) * Improvements to MQTT-node * ⚡ Small improvements done * ⚡ Improvements Co-authored-by: LEE SANG JUN <[email protected]> * ⚡ Minor improvements to MQTT Trigger-Node * ⚡ Update buffer code to not get deprecation message * Remove code duplication * Trim whitespaces in AWS credentials * Trim whitespaces in the AWS credentials in other nodes * ⚡ Minor improvements to S3-Node * 📚 Fix link to docker images * ✨ Recursive listing for FTP/SFTP (n8n-io#903) * ⚡ Add allowUnauthorizedCerts to Postgres-Node * ⚡ Added recursive directory listing for SFTP * ⚡ Added recursive listing for FTP * Removed unused imports * ⚡ Fixed creating an instance of both ftp/sftp both regardless of which is used Co-authored-by: Jan Oberhauser <[email protected]> * ⚡ Normalize FTP-Data Co-authored-by: Denis Palashevskii <[email protected]> Co-authored-by: Jan Oberhauser <[email protected]> Co-authored-by: Rupenieks <[email protected]> Co-authored-by: Ricardo Espinoza <[email protected]> Co-authored-by: vvvictor07 <[email protected]> Co-authored-by: Tanay Pant <[email protected]> Co-authored-by: LEE SANG JUN <[email protected]> Co-authored-by: Rupenieks <[email protected]>
dominion-solutions · Sep 2, 2020 · e30fc2b · e30fc2b
1 parent 2eaad29
commit e30fc2b
Show file tree

Hide file tree

Showing 33 changed files with 1,447 additions and 259 deletions.
diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml
@@ -23,5 +23,6 @@ jobs:
         npm run bootstrap
         npm run build --if-present
         npm test
+        npm run tslint
       env:
         CI: true
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -30,7 +30,7 @@ n8n is split up in different modules which are all in a single mono repository.
 
 The most important directories:
 
- - [/docker/image](/docker/image) - Dockerfiles to create n8n containers
+ - [/docker/image](/docker/images) - Dockerfiles to create n8n containers
  - [/docker/compose](/docker/compose) - Examples Docker Setups
  - [/packages](/packages) - The different n8n modules
  - [/packages/cli](/packages/cli) - CLI code to run front- & backend
@@ -119,6 +119,10 @@ To start n8n execute:
 npm run start
 ```
 
+To start n8n with tunnel:
+```
+./packages/cli/bin/n8n start --tunnel
+```
 
 ## Development Cycle
 
@@ -213,23 +217,7 @@ If you'd like to submit a new node, please go through the following checklist. T
 
 ## Extend Documentation
 
-All the files which get used in the n8n documentation on [https://docs.n8n.io](https://docs.n8n.io)
-can be found in the [/docs](https://github.com/n8n-io/n8n/tree/master/docs) folder. So all changes
-and additions can directly be made in there
-
-That the markdown docs look pretty we use [docsify](https://docsify.js.org). It is possible to test
-locally how it looks like rendered with the following commands:
-
-```bash
-# 1. Install docisify
-npm i docsify-cli -g
-
-# 2. Go into n8n folder (the same folder which contains this file). For example:
-cd /data/n8n
-
-# 3. Start docsificy
-docsify serve ./docs
-```
+The repository for the n8n documentation on https://docs.n8n.io can be found [here](https://github.com/n8n-io/n8n-docs).
 
 
 ## Contributor License Agreement

diff --git a/README.md b/README.md
@@ -2,7 +2,7 @@
 
 ![n8n.io - Workflow Automation](https://raw.githubusercontent.com/n8n-io/n8n/master/assets/n8n-logo.png)
 
-n8n is a free and open [fair-code](http://faircode.io) licensed node based Workflow Automation Tool. It can be self-hosted, easily extended, and so also used with internal tools.
+n8n is an extendable workflow automation tool. With a [fair-code](http://faircode.io) distribution model, n8n will always have visible source code, be available to self-host, and allow you to add your own custom functions, logic and apps. n8n's node-based approach makes it highly versatile, enabling you to connect anything to everything. 
 
 <a href="https://raw.githubusercontent.com/n8n-io/n8n/master/assets/n8n-screenshot.png"><img src="https://raw.githubusercontent.com/n8n-io/n8n/master/assets/n8n-screenshot.png" width="550" alt="n8n.io - Screenshot"></a>
 
@@ -16,7 +16,7 @@ received or lost a star.
 
 ## Available integrations
 
-n8n has 100+ different nodes to automate workflows. The list can be found on: [https://n8n.io/nodes](https://n8n.io/nodes)
+n8n has 170+ different nodes to automate workflows. The list can be found on: [https://n8n.io/nodes](https://n8n.io/nodes)
 
 
 ## Documentation
@@ -67,16 +67,15 @@ check out our job posts:
 
 **Short answer:** It means "nodemation" and it is pronounced as n-eight-n.
 
-**Long answer:** I get that question quite often (more often than I expected)
+**Long answer:** "I get that question quite often (more often than I expected)
 so I decided it is probably best to answer it here. While looking for a
 good name for the project with a free domain I realized very quickly that all the
 good ones I could think of were already taken. So, in the end, I chose
-nodemation. "node-" in the sense that it uses a Node-View and that it uses
-Node.js and "-mation" for "automation" which is what the project is supposed to help with.
+nodemation. 'node-' in the sense that it uses a Node-View and that it uses
+Node.js and '-mation' for 'automation' which is what the project is supposed to help with.
 However, I did not like how long the name was and I could not imagine writing
 something that long every time in the CLI. That is when I then ended up on
-"n8n". Sure does not work perfectly but does neither for Kubernetes (k8s) and
-did not hear anybody complain there. So I guess it should be ok.
+'n8n'." - **Jan Oberhauser, Founder and CEO, n8n.io**
 
 
 
@@ -88,6 +87,6 @@ Have you found a bug :bug: ? Or maybe you have a nice feature :sparkles: to cont
 
 ## License
 
-n8n is [fair-code](http://faircode.io) licensed under [**Apache 2.0 with Commons Clause**](https://github.com/n8n-io/n8n/blob/master/packages/cli/LICENSE.md)
+n8n is [fair-code](http://faircode.io) licensed under [**Apache 2.0 with Commons Clause**](https://github.com/n8n-io/n8n/blob/master/packages/cli/LICENSE.md).
 
-Additional information about license can be found in the [FAQ](https://docs.n8n.io/#/faq?id=license)
+Additional information about license can be found in the [FAQ](https://docs.n8n.io/#/faq?id=license).
diff --git a/packages/cli/package.json b/packages/cli/package.json
@@ -1,6 +1,6 @@
 {
     "name": "n8n",
-    "version": "0.79.0",
+    "version": "0.79.3",
     "description": "n8n Workflow Automation Tool",
     "license": "SEE LICENSE IN LICENSE.md",
     "homepage": "https://n8n.io",
@@ -102,7 +102,7 @@
         "mysql2": "^2.0.1",
         "n8n-core": "~0.43.0",
         "n8n-editor-ui": "~0.55.0",
-        "n8n-nodes-base": "~0.74.0",
+        "n8n-nodes-base": "~0.74.1",
         "n8n-workflow": "~0.39.0",
         "oauth-1.0a": "^2.2.6",
         "open": "^7.0.0",

diff --git a/packages/cli/src/ActiveWorkflowRunner.ts b/packages/cli/src/ActiveWorkflowRunner.ts
@@ -52,29 +52,24 @@ export class ActiveWorkflowRunner {
 		// so intead of pulling all the active wehhooks just pull the actives that have a trigger
 		const workflowsData: IWorkflowDb[] = await Db.collections.Workflow!.find({ active: true }) as IWorkflowDb[];
 
+		// Clear up active workflow table
+		await Db.collections.Webhook?.clear();
+
 		this.activeWorkflows = new ActiveWorkflows();
 
 		if (workflowsData.length !== 0) {
 			console.log('\n ================================');
 			console.log('   Start Active Workflows:');
 			console.log(' ================================');
 
-			const nodeTypes = NodeTypes();
-
 			for (const workflowData of workflowsData) {
-
-				const workflow = new Workflow({ id: workflowData.id.toString(), name: workflowData.name, nodes: workflowData.nodes, connections: workflowData.connections, active: workflowData.active, nodeTypes, staticData: workflowData.staticData, settings: workflowData.settings});
-
-				if (workflow.getTriggerNodes().length !== 0
-					|| workflow.getPollNodes().length !== 0) {
-					console.log(`   - ${workflowData.name}`);
-					try {
-						await this.add(workflowData.id.toString(), workflowData);
-						console.log(`     => Started`);
-					} catch (error) {
-						console.log(`     => ERROR: Workflow could not be activated:`);
-						console.log(`               ${error.message}`);
-					}
+				console.log(`   - ${workflowData.name}`);
+				try {
+					await this.add(workflowData.id.toString(), workflowData);
+					console.log(`     => Started`);
+				} catch (error) {
+					console.log(`     => ERROR: Workflow could not be activated:`);
+					console.log(`               ${error.message}`);
 				}
 			}
 		}
@@ -87,14 +82,18 @@ export class ActiveWorkflowRunner {
 	 * @memberof ActiveWorkflowRunner
 	 */
 	async removeAll(): Promise<void> {
-		if (this.activeWorkflows === null) {
-			return;
+		const activeWorkflowId: string[] = [];
+
+		if (this.activeWorkflows !== null) {
+			// TODO: This should be renamed!
+			activeWorkflowId.push.apply(activeWorkflowId, this.activeWorkflows.allActiveWorkflows());
 		}
 
-		const activeWorkflows = this.activeWorkflows.allActiveWorkflows();
+		const activeWorkflows = await this.getActiveWorkflows();
+		activeWorkflowId.push.apply(activeWorkflowId, activeWorkflows.map(workflow => workflow.id));
 
 		const removePromises = [];
-		for (const workflowId of activeWorkflows) {
+		for (const workflowId of activeWorkflowId) {
 			removePromises.push(this.remove(workflowId));
 		}
 
@@ -183,7 +182,7 @@ export class ActiveWorkflowRunner {
 	 * @memberof ActiveWorkflowRunner
 	 */
 	getActiveWorkflows(): Promise<IWorkflowDb[]> {
-		return Db.collections.Workflow?.find({ select: ['id'] }) as Promise<IWorkflowDb[]>;
+		return Db.collections.Workflow?.find({ where: { active: true }, select: ['id'] }) as Promise<IWorkflowDb[]>;
 	}
 
 

diff --git a/packages/cli/src/Server.ts b/packages/cli/src/Server.ts
@@ -249,7 +249,7 @@ class App {
 				if (token === undefined || token === '') {
 					return ResponseHelper.jwtAuthAuthorizationError(res, "Missing token");
 				}
-				if (jwtHeaderValuePrefix != '' && token.startsWith(jwtHeaderValuePrefix)) {
+				if (jwtHeaderValuePrefix !== '' && token.startsWith(jwtHeaderValuePrefix)) {
 					token = token.replace(jwtHeaderValuePrefix + ' ', '').trimLeft();
 				}
 
@@ -298,7 +298,7 @@ class App {
 		this.app.use((req: express.Request, res: express.Response, next: express.NextFunction) => {
 			(req as ICustomRequest).parsedUrl = parseUrl(req);
 			// @ts-ignore
-			req.rawBody = new Buffer('', 'base64');
+			req.rawBody = Buffer.from('', 'base64');
 			next();
 		});
 
@@ -340,7 +340,12 @@ class App {
 		}));
 
 		//support application/x-www-form-urlencoded post data
-		this.app.use(bodyParser.urlencoded({ extended: false }));
+		this.app.use(bodyParser.urlencoded({ extended: false,
+			verify: (req, res, buf) => {
+				// @ts-ignore
+				req.rawBody = buf;
+			}
+		}));
 
 		if (process.env['NODE_ENV'] !== 'production') {
 			this.app.use((req: express.Request, res: express.Response, next: express.NextFunction) => {

diff --git a/packages/cli/src/databases/mongodb/migrations/1592679094242-WebhookModel.ts b/packages/cli/src/databases/mongodb/migrations/1592679094242-WebhookModel.ts
@@ -2,20 +2,6 @@ import {
 	MigrationInterface,
 } from 'typeorm';
 
-import {
-	IWorkflowDb,
-	NodeTypes,
-	WebhookHelpers,
-} from '../../..';
-
-import {
-	Workflow,
-} from 'n8n-workflow/dist/src/Workflow';
-
-import {
-	IWebhookDb,
-} from '../../../Interfaces';
-
 import * as config from '../../../../config';
 
 import {
@@ -27,26 +13,6 @@ export class WebhookModel1592679094242 implements MigrationInterface {
 
 	async up(queryRunner: MongoQueryRunner): Promise<void> {
 		const tablePrefix = config.get('database.tablePrefix');
-		const workflows = await queryRunner.cursor( `${tablePrefix}workflow_entity`, { active: true }).toArray() as IWorkflowDb[];
-		const data: IWebhookDb[] = [];
-		const nodeTypes = NodeTypes();
-		for (const workflow of workflows) {
-			const workflowInstance = new Workflow({ id: workflow.id as string, name: workflow.name, nodes: workflow.nodes, connections: workflow.connections, active: workflow.active, nodeTypes, staticData: workflow.staticData, settings: workflow.settings });
-			const webhooks = WebhookHelpers.getWorkflowWebhooksBasic(workflowInstance);
-			for (const webhook of webhooks) {
-				data.push({
-					workflowId: workflowInstance.id as string,
-					webhookPath: webhook.path,
-					method: webhook.httpMethod,
-					node: webhook.node,
-				 });
-			}
-		}
-
-		if (data.length !== 0) {
-			await queryRunner.manager.insertMany(`${tablePrefix}webhook_entity`, data);
-		}
-
 		await queryRunner.manager.createCollectionIndex(`${tablePrefix}webhook_entity`, ['webhookPath',  'method'], { unique: true, background: false });
 	}
 

diff --git a/packages/cli/src/databases/mysqldb/migrations/1592447867632-WebhookModel.ts b/packages/cli/src/databases/mysqldb/migrations/1592447867632-WebhookModel.ts
@@ -5,51 +5,13 @@ import {
 
 import * as config from '../../../../config';
 
-import {
-	IWorkflowDb,
-	NodeTypes,
-	WebhookHelpers,
-} from '../../..';
-
-import {
-	Workflow,
-} from 'n8n-workflow';
-
-import {
-	IWebhookDb,
-} from '../../../Interfaces';
-
 export class WebhookModel1592447867632 implements MigrationInterface {
 	name = 'WebhookModel1592447867632';
 
 	async up(queryRunner: QueryRunner): Promise<void> {
 		const tablePrefix = config.get('database.tablePrefix');
 
 		await queryRunner.query(`CREATE TABLE IF NOT EXISTS ${tablePrefix}webhook_entity (workflowId int NOT NULL, webhookPath varchar(255) NOT NULL, method varchar(255) NOT NULL, node varchar(255) NOT NULL, PRIMARY KEY (webhookPath, method)) ENGINE=InnoDB`);
-
-		const workflows = await queryRunner.query(`SELECT * FROM ${tablePrefix}workflow_entity WHERE active=true`) as IWorkflowDb[];
-		const data: IWebhookDb[] = [];
-		const nodeTypes = NodeTypes();
-		for (const workflow of workflows) {
-			const workflowInstance = new Workflow({ id: workflow.id as string, name: workflow.name, nodes: workflow.nodes, connections: workflow.connections, active: workflow.active, nodeTypes, staticData: workflow.staticData, settings: workflow.settings });
-			const webhooks = WebhookHelpers.getWorkflowWebhooksBasic(workflowInstance);
-			for (const webhook of webhooks) {
-				data.push({
-					workflowId: workflowInstance.id as string,
-					webhookPath: webhook.path,
-					method: webhook.httpMethod,
-					node: webhook.node,
-				 });
-			}
-		}
-
-		if (data.length !== 0) {
-			await queryRunner.manager.createQueryBuilder()
-			.insert()
-			.into(`${tablePrefix}webhook_entity`)
-			.values(data)
-			.execute();
-		}
 	}
 
 	async down(queryRunner: QueryRunner): Promise<void> {

diff --git a/packages/cli/src/databases/postgresdb/migrations/1589476000887-WebhookModel.ts b/packages/cli/src/databases/postgresdb/migrations/1589476000887-WebhookModel.ts
@@ -3,20 +3,6 @@ import {
 	QueryRunner,
 } from 'typeorm';
 
-import {
-	IWorkflowDb,
-	NodeTypes,
-	WebhookHelpers,
-} from '../../..';
-
-import {
-	Workflow,
-} from 'n8n-workflow';
-
-import {
-	IWebhookDb,
-} from '../../../Interfaces';
-
 import * as config from '../../../../config';
 
 export class WebhookModel1589476000887 implements MigrationInterface {
@@ -31,30 +17,6 @@ export class WebhookModel1589476000887 implements MigrationInterface {
 		}
 
 		await queryRunner.query(`CREATE TABLE ${tablePrefix}webhook_entity ("workflowId" integer NOT NULL, "webhookPath" character varying NOT NULL, "method" character varying NOT NULL, "node" character varying NOT NULL, CONSTRAINT "PK_${tablePrefixIndex}b21ace2e13596ccd87dc9bf4ea6" PRIMARY KEY ("webhookPath", "method"))`, undefined);
-
-		const workflows = await queryRunner.query(`SELECT * FROM ${tablePrefix}workflow_entity WHERE active=true`) as IWorkflowDb[];
-		const data: IWebhookDb[] = [];
-		const nodeTypes = NodeTypes();
-		for (const workflow of workflows) {
-			const workflowInstance = new Workflow({ id: workflow.id as string, name: workflow.name, nodes: workflow.nodes, connections: workflow.connections, active: workflow.active, nodeTypes, staticData: workflow.staticData, settings: workflow.settings });
-			const webhooks = WebhookHelpers.getWorkflowWebhooksBasic(workflowInstance);
-			for (const webhook of webhooks) {
-				data.push({
-					workflowId: workflowInstance.id as string,
-					webhookPath: webhook.path,
-					method: webhook.httpMethod,
-					node: webhook.node,
-				 });
-			}
-		}
-
-		if (data.length !== 0) {
-			await queryRunner.manager.createQueryBuilder()
-			.insert()
-			.into(`${tablePrefix}webhook_entity`)
-			.values(data)
-			.execute();
-		}
 	}
 
 	async down(queryRunner: QueryRunner): Promise<void> {