Implement console.log in Truffle (#5687)

* Enable Ganache console log

 - Introduce a config namespace for `solidityLog` (see below)
 - Make managed Ganache instances aware of truffle-config because
   `connectOrStart` (ganache) needs to know relevant solidityLog
   settings to hook up Ganache events.
 - Use asynchronous swawn instead of synchronous option in core(console). It is
   necessary to interleave ganache's console stdout events (child process),
   with mocha's test runner (parent process). 
 - Modify the GanacheMixin in `chain.js` to register for and, forward
   Ganache's console log events using the `truffle.solidity.log`
   channel/topic.
 - Use chalk decorated `console.log` to display solidity console log which will
   allow users to redirect `truffle test` output to a file. The debug module
   uses `stderr` insted of stdout which would make redirect to file awkward and
   complicated.
 - Update truffle resolver to handle `@truffle/Console` or `@truffle/console`
   Truffle will include @ganache/console.log/console.sol in its asset and allow
   smart contract writers users to integrate console.log by importing
   "truffle/Console.sol". This dependency is managed through npm and is
   currently pinned. The user does not have to import any console.log packages.
   yay!
  - Add @truffle/[Cc]onsole.log as a resolver dependency. This dependency is
    pinned for the time being
  - Make `includeTruffleSources` the default resolver option.

Settings
========

The settings are namespaced in solidityLog
```
  solidityLog: {
    displayPrefix: string;
    preventConsoleLogMigration: boolean;
  }
```

- solidityLog.displayPrefix - string [ default: "" ]. it is the display prefix
  for all detected console-log messages. NOTE: there is some defensive guards
  that resolve, null, undefined to behave exactly like the default setting ()

- solidityLog.preventConsoleLogMigration - boolean [ default: false]. If set to
  true, `truffle migrate` will not allow deployment on Mainnet.

File changes
============

packages/config/src/configDefaults.ts
packages/config/test/unit.test.ts
  - add defaults and tests for
    solidityLog.{displayPrefix,preventConsoleLogMigration}

packages/core/lib/commands/develop/run.js
  - pass configOptions to connectOrStart

packages/core/lib/commands/migrate/runMigrations.js
  - add migration guards to log when a deployment set has consoleLog assets and
    additionally prevent deployment based on settings.

packages/core/lib/commands/test/run.js
  - hook up consoleLog for test command

packages/core/lib/console.js
  - use spawn instead of spawnSync so that child process commands can be
    printed in-order instead of buffering and printing when process ends. This
    allows consoleLog events from the child process to be interleaved correctly
    with test outline from parent process.
  - modify provision method to eliminate need for filter loop

packages/core/package.json
  - add JSONStream dependency

packages/environment/chain.js
packages/environment/develop.js
packages/environment/package.json
packages/environment/test/connectOrStartTest.js
  - hook into the "ganache:vm:tx:console.log" provider event and forward it
    across IPC infra as SolidityConsoleLogs
  - hook up client side IPC to listen for SolidityConsoleLogs
  - add @truffle/config and chalk dependencies
  - update tests to handle updated interfaces

packages/resolver/.eslintrc.json
  - add eslintrc settings for truffle packages

packages/core/lib/debug/compiler.js
packages/test/src/Test.ts
packages/test/src/TestRunner.ts
  - allow resolver to includeTruffleSources by default

packages/resolver/lib/sources/truffle/index.ts
packages/resolver/lib/resolver.ts
packages/resolver/test/truffle.js
packages/resolver/package.json
  - resolve "truffle/[cC]onsole.sol"
  - add tests for console.sol resolutions
  - add pinned @ganache/console.log/console.sol dependency
  - use for-of instead of forEach to utilize short circuit breakout. for-each
    will visit every item, even after a match has been found because there's no
    way to [1] break out of a for-each loop.
    >
    >There is no way to stop or break a forEach() loop other than by
    >throwing an exception. If you need such behavior, the forEach() method
    >is the wrong tool.
    >
    >Early termination may be accomplished with looping statements like for,
    >for...of, and for...in. Array methods like every(), some(), find(), and
    >findIndex() also stops iteration immediately when further iteration is
    >not necessary.

packages/test/src/SolidityTest.ts
  - include `console.sol` in set of truffle assets to deploy for testing

packages/truffle/package.json
  - add @ganache/console.log dependency

packages/truffle/test/scenarios/solidity_console/Printf.sol
packages/truffle/test/scenarios/solidity_console/printf.js
packages/truffle/test/sources/init/config-disable-migrate-false.js
packages/truffle/test/sources/init/config-disable-migrate-true.js
packages/truffle/test/sources/init/truffle-config.js
  - integration tests for consoleLog

packages/truffle/test/scenarios/sandbox.js
  - return tempDirPath when creating a sandbox. Printf tests depend on that.

packages/truffle/webpack.config.js
  - bundle @ganache/console.log/console.sol

Links
=====

1: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/forEach#:~:text=There%20is%20no%20way%20to,and%20for...in%20.

Co-authored-by: David Murdoch <[email protected]>
Co-authored-by: Micaiah Reid <[email protected]>

packages/config/src/configDefaults.ts

@@ -28,6 +28,10 @@ export const getInitialConfig = ({
       }
     },
     verboseRpc: false,
+    solidityLog: {
+      displayPrefix: "",
+      preventConsoleLogMigration: false
+    },
     gas: null,
     gasPrice: null,
     maxFeePerGas: null,
@@ -93,6 +97,7 @@ export const configProps = ({
     network() {},
     networks() {},
     verboseRpc() {},
+    solidityLog() {},
     build() {},
     resolver() {},
     artifactor() {},

packages/config/test/unit.test.ts

@@ -3,52 +3,65 @@ import TruffleConfig from "../dist";
 import { describe, it } from "mocha";
 
 describe("TruffleConfig unit tests", async () => {
-  describe("with", async () => {
-    let truffleConfig: TruffleConfig;
+  let truffleConfig: TruffleConfig;
 
-    beforeEach(() => {
+  describe("Defaults", async () => {
+    before(() => {
       truffleConfig = TruffleConfig.default();
     });
 
-    it("a simple object", async () => {
-      const expectedRandom = 42;
-      const expectedFoo = "bar";
-      const obj = {
-        random: expectedRandom,
-        foo: expectedFoo
+    it("solidityLog", () => {
+      const expectedSolidityLog = {
+        displayPrefix: "",
+        preventConsoleLogMigration: false
       };
-      const newConfig = truffleConfig.with(obj);
-      assert.equal(expectedRandom, newConfig.random);
-      assert.equal(expectedFoo, newConfig.foo);
+      assert.deepStrictEqual(truffleConfig.solidityLog, expectedSolidityLog);
     });
+  }),
+    describe("with", async () => {
+      beforeEach(() => {
+        truffleConfig = TruffleConfig.default();
+      });
 
-    it("overwrites a known property", () => {
-      const expectedProvider = { a: "propertyA", b: "propertyB" };
-      const newConfig = truffleConfig.with({ provider: expectedProvider });
-      assert.deepEqual(expectedProvider, newConfig.provider);
-    });
+      it("a simple object", async () => {
+        const expectedRandom = 42;
+        const expectedFoo = "bar";
+        const obj = {
+          random: expectedRandom,
+          foo: expectedFoo
+        };
+        const newConfig = truffleConfig.with(obj);
+        assert.strictEqual(expectedRandom, newConfig.random);
+        assert.strictEqual(expectedFoo, newConfig.foo);
+      });
 
-    it("ignores properties that throw", () => {
-      const expectedSurvivor = "BatMan";
-      const minefield = { who: expectedSurvivor };
-
-      const hits = ["boom", "pow", "crash", "zonk"];
-      hits.forEach(hit => {
-        Object.defineProperty(minefield, hit, {
-          get() {
-            throw new Error("BOOM!");
-          },
-          enumerable: true //must be enumerable
-        });
+      it("overwrites a known property", () => {
+        const expectedProvider = { a: "propertyA", b: "propertyB" };
+        const newConfig = truffleConfig.with({ provider: expectedProvider });
+        assert.deepStrictEqual(expectedProvider, newConfig.provider);
       });
 
-      const newConfig = truffleConfig.with(minefield);
+      it("ignores properties that throw", () => {
+        const expectedSurvivor = "BatMan";
+        const minefield = { who: expectedSurvivor };
 
-      //one survivor
-      assert.equal(expectedSurvivor, newConfig.who);
+        const hits = ["boom", "pow", "crash", "zonk"];
+        hits.forEach(hit => {
+          Object.defineProperty(minefield, hit, {
+            get() {
+              throw new Error("BOOM!");
+            },
+            enumerable: true //must be enumerable
+          });
+        });
+
+        const newConfig = truffleConfig.with(minefield);
 
-      //these jokers shouldn't be included
-      hits.forEach(hit => assert.equal(undefined, newConfig[hit]));
+        //one survivor
+        assert.strictEqual(expectedSurvivor, newConfig.who);
+
+        //these jokers shouldn't be included
+        hits.forEach(hit => assert.strictEqual(undefined, newConfig[hit]));
+      });
     });
-  });
 });

packages/core/lib/commands/develop/run.js

@@ -45,14 +45,23 @@ module.exports = async options => {
     "This mnemonic was created for you by Truffle. It is not secure.\n" +
     "Ensure you do not use it on production blockchains, or else you risk losing funds.";
 
-  const ipcOptions = { log: options.log };
+  const ipcOptions = {};
+
+  if (options.log) {
+    ipcOptions.log = options.log;
+  }
+
   const ganacheOptions = configureManagedGanache(
     config,
     customConfig,
     mnemonic
   );
 
-  const { started } = await Develop.connectOrStart(ipcOptions, ganacheOptions);
+  const { started } = await Develop.connectOrStart(
+    ipcOptions,
+    ganacheOptions,
+    config
+  );
   const url = `http://${ganacheOptions.host}:${ganacheOptions.port}/`;
 
   if (started) {

packages/core/lib/commands/migrate/runMigrations.js

@@ -1,6 +1,110 @@
+const { createReadStream } = require("fs");
+const { readdir } = require("fs/promises");
+const path = require("path");
+const JSONStream = require("JSONStream");
 const Migrate = require("@truffle/migrate").default;
+const TruffleError = require("@truffle/error");
+const os = require("os");
+const debug = require("debug")("migrate:run");
+
+// Search for Push1 (60) to Push(32) 7F + console.log
+//                      60 ---- 7F  C O N S O L E . L O G
+const consoleLogRex = /[67][0-9a-f]636F6e736F6c652e6c6f67/i;
+
+async function usesConsoleLog(artifactJson) {
+  const debugLog = debug.extend("test");
+  debugLog("Artifact: %o", artifactJson);
+
+  //create a parser to get the value of jsonpath .deployedBytecode
+  const parser = JSONStream.parse(["deployedBytecode"]);
+  const stream = createReadStream(artifactJson).pipe(parser);
+
+  return new Promise((resolve, reject) => {
+    stream.on("data", data => {
+      //JSONParse will emit the entire string/value
+      //so initiate stream cleanup here
+      stream.destroy();
+      const usesConsoleLog = consoleLogRex.test(data);
+      debugLog("usesConsoleLog:", usesConsoleLog);
+      resolve(usesConsoleLog);
+    });
+
+    stream.on("error", err => {
+      stream.destroy();
+      debugLog("onError: %o", err);
+      reject(err);
+    });
+  });
+}
+
+async function findArtifactsThatUseConsoleLog(buildDir) {
+  const debugLog = debug.extend("dirty-files");
+  const filenames = await readdir(buildDir);
+
+  const artifacts = [];
+  await Promise.allSettled(
+    filenames.map(async filename => {
+      if (filename.endsWith(".json")) {
+        try {
+          const itLogs = await usesConsoleLog(path.join(buildDir, filename));
+          if (itLogs) {
+            artifacts.push(filename);
+          }
+        } catch (e) {
+          debugLog("Promise failure: %o", e.message);
+        }
+      }
+    })
+  );
+  return artifacts;
+}
 
 module.exports = async function (config) {
+  const debugLog = debug.extend("guard");
+  // only check if deploying on MAINNET
+  // NOTE: this includes Ethereum Classic as well as Ethereum as they're only
+  // distinguishable by checking their chainIds, 2 and 1 respectively.
+  if (config.network_id === 1) {
+    debugLog("solidityLog guard for mainnet");
+    try {
+      const buildDir = config.contracts_build_directory;
+      const loggingArtifacts = await findArtifactsThatUseConsoleLog(buildDir);
+
+      debugLog(`${loggingArtifacts.length} consoleLog artifacts detected`);
+      debugLog(
+        "config.solidityLog.preventConsoleLogMigration: " +
+          config.solidityLog.preventConsoleLogMigration
+      );
+
+      if (loggingArtifacts.length) {
+        console.warn(
+          `${os.EOL}Solidity console.log detected in the following assets:`
+        );
+        console.warn(loggingArtifacts.join(", "));
+        console.warn();
+
+        if (config.solidityLog.preventConsoleLogMigration) {
+          throw new TruffleError(
+            "You are trying to deploy contracts that use console.log." +
+              os.EOL +
+              "Please fix, or disable this check by setting solidityLog.preventConsoleLogMigration to false" +
+              os.EOL
+          );
+        }
+      }
+    } catch (err) {
+      if (err instanceof TruffleError) throw err;
+
+      debugLog("Unexpected error %o:", err);
+      // Something went wrong while inspecting for console log.
+      // Log warning and skip the remaining logic in this branch
+      console.warn();
+      console.warn(
+        "Failed to detect Solidity console.log usage:" + os.EOL + err
+      );
+    }
+  }
+
   if (config.f) {
     return await Migrate.runFrom(config.f, config);
   } else {

packages/core/lib/commands/test/run.js

@@ -38,16 +38,21 @@ module.exports = async function (options) {
   }
 
   // Start managed ganache network
-  async function startGanacheAndRunTests(ipcOptions, ganacheOptions, config) {
+  async function startGanacheAndRunTests(
+    ipcOptions,
+    ganacheOptions,
+    truffleConfig
+  ) {
     const { disconnect } = await Develop.connectOrStart(
       ipcOptions,
-      ganacheOptions
+      ganacheOptions,
+      truffleConfig
     );
     const ipcDisconnect = disconnect;
-    await Environment.develop(config, ganacheOptions);
-    const { temporaryDirectory } = await copyArtifactsToTempDir(config);
+    await Environment.develop(truffleConfig, ganacheOptions);
+    const { temporaryDirectory } = await copyArtifactsToTempDir(truffleConfig);
     const numberOfFailures = await prepareConfigAndRunTests({
-      config,
+      config: truffleConfig,
       files,
       temporaryDirectory
     });
@@ -103,6 +108,7 @@ module.exports = async function (options) {
     );
 
     const ipcOptions = { network: "test" };
+
     numberOfFailures = await startGanacheAndRunTests(
       ipcOptions,
       ganacheOptions,