Transpile in the main process #945
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,36 @@ | ||
| The following features are considered "Experimental", in that their exact behavior is not yet considered locked. In general, we want to keep these features around long term (they are unlikely to disappear completely), but significant breaking changes are fair game until we are confident in the implementation. For core features, we generally try to accompany any breaking changes with codemods to ease the pain of upgrading, but users should not expect the same for experimental features. | ||
|
|
||
| Our recommendation is to go ahead and use these features *if* they provide significant benefits to your build. | ||
|
|
||
| # Limited Concurrency | ||
|
|
||
| Enabled via a CLI flag (`--concurrency=5`) or via the `ava` section of `package.json` (`concurrency:5`). | ||
|
There was a problem hiding this comment.
to make it copy-pasteable. |
||
|
|
||
| This will limit the number of test files spawned concurrently during the test run. The default is unlimited, and will cause all test files to be spawned at once. This can overload some systems if there are many, many test files. Especially on CI machines. Use the concurrency config to limit the number of processes. You may need to experiment to determine the optimal number of processes. | ||
|
|
||
| ## Caveats | ||
|
|
||
| Use of `test.only` does not work correctly when concurrency is limited. We think this is a solvable problem. | ||
|
There was a problem hiding this comment. Can you linkify the text here to the relevant issue? There was a problem hiding this comment. I know it's linked below, but would be useful to linkify it here too. |
||
|
|
||
| ## Stability | ||
|
|
||
| Fixing the `test.only` bug is the main barrier to this becoming a stable core feature. It's proven invaluable to many users, so we aren't likely to remove it unless some significantly better solution is discovered. | ||
|
|
||
| We believe limited concurrency is valuable enough to eventually become the default. We would love your help determining what the default concurrency limit should be (See [#966](https://github.com/avajs/ava/issues/966)). | ||
|
|
||
|
|
||
| # Precompiling Sources | ||
|
There was a problem hiding this comment. Worth adding a note in the Babel recipe about this and linking here? |
||
|
|
||
| AVA already precompiles your *test* files in the main process. This removes the need for loading `babel-register` in every child process just to compile the test. This has significant performance benefits. Unfortunately, this does not help users who want to write their project *source* files using the latest language features, as they often still need to use the `--require=babel-register` flag. | ||
|
|
||
| Enabled via a CLI flag (`--precompile`) or via the `ava` section of `packages.json` (`precompile:true`), this feature will cause AVA to precompile all required sources via Babel in the main process before launching the child process. If you use this feature, you **must** remove `--require=babel-register` from your config to see any benefits. | ||
|
There was a problem hiding this comment.
|
||
|
|
||
| ## Caveats | ||
|
|
||
| It does not work with relative requires. You must use strings in your require statements, expressions will not work. This is very similar to the restrictions imposed by packagers like `browserify`. If you need to make relative requires, you must still use `babel-register`, so this is unlikely to be helpful. | ||
|
|
||
| It only works with Babel 6. We still need to find a solution for typescript users. | ||
|
|
||
|
|
||
| ## Stability | ||
|
|
||
| This is a fairly new feature, and the details may change significantly. Initial tests indicate some very significant performance increases over `babel-register`. It is possible that `babel-register` could be further optimized with more a more aggressive caching mechanism that would obviate the need for this feature. As long as it proves to provide significant performance benefits, it is likely this feature will stick around in one way or another. | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,116 @@ | ||
| var path = require('path'); | ||
| var resolveFrom = require('resolve-from'); | ||
| var resolve = require('resolve'); | ||
| var md5hex = require('md5-hex'); | ||
| var detective = require('babel-plugin-detective'); | ||
| var fs = require('graceful-fs'); | ||
|
|
||
| var base = path.join(__dirname, '..', 'index.js'); | ||
|
|
||
| var _babelCore; | ||
|
|
||
| function babelCore() { | ||
| if (!_babelCore) { | ||
| _babelCore = require('babel-core'); | ||
| } | ||
| return _babelCore; | ||
| } | ||
|
|
||
| function Precompiler(cacheDir) { | ||
| this._cacheDir = cacheDir; | ||
| this._cache = {}; | ||
| } | ||
|
|
||
| Precompiler.prototype.cachePath = function (filename) { | ||
| return path.join(this._cacheDir, filename); | ||
| }; | ||
|
|
||
| Precompiler.prototype.buildEntry = function (filename) { | ||
| if (this._cache[filename]) { | ||
| return; | ||
| } | ||
|
|
||
| var entry = this._cache[filename] = {}; | ||
|
|
||
| var result = babelCore().transformFileSync(filename, { | ||
| plugins: [detective] | ||
|
There was a problem hiding this comment. I think if you don't merge this with the babelConfig already defined, this will implicitly be merged with the var config = objectAssign({}, this.babelConfig, {
plugins: this.babelConfig.plugins.concat(detective)
})There was a problem hiding this comment. See this repo failing with this config There was a problem hiding this comment.
Yes. That's the idea. That is only precompiling your sources. We already precompile your tests in There was a problem hiding this comment. @geowarin - see https://github.com/geowarin/repro-ava-precompile/issues/1 to fix yours. |
||
| }); | ||
|
|
||
| var hash = md5hex(result.code); | ||
| fs.writeFileSync(this.cachePath(hash + '.js'), result.code); | ||
|
|
||
| var metadata = detective.metadata(result); | ||
|
|
||
| var dependencies = this.normalizeDependencies(filename, metadata); | ||
|
|
||
| entry.hash = hash; | ||
| entry.dependencies = dependencies; | ||
|
|
||
| dependencies.forEach(this.buildEntry, this); | ||
| }; | ||
|
|
||
| Precompiler.prototype.normalizeDependencies = function (filename, metadata) { | ||
| if (metadata && metadata.expressions && (metadata.expressions === true || metadata.expressions.length)) { | ||
| console.warn(filename + ' has a dynamic require - precompilation may not work'); | ||
| } | ||
|
|
||
| if (!metadata || !metadata.strings || !metadata.strings.length) { | ||
| return []; | ||
| } | ||
|
|
||
| var dir = path.dirname(filename); | ||
|
|
||
| return metadata.strings | ||
| .filter(function (dep) { | ||
| return !resolve.isCore(dep); | ||
| }) | ||
| .map(function (dep) { | ||
| try { | ||
| return resolveFrom(dir, dep); | ||
| } catch (err) { | ||
| err.message = 'Failed to resolve dependency ' + JSON.stringify(dep) + ' for file ' + JSON.stringify(filename) + ' ' + err.message; | ||
| throw err; | ||
| } | ||
| }) | ||
| .filter(Boolean) | ||
| .filter(this.shouldTranspile, this); | ||
| }; | ||
|
|
||
| Precompiler.prototype.shouldTranspile = function (filename) { | ||
| return ( | ||
| (filename !== base) && | ||
| !/[\/\\]node_modules[\/\\]/.test(filename) && | ||
| /\.js$/.test(filename) | ||
| ); | ||
| }; | ||
|
|
||
| Precompiler.prototype.createHash = function (filename, hash, metadata) { | ||
| var hashMap = {}; | ||
| hashMap[filename] = hash; | ||
|
|
||
| var dependencies = this.normalizeDependencies(filename, metadata); | ||
|
|
||
| dependencies.forEach(this.buildEntry, this); | ||
|
|
||
| dependencies.forEach(function (filename) { | ||
| this.attach(filename, hashMap); | ||
| }, this); | ||
|
|
||
| return hashMap; | ||
| }; | ||
|
|
||
| Precompiler.prototype.attach = function (filename, hashMap) { | ||
| if (hashMap[filename] || !this._cache[filename]) { | ||
| return; | ||
| } | ||
|
|
||
| var entry = this._cache[filename]; | ||
|
|
||
| hashMap[filename] = entry.hash; | ||
|
|
||
| entry.dependencies.forEach(function (filename) { | ||
| this.attach(filename, hashMap); | ||
| }, this); | ||
| }; | ||
|
|
||
| module.exports = Precompiler; | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you add a heading
# Experimental Features?