Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parcel 2: Simple working JS packager #2239

Merged
merged 1 commit into from
Nov 2, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions packages/core/core/src/AssetGraph.js
Original file line number Diff line number Diff line change
Expand Up @@ -131,9 +131,10 @@ export default class AssetGraph extends Graph {

for (let assetNode of assetNodes) {
// TODO: dep should already have sourcePath
let depNodes = assetNode.value.dependencies.map(dep =>
nodeFromDep({...dep, sourcePath: file.filePath})
);
let depNodes = assetNode.value.dependencies.map(dep => {
dep.sourcePath = file.filePath;
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To discuss: do we even need this field on a dependency? An Asset node points to a Dependency in the graph, so we should be able to infer the file the dep came from based on its location in the graph.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's used to generate the id when creating the node (

id: `${dep.sourcePath}:${dep.moduleSpecifier}`,
). I like the idea of the value being the only input to create a node, but I suppose we could consider removing it if it really isn't useful and is taking up space.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I saw that. But we could pass the source path as a second argument to nodeFromDep to create the id... Perhaps it is useful information to store on a dep. I'm not sure.

return nodeFromDep(dep);
});
let {removed, added} = this.updateDownStreamNodes(assetNode, depNodes);
prunedFiles = prunedFiles.concat(getFilesFromGraph(removed));
newDepNodes = newDepNodes.concat(getDepNodesFromGraph(added));
Expand Down
1 change: 1 addition & 0 deletions packages/core/core/src/Parcel.js
Original file line number Diff line number Diff line change
Expand Up @@ -174,6 +174,7 @@ export default class Parcel {

let file = {filePath: resolvedPath};
if (signal && !signal.aborted) {
dep.resolvedPath = resolvedPath;
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure about mutating the dep here. Also we might not want to store this in the dep anyway - we could just figure it out based on where it is in the graph. That wouldn't work if we want to cache the resolution though, so perhaps we do want it in the end. Either way, it cannot be on the dep when it is initially created - it will need to be filled in later post-resolution.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we are going to add this field to the dep, it might make more sense to do it in the AssetGraph's updateDependency function.

That wouldn't work if we want to cache the resolution though

Are talking about the parcel cache or like the in memory "cache" on the v1 Resolver? Caching in memory is pretty much already taken care of by the way the graph is built. If a file is transformed a second time and finds some of the same dependencies it won't try to resolve them again. As for the parcel cache, I'm wondering if we want to cache resolved paths, at least to begin with. Seems like cache invalidation for that could get hairy.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm talking about the filesystem cache here. In Parcel 1 we do cache the resolved paths, and it increases performance very significantly.

let {newFile} = this.graph.updateDependency(dep, file);

if (newFile) {
Expand Down
38 changes: 37 additions & 1 deletion packages/packagers/js/src/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,45 @@
'use strict';

import {Packager} from '@parcel/plugin';
import fs from 'fs';

const PRELUDE = fs.readFileSync(__dirname + '/prelude.js');

export default new Packager({
async package(bundle) {
return bundle.assets.map(asset => asset.output.code).join('\n\n');
let assets = bundle.assets
.map((asset, i) => {
let deps = {};

for (let dep of asset.dependencies) {
let resolvedAsset = bundle.assets.find(
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't a great way to find the resolved ids for deps. It will be slower than necessary for sure. Perhaps another reason to have a bundle contain a Graph instead of a flat list of assets.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For sure, I can start looking into that. We'll have to be able serialize/deserialize graphs since the packagers will be running in workers. Do you think that could possibly end up making it not worth it performance wise?

a => a.filePath === dep.resolvedPath
);
deps[dep.moduleSpecifier] = resolvedAsset.id;
}

let wrapped = i === 0 ? '' : ',';
wrapped +=
JSON.stringify(asset.id) +
':[function(require,module,exports) {\n' +
(asset.output.code || '') +
'\n},';
wrapped += JSON.stringify(deps);
wrapped += ']';

return wrapped;
})
.join('');

return (
PRELUDE +
'({' +
assets +
'},{},' +
JSON.stringify([bundle.assets[0].id]) +
', ' +
'null' +
')'
);
}
});
117 changes: 117 additions & 0 deletions packages/packagers/js/src/prelude.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
// modules are defined as an array
// [ module function, map of requires ]
//
// map of requires is short require name -> numeric require
//
// anything defined in a previous bundle is accessed via the
// orig method which is the require for previous bundles

// eslint-disable-next-line no-global-assign
parcelRequire = function(modules, cache, entry, globalName) {
// Save the require from previous bundle to this closure if any
var previousRequire = typeof parcelRequire === 'function' && parcelRequire;
var nodeRequire = typeof require === 'function' && require;

function newRequire(name, jumped) {
if (!cache[name]) {
if (!modules[name]) {
// if we cannot find the module within our internal map or
// cache jump to the current global require ie. the last bundle
// that was added to the page.
var currentRequire =
typeof parcelRequire === 'function' && parcelRequire;
if (!jumped && currentRequire) {
return currentRequire(name, true);
}

// If there are other bundles on this page the require from the
// previous one is saved to 'previousRequire'. Repeat this as
// many times as there are bundles until the module is found or
// we exhaust the require chain.
if (previousRequire) {
return previousRequire(name, true);
}

// Try the node require function if it exists.
if (nodeRequire && typeof name === 'string') {
return nodeRequire(name);
}

var err = new Error("Cannot find module '" + name + "'");
err.code = 'MODULE_NOT_FOUND';
throw err;
}

localRequire.resolve = resolve;
localRequire.cache = {};

var module = (cache[name] = new newRequire.Module(name));

modules[name][0].call(
module.exports,
localRequire,
module,
module.exports,
this
);
}

return cache[name].exports;

function localRequire(x) {
return newRequire(localRequire.resolve(x));
}

function resolve(x) {
return modules[name][1][x] || x;
}
}

function Module(moduleName) {
this.id = moduleName;
this.bundle = newRequire;
this.exports = {};
}

newRequire.isParcelRequire = true;
newRequire.Module = Module;
newRequire.modules = modules;
newRequire.cache = cache;
newRequire.parent = previousRequire;
newRequire.register = function(id, exports) {
modules[id] = [
function(require, module) {
module.exports = exports;
},
{}
];
};

for (var i = 0; i < entry.length; i++) {
newRequire(entry[i]);
}

if (entry.length) {
// Expose entry point to Node, AMD or browser globals
// Based on https://github.com/ForbesLindesay/umd/blob/master/template.js
var mainExports = newRequire(entry[entry.length - 1]);

// CommonJS
if (typeof exports === 'object' && typeof module !== 'undefined') {
module.exports = mainExports;

// RequireJS
} else if (typeof define === 'function' && define.amd) {
define(function() {
return mainExports;
});

// <script>
} else if (globalName) {
this[globalName] = mainExports;
}
}

// Override the current require with this new one
return newRequire;
};