-
Notifications
You must be signed in to change notification settings - Fork 16
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CB-14140 WIP fs-extra instead of shelljs #14
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The usage of fs-extra.copySync
is a bit different than shelljs.copy
, particularly around directories.
fs.copySync
always copies the contents of a directory, so you never need /*
at the end of paths. It will also automatically ensure that the destination directory exists.
So if you want to copy template/www/*
to project/www/
:
shelljs.copy('-R', path.join(template, 'www', '*'), path.join(project, 'www'));
fs.copySync(path.join(template, 'www'), path.join(project, 'www'));
Note that if you want to copy an entire directory, you'll need to make sure the destination path is set up properly:
shelljs.copy('-R', path.join(template, 'www'), project);
fs.copySync(path.join(template, 'www'), path.join(project, 'www'));
index.js
Outdated
* @param {string} dst for copying | ||
* @return No return value | ||
*/ | ||
function copyFile (src, dst) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You should be able to safely use fs.copySync(src, dst)
in the two places this is getting called
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You should be able to safely use
fs.copySync(src, dst)
in the two places this is getting called
With the following change:
--- a/index.js
+++ b/index.js
@@ -235,7 +235,7 @@ module.exports = function (dir, optionalId, optionalName, cfg, extEvents) {
copyContentsIfNotExists(stockAssetPath('hooks'), path.join(dir, 'hooks'));
var configXmlExists = projectConfig(dir); // moves config to root if in www
if (!configXmlExists) {
- copyFile(stockAssetPath('config.xml'), path.join(dir, 'config.xml'));
+ fs.copySync(stockAssetPath('config.xml'), path.join(dir, 'config.xml'));
}
} catch (e) {
if (!dirAlreadyExisted) {
@@ -429,7 +429,7 @@ function linkFromTemplate (templateDir, projectDir) {
// if template/www/config.xml then copy to project/config.xml
copyDst = path.join(projectDir, 'config.xml');
if (!fs.existsSync(copyDst) && fs.existsSync(copySrc)) {
- copyFile(copySrc, projectDir);
+ fs.copySync(copySrc, projectDir);
}
}
I get the following result:
> [email protected] jasmine /Users/brodybits/Documents/cbwork/cordova-create
> jasmine spec/create.spec.js
Started
.............F.......
Failures:
1) create end-to-end when --link-to is provided when passed www folder should not move www/config.xml, only copy and update
Message:
Error: EPERM: operation not permitted, unlink '/var/folders/0j/4p0hqtr54914lbts82d2z9vh0000gn/T/cordova-create-tests-jR6vV0/TestBase'
Stack:
error properties: Object({ errno: -1, syscall: 'unlink', code: 'EPERM', path: '/var/folders/0j/4p0hqtr54914lbts82d2z9vh0000gn/T/cordova-create-tests-jR6vV0/TestBase' })
Error: EPERM: operation not permitted, unlink '/var/folders/0j/4p0hqtr54914lbts82d2z9vh0000gn/T/cordova-create-tests-jR6vV0/TestBase'
at Object.fs.unlinkSync (fs.js:1029:3)
at mayCopyFile (/Users/brodybits/Documents/cbwork/cordova-create/node_modules/fs-extra/lib/copy-sync/copy-sync.js:59:8)
at onFile (/Users/brodybits/Documents/cbwork/cordova-create/node_modules/fs-extra/lib/copy-sync/copy-sync.js:53:47)
at getStats (/Users/brodybits/Documents/cbwork/cordova-create/node_modules/fs-extra/lib/copy-sync/copy-sync.js:47:39)
at startCopy (/Users/brodybits/Documents/cbwork/cordova-create/node_modules/fs-extra/lib/copy-sync/copy-sync.js:37:10)
at Object.copySync (/Users/brodybits/Documents/cbwork/cordova-create/node_modules/fs-extra/lib/copy-sync/copy-sync.js:32:10)
at linkFromTemplate (/Users/brodybits/Documents/cbwork/cordova-create/index.js:432:12)
at /Users/brodybits/Documents/cbwork/cordova-create/index.js:229:21
at _fulfilled (/Users/brodybits/Documents/cbwork/cordova-create/node_modules/q/q.js:854:54)
at /Users/brodybits/Documents/cbwork/cordova-create/node_modules/q/q.js:883:30
21 specs, 1 failure
Finished in 3.952 seconds
Any clues?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You should be able to safely use
fs.copySync(src, dst)
in the two places this is getting called
It actually works for me in one place, just pushed the change. The following change still gives me the EPERM: operation not permitted
error:
--- a/index.js
+++ b/index.js
@@ -429,7 +429,7 @@ function linkFromTemplate (templateDir, projectDir) {
// if template/www/config.xml then copy to project/config.xml
copyDst = path.join(projectDir, 'config.xml');
if (!fs.existsSync(copyDst) && fs.existsSync(copySrc)) {
- copyFile(copySrc, projectDir);
+ fs.copySync(copySrc, projectDir);
}
}
e67ed99
to
b5ddb10
Compare
@dpogue I was able to use However the following change persistently fails with diff --git a/index.js b/index.js
index 9777240..d8f34d8 100644
--- a/index.js
+++ b/index.js
@@ -337,7 +337,7 @@ function copyTemplateFiles (templateDir, projectDir, isSubDir) {
// Copy each template file after filter
for (var i = 0; i < templateFiles.length; i++) {
copyPath = path.resolve(templateDir, templateFiles[i]);
- copyFolderWithSymlinks(copyPath, projectDir);
+ fs.copySync(copyPath, projectDir);
}
}
} For example:
I am starting to suspect an issue with how fs-extra deals with symbolic links, will investigate when I get a chance. I hope it would be possible to include these changes, since they are needed for me to rebase and rework #8 for CB-12397 without the undesired use of shelljs as discussed in apache/cordova-common#21 (comment) and apache/cordova-discuss#69 (comment). I would be happy to rebase or squash these changes if needed. |
4cb86ca
to
faa87b3
Compare
I just did one more bit of cleanup and squashed down to two commits:
As I said before there is still one place where I could not replace |
faa87b3
to
c11756a
Compare
Sorry guys for the bother, these changes will be part of a bigger PR for CB-12397. |
What's the matter? I think this should be its own PR. |
I would like to make a couple more updates before rebasing the change in #8: I was thinking it might be easiest to contribute all in a single cb-12397-updates-before-fix PR before issuing the updated change in #8. One less merge to wait for. I would also be happy to reopen to keep this change by itself if you think we can get it integrated within the next day or so. |
I see. I can't promise anything about integration, since I cannot merge anything yet and wouldn't want to rush anything either. I just made some changes to this branch of yours to make it work without shelljs. I'm about to review it right now, but all in all it looks good to me. Maybe just rename this PR and build your test updates on top of it? And I must say I'd prefer you save the var to const for a different day. Even if it bugs me too, but it's really not that urgent. I'd rather start a separate ES201x-ify initiative later. |
@raphinesse I can think of several ways forward:
Please let me know which way you would like to proceed. Reopening for now. |
I filed a PR against your PR branch: brody4hire#1. I think if you merge that, my changes should appear here 🤔 |
Thanks @raphinesse, the changes you contributed in brody4hire#1 look really good in general and pass the spec test suite on both my mac and Windows PC. I have the following questions: The changes pass the spec test suite whether or not the changes you proposed in eec80ce to fix the destination path are included. I wonder if this is because fs.copySync is able to deal with this difference or if we may be missing something in the test suite? How much should I squash the changes (if at all)? In general I would favor having one commit with the source changes, shown to pass the existing spec suite, and another commit with the spec test changes, would be open for input. I will definitely include @raphinesse as co-author (thanks for the link). |
If you merge/rebase/squash the PR to this branch using the GitHub interface, it should keep the author info intact without you needing to do anything extra 🙂 |
As for splitting things up, I'd be inclined to squash this whole PR to a single commit when merging to master, so I don't think it matters how many commits you add to the branch. |
I'm fine with having two commits. Seeing the existing tests pass is a valid point. Given the current set of changes I'd leave it at that. I think I have to call it a day/night now. I give this my go with my changes included. |
c11756a
to
f89950f
Compare
I just pushed the changes in 2 commits, naming myself and @raphinesse as co-authors. I would side with @raphinesse in favor of getting this merged in 2 commits but not so strongly in this case. I am becoming a bit concerned about the statement by @raphinesse:
In general I would rather have spec verify the existing behavior as it may be affected before applying the changes discussed here. I would really like to avoid risk breaking existing behavior if we can avoid it. I will take another look. |
f89950f
to
538a345
Compare
@brodybits I updated raphinesse:introduce-fs-extra-usage with a failing test for one of my changes |
0cd13c1 - looks like a nice find I will probably need a few hours to look at this one, thinking it would be good to update spec to check for this find and a few more artifacts before merging this change. P.S. +1 for 89211c2 to fix the test error message |
5053bf5
to
92a56ed
Compare
I have updated the changes with a better check for the www artifacts, along the lines of the test in 0cd13c1 proposed by @raphinesse. I have verified that both 68d4c9b (replace shelljs in index.js only) and 92a56ed (shelljs also completely replaced in spec & package.json) pass on both mac and Windows. The proposed changes are now in 5 commits and I would prefer to keep it that way. (I would also be fine if we squash the first 3 commits for spec updates and keep the last 2 commits separate.) P.S. The changes include a much simpler fix for the toExist matcher. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I left some remarks, but these might be a matter of taste.
Thanks @brodybits! For me, this is good to go 👍
PS: I would squash commits 2&3 and 4&5 but to be frank, I don't really care.
expect(path.join(project, 'www', '.gitignore')).not.toExist(); | ||
|
||
// Check that .npmignore does not exist inside of www | ||
expect(path.join(project, 'www', '.npmignore')).not.toExist(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would remove all mentions of .npmignore
from the specs. I think we should really treat it like any other file. To me, these explicit checks convey that we are handling these files specially.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would personally favor keeping this check to keep the related behavior as clear as possible.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
But our intended behavior is Treat it like any other file. So if a template wanted to define a .npmignore
, it could. IMO, the only tests related to .npmignore
(if any) should be to check that it is preserved in the project if we have one in the template (or a .gitignore
that is being transformed to one).
That being said, this is no show stopper for me.
spec/helpers.js
Outdated
@@ -84,7 +83,7 @@ beforeEach(function () { | |||
var result = {}; | |||
result.pass = fs.existsSync(testPath); | |||
|
|||
if (result.pass) { | |||
if (!result.pass) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's a shorter diff, but I find the code to be less clear.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am starting to agree with you, will do as you had proposed in 0cd13c1.
@raphinesse I will rework the first commit to do as you had proposed in 0cd13c1, would like to keep the check for .npmignore. I will definitely squash the first 3 commits with the spec updates. For the last 2 I am undecided whether or not to squash them together. I think we discussed before keeping them separate to show that the changes to index.js pass the existing spec suite, not sure how important it really is (very few changes needed to migrate spec from shelljs to fs-extra). |
I'd keep the first commit separate, since it's an actual fix. Regarding the last two: I think having been able to see the original specs pass was nice during development, but we don't need it in the history. |
(some redundant test code removed) Co-authored-by: Christopher J. Brody <[email protected]> Co-authored-by: Raphael von der Grün <[email protected]>
92a56ed
to
bbe4af9
Compare
Co-authored-by: Christopher J. Brody <[email protected]> Co-authored-by: Raphael von der Grün <[email protected]>
bbe4af9
to
754076d
Compare
Now squashed down to 3 commits as discussed, first commit with the toExists message fix as originally proposed by @raphinesse (with slight rewording of commit message). @raphinesse it would be great if you can give a final approval for this. I think we both want these commits to be kept as 3 commits in the final merge. P.S. @raphinesse I see your point that we should not consider |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. And yes, I would merge these commits without squashing.
I could merge, I just don't know how much approval one should await before merging 🤔
@dpogue Are you fine with this getting merged?
XXX TODO: USE fs-extra function instead, XXX WAITING FOR apache#14 to be merged NOTE: This implementation is a workaround for the npm .gitignore/.npmignore behavior discussed in: - npm/npm#1862 - npm/npm#3763 - npm/npm#7252
Another thing is that I would favor introducing this change in the next major release. While I do not expect this change to break anything I cannot be 100% sure. See also apache/cordova-common#21 (comment). |
I think if this is too dangerous to release in a minor version, then master is already on track of a new major version after my latest cleanup. |
Given the delay of the #8 I'm giving this another overhaul to remove any related changes so that we can merge this. Please don't merge until further notice. |
Waiting for #16 to be merged so this can be rebased onto it |
Well, actually this does not really depend on #16 so we could at least reduce this to the fs-extra commit. Can you do this @brodybits? The fs-extra part is fine by me. Would like one other approval before merging. |
Hmm...
To be honest I would rather see the test changes integrated first,
including the .giignore and .npmignore checks that I had proposed.
On Jun 4, 2018 5:26 AM, "Raphael von der Grün" <[email protected]> wrote:
Well, actually this does not really depend on #16
<#16> so we could at least
reduce this to the fs-extra commit. Can you do this @brodybits
<https://github.com/brodybits>?
The fs-extra part is fine by me. Would like one other approval before
merging.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#14 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ABfNUHhovsJoHG0LYkg-sAEztp7JrCSEks5t5P1VgaJpZM4UR3xM>
.
|
@brodybits I heard you and I still disagree with you on the As soon as #16 is merged you can create a separate PR to discuss your |
@raphinesse I would like to take the following approach:
|
@brodybits I'm fine with that. I just thought we might accelerate the overall process if we make this PR reviewable ASAP. |
Linking this change in progress to Apache CB-14140 (https://issues.apache.org/jira/browse/CB-14140). |
P.S. I would like to see the actual shelljs --> fs-extra commits marked for CB-14140 here and in the other components. |
Platforms affected
All
What does this PR do?
Introduce usage of
fs-extra
and isolateshelljs
usage, as a first step towards completely droppingshelljs
dependency as discussed in apache/cordova-common#21 (comment) and apache/cordova-discuss#69 (comment)What testing has been done on this change?
index.js
pass existingspec
suiteChecklist
Reported an issue in the JIRA databaseCommit message follows the format: "CB-3232: (android) Fix bug with resolving file paths", where CB-xxxx is the JIRA ID & "android" is the platform affected.Added automated test coverage as appropriate for this change.FUTURE TODO
As discussed in apache/cordova-common#21 (comment) and apache/cordova-discuss#69 (comment) it is desired to complete drop dependency on
shelljs
. My first attempt to replaceshell.cp
call withfs-extra
call did not pass the existing test suite. If someone else can resolve this issue I would be happy to amend this PR or maybe even drop it in favor of another solution.