Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Gatsby copy linked files, video tag support, a tag support and option to ignore default ignored file types #1571

Merged

Conversation

chiedo
Copy link

@chiedo chiedo commented Jul 19, 2017

I added the ability to copy linked images in HTML nodes and also the ability to override which file types are ignored so you can now copy png and svg files if you would like, etc.

@chiedo
Copy link
Author

chiedo commented Jul 19, 2017

video and a tag support coming soon.

@gatsbybot
Copy link
Collaborator

gatsbybot commented Jul 19, 2017

Deploy preview ready!

Built with commit 2421ddc

https://deploy-preview-1571--using-drupal.netlify.com

@gatsbybot
Copy link
Collaborator

gatsbybot commented Jul 19, 2017

Deploy preview failed.

Built with commit 76bbfc3f34cb85911d7c344a9537214efb3632a3

https://app.netlify.com/sites/gatsbyjs/deploys/5978b2550752d0375d5f691c

@gatsbybot
Copy link
Collaborator

gatsbybot commented Jul 19, 2017

Deploy preview ready!

Built with commit adee534

https://deploy-preview-1571--gatsbygram.netlify.com

@chiedo chiedo force-pushed the gatsby-copy-linked-files/more-features branch from 14a94c4 to 44707dc Compare July 20, 2017 12:35
@chiedo
Copy link
Author

chiedo commented Jul 20, 2017

I think this is officially ready. :) All features I wanted to add are now there.

@chiedo chiedo changed the title Gatsby copy linked files/more features Gatsby copy linked files, video tag support, a tag support and option to ignore default ignored file types Jul 20, 2017
{
resolve: 'gatsby-remark-copy-linked-files',
options: {
ignoreFileExtensions: ['png', 'jpg', 'jpeg', 'bmp', 'tiff'],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you add a comment here saying this is the default options?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So you don't need to add it.

Or even better, show the simple default way to add the plugin to gatsby-config and what to do if you want to copy images over (e.g. an empty ignoreFileExtensions.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes sir!

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me know what you think about the changes I just made.

@KyleAMathews
Copy link
Contributor

KyleAMathews commented Jul 25, 2017

Deploy preview failed.

Built with commit a37d48ac527a8fb669cb078578e547c6df0f74ba

https://app.netlify.com/sites/using-styled-components/deploys/5977c3e3424ef2111748eabf

@KyleAMathews
Copy link
Contributor

Deploy preview failed.

Built with commit 12a333472c97b480824fa36b3c720c274faf85c0

https://app.netlify.com/sites/using-remark/deploys/5977994b6f4c50758d7ec9b7

@KyleAMathews
Copy link
Contributor

KyleAMathews commented Jul 25, 2017

Deploy preview failed.

Built with commit adee534

https://app.netlify.com/sites/using-glamor/deploys/59dfebeca6188f3de3a76b0a

@KyleAMathews
Copy link
Contributor

Deploy preview failed.

Built with commit 12a333472c97b480824fa36b3c720c274faf85c0

https://app.netlify.com/sites/image-processing/deploys/5977994b6f4c50758d7ec9a8

@KyleAMathews
Copy link
Contributor

Deploy preview failed.

Built with commit 12a333472c97b480824fa36b3c720c274faf85c0

https://app.netlify.com/sites/using-contentful/deploys/5977994b6f4c50758d7ec9b2

@KyleAMathews
Copy link
Contributor

KyleAMathews commented Jul 25, 2017

Deploy preview failed.

Built with commit 12a333472c97b480824fa36b3c720c274faf85c0

https://app.netlify.com/sites/using-postcss-sass/deploys/597799496f4c50758d7ec9a1

@KyleAMathews
Copy link
Contributor

Getting a bunch of errors now when building gatsbyjs.org

3:50:36 PM: UNHANDLED REJECTION TypeError: Expected a string
at module.exports (/opt/build/repo/www/node_modules/is-absolute-url/index.js:4:9)
at module.exports (/opt/build/repo/www/node_modules/is-relative-url/index.js:5:10)
at Node.<anonymous> (/opt/build/repo/www/node_modules/gatsby-remark-copy-linked-files/index.js:176:21)
at initialize.exports.each (/opt/build/repo/www/node_modules/cheerio/lib/api/traversing.js:300:24)
at _callee2$ (/opt/build/repo/www/node_modules/gatsby-remark-copy-linked-files/index.js:175:24)
at tryCatch (/opt/build/repo/www/node_modules/regenerator-runtime/runtime.js:65:40)
at Generator.invoke [as _invoke] (/opt/build/repo/www/node_modules/regenerator-runtime/runtime.js:303:22)
at Generator.prototype.(anonymous function) [as next] (/opt/build/repo/www/node_modules/regenerator-runtime/runtime.js:117:21)
at step (/opt/build/repo/www/node_modules/babel-runtime/helpers/asyncToGenerator.js:17:30)
at /opt/build/repo/www/node_modules/babel-runtime/helpers/asyncToGenerator.js:35:14
at new Promise (/opt/build/repo/www/node_modules/core-js/library/modules/es6.promise.js:191:7)
at /opt/build/repo/www/node_modules/babel-runtime/helpers/asyncToGenerator.js:14:12
at /opt/build/repo/www/node_modules/gatsby-remark-copy-linked-files/index.js:454:20
at one (/opt/build/repo/www/node_modules/unist-util-visit/index.js:23:16)
at all (/opt/build/repo/www/node_modules/unist-util-visit/index.js:44:20)
at one (/opt/build/repo/www/node_modules/unist-util-visit/index.js:27:14)
3:50:36 PM: UNHANDLED REJECTION TypeError: Expected a string
at module.exports (/opt/build/repo/www/node_modules/is-absolute-url/index.js:4:9)
at module.exports (/opt/build/repo/www/node_modules/is-relative-url/index.js:5:10)
at Node.<anonymous> (/opt/build/repo/www/node_modules/gatsby-remark-copy-linked-files/index.js:353:21)
at initialize.exports.each (/opt/build/repo/www/node_modules/cheerio/lib/api/traversing.js:300:24)
at _callee2$ (/opt/build/repo/www/node_modules/gatsby-remark-copy-linked-files/index.js:352:22)
at tryCatch (/opt/build/repo/www/node_modules/regenerator-runtime/runtime.js:65:40)
at Generator.invoke [as _invoke] (/opt/build/repo/www/node_modules/regenerator-runtime/runtime.js:303:22)
at Generator.prototype.(anonymous function) [as next] (/opt/build/repo/www/node_modules/regenerator-runtime/runtime.js:117:21)
at step (/opt/build/repo/www/node_modules/babel-runtime/helpers/asyncToGenerator.js:17:30)
at /opt/build/repo/www/node_modules/babel-runtime/helpers/asyncToGenerator.js:35:14
at new Promise (/opt/build/repo/www/node_modules/core-js/library/modules/es6.promise.js:191:7)
at /opt/build/repo/www/node_modules/babel-runtime/helpers/asyncToGenerator.js:14:12
at /opt/build/repo/www/node_modules/gatsby-remark-copy-linked-files/index.js:454:20
at one (/opt/build/repo/www/node_modules/unist-util-visit/index.js:23:16)
at all (/opt/build/repo/www/node_modules/unist-util-visit/index.js:44:20)
at one (/opt/build/repo/www/node_modules/unist-util-visit/index.js:27:14)
3:50:46 PM: UNHANDLED REJECTION TypeError: Cannot read property 'slice' of undefined
at _callee3$ (/opt/build/repo/www/node_modules/gatsby-remark-images/index.js:258:51)
at tryCatch (/opt/build/repo/www/node_modules/regenerator-runtime/runtime.js:65:40)
at Generator.invoke [as _invoke] (/opt/build/repo/www/node_modules/regenerator-runtime/runtime.js:303:22)
at Generator.prototype.(anonymous function) [as next] (/opt/build/repo/www/node_modules/regenerator-runtime/runtime.js:117:21)
at step (/opt/build/repo/www/node_modules/babel-runtime/helpers/asyncToGenerator.js:17:30)
at /opt/build/repo/www/node_modules/babel-runtime/helpers/asyncToGenerator.js:35:14
at new Promise (/opt/build/repo/www/node_modules/core-js/library/modules/es6.promise.js:191:7)
at /opt/build/repo/www/node_modules/babel-runtime/helpers/asyncToGenerator.js:14:12
at /opt/build/repo/www/node_modules/gatsby-remark-images/index.js:338:26
at Promise._execute (/opt/build/repo/www/node_modules/bluebird/js/release/debuggability.js:300:9)
at Promise._resolveFromExecutor (/opt/build/repo/www/node_modules/bluebird/js/release/promise.js:483:18)
at new Promise (/opt/build/repo/www/node_modules/bluebird/js/release/promise.js:79:10)
at /opt/build/repo/www/node_modules/gatsby-remark-images/index.js:205:16
at Array.map (native)
at /opt/build/repo/www/node_modules/gatsby-remark-images/index.js:204:20
at tryCatcher (/opt/build/repo/www/node_modules/bluebird/js/release/util.js:16:23)
3:50:47 PM: UNHANDLED REJECTION TypeError: Expected a string
at module.exports (/opt/build/repo/www/node_modules/is-absolute-url/index.js:4:9)
at module.exports (/opt/build/repo/www/node_modules/is-relative-url/index.js:5:10)
at Node.<anonymous> (/opt/build/repo/www/node_modules/gatsby-remark-copy-linked-files/index.js:176:21)
at initialize.exports.each (/opt/build/repo/www/node_modules/cheerio/lib/api/traversing.js:300:24)
at _callee2$ (/opt/build/repo/www/node_modules/gatsby-remark-copy-linked-files/index.js:175:24)
at tryCatch (/opt/build/repo/www/node_modules/regenerator-runtime/runtime.js:65:40)
at Generator.invoke [as _invoke] (/opt/build/repo/www/node_modules/regenerator-runtime/runtime.js:303:22)
at Generator.prototype.(anonymous function) [as next] (/opt/build/repo/www/node_modules/regenerator-runtime/runtime.js:117:21)
at step (/opt/build/repo/www/node_modules/babel-runtime/helpers/asyncToGenerator.js:17:30)
at /opt/build/repo/www/node_modules/babel-runtime/helpers/asyncToGenerator.js:35:14
at new Promise (/opt/build/repo/www/node_modules/core-js/library/modules/es6.promise.js:191:7)
at /opt/build/repo/www/node_modules/babel-runtime/helpers/asyncToGenerator.js:14:12
at /opt/build/repo/www/node_modules/gatsby-remark-copy-linked-files/index.js:454:20
at one (/opt/build/repo/www/node_modules/unist-util-visit/index.js:23:16)
at all (/opt/build/repo/www/node_modules/unist-util-visit/index.js:44:20)
at one (/opt/build/repo/www/node_modules/unist-util-visit/index.js:27:14)
3:50:47 PM: UNHANDLED REJECTION TypeError: Expected a string

@chiedo
Copy link
Author

chiedo commented Jul 25, 2017

I'll look into it! Worked on all my sites. I'm assuming the deploy preview will tell me when I got it right?

@chiedo
Copy link
Author

chiedo commented Jul 25, 2017

Where can I find the gatsbyjs.org code base so I can do some test builds and debug? Grep and find turned up nothing. Sorry if I'm missing something obvious.

@KyleAMathews
Copy link
Contributor

It's in the www directory.

@chiedo chiedo force-pushed the gatsby-copy-linked-files/more-features branch 2 times, most recently from 5efa155 to 76bbfc3 Compare July 26, 2017 15:16
@chiedo
Copy link
Author

chiedo commented Jul 26, 2017

I haven't had any luck. If anyone else sees this, feel free to dig in.

This plugin works perfectly on my sites. I can't figure out why it isn't working on gatsby.org.

At this point the only errors I'm getting are gatsby-remark-images errors which doesn't make sense to me seeing as I've changed nothing in those files in this pull request.

It seems like there's some sort of connection between the two plugins even when this plugin ignores the file types that the remark-images plugin works on.

Not sure. I'm using this in production as a local plugin on my sites but can't figure out why this is breaking gatsby.org. Spent about 3 hours digging into it with no progress.

Open to answer any questions for anyone that sees this and has them

@KyleAMathews
Copy link
Contributor

If you remove this plugin does Gatsbyjs build? If you change the order of plugins does that change anything?

@chiedo
Copy link
Author

chiedo commented Jul 26, 2017

The issues remain if I comment out this plugin. The exact same error (all gatsby-remark-images errors after adding some try catches) still displays.

The issues go away if I comment out gatsby-remark-images.

That's why I'm confused. I haven't changed anything in remark-images as far as this branch is concerned... lol

It appears to be a gatsby-remark-images bug but I don't understand how that makes sense.

@KyleAMathews
Copy link
Contributor

Hmmm that doesn't make sense. Can you reproduce these errors if you switch to master?

@chiedo
Copy link
Author

chiedo commented Jul 26, 2017

First thing I tried this morning. Master worked fine.

Just tested again. Master is still error free.

How are you testing plugins by the way that aren't public yet? I'm manually overwriting node_modules to test how they would function from npm. I'm using the plugins directory fore my projects while waiting for pull requests to be merged but while testing before doing a pull request, is there an easier way than copying to and clearing node_modules?

@KyleAMathews
Copy link
Contributor

Update to master then?

On developing, gatsby-dev-cli https://www.gatsbyjs.org/docs/how-to-contribute/

@chiedo
Copy link
Author

chiedo commented Jul 27, 2017

On developing, gatsby-dev-cli https://www.gatsbyjs.org/docs/how-to-contribute/

Thanks, I'll have to read that section more closely!

Update to master then?

Same issue as soon as I add my new version of the gatsby-remark-copy-linked-files plugin. As soon as I change that, gatsby-remark-images starts having issues.

@KyleAMathews
Copy link
Contributor

Have you tried changing the order of the two plugins? If there's some sort of interaction between the two, changing the order probably would affect that.

@chiedo
Copy link
Author

chiedo commented Jul 27, 2017

Yeah, no difference. I've tried a lot and I'm honestly quite confused. I don't even see how interaction could occur. My code isn't doing anything that should be affected by any interaction. It's ultimately just reading files and then copying data.

Added

- ability to ignore specific file types and override defaults
- Ability to copy all image types if desired
- parsing of html a tags
- parsing of html video tags
@chiedo chiedo force-pushed the gatsby-copy-linked-files/more-features branch from 9c25e46 to 1ffe64c Compare July 30, 2017 16:25
@KyleAMathews KyleAMathews dismissed their stale review October 12, 2017 21:06

Not sure anymore what it was :-)

@KyleAMathews
Copy link
Contributor

Yeah!!! Tests are passing and gatsbyjs.org is building :-D

Merging this overly old PR now!

@KyleAMathews KyleAMathews merged commit 82e0034 into gatsbyjs:master Oct 12, 2017
@chiedo
Copy link
Author

chiedo commented Oct 12, 2017

YAY!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! :)

@chiedo chiedo deleted the gatsby-copy-linked-files/more-features branch October 12, 2017 22:38

- <img />
- <video />
- <a />
Copy link
Contributor

@jacobdeichert jacobdeichert Oct 17, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are not wrapped in code tags (<img/>) and are attempting to render on https://www.gatsbyjs.org/packages/gatsby-remark-copy-linked-files/

@@ -3,9 +3,19 @@ const isRelativeUrl = require(`is-relative-url`)
const fsExtra = require(`fs-extra`)
const path = require(`path`)
const _ = require(`lodash`)
const $ = require(`cheerio`)
const cheerio = require(`cheerio`)
const sizeOf = require(`image-size`)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This new dependency image-size was added yet not included in the package.json which causes this plugin to fail for me

@jacobdeichert
Copy link
Contributor

Hey guys, found two issues that I just ran into.

I'll submit a PR tonight or tomorrow.

KyleAMathews pushed a commit that referenced this pull request Oct 17, 2017
* Add new dependency

This dependency was starting to be used from #1571
but it was never added as a dependency.

* Wrap html tags as code
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants