Support for embedded MP4s

[mstenta]

I was trying to embed MP4 files in a blog post PR (farmOS/farmOS-community-blog#16) and had trouble getting it to work. That PR is against the farmOS-community-blog repo, not this one, but this repository is responsible for converting those blogs posts into HTML files, so I think this is the right place for this issue.

I first attempted to embed the MP4 in the same way we embed images, like:

![Demo of Birth quick form](./birth.mp4)

... but that caused a build error:

3:24:23 PM: error Failed to retrieve metadata from image /opt/build/repo/farmOS.org/.cache/gatsby-source-git/blog/posts/2023/birth-quick-form/birth.mp4
3:24:23 PM: 
3:24:23 PM: 
3:24:23 PM:   Error: Input buffer contains unsupported image format

Full log: https://app.netlify.com/sites/farmos-community-blog-preview/deploys/646286a3eb759b0008fb9494

Then I tried with a <video> tag:

<video control>
  <source src="./birth.mp4" type="video/mp4">
</video>

... that went through the build process successfully, but MP4 file itself is not being copied into the build output, so it results in a 404 on the MP4 files and a blank area in the blog post where they would have been embedded.

Do we need to add a Gatsby.js plugin, or configure something differently, to make this possible?

[mstenta]

Experimenting in this PR: #76

I tried adding this plugin: https://www.gatsbyjs.com/plugins/gatsby-remark-video/

And using it to embed a test MP4 in markdown via the syntax described in the link above, but it just results in the code being printed to the page like:

video: ./test.mp4

It also doesn't put test.mp4 in the same directory in the finished build. And @paul121 noticed that it hasn't been updated in years, so abandoning that idea...

[mstenta]

@paul121 found this: https://www.gatsbyjs.com/plugins/gatsby-remark-videos/

It appears to be actively maintained, and purports to do exactly what I want. However, it requires a higher version of Gatsby than we are using:

$ npm install gatsby-remark-videos
npm ERR! code ERESOLVE
npm ERR! ERESOLVE unable to resolve dependency tree
npm ERR! 
npm ERR! While resolving: [email protected]
npm ERR! Found: [email protected]
npm ERR! node_modules/gatsby
npm ERR!   gatsby@"^3.14.1" from the root project
npm ERR! 
npm ERR! Could not resolve dependency:
npm ERR! peer gatsby@"^5.9.1" from [email protected]
npm ERR! node_modules/gatsby-remark-videos
npm ERR!   gatsby-remark-videos@"*" from the root project
npm ERR! 
npm ERR! Fix the upstream dependency conflict, or retry
npm ERR! this command with --force or --legacy-peer-deps
npm ERR! to accept an incorrect (and potentially broken) dependency resolution.
npm ERR! 
npm ERR! 
npm ERR! For a full report see:
npm ERR! /home/node/.npm/_logs/2023-05-18T17_04_21_763Z-eresolve-report.txt

npm ERR! A complete log of this run can be found in: /home/node/.npm/_logs/2023-05-18T17_04_21_763Z-debug-0.log

Curious if we can update Gatsby... but that should be a separate issue IMO.

[mstenta]

We talked about this on the dev call today and realized there are two separate issues/requests here:

1. The ability to embed MP4/Webm/etc in the same way that we embed images (eg: ![My cool video](./video.mp4))
2. The ability for Gatsby to automatically copy files into the finished build directory.

It seems like gatsby-remark-videos can solve 1, but it states on the plugin page that you need another plugin to solve 2:

Also make sure you have a plugin that copies the files you are referencing, for example gatsby-remark-copy-linked-files.

So maybe we can just use gatsby-remark-copy-linked-files and manually add <video> tags as a first step, and then start using gatsby-remark-videos after we upgrade Gatsby.

I'll give that a try in my PR.

[mstenta]

Bah! gatsby-remark-copy-linked-files requires Gatsby 5+ too!

$ npm install gatsby-remark-copy-linked-files
npm ERR! code ERESOLVE
npm ERR! ERESOLVE unable to resolve dependency tree
npm ERR! 
npm ERR! While resolving: [email protected]
npm ERR! Found: [email protected]
npm ERR! node_modules/gatsby
npm ERR!   gatsby@"^3.14.1" from the root project
npm ERR! 
npm ERR! Could not resolve dependency:
npm ERR! peer gatsby@"^5.0.0-next" from [email protected]
npm ERR! node_modules/gatsby-remark-copy-linked-files
npm ERR!   gatsby-remark-copy-linked-files@"*" from the root project
npm ERR! 
npm ERR! Fix the upstream dependency conflict, or retry
npm ERR! this command with --force or --legacy-peer-deps
npm ERR! to accept an incorrect (and potentially broken) dependency resolution.
npm ERR! 
npm ERR! 
npm ERR! For a full report see:
npm ERR! /home/node/.npm/_logs/2023-05-18T18_16_42_761Z-eresolve-report.txt

npm ERR! A complete log of this run can be found in: /home/node/.npm/_logs/2023-05-18T18_16_42_761Z-debug-0.log

[mstenta]

Chatted about this with @jgaehring - copying some of his thoughts here...

@jgaehring:

oh, just occurred to me, is the mp4 file in the farmOS repo? might need to tweak the config to make sure that file type is recognized when loading the source repo files
it would also be good to inspect if that file was actually included in the build

@mstenta:

ah yea i wondered that too

@jgaehring:

oh is that why you were using that copy linked file plugin as well?

@mstenta:

yea, according to https://www.gatsbyjs.com/plugins/gatsby-remark-videos/:

Also make sure you have a plugin that copies the files you are referencing, for example gatsby-remark-copy-linked-files.
We were talking about it on the dev call - and wondered if we had that already (because image files work) - but guessed that maybe the plugin for scaling images handles that?

@jgaehring:

ah, ok, we probably shouldn't add that one just for that sake

@mstenta:

oh ok - is there something in our code already that handles copying files?
how do the images get copied?

@jgaehring:

we're already using gatsby-source-git to copy the files
in conjunction w/ gatsby-remark-images
gatsby-source-git actually downloads the full source repo to .cache/gatsby-source-git/ then remark pulls from there to copy the files to the proper public or static directory and also formats the HTML for the embed

...

@jgaehring:

might just need to tweak the pattern glob option in gatsby-source-git: https://www.gatsbyjs.com/plugins/gatsby-source-git/?=gatsby-source-git
https://github.com/farmOS/farmOS.org/blob/main/lib/plugins.js#L29
I would have thought that line would ensure all files were fetched, and maybe it does and there's something else going wrong, but that'd be the first place to check, to make sure those files are ending up in the final build
and that source parameter in createSourceGitPlugins is the config for each source repo as specified in /site-data.js
and after ensuring that the file ends up in .cache, the next thing would be to make sure whichever remark plugin you're going with does indeed copy it to public
I could be wrong in assuming the remark plugins are meant to do that
that might just be what gatsby-remark-images does, but not necessarily all others

[mstenta]

Ah another thing I just learned: gatsby-remark-videos requires ffmpeg to be installed on the server performing the build. This is probably doable, but it's another layer of complexity.

I'm going to focus on problem 2 (copying the MP4 file during build) first.

[mstenta]

OK! I was able to install gatsby-remark-copy-linked-files version 3, and that worked to copy the test.mp4 file where it needed to go in my test PR! #76

Only problem now is that it's too big:

I imaging gatsby-remark-videos would handle resizing it. I might try that on top of the Gatsby5 PR (#79) next...

[mstenta]

I imaging gatsby-remark-videos would handle resizing it. I might try that on top of the Gatsby5 PR (#79) next...

Hmm for some reason I can't get gatsby-remark-videos to work properly... it seems that maybe the gatsby-remark-images plugin is taking precedence and replacing ![](./test.mp4) with an <img> tag before gatsby-remark-videos has a chance?

This is the markdown:

![](./test.mp4)

And this is the output:

<img src="/static/0efd4fe24d5c6baccb97a544830beded/test.mp4" alt="">

The good news is the MP4 actually works when you navigate to it directly!

[mstenta]

Only problem now is that it's too big:

Manually adding style="width: 100%; height: auto;" to the <video> tag works (per https://www.w3schools.com/Css/css_rwd_videos.asp):

[mstenta]

I opened this pull request to add gatsby-remark-copy-linked-files: #81

I think that is all that's needed in order to merge the blog pull request that motivated all of this: farmOS/farmOS-community-blog#16

We can consider adding gatsby-remark-videos in a follow-up, after the Gatsby 5 upgrade (#78).

[mstenta]

These two PRs have been merged, which serve as a first step (MVP) for supporting embedded MP4s:

• Install and configure gatsby-remark-copy-linked-files #81
• Set default height and width of videos #83

I will leave this open for now so we can consider adding gatsby-remark-videos as a next step (after #78 is merged).

[mstenta]

I will leave this open for now so we can consider adding gatsby-remark-videos as a next step (after #78 is merged).

#78 has been merged.