Ability to deploy non-html files to separate domain

[davidhouweling]

Doing some search, there was an initial issue which was closed by the OP: #1369.

However, the work I'm presently doing means the gatsby page will sit behind a company wide domain with the assets ideally sitting behind our static domain. Think... mywebiste.com/gatsby-page should reference the non-html files that sit at mystaticcontent.com. (so something like mystaticcontent.com/gatsby-page/js-file.js).

I understand that from your point of view it should all just sit in 1 domain, but we have a very large website with a lot of applications and routing being fronted by a load balancer.

At present, it isn't feasible as well because the URL we want to use for the present page we have build is a top level URL, such as mywebsite.com/gatsby-page, but we can't really serve the assets from the top level domain as well.

We did look at the gatsby documentation which has path prefix (so not allowing us to put it in a different domain altogether, but at least trialing putting the assets somewhere else), which puts the whole project under a certain path, but this also does it for react router. We added two rules into our load balancer, one for the URL we want the HTML page to resolve to, and another for the path prefix resolving to the public folder. But when viewing the page in the browser, react-router kicks in and doesn't understand the URL and send you off to the 404 page defined with gatsby.

We also looked into using modifyWebpackConfig and setting the public path just for webpack, but this doesn't seem to be picked up as part of headComponents in the html.jsx file (as it is still setting the link refs to /some-js-file.js.

[KyleAMathews]

Wouldn't using mywebsite.com/gatsby-page/ and setting the pathPrefix to /gatsby-page work?

[davidhouweling]

It isn't flexible enough really. Imagine you had the following pages...

mywebsite.com/page1
mywebsite.com/page2
mywebsite.com/path1/page3
mywebsite.com/path2/path3/page4

It puts all assets with a path as / at the start. So it means all assets are root level.

If you then use pathPrefix... that is also given to react router for prod build, so it would look like this...

mywebsite.com/pathPrefix/page1
mywebsite.com/pathPrefix/page2
mywebsite.com/pathPrefix/path1/page3
mywebsite.com/pathPrefix/path2/path3/page4

Meaning the URLs we really wanted won't work, as the html renders, then react-router kicks in and says it doesn't know the URL it's on, and goes to 404.

[jbrantly]

Throwing my hat into the ring, I have this need as well. Totally understand the argument that with a static site generator everything should be hosted on the CDN, but sometimes that's not practical. Would be great to have a configuration to load JS components and such from a different host.

[vovacodes]

@KyleAMathews if you can advise on the API you think can work for it, I'd be happy to contribute this feature. This is a dealbreaker for our project as well.

[KyleAMathews]

@wzrdzl could you write in detail what you need for your project?

[vovacodes]

I need to be able to serve the /{page}/index.html from my own nginx, e.g. https://example.com/{page}/ whereas all the resources (js files and other static) from a CDN, e.g. https://my-fancy-cdn.com/resources/js-file-for-my-page.js.

In our SPA we simply use webpack's:

publicPath: `//akamaized.net/w/${version}/resources/`,

so we need something similar for the static website

[KyleAMathews]

@wzrdzl Gatsby is using webpack as well so a Gatsby plugin could probably make the same tweaks to our webpack setup that you're doing for your SPA?

[Rob3rtH]

@KyleAMathews This is also a sticking point for my use-case as well, need pretty much what @wzrdzl describes.

HTML in:
/{page}/index.html
/{page}/{sub-page}/index.html

Should come from my server via:
https://my-website.com/{page}/
https://my-website.com/{page}/{sub-page}/

All assets (js, images etc) should come from (probably) something like:
https://my-cdn.com/{whatever asset}
https://my-cdn.com/static/{whatever asset}

[KyleAMathews]

@Rob3rtH Gatsby is designed so you can put all files on the CDN. Perhaps give that a go?

[szimek]

This would help us as well, though in our case it could be a separate domain or a path.

Our path structure currently looks like this: /:locale/*. We've got an old Angular app that handles almost all requests, with a few exceptions. One of these exceptions is /:locale/blog/*, which redirects to WP Engine, where we host our current (soon to be replaced by Gatsby) Wordpress blog.

Our blog is translated into 6 languages. When generating pages in Gatsby, we simply generate paths like /en/blog/posts/:slug, /de/blog/posts/:slug etc. It all worked fine, till we tried to integrate our new blog together with our old Angular app, because Gatsby generates some common files (app-[hash].js, commons-[hash].js etc) in the root folder and expects them to be accessible at root path, e.g. /app-[hash].js.

Thus, we can't use the simple the rule we had so far (i.e. redirect everything that starts with /:locale/blog/ to the blog app), because Gatsby generates some files that don't match this rule.

Michał Piechowiak suggested using stats.json file to generate config file with redirect rules for our web server, but then there's an issue how to put this redirect config generated in Gatsby app into a completely different app that has nginx server that proxies requests to all our web apps.

Our current idea (not yet tested) is to build our app 6 times, once for each language, and set pathPrefix each time based on the language, e.g. /en/blog, /de/blog etc.

An option to serve all non-html files from a different domain or a specific path, would be much simpler solution for us. Especially, that after we finish working on our blog, we want to create 2 other multi-language Gatsby apps and we'll have the same issue then.

[szimek]

@KyleAMathews I'm investigating some issues we have with internal React errors showing up after every deployment and I'm trying to come up with possible scenarios when stuff can break ;) Here are 2 that can happen if someone deploys Gatsby app to a domain not backed up by CDN:

• when using rolling deployments, even if you have only one server, at some point during deployment there will be 2 servers - one serving the new version of the code and one serving the old one; it's possible that a user will get an HTML file from a server serving the new version, but requests for JS files will be handled by server serving the old version of the code, where these files don't exist anymore; with more servers, this issue is even more likely to happen, because replacing more servers obviously takes more time
• if a new version of the app is deployed after user opened a page, then when user clicks any "internal" link, the required JS files won't be present anymore on the servers, because they'll already serve only the new version of the code

Using CDN would solve these problems, because it would still keep old files in its cache. However, just like the author of this issue, we host many apps at the same domain and Gatsby is just one of them, so I don't think we can easily make it, so that only Gatsby app uses our CDN. I'm not an ops person, so there might be other and better ways to solve these issues without changing Gatsby code, but allowing to specify a domain (and preferably a path as well) for non-html files would be pretty useful.

[KyleAMathews]

On the first bullet point — you're always going to run into trouble when serving from two out-of-sync servers. You should build somewhere else and then copy the files onto the web server(s). This way you could also keep old files pretty easily.

On the second bullet — I think there's an issue about Gatsby catching js load errors that'd handle this sort of thing. Ideally though you keep files from older builds around so old versions still work.

[nakedible]

Probably #7808 could be used to solve this issue as well.

[xjamundx]

We're looking at adoption of Gatsby at PayPal and have the requirement that we need to load gatsby from our app servers, but would like to load static assets from a CDN.

We currently manage this with our webpack apps by setting the __webpack_public_path__ variable, which configures where chunks are loaded from, but that doesn't appear to work with gatsby. Any suggested solution for this problem?

[DSchau]

@xjamundx could you give a concrete example of what you're trying to do? How I'm interpreting this (and correct me if I'm wrong) is something like the following:

• You have a CDN for assets, e.g. cdn.paypal.com/assets
• You want to use that path for all static content, e.g. images, js, css, etc.
• You run gatsby build (with some type of asset path enabled)
• All links to assets are prefixed with https://cdn.paypal.com/assets/, e.g. https://cdn.paypal.com/assets/asdfadsf-1234.js is a link to a JS file processed via webpack
• Your Gatsby site will be deployed to https://some-gatsby-site.paypal.com (not as important, but the idea is that it's deployed somewhere separate from the cdn) and will serve all assets from the CDN path configured at build time

Does that about line up with what you're looking for here? Note: we don't currently support this, but just want to make sure we're on the same page before we discuss further!

[xjamundx]

@DSchau Yeah that could work 👍

(Our current system wouldn't have the full path until AFTER gatsby build is run, but we could probably modify it to support the approach you outlined.)

My current thought was to modify the servers that will be serving the gatsby-generated HTML and injecting __webpack_public_path == '<CDN_ASSET_PATH>' or whatever into them, but that may not help for images and other non-JS things. 🤔

[xjamundx]

@DSchau This proposal may cover it? #7808 (comment)

[szimek]

Wouldn’t it require 2 different options? One would be dataPrefixPath that would cause the build process to physically put files into specified path (e.g. /blog-assets would output non-html files to ./public/blog-assets), the other would be e.g. dataDomainPrefix, that would just add prefix to generated paths (e.g. https://cdn.mydomain.com).

[Daniel15]

I'm also wondering how to do this. I want to use Gatsby for an existing site that has a bunch of endpoints with server-side logic. So, I can't load the entire site from the CDN, but want to use a CDN for all the static files.

[trickydisco78]

Also would benefit my use case. I'm forced to serve our site internally at the moment and getting marked down for not using a CDN so would rather just host the index page internally and all the static assets via a CDN

[DSchau]

@Daniel15 @trickydisco78 I'm working on this now.

We want to do this in a clean way that doesn't introduce subtle edge cases and can be tapped into cleanly via plugin authors with an exposed API (e.g. withAssetPath helper or something). We've had some weirdness around a similar feature (pathPrefix) so want to introduce this feature and not break in subtle ways similarly to how pathPrefix has sometimes been broken.

You can follow along on this branch for now, and I'll hope to get something up and ready as soon as possible.

[revskill10]

I really need this feature to integrate Gatsby to my existing web application.
More details:

• I intend to build locally assets files, so that when deploying, i'll put those non-html assets to separate domain on a CDN.
• I'll serve html files as static assets on a server normally.

So when building, i need to have ability to:

• Set assetPrefix for all non-html assets based on environments (dev/production,..)
• Building html files into its own html folder to be served, for example from server.

Basically, this feature will separate building non-html assets and html assets into its own folder.

[ethagnawl]

An application I'm working on requires this functionality, too.

#10933 looks very promising, but (based on its description) likely won't be ready until after my initial rollout. In the meantime, I'm thinking I'll use sed to manipulate the asset paths after the production build and before the site has been deployed. Has anyone else come up with a similar solution? Were there any surprising edge cases you ran into?

[DSchau]

@ethagnawl you could probably use cheerio and the onPostBuild Node API to implement that a little more cleanly.

The basic idea is you'd query for links, scripts, and any other external links, and prefix them with your asset prefix. We can't highly encourage this technique, but in theory that should kinda work.

I think there will be lots of edge cases. Anywhere we add a URL dynamically (e.g. with JS) will probably fail this check. Things like images (using gatsby-image), preloaded content, etc. will fail with this technique.

[ethagnawl]

@DSchau I looked into using onPostBuild but, compared to using sed and manipulating the asset paths during my build, found that approach to be very complicated. (If I'm straying from the golden path, why not take the shortest route?) I was hoping to use a dummy pathPrefix (e.g. ASSET-PATH), which could be easily subbed out during the build, and that does get me most of the way there. Unfortunately, that has the unintended side effect of the page being redirected to site.com/ASSET-PATH after the client-side js takes over -- I could also strip all references to that pathPrefix but I feel like I'm spinning in circles here. (I am following #4337, which discusses the redirect issue, but it doesn't look like there's a simple way to opt out of that behavior.)

[id0Sch]

@Daniel15 @trickydisco78 I'm working on this now.

We want to do this in a clean way that doesn't introduce subtle edge cases and can be tapped into cleanly via plugin authors with an exposed API (e.g. withAssetPath helper or something). We've had some weirdness around a similar feature (pathPrefix) so want to introduce this feature and not break in subtle ways similarly to how pathPrefix has sometimes been broken.

You can follow along on this branch for now, and I'll hope to get something up and ready as soon as possible.

thanks for making an effort to support this, it is super relevant for a project I'm working on.
Same case, I need to serve assets and chunks from cdn and pages from main server.
Is there an ETA for this feature?
I've tried setting output.publicPath as I did with my old webpack config and it seems to not work at all.
Thanks again!

[revskill10]

Don't forget about the service-worker.js thing.
This feature should support PWA + Offline out of the box !
So, we need service-worker.js + html assets on one domain.
Other assets are on real CDN with production build.

[nsisodiya]

We are also stuck at this issue. This should be solved. It is quite common for an API server to send initial HTML file and rest of the js/css loads from other CDN server. Please let us know the workaround till the issue get solved.

[ethagnawl]

@nsisodiya Perhaps you should try using @DSchau's proposed solution and, if you find anything unexpected, provide constructive feedback? (I used it on a simple project recently and it worked very nicely.)

[gatsbot]

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 30 days of inactivity. It’s been at least 20 days since the last update here.

If we missed this issue or if you want to keep it open, please reply here. You can also add the label "not stale" to keep this issue open!

Thanks for being a part of the Gatsby community! 💪💜

[nyedidikeke]

@xjamundx could you give a concrete example of what you're trying to do? How I'm interpreting this (and correct me if I'm wrong) is something like the following:

• You have a CDN for assets, e.g. cdn.paypal.com/assets
• You want to use that path for all static content, e.g. images, js, css, etc.
• You run gatsby build (with some type of asset path enabled)
• All links to assets are prefixed with https://cdn.paypal.com/assets/, e.g. https://cdn.paypal.com/assets/asdfadsf-1234.js is a link to a JS file processed via webpack
• Your Gatsby site will be deployed to https://some-gatsby-site.paypal.com (not as important, but the idea is that it's deployed somewhere separate from the cdn) and will serve all assets from the CDN path configured at build time

Does that about line up with what you're looking for here? Note: we don't currently support this, but just want to make sure we're on the same page before we discuss further!

Exactly what I'm looking for.
Is there any effective way of achieving the current as described with Gatsby?
For a self-hosted solution (HTML files on premise) but with assets such as JavaScript, fonts, images and CSS served from a commercial CDN (Akamai, Cloudflare, MaxCDN, et al.).

[Daniel15]

Hey @nyedidikeke, this is currently being worked on here: #12128. It's not complete yet, but there's instructions here if you'd like to test it out: #12128 (comment)

[ethagnawl]

This is great news! Thanks for your efforts on this feature, @DSchau!

[DSchau]

Published and available in [email protected]. Specifically, the following packages have been augmented with assetPrefix functionality.

• [email protected]
• [email protected] ([email protected] depends upon this, don't need to install manually!)
• [email protected]
• [email protected]
• [email protected]
• [email protected]

Soon enough, the documentation will be available at /docs/asset-prefix/ (waiting for the build to complete), but you can get a sneak peek here

[erodrig]

is there any way to filter what assets will be referenced in this way, for me this only make sense for serving images, that need transformations, and having them done in each build in a CI server is nuts.
css and js, are ok with regular build.