Set up /website Vercel deployment #3092

taneliang · 2020-12-28T12:42:24Z

Goal

Setting up Vercel now, which may be able to replace Netlify (though we're on the old free plan that has unlimited team members but only a meagre 2 build hours) and even our Docker website setup.

The commit history is rather messed up -- it'll be easier to review this just by looking at the diff. ~~Also ignore the changes to the CircleCI config; that has been spun out to #3094 to reduce the mess in this PR.~~ #3094 has been merged

Breakdown

Fix Vercel build failure caused by Packtracker plugin.
Overhaul scripts/build.js to parallelize website and timetable-only builds. There's now a ~15% reduction in build time when built locally.
Nest timetable-only website within the main website's dist folder. This will allow one Vercel project to serve both projects. (Edit: done: https://nusmods-website-git-eliang-vercel.nusmodifications.vercel.app/timetable-only)
Configure caching headers.
- Move long-lived assets into an assets folder.
- Configure Vercel to provide long-lived caching headers for everything in assets.

This should only be merged until after we can confirm that the export server can consume the new timetable-only bundle hosted by Vercel. (EDIT: it can! 🎉 see #3095)

What's purposely not done

We'll no longer proxy /api to api.nusmods.com. Visiting /api in the browser still works as React Router has a redirect configured, ~~but it won't be redirected at the proxy level as imo it's unnecessary.~~ /website will use api.nusmods.com directly. EDIT: we'll redirect /api to api.nusmods.com since it's convenient.

vercel · 2020-12-28T12:42:26Z

This pull request is being automatically deployed with Vercel (learn more).
To see the status of your deployments, click below or on the icon next to each commit.

nusmods-website – ./website

🔍 Inspect: https://vercel.com/nusmodifications/nusmods-website/q9f8zf3vo
✅ Preview: https://nusmods-website-git-eliang-vercel.nusmodifications.vercel.app

nusmods-export – ./export

🔍 Inspect: https://vercel.com/nusmodifications/nusmods-export/mwgktcnya
✅ Preview: https://nusmods-export-git-eliang-vercel.nusmodifications.vercel.app

ZhangYiJiang · 2020-12-29T00:28:58Z

I wonder if we can replace this with the GitHub action or CircleCI orb - https://github.com/packtracker/report

taneliang · 2020-12-29T01:56:28Z

Great idea! Let me try configuring a CircleCI Orb

1. Use new executors key. 2. Also: "workflows no longer have a version number. Make sure to take the version key out of your workflows section or bad things will happen. The only version key in 2.1 is the top-level key." -- https://discuss.circleci.com/t/circleci-2-1-config-overview/26057

Don't use GitHub Action because (I believe) that'll need to rebuild website again, and we don't want to diversify our CI over too many separate systems as that'll make it harder to reason about.

So it's a job that builds and runs independently of our build job... this isn't looking good.

Their docs were a little inconsistent

taneliang · 2020-12-29T04:42:43Z

So I think we can't use either the orb or the GitHub Action:

Both the orb and the action build the app directly from the Webpack config provided to it, independently of our build script. Apart from just being wasteful and requiring more CI time, our prod build script also requires the browser warning bundle to be built first.
The orb is erroring because it tries to look for a package.json before cding into our project_root 🤦

I'll add the Webpack plugin back to our Webpack config after lunch.

This reverts commit 1a2e860.

This reverts commit 9719947.

This reverts commit 7d11aca.

This reverts commit 4fa2671.

Irrelevant again, now that all targets will be built and used by the same worker.

taneliang · 2020-12-29T14:53:29Z

We currently redirect /news to https://blog.nusmods.com in our Caddyfile; I'll add that to our vercel.json tomorrow, but this PR is ready for review.

ZhangYiJiang · 2020-12-29T19:04:21Z

website/src/config/app-config.json

@@ -2,7 +2,7 @@
  "brandName": "NUSMods",
  "defaultDescription": "NUSMods is a timetable builder and knowledge platform, providing students with a better way to plan their school timetable and useful module-related information that are community-driven.",
  "academicYear": "2020/2021",
-  "apiBaseUrl": "https://nusmods.com/api",
+  "apiBaseUrl": "https://api.nusmods.com",


Should add this as a DNS prefetch into either our website or our headers

Never knew this existed! But because we already preload the module list endpoint and thus presumably already perform this DNS resolution very early, will this have any significant effect? I'm not too familiar with prefetching in general, but if the improvement is negligible I'm concerned that the cost of sending down that additional DNS prefetching line will negate any gains. Plus there's some (minimal) eng cost to adding more fields to our HTML Webpack Plugin config.

I was thinking Vercel would have a config for that in the headers field

Yeah we can add a header to our Vercel config, but I'm still not sure if it's worth it. It'll still be one extra line sent down to the client, plus now we'll have api.nusmods.com in multiple places in our codebase. Very minimal issues of course, but if the gains are negligible these may cost more

ZhangYiJiang · 2020-12-29T19:16:48Z

website/vercel.json

+      "headers": [
+        {
+          "key": "Cache-Control",
+          "value": "public, max-age=31536000, immutable"


I think 7 days (604800) is the most I would cache this for. We've definitely had bugs in this file before, and it would be awkward for this file to be borked for an entire year. Also I just took another look at the file and it's somehow 164kb (!?) will investigate

Actually hold on, all the files in the /assets/ folder have content hashes in their filenames. I think we potentially may have a problem if we accidentally produce unhashed files, but I think that'll be unlikely?

My intention with this 1 year cache is to help with our "Error loading chunk xxx" errors, by basically letting the CDN serve them even after they're outdated. Maybe 1 year is too long, but given that students tend to keep NUSMods open for a really long time I think 1 week may be a bit too short. Maybe a month?

I've changed it to a month. Let's discuss this again if there's a better choice :)

I'm not sure why you'd think the browser cache would help with the error? We've always kept the older versions of our assets on our current server and it never seem to help. But yeah the URL must contain hash, in which case a 1 year expiration date actually make sense and we can use immutable https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control#Revalidation_and_reloading

I'm thinking the proxy cache/CDN (i.e. Vercel) can help with this. I'm not sure if our previous Caddy/Traefik set up was ever configured to continue serving cached versions of these files after they were deleted by the build script, but I doubt they were. I know the pre-Docker setup had all the old files sitting around, but I don't recall whether we had these errors then.

And yep this uses immutable. I'll bump the max-age back to 1 year in another PR

ZhangYiJiang · 2020-12-29T19:18:01Z

website/scripts/build.js

+  ]);
+  const errors = results.filter((r) => r.status === 'rejected').map((r) => r.reason);
+  if (errors.length > 0) {
+    throw errors;


A bit awkward since you're throwing an array not an Error, but not a huge issue

Yeah, I'm a bit desensitized to this now after seeing React throw *promises* 😆

Figured it should be fine since we do want to keep track of multiple errors, and creating an Errors extends Error class seemed a bit overkill since these thrown errors are only used by this file.

website/scripts/build.js

taneliang · 2020-12-30T13:29:11Z

/news and /api now redirect to blog.nusmods.com and api.nusmods.com respectively, with a 307 status code. Merging now to unblock #3095, but we can continue discussing the unresolved comments

taneliang added 5 commits December 29, 2020 10:04

Disable Packtracker to fix Vercel builds

e811290

Add packtracker/report CircleCI Orb

4fa2671

Don't use GitHub Action because (I believe) that'll need to rebuild website again, and we don't want to diversify our CI over too many separate systems as that'll make it harder to reason about.

Remove Packtracker Webpack plugin

1a2e860

Add target arg to build.js script

757cb79

taneliang force-pushed the eliang/vercel branch from a98e733 to 757cb79 Compare December 29, 2020 04:09

vercel bot had a problem deploying to Preview – nusmods-website-export December 29, 2020 04:09 Failure

vercel bot had a problem deploying to Preview – nusmods-website December 29, 2020 04:09 Failure

taneliang changed the title ~~Disable Packtracker to fix Vercel builds~~ Update build scripts and CircleCI config version Dec 29, 2020

vercel bot deployed to Preview – nusmods-website-export December 29, 2020 04:14 View deployment

Fix timetable-only build uncaught promise rejection

1456b94

vercel bot deployed to Preview – nusmods-website-export December 29, 2020 04:24 View deployment

vercel bot deployed to Preview – nusmods-website December 29, 2020 04:24 View deployment

Fix Packtracker CircleCI Orb

7d11aca

So it's a job that builds and runs independently of our build job... this isn't looking good.

vercel bot deployed to Preview – nusmods-website December 29, 2020 04:34 View deployment

vercel bot deployed to Preview – nusmods-website-export December 29, 2020 04:34 View deployment

Fix Packtracker CircleCI Orb name

9719947

Their docs were a little inconsistent

vercel bot deployed to Preview – nusmods-website December 29, 2020 04:36 View deployment

vercel bot deployed to Preview – nusmods-website-export December 29, 2020 04:36 View deployment

taneliang added 4 commits December 29, 2020 13:35

Revert "Remove Packtracker Webpack plugin"

dd609ba

This reverts commit 1a2e860.

Revert "Fix Packtracker CircleCI Orb name"

c47ae49

This reverts commit 9719947.

Revert "Fix Packtracker CircleCI Orb"

2ee9d94

This reverts commit 7d11aca.

Revert "Add packtracker/report CircleCI Orb"

4595eff

This reverts commit 4fa2671.

vercel bot deployed to Preview – nusmods-website December 29, 2020 05:39 View deployment

vercel bot deployed to Preview – nusmods-website-export December 29, 2020 05:39 View deployment

Fix prettier lint

85b09fd

taneliang marked this pull request as draft December 29, 2020 08:14

taneliang changed the title ~~Parallelize build script, update CircleCI config to V2.1, fix Packtracker error on Vercel~~ Set up /website Vercel deployment Dec 29, 2020

Remove separate build target param from build.js script

5279b38

Irrelevant again, now that all targets will be built and used by the same worker.

vercel bot deployed to Preview December 29, 2020 08:27 View deployment

taneliang added 2 commits December 29, 2020 16:56

Add aggressive Cache-Control header for /assets folder

9bfb9ee

Make website consume api.nusmods.com instead of /api

a522afd

vercel bot deployed to Preview December 29, 2020 08:59 View deployment

Merge branch 'master' into eliang/vercel

3de25a0

vercel bot deployed to Preview – nusmods-website December 29, 2020 11:11 View deployment

vercel bot had a problem deploying to Preview – nusmods-export December 29, 2020 11:11 Failure

taneliang mentioned this pull request Dec 29, 2020

Turn export service into a Vercel serverless function #3095

Merged

taneliang requested a review from a team December 29, 2020 14:43

taneliang marked this pull request as ready for review December 29, 2020 14:44

ZhangYiJiang approved these changes Dec 29, 2020

View reviewed changes

taneliang added 4 commits December 30, 2020 20:52

Try removing (e) from catch

4eb991e

Move _.castArray call into printErrors to improve resilience

5e57260

Reduce asset cache-control max-age from 1 year to 1 month

15b1fbf

Add /api and /news redirects

9aa4b13

vercel bot deployed to Preview – nusmods-export December 30, 2020 13:15 View deployment

vercel bot deployed to Preview – nusmods-website December 30, 2020 13:15 View deployment

Fix Vercel redirect paths, and change status code from 301 -> 302

aa637f8

vercel bot deployed to Preview – nusmods-export December 30, 2020 13:21 View deployment

vercel bot deployed to Preview – nusmods-website December 30, 2020 13:21 View deployment

taneliang merged commit a76344a into master Dec 30, 2020

taneliang deleted the eliang/vercel branch December 30, 2020 13:29

taneliang mentioned this pull request Dec 30, 2020

Migrate /website and /export from Docker to Vercel #3098

Open

13 tasks

This was referenced Jan 27, 2021

Increase /assets/* cache max-age from 1 month to 1 year #3209

Merged

Implement Module Planning Exercise form for NUS #3035

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Set up /website Vercel deployment #3092

Set up /website Vercel deployment #3092

taneliang commented Dec 28, 2020 •

edited

Loading

vercel bot commented Dec 28, 2020 •

edited

Loading

ZhangYiJiang commented Dec 29, 2020

taneliang commented Dec 29, 2020

taneliang commented Dec 29, 2020

taneliang commented Dec 29, 2020

ZhangYiJiang Dec 29, 2020

taneliang Dec 30, 2020

ZhangYiJiang Dec 30, 2020

taneliang Dec 31, 2020

ZhangYiJiang Dec 29, 2020

taneliang Dec 30, 2020

taneliang Dec 30, 2020

ZhangYiJiang Dec 30, 2020

taneliang Dec 31, 2020

ZhangYiJiang Dec 29, 2020

taneliang Dec 30, 2020

taneliang commented Dec 30, 2020

Set up /website Vercel deployment #3092

Set up /website Vercel deployment #3092

Conversation

taneliang commented Dec 28, 2020 • edited Loading

Goal

Breakdown

What's purposely not done

vercel bot commented Dec 28, 2020 • edited Loading

nusmods-website – ./website

nusmods-export – ./export

ZhangYiJiang commented Dec 29, 2020

taneliang commented Dec 29, 2020

taneliang commented Dec 29, 2020

taneliang commented Dec 29, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

taneliang commented Dec 30, 2020

taneliang commented Dec 28, 2020 •

edited

Loading

vercel bot commented Dec 28, 2020 •

edited

Loading