Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(gatsby): Adapters #38232

Merged
merged 170 commits into from
Jul 24, 2023
Merged
Changes from 1 commit
Commits
Show all changes
170 commits
Select commit Hold shift + click to select a range
5bc0b1f
initial - cache restoration
pieh May 18, 2023
72f6ecd
snapshot
pieh May 19, 2023
ec6e85b
cache.restore() return comment
pieh May 19, 2023
18a6ef8
redirect status
pieh May 19, 2023
cd8d2c8
note about named wildcard paths
pieh May 19, 2023
60dfb1b
don't pass named wildcards to function manifest
pieh May 19, 2023
c70cebb
fix status code resolution for redirects
pieh May 19, 2023
01fb83d
scaffold initial gatsby-adapter-netlify package and use if for dev pu…
pieh May 22, 2023
9e6a35a
build adapter package as ESM and load it as such, so we can use ESM-o…
pieh May 22, 2023
32670e1
no require in esm world
pieh May 22, 2023
671a7d2
add redirect headers
pieh May 24, 2023
100c8d1
start scaffolding autoinstallation of adapters
pieh May 24, 2023
c7676c3
webpack assets + unmanaged assets (start)
pieh May 30, 2023
ceaf740
static queries, app-data.json, minor refactors and initial setup for …
pieh May 30, 2023
62cdff2
move adapter version to 1.0.0
LekoArts May 30, 2023
98da13d
generalize get-latest-apis for adapters
LekoArts May 30, 2023
1975ae9
handle JS files in get-latest-gatsby-files
LekoArts May 31, 2023
606c604
set peerDep
LekoArts May 31, 2023
0292cb2
use other testing pkg
LekoArts May 31, 2023
7da3c66
get installation, discovery, re-using working
LekoArts May 31, 2023
a983541
Merge branch 'master' into feat/adapters
LekoArts May 31, 2023
3f1659a
update versions
LekoArts May 31, 2023
ac6ecf3
move adapter init into its own file
LekoArts May 31, 2023
df07cda
add version checking
LekoArts May 31, 2023
818c1ed
adjust comment
LekoArts May 31, 2023
19828e9
move stuff around
LekoArts May 31, 2023
9499cff
Merge branch 'master' into feat/adapters
LekoArts May 31, 2023
207bddf
feat: add headers to gatsby-config
LekoArts May 31, 2023
c74bd88
misc stuff
LekoArts May 31, 2023
89cbde7
initial engine lambda
pieh May 31, 2023
6089288
Merge branch 'master' into feat/adapters
LekoArts Jun 1, 2023
0947ec2
start headers utils
LekoArts Jun 1, 2023
a9c37fb
Merge branch 'master' into feat/adapters
LekoArts Jun 2, 2023
ead3775
update deps
LekoArts Jun 2, 2023
afde61d
rewrite util
LekoArts Jun 2, 2023
d50aee2
linting
LekoArts Jun 2, 2023
ec3c643
convert to obj args
LekoArts Jun 2, 2023
f839663
remove todo comment
pieh Jun 2, 2023
f46f850
add requiredFiles to functions manifest
pieh Jun 2, 2023
a7c3746
make headers default to []
LekoArts Jun 2, 2023
9ac1445
move constants to own file
LekoArts Jun 2, 2023
53be6ec
export rankRoute
LekoArts Jun 2, 2023
0fc101f
delete unneeded util
LekoArts Jun 5, 2023
dd2dd60
createHeadersMatcher initial impl
LekoArts Jun 5, 2023
63913a7
use createHeadersMatcher
LekoArts Jun 5, 2023
a03b176
Merge branch 'master' into feat/adapters
LekoArts Jun 5, 2023
20e5796
fix types
LekoArts Jun 5, 2023
e805df7
add http status code type
LekoArts Jun 5, 2023
3eba52c
improve createHeadersMatcher and add tests
LekoArts Jun 5, 2023
4ca653b
move adapterManager init to initialize func
LekoArts Jun 5, 2023
610d3c0
adjust func args to move reporter and allow adapter options
LekoArts Jun 5, 2023
8bc6351
add "adapter" option to gatsby-config
LekoArts Jun 5, 2023
3916d79
put netlify adapter first in the list - it will only match when env v…
pieh Jun 2, 2023
86429b1
kebabcase function name as function id
pieh Jun 5, 2023
b8bc194
export FunctionDefinition type
pieh Jun 5, 2023
8fcca54
req.path -> req.url
pieh Jun 5, 2023
657659a
initial functions wrapping/bundling in gatsby-adapter-netlify
pieh Jun 5, 2023
fee841d
remove netlify adapter from gatsby deps, add gatsby as devDep to adap…
pieh Jun 6, 2023
be62f1d
fix bundling function files containing [ ]
pieh Jun 6, 2023
90423bf
unify tsconfig for adapter
LekoArts Jun 6, 2023
900ed5e
add joi testing for adapter setting
LekoArts Jun 6, 2023
eb9f676
typescript: make bootstrap work again
LekoArts Jun 6, 2023
9d0e6f8
generate redirect/rewrite rules, generate 2 variants of ssr-engine (…
pieh Jun 6, 2023
80c45a4
move routes manifest handling into its own module
pieh Jun 6, 2023
b16febb
generate _headers rules
pieh Jun 6, 2023
ff7575f
add sorting to routesManifest
LekoArts Jun 6, 2023
dcec323
adjust graphql-engine bundling to not leave unreasolvable imports
pieh Jun 7, 2023
e2c2ed2
ssr lambda handling when it executes in read-only dir (use tmpdir() t…
pieh Jun 7, 2023
05e82f6
inject functions matchPath into function bundle and generate req.para…
pieh Jun 12, 2023
12410e3
serve api from path prefixed path as well
pieh Jun 12, 2023
d603ebb
add path prefix stripping in function wrapper
pieh Jun 12, 2023
a3db597
add cache store and restore in gatsby-adapter-netlify
pieh Jun 12, 2023
fd1e610
adjust internal 'lambda' name to 'function'
LekoArts Jun 12, 2023
7b05c89
format lambda-handler
LekoArts Jun 12, 2023
591a26b
misc changes
LekoArts Jun 12, 2023
d656565
missing rename
pieh Jun 12, 2023
d6b252c
compile gatsby-adapter-netlify to cjs
pieh Jun 12, 2023
5f65225
add generator field
pieh Jun 13, 2023
27b1eff
use netlify adpter when NETLIFY or NETLIFY_LOCAL env var is defined
pieh Jun 13, 2023
94b9e9d
use headers from config for ssg/dsg
pieh Jun 13, 2023
80c0142
allow specyfing different lmdb binary than current process, use abi83…
pieh Jun 13, 2023
c93dfde
get-route-path tests
LekoArts Jun 14, 2023
0a869b6
manager refactoring + typo fix
LekoArts Jun 14, 2023
b46a390
initialize adapters e2e test
LekoArts Jun 14, 2023
8c7d99e
cypress: remove viewPort configs
LekoArts Jun 14, 2023
5f2f652
use next preid for netlify adapter
pieh Jun 14, 2023
cc34cc7
remove test adapter from adapters manifest
pieh Jun 14, 2023
8bdaa2e
don't log errors when testing for user installed adapters
pieh Jun 14, 2023
6956ad5
update adapters manifest
pieh Jun 14, 2023
a3d4278
cleanup gatsby-adapter-netlify a bit, add more public adapter related…
pieh Jun 14, 2023
083fd85
e2e: update gitignore
LekoArts Jun 14, 2023
0a7b663
resolve lmdb binary from lmdb package and not hardcode the forced path
pieh Jun 14, 2023
5dc6576
gatsby-plugin-image add downlevelIteration
pieh Jun 14, 2023
8ff7b22
fix persisted redux keys
pieh Jun 14, 2023
ae403c0
update snapshot and mocks
pieh Jun 14, 2023
f5711f3
fix: only run adapters during gatsby build
LekoArts Jun 14, 2023
c615f04
resolve netlify functions runtime deps from adapter context
pieh Jun 14, 2023
62a298a
improve public typings
LekoArts Jun 15, 2023
36df358
adding adapter to e2e test so that dev-cli copies stuff over
LekoArts Jun 15, 2023
5aa1126
Merge branch 'master' into feat/adapters
LekoArts Jun 15, 2023
f6326af
update babel-preset-gatsby-package dep
LekoArts Jun 15, 2023
9791bd2
e2e: test functions and assets
LekoArts Jun 15, 2023
0fc4bd1
e2e: client-only WIP
LekoArts Jun 15, 2023
047be98
e2e: improve basics
LekoArts Jun 16, 2023
c532978
e2e: improve client-only
LekoArts Jun 16, 2023
1740f9e
e2e: redirects
LekoArts Jun 16, 2023
1e31d19
merge _headers and _redirects instead of overwriting it
pieh Jun 16, 2023
58be681
Merge branch 'master' into feat/adapters
LekoArts Jun 26, 2023
30f12b0
apply trailing slash option + pass through trailingSlash & pathPrefix
LekoArts Jun 26, 2023
cbd0374
add unit tests for manager
LekoArts Jun 26, 2023
f358c43
improve manager tests
LekoArts Jun 26, 2023
d5d1720
update types
LekoArts Jun 28, 2023
faa4b2a
improve e2e tests
LekoArts Jun 28, 2023
9e2935c
add excludeDatastoreFromEngineFunction flow
pieh Jun 29, 2023
903dd04
normalize path after globbing
pieh Jun 29, 2023
d288e86
mock shouldBundleDatastore
pieh Jun 29, 2023
46ee1f3
rename adapter config types to be less confusing with gatsby-config
pieh Jun 29, 2023
a00a634
keep same obfuscated path between builds
pieh Jun 29, 2023
140f62f
normalize more paths
pieh Jun 29, 2023
ce2f692
support custom 404/500 page for serverless functions
LekoArts Jun 29, 2023
63d3f9d
update snapshot
LekoArts Jun 29, 2023
305e96f
generate relative imports in function
pieh Jun 29, 2023
b4e5c56
skip trying to copy data to tmp if we are downloading from cdn
pieh Jun 29, 2023
e9624aa
improve TS types
LekoArts Jun 29, 2023
e42ab8f
mock uuid
LekoArts Jun 29, 2023
ddda443
put requiredFiles in correct place heh
LekoArts Jun 29, 2023
56824a1
typo
LekoArts Jun 29, 2023
3e88e97
snapshot
LekoArts Jun 29, 2023
016d697
handle GATSBY_EXCLUDE_DATASTORE_FROM_BUNDLE env var
pieh Jun 29, 2023
342f2f9
improve README & update types
LekoArts Jun 30, 2023
81780af
code block
LekoArts Jun 30, 2023
6a18b5c
handle partytown routes
pieh Jun 30, 2023
bac5412
handle slices (html and slice-data)
pieh Jun 30, 2023
363e6c5
handle chunk-map and webpack.stats
pieh Jun 30, 2023
4ddabf3
feat: add name to functionsManifest & displayName to Netlify
LekoArts Jun 30, 2023
e1e2201
update snapshot
LekoArts Jun 30, 2023
9b2e632
rename headers constants
pieh Jun 30, 2023
c0a5402
handle image-cdn and file-cdn
pieh Jun 30, 2023
d294c7d
update routesManifest test fixture and snapshot
pieh Jun 30, 2023
15ec313
don't log unmanaged static assets anymore
pieh Jun 30, 2023
b5ac1e3
handle some TODOs
LekoArts Jun 30, 2023
6496cf9
add 'adapters' to feature list
pieh Jun 30, 2023
6662f45
tmp: make peer dependency allow to use with canaries
pieh Jul 4, 2023
8ba5fae
feat: add 'supports' to config to let adapters provide some capabilit…
pieh Jul 4, 2023
7f1be53
Merge branch 'master' into feat/adapters
LekoArts Jul 5, 2023
2adc306
adjust some text
LekoArts Jul 5, 2023
46cbbff
apply trailingSlash to tests and other stuff, add utils
LekoArts Jul 5, 2023
d68acb2
readme and package.json update
LekoArts Jul 5, 2023
d1db998
fix ts versions
LekoArts Jul 5, 2023
2cef296
enable typecheck for adapters.js
pieh Jul 10, 2023
24bae2a
don't try to use install adapter if no version matches, install versi…
pieh Jul 10, 2023
6d76ebc
allow adapter to disable prior deployment plugins
pieh Jul 10, 2023
f837c54
disable gatsby-plugin-netlify when using gatsby-adapter-netlify
pieh Jul 10, 2023
8bb775e
fix ts
pieh Jul 11, 2023
7d848f6
handle non-alpha-numerc paths
pieh Jul 11, 2023
ec8839f
verbose log adapter package that is being installed
pieh Jul 11, 2023
f9857be
only try to restore cache if payload provided
pieh Jul 11, 2023
36891f6
memoize cache utils
pieh Jul 12, 2023
6436295
log adapter version
pieh Jul 12, 2023
d07fe0d
handle body parsing in produced api functions
pieh Jul 12, 2023
81bc037
Merge remote-tracking branch 'origin/master' into feat/adapters
pieh Jul 12, 2023
ce9e7a8
properly handle cases when body parsing already happened and when the…
pieh Jul 12, 2023
902d30f
handle body parsing config in generated function
pieh Jul 12, 2023
ca225bf
drop unused
pieh Jul 12, 2023
092b92e
remove redirects created by previous deployment plugins
pieh Jul 13, 2023
fa7dc92
properly handle default body for status responses in api functions
pieh Jul 14, 2023
17d4a58
put some dev logs as verbose to limit regular terminal output
pieh Jul 19, 2023
3787d92
put some dev logs as verbose to limit regular terminal output vol2
pieh Jul 19, 2023
3dbe567
maybe deflake function tests
pieh Jul 21, 2023
92a7996
maybe precompile api functions in develop function tests
pieh Jul 24, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
static queries, app-data.json, minor refactors and initial setup for …
…having routes sorted by specificity
pieh committed May 30, 2023

Verified

This commit was signed with the committer’s verified signature.
thaJeztah Sebastiaan van Stijn
commit ceaf74081571ec48d8c07e6d02f14fea607e9fa9
89 changes: 51 additions & 38 deletions packages/gatsby/src/utils/adapter/manager.ts
Original file line number Diff line number Diff line change
@@ -6,6 +6,7 @@ import {
IAdaptContext,
AdapterInit,
RoutesManifest,
Route,
} from "./types"
import { preferDefault } from "../../bootstrap/prefer-default"
import { generateHtmlPath } from "gatsby-core-utils/page-html"
@@ -14,6 +15,7 @@ import { getPageMode } from "../page-mode"
import { generatePageDataPath } from "gatsby-core-utils/page-data"
import { satisfies } from "semver"
import { sync as globSync } from "glob"
import { getStaticQueryPath } from "../static-query-utils"

interface IAdapterManager {
restoreCache: () => Promise<void> | void
@@ -150,6 +152,8 @@ export function setWebpackAssets(assets: Set<string>): void {
}

function getRoutesManifest(): RoutesManifest {
// TODO: have routes list sorted by specifity so more specific ones are before less specific ones (/static should be before /:param and that should be before /*),
// so routing can just handle first match
const routes = [] as RoutesManifest

const fileAssets = new Set(
@@ -160,12 +164,31 @@ function getRoutesManifest(): RoutesManifest {
})
)

function removeFromAssets(path: string): void {
if (fileAssets.has(path)) {
fileAssets.delete(path)
function addSortedRoute(route: Route): void {
if (!route.path.startsWith(`/`)) {
route.path = `/${route.path}`
}
// TODO: calculate specifity of route's path and insert route in correct place
routes.push(route)
}

function addStaticRoute(
path: string,
pathToFilInPublicDir: string,
headers: Array<string>
): void {
addSortedRoute({
path,
type: `static`,
filePath: posix.join(`public`, pathToFilInPublicDir),
headers,
})

if (fileAssets.has(pathToFilInPublicDir)) {
fileAssets.delete(pathToFilInPublicDir)
} else {
reporter.warn(
`[dev-adapter-manager] tried to remove "${path}" from assets but it wasn't there`
`[dev-adapter-manager] tried to remove "${pathToFilInPublicDir}" from assets but it wasn't there`
)
}
}
@@ -180,23 +203,8 @@ function getRoutesManifest(): RoutesManifest {
const htmlFilePath = generateHtmlPath(``, page.path)
const pageDataFilePath = generatePageDataPath(``, page.path)

routes.push({
path: htmlRoutePath,
type: `static`,
filePath: posix.join(`public`, htmlFilePath),
headers: STATIC_PAGE_HEADERS,
})

removeFromAssets(htmlFilePath)

routes.push({
path: pageDataRoutePath,
type: `static`,
filePath: posix.join(`public`, pageDataFilePath),
headers: STATIC_PAGE_HEADERS,
})

removeFromAssets(pageDataFilePath)
addStaticRoute(htmlRoutePath, htmlFilePath, STATIC_PAGE_HEADERS)
addStaticRoute(pageDataRoutePath, pageDataFilePath, STATIC_PAGE_HEADERS)
} else {
// TODO: generate lambda function for SSR/DSG
// TODO: figure out caching behavior metadata - maybe take a look at https://vercel.com/docs/build-output-api/v3/primitives#prerender-functions for inspiration
@@ -213,28 +221,38 @@ function getRoutesManifest(): RoutesManifest {
}
}

// static query json assets
for (const staticQueryComponent of store
.getState()
.staticQueryComponents.values()) {
const staticQueryResultPath = getStaticQueryPath(staticQueryComponent.hash)
addStaticRoute(
staticQueryResultPath,
staticQueryResultPath,
STATIC_PAGE_HEADERS
)
}

// app-data.json
{
const appDataFilePath = posix.join(`page-data`, `app-data.json`)
addStaticRoute(appDataFilePath, appDataFilePath, STATIC_PAGE_HEADERS)
}

// webpack assets
if (!webpackAssets) {
throw new Error(`[dev-adapter-manager] webpackAssets is not defined`)
}

for (const asset of webpackAssets) {
routes.push({
path: asset,
type: `static`,
filePath: posix.join(`public`, asset),
headers: WEBPACK_ASSET_HEADERS,
})
removeFromAssets(asset)
addStaticRoute(asset, asset, WEBPACK_ASSET_HEADERS)
}

// TODO: slices

// TODO: static query json assets

// redirect routes
for (const redirect of store.getState().redirects.values()) {
routes.push({
addSortedRoute({
path: redirect.fromPath,
type: `redirect`,
toPath: redirect.toPath,
@@ -246,7 +264,7 @@ function getRoutesManifest(): RoutesManifest {

// function routes
for (const functionInfo of store.getState().functions.values()) {
routes.push({
addSortedRoute({
path: `/api/${
maybeDropNamedPartOfWildcard(functionInfo.matchPath) ??
functionInfo.functionRoute
@@ -262,12 +280,7 @@ function getRoutesManifest(): RoutesManifest {
)

for (const fileAsset of fileAssets) {
routes.push({
type: `static`,
path: fileAsset,
filePath: posix.join(`public`, fileAsset),
headers: ASSET_HEADERS,
})
addStaticRoute(fileAsset, fileAsset, ASSET_HEADERS)
}

return routes
2 changes: 1 addition & 1 deletion packages/gatsby/src/utils/adapter/types.ts
Original file line number Diff line number Diff line change
@@ -43,7 +43,7 @@ interface IRedirectRoute extends IBaseRoute {
[key: string]: unknown
}

type Route = IStaticRoute | ILambdaRoute | IRedirectRoute
export type Route = IStaticRoute | ILambdaRoute | IRedirectRoute

export type RoutesManifest = Array<Route>