Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

testing matrix #3

Merged
merged 33 commits into from
Dec 3, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
7cd2b27
init testing matrix
Ethan-Arrowood Nov 21, 2024
203b39c
add hdb stop
Ethan-Arrowood Nov 21, 2024
3a54320
try timeout
Ethan-Arrowood Nov 21, 2024
f4e6afe
add || true
Ethan-Arrowood Nov 21, 2024
2718f1e
try this
Ethan-Arrowood Nov 21, 2024
68ef8ff
try sleep 30
Ethan-Arrowood Nov 21, 2024
ef7a747
whoops
Ethan-Arrowood Nov 21, 2024
e64e534
oh actions I love you
Ethan-Arrowood Nov 21, 2024
ebb2cd3
.
Ethan-Arrowood Nov 21, 2024
1737633
use docker and js. wip
Ethan-Arrowood Nov 22, 2024
10e3db3
update dockerfil to copy the base component
Ethan-Arrowood Nov 22, 2024
610617b
fix arg order in dockerfile
Ethan-Arrowood Nov 22, 2024
946180b
Merge branch 'main' into test-matrix
Ethan-Arrowood Nov 26, 2024
0c43aba
use node test runner
Ethan-Arrowood Nov 26, 2024
8a7bd2c
try out some concurrency
Ethan-Arrowood Nov 26, 2024
808280a
add debugging and make operations non-blocking
Ethan-Arrowood Nov 27, 2024
4041498
DRY
Ethan-Arrowood Nov 27, 2024
dc290a6
progress
Ethan-Arrowood Nov 30, 2024
25ceca3
ci?
Ethan-Arrowood Nov 30, 2024
53469ca
debug in ci
Ethan-Arrowood Nov 30, 2024
e73ae75
try localhost:
Ethan-Arrowood Nov 30, 2024
fcf0f96
okay working(ish). component loading issue related to dep install.
Ethan-Arrowood Nov 30, 2024
ca16551
install install
Ethan-Arrowood Nov 30, 2024
f84b972
eureka! separate out build step to improve test speed
Ethan-Arrowood Dec 1, 2024
ce61a75
lots of progress. but why do the tests fail in concurrency mode?
Ethan-Arrowood Dec 2, 2024
4270782
no more concurrency. breaks computer
Ethan-Arrowood Dec 2, 2024
4371ab3
prepare for review
Ethan-Arrowood Dec 2, 2024
ba18a38
fix CONTRIBUTING
Ethan-Arrowood Dec 2, 2024
0903bca
fix workflow
Ethan-Arrowood Dec 2, 2024
9483090
come on docker
Ethan-Arrowood Dec 2, 2024
a0e0e1f
podman?
Ethan-Arrowood Dec 2, 2024
8026d2a
ugh
Ethan-Arrowood Dec 2, 2024
65b5628
playwright this time
Ethan-Arrowood Dec 3, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 36 additions & 0 deletions .github/workflows/test.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
name: Test

on:
push:
branches: [main]
pull_request:
branches: [main]

jobs:
test:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4

- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '22'
cache: 'npm'

- name: Install dependencies
run: npm ci

- name: Install Playwright Browsers
run: npx playwright install --with-deps

- name: Run tests
run: npm run test

- uses: actions/upload-artifact@v4
if: ${{ !cancelled() }}
with:
name: playwright-report
path: playwright-report/
retention-days: 30
4 changes: 3 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1 +1,3 @@
node_modules/
node_modules/
.next/
test-results/
1 change: 1 addition & 0 deletions .node-version
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
22.11.0
38 changes: 38 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# Contributing

## Code Organization

All code should abide by the following organization rules:

- `fixtures/` is for directories that will be utilized in testing.
- They should be fully executable locally for debugging purposes
- They should be as minimal as possible
- `test/` is for Playwright based test files that are executed by `npm run test`
- Test files should use `import { test, expect } from '../util/test-fixture.js';` so that the correct **Fixture** is managed for the test script
- Test files should execute serially, and relevant to the given Next.js versions
- `util/` is for any non-module source code. This includes scripts (`util/scripts/`), docker configurations (`util/docker/`), or any other utility based code.
- Prime examples include the source code responsible for [_building_](./util/scripts/pretest.js) the **Fixtures**, or the custom Playwright [test fixture](./util/test-fixture.js)

The key source files for the repo are:

- `cli.js`
- `extension.js`
- `config.yaml`
- `schema.graphql`

## Testing

Testing for this repo uses containers in order to generate stable, isolated environments containing:

- HarperDB
- Node.js
- A HarperDB Base Component (responsible for seeding the database)
- A Next.js application Component (which uses this `@harperdb/nextjs` extension)

To execute tests, run `npm run test`

The first run may take some time as the pretest script is building 12 separate images (3 Node.js ones, 9 Next.js ones). Note, at the moment this operation is parallelized as building is very expensive and can result in the system running out of resources (and crashing the build processes). Subsequent runs utilize the Docker build step cache and are very fast.

After the images are built, [Playwright](https://playwright.dev/) will run the tests. These tests each utilize an image, and will manage a container instance relevant to the given Next.js and Node.js pair.

The tests are configured with generous timeouts and limited to 3 workers at a time to not cause the system to run out of resources.
5 changes: 5 additions & 0 deletions fixtures/harperdb-base-component/config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
rest: true
graphqlSchema:
files: './schema.graphql'
jsResource:
files: './resources.js'
4 changes: 4 additions & 0 deletions fixtures/harperdb-base-component/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
{
"name": "harperdb-base-component",
"private": true
}
18 changes: 18 additions & 0 deletions fixtures/harperdb-base-component/resources.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
const dogs = [
{ id: '0', name: 'Lincoln', breed: 'Shepherd' },
{ id: '1', name: 'Max', breed: 'Cocker Spaniel' },
{ id: '2', name: 'Bella', breed: 'Lab' },
{ id: '3', name: 'Charlie', breed: 'Great Dane' },
{ id: '4', name: 'Lucy', breed: 'Newfoundland' },
{ id: '5', name: 'Cooper', breed: 'Pug' },
{ id: '6', name: 'Daisy', breed: 'Bull Dog' },
{ id: '7', name: 'Rocky', breed: 'Akita' },
{ id: '8', name: 'Luna', breed: 'Wolf' },
{ id: '9', name: 'Buddy', breed: 'Border Collie' },
{ id: '10', name: 'Bailey', breed: 'Golden Retriever' },
{ id: '11', name: 'Sadie', breed: 'Belgian Malinois' },
];

for (const dog of dogs) {
tables.Dog.put(dog);
}
5 changes: 5 additions & 0 deletions fixtures/harperdb-base-component/schema.graphql
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
type Dog @table @export {
id: ID @primaryKey
name: String
breed: String
}
1 change: 1 addition & 0 deletions fixtures/next-13/.npmrc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
package-lock=false
11 changes: 11 additions & 0 deletions fixtures/next-13/app/layout.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
export const metadata = {
title: 'HarperDB - Next.js v13 App',
};

export default function RootLayout({ children }) {
return (
<html>
<body>{children}</body>
</html>
);
}
7 changes: 7 additions & 0 deletions fixtures/next-13/app/page.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
export default async function Page() {
return (
<div>
<h1>Next.js v13</h1>
</div>
);
}
4 changes: 4 additions & 0 deletions fixtures/next-13/config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
'@harperdb/nextjs':
package: '@harperdb/nextjs'
files: '/*'
port: 9926
1 change: 1 addition & 0 deletions fixtures/next-13/next.config.js
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
module.exports = {};
17 changes: 17 additions & 0 deletions fixtures/next-13/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
{
"name": "next-13",
"private": true,
"scripts": {
"dev": "next dev",
"build": "next build",
"start": "next start",
"lint": "next lint",
"postinstall": "npm link @harperdb/nextjs"
},
"dependencies": {
"@harperdb/nextjs": "*",
"react": "^18",
"react-dom": "^18",
"next": "^13"
}
}
1 change: 1 addition & 0 deletions fixtures/next-14/.npmrc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
package-lock=false
11 changes: 11 additions & 0 deletions fixtures/next-14/app/layout.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
export const metadata = {
title: 'HarperDB - Next.js v14 App',
};

export default function RootLayout({ children }) {
return (
<html>
<body>{children}</body>
</html>
);
}
7 changes: 7 additions & 0 deletions fixtures/next-14/app/page.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
export default async function Page() {
return (
<div>
<h1>Next.js v14</h1>
</div>
);
}
4 changes: 4 additions & 0 deletions fixtures/next-14/config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
'@harperdb/nextjs':
package: '@harperdb/nextjs'
files: '/*'
port: 9926
1 change: 1 addition & 0 deletions fixtures/next-14/next.config.js
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
module.exports = {};
17 changes: 17 additions & 0 deletions fixtures/next-14/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
{
"name": "next-14",
"private": true,
"scripts": {
"dev": "next dev",
"build": "next build",
"start": "next start",
"lint": "next lint",
"postinstall": "npm link @harperdb/nextjs"
},
"dependencies": {
"@harperdb/nextjs": "*",
"react": "^18",
"react-dom": "^18",
"next": "^14"
}
}
1 change: 1 addition & 0 deletions fixtures/next-15/.npmrc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
package-lock=false
11 changes: 11 additions & 0 deletions fixtures/next-15/app/layout.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
export const metadata = {
title: 'HarperDB - Next.js v15 App',
};

export default function RootLayout({ children }) {
return (
<html>
<body>{children}</body>
</html>
);
}
7 changes: 7 additions & 0 deletions fixtures/next-15/app/page.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
export default async function Page() {
return (
<div>
<h1>Next.js v15</h1>
</div>
);
}
4 changes: 4 additions & 0 deletions fixtures/next-15/config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
'@harperdb/nextjs':
package: '@harperdb/nextjs'
files: '/*'
port: 9926
1 change: 1 addition & 0 deletions fixtures/next-15/next.config.js
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
module.exports = {};
17 changes: 17 additions & 0 deletions fixtures/next-15/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
{
"name": "next-15",
"private": true,
"scripts": {
"dev": "next dev",
"build": "next build",
"start": "next start",
"lint": "next lint",
"postinstall": "npm link @harperdb/nextjs"
},
"dependencies": {
"@harperdb/nextjs": "*",
"react": "19.0.0-rc-66855b96-20241106",
"react-dom": "19.0.0-rc-66855b96-20241106",
"next": "^15"
}
}
64 changes: 64 additions & 0 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

7 changes: 6 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -33,13 +33,18 @@
"schema.graphql"
],
"scripts": {
"format": "prettier --write ."
"format": "prettier .",
"format:check": "npm run format -- --check",
"format:fix": "npm run format -- --write",
"pretest": "node util/scripts/pretest.js",
"test": "playwright test --workers 3"
},
"dependencies": {
"shell-quote": "^1.8.1"
},
"devDependencies": {
"@harperdb/code-guidelines": "^0.0.2",
"@playwright/test": "^1.49.0",
"prettier": "^3.3.3"
},
"prettier": "@harperdb/code-guidelines/prettier"
Expand Down
22 changes: 22 additions & 0 deletions playwright.config.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
import { defineConfig, devices } from '@playwright/test';

const NEXT_MAJORS = ['13', '14', '15'];
const NODE_MAJORS = ['18', '20', '22'];

export default defineConfig({
testDir: 'test',
fullyParallel: true,
forbidOnly: !!process.env.CI,
workers: 3,
retries: 2,
expect: {
timeout: 30000,
},
projects: NEXT_MAJORS.flatMap((nextMajor) =>
NODE_MAJORS.map((nodeMajor) => ({
name: `Next.js v${nextMajor} - Node.js v${nodeMajor}`,
use: { versions: { nextMajor, nodeMajor }, ...devices['Desktop Chrome'] },
testMatch: [`test/next-${nextMajor}.test.js`],
}))
),
});
2 changes: 1 addition & 1 deletion schema.graphql
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
type NextCache @table(database: "cache" expiration: 3600) @export {
type NextCache @table(database: "cache", expiration: 3600) @export {
id: ID @primaryKey
headers: Any
content: Bytes
Expand Down
Loading
Loading