Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PeprStore, K8s Fluent API & Watch Action #180

Merged
merged 124 commits into from
Oct 2, 2023
Merged
Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
124 commits
Select commit Hold shift + click to select a range
028c990
start some trouble
jeff-mccoy Jul 7, 2023
7ab1a68
simplify observe, change validate from boolean -> resp obj
jeff-mccoy Jul 16, 2023
2e8da69
update deps
jeff-mccoy Jul 22, 2023
562eb0e
update prettier to use promise
jeff-mccoy Jul 22, 2023
f60fe8b
Finish Mutate/validate/Watch bindings
jeff-mccoy Jul 23, 2023
7606a6e
simplify watch code (non-fluent as it wont be user-facing)
jeff-mccoy Jul 23, 2023
68fef05
wip leaderelection things
jeff-mccoy Jul 23, 2023
79d0c46
actually implement the Pepr Store wiring 😭
jeff-mccoy Jul 31, 2023
400a01f
fix global namespace ignores
jeff-mccoy Jul 31, 2023
a91ce1d
add /healthz tests
jeff-mccoy Jul 31, 2023
518ff5b
Merge branch 'main' into shifting-sands
jeff-mccoy Jul 31, 2023
4e2c6f4
lint
jeff-mccoy Jul 31, 2023
e455ff8
leader election -> dedicated pod
jeff-mccoy Aug 1, 2023
d5a316f
Add store subscribe
jeff-mccoy Aug 4, 2023
ca764a4
watcher statefulset -> deployment + more webhook reorg
jeff-mccoy Aug 4, 2023
d481d31
Merge branch 'main' into shifting-sands
jeff-mccoy Aug 5, 2023
a1de123
conditionally generate the watch deployment
jeff-mccoy Aug 5, 2023
d374163
pass errors from store write operations
jeff-mccoy Aug 5, 2023
26f64fd
introduce debounced/throttled read/write sync store + onready handler
jeff-mccoy Aug 6, 2023
855bf01
wip crd madness
jeff-mccoy Aug 6, 2023
51cae98
Merge branch 'main' into shifting-sands
jeff-mccoy Aug 6, 2023
8eda32e
Merge remote-tracking branch 'origin/shifting-sands' into shifting-sands
jeff-mccoy Aug 6, 2023
7175435
wip: move to Kube() api + PeprStore CRD
jeff-mccoy Aug 7, 2023
bb91655
add jitter for controller store creation
jeff-mccoy Aug 13, 2023
e2c614b
continue cleanup rbac/raw api
jeff-mccoy Aug 13, 2023
d623dd8
update e2e to match capability log changes
jeff-mccoy Aug 14, 2023
d350d2f
add readiness probes to wait for pepr store
jeff-mccoy Aug 14, 2023
02bdb50
Merge branch 'main' into shifting-sands
jeff-mccoy Aug 14, 2023
f9597fa
simplify fluent k8s fluent interfaces
jeff-mccoy Aug 15, 2023
3cef70c
start switching tests to kube fluent api
jeff-mccoy Aug 15, 2023
e340dac
Merge branch 'main' into shifting-sands
jeff-mccoy Aug 15, 2023
714080d
move more k8s imports to k8s from client-node
jeff-mccoy Aug 15, 2023
20d3609
Merge remote-tracking branch 'origin/shifting-sands' into shifting-sands
jeff-mccoy Aug 15, 2023
1977843
Merge branch 'main' into shifting-sands
jeff-mccoy Aug 24, 2023
501f240
add errors class for onError handling
jeff-mccoy Sep 1, 2023
d22151d
add module.ts unit test
jeff-mccoy Sep 1, 2023
63f3823
add unit test mocks/coverage/ts testing
jeff-mccoy Sep 1, 2023
cd0b737
wip: standardize ES2020 class structure
jeff-mccoy Sep 1, 2023
59117dc
e2e -> journey & class method this binding
jeff-mccoy Sep 2, 2023
4c03a47
lame wildcard issue in npm scripts
jeff-mccoy Sep 2, 2023
89c6d23
convert stdout Buffer -> string again
jeff-mccoy Sep 2, 2023
03b89c8
more journey test cleanup
jeff-mccoy Sep 2, 2023
3fc247a
validate tsx loader in ci
jeff-mccoy Sep 2, 2023
e0addce
try without nyc
jeff-mccoy Sep 2, 2023
f90ea54
but whyyy
jeff-mccoy Sep 3, 2023
88c235e
ava/sinon/nyc -> jest
jeff-mccoy Sep 3, 2023
8f7d408
dep updates
jeff-mccoy Sep 3, 2023
2ae2e0c
add test adr
jeff-mccoy Sep 3, 2023
4a6411b
use json instead of js config due to gitignore
jeff-mccoy Sep 3, 2023
4e221c6
always generate the data.json for unit tests
jeff-mccoy Sep 3, 2023
296421a
standardize private/public class methods
jeff-mccoy Sep 3, 2023
d2934c5
Merge branch 'main' into 235-pepr-always-fails-when-you-select-audit-…
jeff-mccoy Sep 3, 2023
894cb68
remove need for disable node-wide tls verification
jeff-mccoy Sep 4, 2023
ac38bdf
Merge remote-tracking branch 'origin/235-pepr-always-fails-when-you-s…
jeff-mccoy Sep 4, 2023
c27b91e
Merge branch 'main' into shifting-sands
jeff-mccoy Sep 5, 2023
593bf64
Merge branch 'main' into shifting-sands
jeff-mccoy Sep 5, 2023
dc033d4
continue post-merge cleanup
jeff-mccoy Sep 5, 2023
749fc7b
manage delayed controller boot for PeprStore in journey tests
jeff-mccoy Sep 5, 2023
6d205d3
Merge branch 'main' into shifting-sands
Sep 6, 2023
a8b5e66
re-add k8s fluent api for journey tests
jeff-mccoy Sep 7, 2023
817d875
Merge branch 'main' into shifting-sands
jeff-mccoy Sep 7, 2023
347e709
Merge branch 'main' into shifting-sands
jeff-mccoy Sep 7, 2023
c12c91c
cleanup uuid import for templates
jeff-mccoy Sep 10, 2023
6c91e9d
Merge branch 'main' into shifting-sands
jeff-mccoy Sep 10, 2023
23cda48
more unit tests
jeff-mccoy Sep 10, 2023
ecf558d
Kube() refactors + some unit tests
jeff-mccoy Sep 11, 2023
c3f4ec3
revert uuid rename
jeff-mccoy Sep 11, 2023
b170f63
Merge branch 'main' into shifting-sands
Sep 11, 2023
1d67119
more unit tests
jeff-mccoy Sep 13, 2023
d18dfc8
Merge remote-tracking branch 'origin/shifting-sands' into shifting-sands
jeff-mccoy Sep 13, 2023
49844a2
mock k8s client
jeff-mccoy Sep 14, 2023
482f33e
CreateOrReplace()/Replace() -> Apply()
jeff-mccoy Sep 14, 2023
7356a9b
Merge branch 'main' into shifting-sands
jeff-mccoy Sep 14, 2023
da9c2fa
Remove Kube().Patch(oldObj, newObj) in favor of SSA, Kube().Apply()
jeff-mccoy Sep 14, 2023
2bee372
try to set apiVersion if not passed into the Kube() api in the object
jeff-mccoy Sep 14, 2023
337112e
experiment, maybe terrible but add `given.*` for better Kube() readab…
jeff-mccoy Sep 14, 2023
bc9dd68
Merge remote-tracking branch 'origin/shifting-sands' into shifting-sands
jeff-mccoy Sep 14, 2023
5d21c67
Merge branch 'main' into shifting-sands
jeff-mccoy Sep 14, 2023
5f5d4da
Merge remote-tracking branch 'origin/shifting-sands' into shifting-sands
jeff-mccoy Sep 14, 2023
6be71c0
add pepr store journey tests
jeff-mccoy Sep 15, 2023
28cf066
minor hello-pepr cleanup
jeff-mccoy Sep 15, 2023
921a1f8
add logLevel config to package.json
jeff-mccoy Sep 16, 2023
cebe294
add internal-use destroyModule() function
jeff-mccoy Sep 16, 2023
78d992c
properly manage buildMode/watchMode bindings
jeff-mccoy Sep 16, 2023
8767a83
initial pepr watch binding
jeff-mccoy Sep 16, 2023
b38ba91
add phase to hellopepr watch example
jeff-mccoy Sep 16, 2023
fb3ffda
Merge branch 'main' into shifting-sands
jeff-mccoy Sep 19, 2023
c62d07d
remove prettier + update transitive deps
jeff-mccoy Sep 19, 2023
99e0516
readme updates
jeff-mccoy Sep 20, 2023
27397de
move to node-fetch / k8s client node 1
jeff-mccoy Sep 22, 2023
37758bf
move tls files to lib
jeff-mccoy Sep 22, 2023
7026845
merge watch()/kube() + migrate to internal watch logic
jeff-mccoy Sep 22, 2023
aa40641
migrate to kubernetes-fluent-client
jeff-mccoy Sep 25, 2023
26e74c3
remove k8s mock (now in fluent client)
jeff-mccoy Sep 25, 2023
59e2ae2
Merge branch 'main' into shifting-sands
jeff-mccoy Sep 25, 2023
3f573e5
Add Pepr Store readme
jeff-mccoy Sep 25, 2023
393b4a0
Merge remote-tracking branch 'origin/shifting-sands' into shifting-sands
jeff-mccoy Sep 25, 2023
6f63ace
add fluent client example
jeff-mccoy Sep 25, 2023
c3af6d0
match phases for watch actions
jeff-mccoy Sep 26, 2023
01f04f6
update readme for watch
jeff-mccoy Sep 26, 2023
f956eac
update pepr diagram for watch + store
jeff-mccoy Sep 26, 2023
d8f6b07
fix font color issue in diagram
jeff-mccoy Sep 26, 2023
fdbe7a5
remove async from when() example
jeff-mccoy Sep 26, 2023
92365f6
readme updates
jeff-mccoy Sep 26, 2023
565ad32
readme code sample validation
jeff-mccoy Sep 26, 2023
edf2cd0
lint
jeff-mccoy Sep 26, 2023
db83c42
Update README.md
jeff-mccoy Sep 26, 2023
958ace3
Update docs/actions.md
jeff-mccoy Sep 26, 2023
a23b6d6
Update docs/store.md
Sep 27, 2023
9572c8f
Update README.md
Sep 27, 2023
da99d56
Update docs/actions.md
Sep 27, 2023
2d06b29
Update docs/store.md
jeff-mccoy Sep 27, 2023
c8f3df2
Update docs/store.md
jeff-mccoy Sep 27, 2023
db055f8
Update journey/k8s.ts
jeff-mccoy Sep 29, 2023
ffcbb03
add watch retry / crash handler
jeff-mccoy Sep 30, 2023
f16c0b1
Merge remote-tracking branch 'origin/shifting-sands' into shifting-sands
jeff-mccoy Sep 30, 2023
023d7c1
fix sleep rename in journey tests
jeff-mccoy Sep 30, 2023
23e45a8
add watch processor unit test
jeff-mccoy Sep 30, 2023
3607b30
update deps
jeff-mccoy Oct 1, 2023
d5b853f
delay watch actions until the peprstore is ready
jeff-mccoy Oct 1, 2023
11ca941
add store watch action journey tests
jeff-mccoy Oct 1, 2023
bcb28a1
small readme updates
jeff-mccoy Oct 1, 2023
26c9d1e
fix text escaping on journey test match
jeff-mccoy Oct 1, 2023
0024c29
Merge branch 'main' into shifting-sands
cmwylie19 Oct 2, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ When(a.ConfigMap)
.IsCreatedOrUpdated()
.InNamespace("pepr-demo")
.WithLabel("unicorn", "rainbow")
.Then(request => {
.Mutate(request => {
// Add a label and annotation to the ConfigMap
request
.SetLabel("pepr", "was-here")
Expand Down
38 changes: 31 additions & 7 deletions hack/e2e.test.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -213,11 +213,11 @@ test.serial("E2E: `pepr deploy`", async t => {
t.is(s1.data["magic"], "Y2hhbmdlLXdpdGhvdXQtZW5jb2Rpbmc=");
t.is(
s1.data["binary-data"],
"iCZQUg8xYucNUqD+8lyl2YcKjYYygvTtiDSEV9b9WKUkxSSLFJTgIWMJ9GcFFYs4T9JCdda51u74jfq8yHzRuEASl60EdTS/NfWgIIFTGqcNRfqMw+vgpyTMmCyJVaJEDFq6AA=="
"iCZQUg8xYucNUqD+8lyl2YcKjYYygvTtiDSEV9b9WKUkxSSLFJTgIWMJ9GcFFYs4T9JCdda51u74jfq8yHzRuEASl60EdTS/NfWgIIFTGqcNRfqMw+vgpyTMmCyJVaJEDFq6AA==",
);
t.is(
s1.data["ascii-with-white-space"],
"VGhpcyBpcyBzb21lIHJhbmRvbSB0ZXh0OgoKICAgIC0gd2l0aCBsaW5lIGJyZWFrcwogICAgLSBhbmQgdGFicw=="
"VGhpcyBpcyBzb21lIHJhbmRvbSB0ZXh0OgoKICAgIC0gd2l0aCBsaW5lIGJyZWFrcwogICAgLSBhbmQgdGFicw==",
);
t.log("Validated secret-1 Secret data");

Expand All @@ -244,6 +244,9 @@ test.serial("E2E: `pepr dev`", async t => {
try {
const cmd = await new Promise(peprDev);

const healthz = await testHealthz();
t.is(healthz, true);

await testAPIKey();

cmd.kill();
Expand All @@ -270,6 +273,25 @@ test.serial("E2E: `pepr metrics`", async t => {
}
});

async function testHealthz() {
// Ignore TLS verification
process.env.NODE_TLS_REJECT_UNAUTHORIZED = "0";
Fixed Show fixed Hide fixed

const base = "https://localhost:3000/healthz";

const healthzOk = await fetch(base);

// Restore TLS verification
process.env.NODE_TLS_REJECT_UNAUTHORIZED = "1";

if (healthzOk.status !== 200) {
await delay2Secs();
return testHealthz();
}

return true;
}

async function testAPIKey() {
// Ignore TLS verification
process.env.NODE_TLS_REJECT_UNAUTHORIZED = "0";
Expand Down Expand Up @@ -325,10 +347,12 @@ function peprDev(resolve, reject) {
return !data.replace(/\s+/g, " ").includes(expectedLine);
});

console.log(
"\x1b[36m%s\x1b[0m'",
"Remaining expected lines:" + JSON.stringify(expectedLines, null, 2)
);
if (expectedLines.length > 0) {
console.log(
"\x1b[36m%s\x1b[0m'",
"Remaining expected lines:" + JSON.stringify(expectedLines, null, 2),
);
}

// If all expected lines are found, resolve the promise
if (expectedLines.length < 1) {
Expand Down Expand Up @@ -414,7 +438,7 @@ async function getPodLogs(namespace, labelSelector) {
undefined,
undefined,
undefined,
labelSelector
labelSelector,
);
const pods = res.body.items;

Expand Down
Loading