[nexus] webhooks #7277
[nexus] webhooks #7277
Conversation
This commit adds (unimplemented) public API endpoints for managing Nexus webhooks, as described in [RFD 364][1]. [1]: https://rfd.shared.oxide.computer/rfd/364#_external_api
Co-authored-by: Augustus Mayo <[email protected]>
hawkw
force-pushed
the
eliza/webhook-models
branch
from
51f7f8e to
139cfe6
Compare
hawkw
force-pushed
the
eliza/webhook-models
branch
from
df87c31 to
326b3f7
Compare
needs testing, but i'd like to do that after finishing the DB queries that use it...
gotta redo this to take the params but need to rebase
hawkw
force-pushed
the
eliza/webhook-models
branch
from
140aea4 to
0b80c8f
Compare
this way, rather than transforming the globs into patterns that we can match in the database every time an event is dispatched, we instead use globs to generate "exact" subscriptions to specific named event classes. this avoids doing a full scan over the subscription table every time an event is dispatched, since we can look up subscribed receivers by event class name instead. we'll have to do this processing on receiver creation or when a subscription is added, and when new event classes are added (e.g. on software updates). thanks @andrewjstone for pointing me in this direction!
hawkw
force-pushed
the
eliza/webhook-models
branch
from
41cf0b0 to
2bc5925
Compare
|
I think I've come around a bit to @andrewjstone's proposal that the event classes be a DB enum, so I'm planning to change that. I'd like to have a way to include a couple "test" variants in there that aren't exposed in the public API, so I'll be giving some thought to how to deal with that. |
Glob subscription entries in |
|
As far as GCing old events from the event table, dispatching an event should probably add a count of the number of receivers it was dispatched to, and then when we successfully deliver the event, we increment a count of successes. That way, we would not consider an event entry eligible to be deleted unless the two counts are equal; we want to hang onto events that weren't successfully delivered so any failed deliveries can be re-triggered. GCing an event would also clean up any child delivery attempt records. |
| // This performs a `SELECT ... FOR UPDATE SKIP LOCKED` on the | ||
| // `webhook_event` table, returning the oldest webhook event which has not | ||
| // yet been dispatched to receivers and which is not actively being | ||
| // dispatched in another transaction. | ||
| // NOTE: it would be kinda nice if this query could also select the | ||
| // webhook receivers subscribed to this event, but this requires | ||
| // a `FOR UPDATE OF webhook_event` clause to indicate that we only wish | ||
| // to lock the `webhook_event` row and not the receiver. | ||
| // Unfortunately, I don't believe Diesel supports this at present. |
There was a problem hiding this comment.
Unfortunately, this isn't going to work with CRDB, as it doesn't actually implement SKIP LOCKED. We're going to have to redesign the mechanism that prevents duplicate delivery creation. Another option could be to change the webhook_delivery table to have a UNIQUE constraint on (rx_id, event_id), but this doesn't work with the way I wanted to implement the resend event API (creating a new entry in webhook_delivery.
I think the correct thing to do here is something like:
INSERT INTO omicron.public.webhook_delivery (
{rx_id},
{event_id},
/* ... */
)
WHERE NOT EXISTS (
SELECT 1 FROM omicron.public.webhook_delivery
WHERE rx_id == {rx_id} AND event_id == {event_id}
)so that we can prevent creating duplicate deliveries without a lock. But, I can't seem to find a way to do INSERT ... WHERE NOT EXISTS in Diesel, so this probably needs to be a CTE...
No description provided.