Release 2024-12-11 - (expected chart version 5.8.0) #4371
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Move user email updating functions to wire-subsystems * DELETE /self/email: ZLocalUser instead of ZUser * Wrap a function signature * Move 'deleteEmail' to UserStore * Migrate 'removeEmail' to UserSubsystem * Add a change log * Rename an activation code action * Rename UserSubsystem helpers * Remove redundant error interpretation * Elaborate a recusive interpretation cycle * Drop an unused no-password error * Implement removeEmail via RemoveEmailEither This also changes the error type in RemoveEmailEither to the general UserSubsystemError type. Handler-specific reinterpretations are done in the handler instead. * "Fix" timing issue in (old) galley integration tests. * Move reading from the environment to a helper --------- Co-authored-by: Matthias Fischmann <[email protected]>
Master->Develop after release
* Add a unit test for DeleteEmail * Add a unit test for UpdateEmailUnvalidated Co-authored-by: Sven Tennie <[email protected]> * Pull out a mock interpreter function * Add a unit test for LookupActivationCode * Add a unit test for NewActivationCode * Add unit tests for RemoveEmailEither * Add unit tests for requestEmailChange * Update a change log --------- Co-authored-by: Sven Tennie <[email protected]>
…n of the team owner (#4333)
* Add allowExisting field to invitation request * Honor allowExisting flag in InvitationRequest * Test legacy invitation behaviour * Add CHANGELOG entry * Update golden tests
* create rabbitmq exchange * set up DLX * wip * create user queues on client add/update * make it internal server error * make galley compile * make rabbitmq mandatory * fix integration tests * set correct DLX headers * wip * Add a WIP test to consume notifs via `GET /events` * cannon: Roughly implement subscribing notifs from RabbitMQ * gundeck: Start implementing push to rabbitmq * integration: Fix assertion to assert on a real event * integration: Use correct vHost in cannon * cannon: Ensure exchange exists and publish event correctly on WS * NotificationSubsystem: Align names for queues and extract them as top level bindings * gundeck: Push events to RabbitMQ for compatible clients * integration: Assert that acked events don't come back * cannon: Forward client acks to rabbitmq * WIP: Get rid of channel as an explicit param for NotificationSubsystem actions * Get galley to compile. * Fix some easy TODOs. * Use these library better. * resolve rebase conflict better. * fix ghc errors. * Extract rabbitmq channel lookup into helper. * Move setupConsumableNotificationsClient from subsystems to gundeck. * Revert "Get galley to compile." This reverts commit 8a1840d. * wire-subsystems: Fix compile errors in tests * gundeck: doesn't depend on wire-subsystems (yet?) * brig: Remove unnecessary import * cannon: Don't create the queue for clients, expect it to already be there * cannon: close ws connection when something goes wrong * Funky! * Undo funkiness: Create Wire.API.WebSocket with types for comms on the new websocket * WIP: cannon: try to use the new types from wire-api The case split is not the nicest, perhaps we can solve it with one these things: - Bring back the funkiness for couple of commits ago - Use separate types for server to client and client to server messages * wire-api: Cabal file * gen nix stuff * ormolu * Is this a good way of representing websocket messages? * Resolve TODO. amqp doesn't offer a bulk push operation. instead it makes individual pushes performant enough. https://www.rabbitmq.com/docs/publishers * Refactor test, extend coverage. * Suggestions for better module name; removed "websocket" reference from some names. * Roundtrip tests for Message*To*. * Source comments. * debug failing test [WIP] * refactor tests a bit, fix ack, fix typos * fix test, handle connection closed, format * ping pong test * Maintain stable connection to rabbitmq from cannon. [WIP] * Fix typo. * Tune tests. * Remove ping-pong stuff. * Revert "Maintain stable connection to rabbitmq from cannon. [WIP]" This reverts commit 861ee10. problems with this approach: - there is a maximum number of chans / conn. - this is all very complicated and should be done separately. * Test multiple acks and out of order acks * cannon: Refactor code a little and more logging * integration: Deal with events websocket more gracefully, ensure all acks are sent * cannon: Easier to understand cleanup code * Add TODO for tomorrow * cannon: Ensure invalid messages don't accumulate * small re-org of code * Avoid using unsafeRange * Reduce top level functions * integration: Test that old and new clients can co-exist * gundeck: Optimize number of calls to brig * gundeck: Try to not kill brig * Fix typo * gundeck: Remove pairing comment * wire-api: Rename Wire.API.WebSocket -> Wire.API.Event.WebSocketProtocol * gundeck: Don't configure dead-lettering while declaring queues This should be done via Policies: https://www.rabbitmq.com/docs/parameters#policies When done with Policies, we can change our mind about how to deal with dead-lettering later because queues cannot be redeclared with with different headers. * More TODOs * Use direct exchange for user notifications Topic exchange is not very useful for our usecase * integration: Test that users only get notifs meant for them * integration/Notifications: Allow waiting for notifs without a client * Deflake newly written tests * cannon: Don't declare the exchange, its not needed * integration: Make cannon logLevel Warn again * integrations: Throw error if websocket responds with invalid JSON * Add ticket number to FUTUREWORK * Add notification Id to rabbitmq notifs This can be used by clients to detect duplicate deliveries. * Makefile/clean-rabbit: Also cleanup exchanges * gundeck: Get preexisting unit tests to pass * Set correct expiration on events pushed to RabbitMQ * fix a typo in the changelog * Fix a release note * Add changelogs for public and internal API changes * Move around routing key helper functions * Remove commented out code * Detect and flag missed notifications from RabbitMQ (#4317) * Update a change log to reflect changes from PR #4317 --------- Co-authored-by: Akshay Mankar <[email protected]> Co-authored-by: Matthias Fischmann <[email protected]> Co-authored-by: Marko Dimjašević <[email protected]> Co-authored-by: Igor Ranieri <[email protected]>
* charts: Brig rabbitmq config was inside federation block. * charts: Added cassandra config to bg-worker deployment. * changelog: Updated info to explain this is a breaking change on charts.
* brig: make rabbitmq field optional again * Make rabbitmq configuration in brig conditional on federation being enabled * Add CHANGELOG entry * Fix brig no-federation test
* nit: Fixed grammar. * Cannon: Add draining for RabbitMQ consumers. * lint: removed dead code. * Changelog. * Deleted outdated todo. * hi ci
Call migration code instead of faking it. These were previously untested.
* Initial implementation of rabbitmq pool * Set and unset inner channel * Integrate rabbitmq pool into cannon * Add rabbitmq channel finaliser * Create connections when necessary * Add rabbitmq options to cannon * Use Codensity in event tests * Refactor web socket test code * Fix connection pool deadlock * Replace TChan with MVar * introduce EventWebSocket * Fix channel limit test * Fix potential channel leak * Minor cleanups * Remove websocket read thread * Catch AsyncCancelled when RabbitMQ channel is closed * Remove more spurious logs * Keep track of channels in ConnectionPool * Implement connection pool draining * Validate rabbitmq pool options * Add CHANGELOG entry * Regenerate nix packages * Rename connectionLimit → numDictSlices * Fix connection creation bracket * Remove unnecessary type annotation * Replace explicit `async` with `concurrently` * Add rabbitmq username and password to cannon env * Fix rabbitmq CA secret in cannon * fixup! Fix rabbitmq CA secret in cannon * Remove unnecessary uses of withModifiedBackend * Add rabbit-mq mount for cannon in integration --------- Co-authored-by: Stefan Berthold <[email protected]>
…#4351) With consumable-notifications we expect a lot of queues to exist in RabbitMQ. Using regex to filter here would ensure this loop doesn't keep going over all of those queues over and over again.
* Log uncaught IO exceptions in cargohold * Add CHANGELOG entry
* A scim token can now be associated with an IDP explicitly on creation * STM-based integration test * bump token limit to 8 for integration tests * Fix: use Versioned for public end-points in integration tests. * Add docs. --------- Co-authored-by: Matthias Fischmann <[email protected]>
------- Co-authored-by: Stefan Berthold <[email protected]> Co-authored-by: Leif Battermann <[email protected]>
The generated file already exists in our repo and won't change.
* move ClientCapability ClientSupportsConsumableNotifications to APIv8 * repair overlapping version bound * Freeze API version 7, create dev API version 8. * Avoid catch-all pattern if it doesn't have any benefit. (This way we get an error if we forget that spot during the next version freeze.) --------- Co-authored-by: Stefan Berthold <[email protected]>
…notifs are not part of the enum (#4366) * ClientAPI: Ensure unnecessary nesting of capabilities is removed in API v7 * ClientAPI: Ensure consumable-notifications capability doesn't leak to API <= V7 * wire-api: Avoid adding double version suffix * wire-api: Name versioned Client consistently in swagger * Regenerate swagger-v7.json * wire-api: Add golden test for ClientCapabilityListV7
* Keep track of unacked messages * Clear inner channel mvar on channel reset * Add channel reset test * Revert "Keep track of unacked messages" This reverts commit ad0b2d5. * Disable channel reset in connection pool * move retryEnabled to pool opts * use correct rabbitMQ credentials * reset log cannon's log level to warn * Add CHANGELOG entry * Restore rabbitmq admin port * Add rabbitmq port to integration configuration * Use servant client to delete connections * Use connection ID for the connection name * Linter * Remove comments Co-authored-by: Akshay Mankar <[email protected]> --------- Co-authored-by: Leif Battermann <[email protected]> Co-authored-by: Akshay Mankar <[email protected]>
* spar: Move changes for named scim auth-tokens to V8 * Regen swagger-v7.json Also includes changes from #4368 * Remove redundant wrapper function in spar. * Clarify internal comment. * Bring back the old constraints on number of idps in create-idp V7. --------- Co-authored-by: Marko Dimjašević <[email protected]> Co-authored-by: Matthias Fischmann <[email protected]>
echoes-hq
bot
added
echoes: technical-roadmap/throughput
zebot
added
the
ok-to-test
fisx
force-pushed
the
release_2024-12-11_16_35
branch
from
bf15031 to
071f34e
Compare
fisx
approved these changes
Dec 12, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
https://wearezeta.atlassian.net/browse/WPB-14609
[2024-12-11] (Chart Release 5.8.0)
Release notes
[RabbitMQ events] Notifications are now also sent via RabbitMQ. Therefore RabbitMQ is now a required dependency for Cannon and Gundeck. Cassandra is now a required dependency for Cannon and Background-Worker. Both of them need access to the Gundeck keyspace. These are breaking changes for Charts. ([WPB-10308] Use RabbitMQ queues for notifications #4272, Fix message acks on wrong rabbitmq channels #4358, brig: make rabbitmq field optional again #4340)
If brig's server values config has the field
emailSMS.team, the correct value for the personal user to team invitation URL must be set underemailSMS.team.tExistingUserInvitationUrl. Otherwise the URL will point to a path under the account pages and therefore a value forexternalUrls.accountPagesis required. (WPB-14284 personal user invitation URL configmap fixed #4341)API changes
POST /teams/:tid/invitationsgained a new optional fieldallow_existing, which controls whether an existing personal user should be invited to the team (Preserve old invitation behaviour in v6 #4336)Features
Welcome email for new team owner. (WPB-11183 confirmation email after team creation and account migration of the team owner #4333)
Added inviter's email to
GET /teams/invitation/infoendpoint. ([WPB-12098] add inviter email to in-game invitation #4332)Bug fixes and other updates
Updated
nginzconfig for personal user to team flow (WPB-10658 invitation and acceptance of individual users to teams fix nginz #4334)Freeze API version 7, create new dev version 8. Also update checklist. ([WPB-11973] Freeze API version 7, create dev API version 8. #4356, [WPB-11973] Freeze API version 7, create dev API version 8. #4356)
Fixed config for personal user to team invitation URL template. (WPB-14284 personal user invitation URL configmap fixed #4341)
Fixed search index after personal user creates team (WPB-14856 fix: Personal user to team owner not listed in team search #4362)
Documentation
Internal changes
charts/wire-server-enterpriseis a Helm chart to run thewire-server-enterpriseservice. This service can only be deployed with an image pull secret (the
registry is not open to public.) (Add helm chart for wire-server-enterprise #4359)
[Polysemy] Move email update and remove operations to effects ([WPB-8881] Move email update and remove operations to effects #4316, [WPB-8881] Move email update and remove operations to effects #4316)
Log uncaught IO exceptions in cargohold (Log uncaught IO exceptions in cargohold #4352)
Updated email templates to v1.0.124 (Update email templates to v1.0.124. #4328)
charts/galley: Make missing mls keys a templating error. Update MLS docs. (MLS: Make missing MLS removal keys a template error #4369)
[RabbitMQ events] New endpoint
GET /eventsfor consuming events is added (in API V8).GET /eventsendpoint whenever the system detects this happening. The next acknowledgement of the message makes this notification not appear anymore until the next notification is missed. ([WPB-10308] Use RabbitMQ queues for notifications #4272)POST /i/users/:uid/clients/:cid/consumable-notificationsis added ([WPB-10308] Use RabbitMQ queues for notifications #4272)