Release 2024-10-25 - (expected chart version 5.6.0)

[zebot]

[2024-10-25] (Chart Release 5.6.0)

Release notes

• To remove phone keys from brig's user_keys table an ad hoc data-migration can be run. See PR [DO NOT MERGE] WPB-10058 remove phone keys ad hoc data migration #4146 which contains the implementation. (WPB-10058 delete phone column in brig's user table #4130)

• Because the phone column is deleted from Brig's user table in a schema
  migration, temporarily there might be 5xx errors during deployment if Wire
  server 5.4.0 was not deployed previously. To avoid these errors, please deploy
  the Wire server 5.4.0 release first. (WPB-10058 delete phone column in brig's user table #4130)

• With this release it will be possible to invite personal users to teams. In brig's config, emailSMS.team.tExistingUserInvitationUrl is required to be set to a value that points to the correct teams/account page.
  If emailSMS.team is not defined at all in the current environment, the value of externalUrls.teamSettings (or, if not present, externalUrls.nginz) will be used to construct the correct url, and no configuration change is necessary. (WPB-10658 invitation and acceptance of individual users to teams #4229)

• charts/wire-server: There is a new config value called background-worker.config.enableFederation which defaults to false. This must be kept in sync with tags.federation. (WPB-10660 Enable and deploy background worker in non federation environments #4243)

• If you are mapping an email address to the externalId field in the
  scim schema, please check the following list for items that apply to
  you and recommended steps before/during/after upgrade.

  • Situation: the emails field of in your scim user records is
    empty.

    What you need to do: change your schema mapping to contain the
    same address in externalId and (as a record with one element) in
    emails.

  • Situation: the emails field of your scim user records is
    non-empty.

    What you need to do: make sure emails contains exactly one
    entry, which is the email from externalId. If there is a
    discrepancy, the address from emails will become the new
    (unvalidated) address of the user, and the user will receive an
    email to validate it. If the email cannot be sent or is ignored
    by the recipient, the valid address will not be changed. (Handle emails field in scim user record #4221)

• A schema migration drops column 'phone' from Brig's 'team_invitation' table. Previous releases were still reading this column. As there is no Team Settings UI action to enter a phone number, this reading will not miss to read actual phone numbers. Therefore, during deployment this will lead to benign 5xx errors. ([WPB-8707] Remove phone functionality in the development client API version #4149)

• Password hashing is now done using argon2id instead of scrypt. The argon2id parameters can be configured using these options:

  brig:
    optSettings:
      setPasswordHashingOptions:
        iterations: ...
        memory: ... # memory needed in KiB
        parallelism: ...
  galley:
    settings:
      passwordHashingOptions:
        iterations: ...
        memory: ... # memory needed in KiB
        parallelism: ...

  These have default values, which should work for most deployments. Please see documentation on config-options for more. ([feat] Allow configuring argon2id parameters #4291)

• Config value gundeck.config.bulkPush has been removed. This is purely an
  internal change, in case the value was overriden to false, operators might see
  more spiky usage of CPU and memory from gundeck due to bulk processing. (gundeck: Remove bulkPush config option #4290)

API changes

• A new endpoint POST /teams/invitations/accept allows a non-team user to accept an invitation to join a team (WPB-10658 invitation and acceptance of individual users to teams #4229)

• Services allowlist are blocked by 409 (mls-services-not-allowed) for teams with default protocol MLS. (Block services whitelist for teams with default protocol MLS #4266)

• The POST /clients and PUT /clients/:cid endpoints support a new capability "consume-notifications" ([WPB-11163] Support for a consumable notifications capability #4259)

• New variant in API version 7 of endpoints for creating and listing SCIM tokens that support a name field. New endpoint in version 7 for updating a SCIM token name. (WPB-685 give SCIM connections human readable names #4307)

• All the phone number-based functionality is removed from the client API v6 ([WPB-8707] Remove phone functionality in the development client API version #4149)

• The team CSV export endpoint has gained two extra columns: last_active and status. The streaming behaviour has also been improved. (Refactor CSV export #4293)

• The changes to the capabilities field of the Client structure, introduced in v6, have now been postponed to v7 (Finalise version 6 #4179)

• Finalise version 6 and introduce new development version 7 (Finalise version 6 #4179, Finalise version 6 #4179)

• From API version 7 the GET /mls/public-key and GET /conversations/one2one/:domain/:uid endpoints now take a format query parameter which can be either raw (default, for raw base64-encoded keys) or jwk (for JWK keys) (Add format paramter to mls public key endpoint #4216, Return MLS public keys as part of getting a 1:1 conversation #4224)

• GET /conversations/one2one/:domain/:uid now returns public_keys along with the conversation containing all MLS public keys for the backend which will host this conversation (since v6). (Return MLS public keys as part of getting a 1:1 conversation #4224)

• Remove the ability to set the TTL of a feature flag. Existing TTLs are still retrieved and returned as before. Note that this only applies to the conferenceCalling feature, as none of the others supported TTL anyway. (One to one SFT feature flag config #4164)

• Add useSFTForOneToOneCalls as a config option for the Conference Calling feature flag and make its lock status explicit. (One to one SFT feature flag config #4164)

• Add endpoint to upgrade a personal user to a team owner ([WPB-10708] personal account to own team #4251)

Features

• DB migration for dropping phone column from user table (WPB-10058 delete phone column in brig's user table #4130)

• A text status field was added to user and user profile (WPB-10204 Add text status field to user (profile) data #4155)

• Allow an existing non-team user to migrate to a team (WPB-10658 invitation and acceptance of individual users to teams #4229, WPB-10658 invitation and acceptance of individual users to teams #4229)

• Makes it impossible for a user to join an MLS conversation while already under legalhold (at least pending)

  This implies two things:

  1. If a user is under legalhold they cannot ever join an MLS conversation, not even an MLS self conversation.
  2. A user has to reject to be put under legalhold when they want to join an MLS conversation (ignoring the request to be put under legalhold is not enough). ([WPB-10772] Make it impossible for a user under legalhold to join an MLS conversation #4242)

• Email template for inviting a personal user to a team added (WPB-11050 email templates for invitation of personal user to existing team #4310)

• Clients can declare to be supporting a capability for consuming notifications ([WPB-11163] Support for a consumable notifications capability #4259)

• New endpoint to revoke an OAuth session (WPB-1333 OAuth endpoint to revoke single refresh tokens #4213)

• Adds a field which contains a list of all active sessions to each OAuth application in the response of GET /oauth/applications (WPB-1334 extend list of OAuth apps with active refresh token ids #4211)

• SCIM's emails field is now handled and the external ID is not restricted to being an email anymore (Handle emails field in scim user record #4221)

• Added human readable names for SCIM tokens (WPB-685 give SCIM connections human readable names #4307)

• allow subconversations for MLS 1-1 conversations (allow subconversations for MLS 1-1 conversation #4133)

• Allow configuring Argon2id parameters ([feat] Allow configuring argon2id parameters #4291)

• Deny requests for a legalhold device for users who are part of any MLS conversations ([WPB-10783] Prevent MLS-Legalhold interactions #4245)

• Allow setting of Kubernetes annotations for the coturn Service. (coturn: Allow setting of K8s annotations at the Service #4189)

• Add initialConfig setting for the mls feature flag (Initial MLS configuration for new teams #4262)

• added open telemetry instrumentation for brig, galley, gundeck and cannon ([WPB-10092] open telemetry instrumentation  #3901)

• Send confirmation email after adding a personal user to a new team (Send confirmation email after upgrade to team owner #4253)

• The SFT and turn usernames returned by /calls/config/v2 are now deterministically computed from the user ID (Use user ID hashes as SFT usernames #4156)

• Use latest stable RabbitMQ version (3.13.7) and Helm chart (14.6.9). Please
  note that this minor RabbitMQ version upgrade (3.11.x to 3.13.x) may need
  special treatment regarding existing RabbitMQ instances. See
  https://www.rabbitmq.com/docs/upgrade#rabbitmq-version-upgradability . The major
  Helm chart version upgrade may (depending on your setup/values) need attention
  as well: https://github.com/bitnami/charts/tree/main/bitnami/rabbitmq#upgrading (upgrade to latest stable rabbitmq #4227)

Bug fixes and other updates

• Fixed API version check. It has now precedence over other checks like e.g. method check. ([fix] API version check comes before method check #4152)

• Fix handling of defaults of mlsE2EID feature config (Fix clrProxy field of MLSE2EId feature flag #4233)

• Match cipher suite tag in query parameters against key packages on replacing key packages (WPB-10207 Match cipher suite tag in query parameters against key packages on replacing key packages #4158)

• Users with SAML-SSO are allowed to delete their email address on the rest api. If they do that, the search indices are not updated correctly, and finding the user by the removed email address is still possible. ([WPB-11122] Disallow searching user by old email #4260)

• Exclude exception message from error response (Stop leaking ES error info. #4153)

• Return HTTP 400 instead of 500 when property key is not printable ASCII (Introduce PropertySubsystem #4148)

• move cipher suite updates into the commit lock (move ciphersuite updates into the commit lock #4151)

• Fix feature flag default calculation for mlsMigration and enforceFileDownloadLocation (Fix feature flag defaults #4265)

• Allow setting existing properties even if we have max properties (Introduce PropertySubsystem #4148)

• removed spam from nginx (nginz) by using the new style http/2 directive ([WPB-10092] open telemetry instrumentation  #3901)

• brig: Make GET /services/tags work again (brig: Make GET /services/tags work again #4250)

• Process bounce and complaint notifications from SES correctly. (Process bounce and complaint notifications from SES correctly #4301)

Documentation

• Call graph of federated endpoints was removed from the docs (WPB-11502 remove transitive-anns #4299)

• Restored LegalHold internal API swagger as part of Brig. ( [chore] Remove more dead code, restore LegalHold internal API swagger #4191)

• Fix: show openapi docs for blocked versions (Fix swagger #4309)

• Deleted proteus-specific test documentation tags and added some new tags to MLS tests (Move test documentation tags from proteus to MLS #4240)

• Fix openapi validation errors (Validate swagger #4295, Validate swagger #4295)

• Re-introduce test case tags for BSI audit (revert [WPB-8628] Clean up syntax of test cases that occur in BSI audit. #4041) (Re-introduce test case tags for BSI audit (revert #4041). #4192)

Internal changes

• Introduced API versioning and version negotiation for external LegalHold Service supporting v0 and v1 ([WPB-11188] LegalHold support V1 #4284)

• Read sftTokenSecret from secrets.yaml and mount to /etc/wire/brig/secrets/sftTokenSecret by default (Read sftToken from secrets.yaml #4214)

• Added node based topology constraint to ensure pods are distributed uniformly on all nodes. (Wpb 10335 | Ensure pods are distributed evenly on each k8s node #4222)

• Move smallstep-accomphelm charts towireapp/helm-charts` (Remove smallstep-accomp helm chart #4204)

• Remove coturn helm chart. It is moved to wireapp/coturn. (WPB-10581: Remove coturn helm chat #4209)

• Additional test for password reset, port tests to new integration test suite (WPB-11000 Test password reset with wrong key/code should fail #4249)

• Remove unused invitation tables from brig. (WPB-11101 remove invitation tables from brig #4263)

• Improve abstraction in the invitation store and hide DB interaction-specific internal types from the application code. ([WPB-11101] Refactor invitation store #4280)

• Move some invitation handling from brig to wire-subsystems.

  • introduce cyclically dependent effects: UserSubsystem, AuthenticationSubsystem (see Brig.CanonicalInterpreter).
  • introduce TeamInvitationSubsystem with operations inviteUser, internalCreateInvitation.
  • add verifyPassword to AuthenticationSubsystem.
  • add sendInvitationMail, sendInvitationMailPersonalUser to EmailSubsystem.
  • add getTeamSize to IndexedUserStore (this is morally internal to wire-subsystems, and making another ES subsystem would mean adding a lot of code everywhere).
  • add updateUserTeam to UserStore.
  • add acceptTeamInvitation, internalFindTeamInvitation to UserSubsystem.
  • make a few small rest api handlers in brig polysemic (Handler -> Sem). (WPB-11217 Move accept team invitation to user subsystem #4264)

• tools/db/team-info: collects last login times of all team members ([WPB-11301] db script for collecting last login times of all team members #4274)

• Introduce length-preserving function mapRange to replace Functor instance for Range data type. ([WPB-11386] Introduce length-preserving function mapRange to replace fmap. #4279)

• TransitiveAnns compiler plugin was removed (WPB-11502 remove transitive-anns #4299)

• Servantify internal routing table for proxy. ([WPB-1220] servantify proxy internal #4296)

• Servantify gundeck internal api ([WPB-1228] Servantify gundeck internal api #4246)

• Removed indexReindex and indexReindexIfSameOrNewer from internal Brig/SearchIndex. (Move search operations to UserSubsystem #4188)

• Introduced ElasticSearch effects related to user search. (Move search operations to UserSubsystem #4188)

• Brig was refactored by pulling out email block-listing into a wire subsystems effect, and its actions are exposed via the user subsystem. ([WPB-8892] Add block list operations to the user subsystem #4167)

• charts/wire-server: Deploy background-worker even when tags.federation is false (Drain rabbitmq consumers slowly from Cannon #4342, Fix FromJSON AmqpEndpoint error message #4248)

• Updated email templates to v1.0.122 (Update email templates to v1.0.122. #4308)

• Refactor feature flags

  • Improved naming slightly. Features types are now called Feature, LockableFeature and LockableFeaturePatch
  • Turned AllFeatures into an extensible record type
  • Removed WithStatusBase barbie.
  • Deleted obsolete computeFeatureConfigForTeamUser
  • Abstracted getFeature and setFeature
  • Abstracted getAllTeamFeatures (Feature flag refactoring (part 1) #4181)

• Clean up and reorganise feature flag endpoints (Feature flag refactoring (part 2) #4193)

• Clean up feature default configuration code (Feature flag refactoring (part 3) #4196)

• Add federation-v1 environment for testing compatibility of the federation API with version 1 (Federation V1 test setup #4125)

• nginz/local-conf: Update list of endpoints (Enable manual usage of locally running wire-server  #4176)

• Expose gundeck internal API on swagger. Mv some types and routes to wire-api. (Gundeck internal API swagger #4247)

• dockerephemeral: Use inbucket for SMTP (Enable manual usage of locally running wire-server  #4176)

• Makefile: Add target crm to run services tuned for manual usage (Enable manual usage of locally running wire-server  #4176)

• Postgresql helm chart is removed from charts/ directory and migrated to wireapp/helm-charts repo (remove postgres wrapper chart from wire-server #4208)

• Simplify NewTeam and related types and remove lenses (NewTeam types refactoring #4257)

• Add openapi validation test to integration ([WPB-10314] validate swagger: add swagger linter to integration tests (so it'll run in CI). #4302)

• Optimize getting a lot of users by concurrently getting target users (Concurrently fetch user profiles from the DB #4140)

• charts/{brig,galley}: Allow setting a preStop hook for the deployments ( charts/{brig,galley}: Allow setting a preStop hook for the deployments #4200)

• Introduce proeprty subsytem (Introduce PropertySubsystem #4148)

• Changed default password hashing from Scrypt to Argon2id. ([Brig] Move password verification to the AuthenticationSubsystem, move to Argon2id with new settings. #4271)

• Factored out our Email type in favour of EmailAddress from email-validate. ([chore] Simplify email types #4206)

• Move CSV export test to integration ([WPB-11368] Test for export team member CSV #4292)

• add the TODO pattern and the todo function to Imports (add the todo function and the TODO pattern #4198)

• Refactor user feature logic (Simplify user feature logic #4178)

• Remove UserAccount and ExtendedUserAccount and their fields to the User type (Refactor user types #4275)

• Started weeding out dead code. ([chore] Weed out dead code. #4170)

• New user subsystem operation getAccountsBy for complex account lookups. ([WPB-8887] wire-subsystems: implement the GetBy* account queries, includes InvitationCodeStore. #4218)

• Added warning when deploying wire-server helm chart with User/Team creation over internet enabled. (add warning when team/user creation is enabled over internet #4212)

[akshaymankar]

Please do not forget to merge https://github.com/zinfra/cailleach/pull/2354 before the release.