Merge branch 'master' into 7168-export-editable-mapping-as-segmentation

scalableminds · Jan 9, 2025 · 314a057 · 314a057
2 parents 721fc70 + 97ce494
commit 314a057
Show file tree

Hide file tree

Showing 80 changed files with 1,211 additions and 803 deletions.
diff --git a/CHANGELOG.unreleased.md b/CHANGELOG.unreleased.md
@@ -14,6 +14,10 @@ For upgrade instructions, please check the [migration guide](MIGRATIONS.released
 - Added the total volume of a dataset to a tooltip in the dataset info tab. [#8229](https://github.com/scalableminds/webknossos/pull/8229)
 - Optimized performance of data loading with “fill value“ chunks. [#8271](https://github.com/scalableminds/webknossos/pull/8271)
 - Added the option to export proofreading as segmentation [#8286](https://github.com/scalableminds/webknossos/pull/8286)
+- The fill tool can now be adapted so that it only acts within a specified bounding box. Use the new "Restrict Floodfill" mode for that in the toolbar. [#8267](https://github.com/scalableminds/webknossos/pull/8267)
+- Added the option for "Selective Segment Visibility" for segmentation layers. Select this option in the left sidebar to only show segments that are currently active or hovered.  [#8281](https://github.com/scalableminds/webknossos/pull/8281)
+- A segment can be activated with doubleclick now. [#8281](https://github.com/scalableminds/webknossos/pull/8281)
+- It is now possible to select the magnification of the layers on which an AI model will be trained. [#8266](https://github.com/scalableminds/webknossos/pull/8266)
 
 ### Changed
 - Renamed "resolution" to "magnification" in more places within the codebase, including local variables. [#8168](https://github.com/scalableminds/webknossos/pull/8168)
@@ -22,22 +26,29 @@ For upgrade instructions, please check the [migration guide](MIGRATIONS.released
 - Improved the default colors for skeleton trees. [#8228](https://github.com/scalableminds/webknossos/pull/8228)
 - Allowed to train an AI model using differently sized bounding boxes. We recommend all bounding boxes to have equal dimensions or to have dimensions which are multiples of the smallest bounding box. [#8222](https://github.com/scalableminds/webknossos/pull/8222)
 - Within the bounding box tool, the cursor updates immediately after pressing `ctrl`, indicating that a bounding box can be moved instead of resized. [#8253](https://github.com/scalableminds/webknossos/pull/8253)
+- Improved the styling of active tools and modes in the toolbar. [#8295](https://github.com/scalableminds/webknossos/pull/8295)
 
 ### Fixed
 - Fixed that listing datasets with the `api/datasets` route without compression failed due to missing permissions regarding public datasets. [#8249](https://github.com/scalableminds/webknossos/pull/8249)
 - A "Locked by anonymous user" banner is no longer shown when opening public editable annotations of other organizations. [#8273](https://github.com/scalableminds/webknossos/pull/8273)
 - Fixed a bug that uploading a zarr dataset with an already existing `datasource-properties.json` file failed. [#8268](https://github.com/scalableminds/webknossos/pull/8268)
 - Fixed the organization switching feature for datasets opened via old links. [#8257](https://github.com/scalableminds/webknossos/pull/8257)
 - Fixed that uploading an NML file without an organization id failed. Now the user's organization is used as fallback. [#8277](https://github.com/scalableminds/webknossos/pull/8277)
-- Fixed that the frontend did not ensure a minium length for annotation layer names. Moreover, names starting with a `.` are also disallowed now. [#8244](https://github.com/scalableminds/webknossos/pull/8244)
+- Fixed that the frontend did not ensure a minimum length for annotation layer names. Moreover, names starting with a `.` are also disallowed now. [#8244](https://github.com/scalableminds/webknossos/pull/8244)
 - Fixed a bug where in the add remote dataset view the dataset name setting was not in sync with the datasource setting of the advanced tab making the form not submittable. [#8245](https://github.com/scalableminds/webknossos/pull/8245)
 - Fix read and update dataset route for versions 8 and lower. [#8263](https://github.com/scalableminds/webknossos/pull/8263)
 - Fixed that task bounding boxes are again converted to user bounding boxes when uploading annotations via nmls. [#8280](https://github.com/scalableminds/webknossos/pull/8280)
 - Added missing legacy support for `isValidNewName` route. [#8252](https://github.com/scalableminds/webknossos/pull/8252)
 - Fixed some layout issues in the upload view. [#8231](https://github.com/scalableminds/webknossos/pull/8231)
 - Fixed `FATAL: role "postgres" does not exist` error message in Docker compose. [#8240](https://github.com/scalableminds/webknossos/pull/8240)
+- Fixed the Zarr 3 implementation not accepting BytesCodec without "configuration" key. [#8282](https://github.com/scalableminds/webknossos/pull/8282)
+- Fixed that reloading the data of a volume annotation layer did not work properly. [#8298](https://github.com/scalableminds/webknossos/pull/8298)
+- Removed the magnification slider for the TIFF export within the download modal if only one magnification is available for the selected layer. [#8297](https://github.com/scalableminds/webknossos/pull/8297)
+- Fixed regression in styling of segment and skeleton tree tab. [#8307](https://github.com/scalableminds/webknossos/pull/8307)
+- Fixed the template for neuron inferral using a custom workflow. [#8312](https://github.com/scalableminds/webknossos/pull/8312)
 
 ### Removed
 - Removed support for HTTP API versions 3 and 4. [#8075](https://github.com/scalableminds/webknossos/pull/8075)
+- Removed that a warning is shown when a dataset is served from a datastore that was marked with isScratch=true. [#8296](https://github.com/scalableminds/webknossos/pull/8296)
 
 ### Breaking Changes
diff --git a/app/WebknossosModule.scala b/app/WebknossosModule.scala
@@ -1,6 +1,6 @@
 import com.google.inject.AbstractModule
 import com.scalableminds.webknossos.datastore.storage.DataVaultService
-import controllers.InitialDataService
+import controllers.{Application, InitialDataService}
 import files.TempFileService
 import mail.MailchimpTicker
 import models.analytics.AnalyticsSessionService
@@ -17,6 +17,7 @@ import utils.sql.SqlClient
 
 class WebknossosModule extends AbstractModule {
   override def configure(): Unit = {
+    bind(classOf[Application]).asEagerSingleton()
     bind(classOf[Startup]).asEagerSingleton()
     bind(classOf[SqlClient]).asEagerSingleton()
     bind(classOf[InitialDataService]).asEagerSingleton()

diff --git a/app/controllers/Application.scala b/app/controllers/Application.scala
@@ -7,7 +7,7 @@ import models.organization.OrganizationDAO
 import models.user.UserService
 import org.apache.pekko.actor.ActorSystem
 import play.api.libs.json.Json
-import play.api.mvc.{Action, AnyContent}
+import play.api.mvc.{Action, AnyContent, Result}
 import play.silhouette.api.Silhouette
 import security.WkEnv
 import utils.sql.{SimpleSQLDAO, SqlClient}
@@ -51,9 +51,12 @@ class Application @Inject()(actorSystem: ActorSystem,
     }
   }
 
+  // This only changes on server restart, so we can cache the full result.
+  private lazy val cachedFeaturesResult: Result = addNoCacheHeaderFallback(
+    Ok(conf.raw.underlying.getConfig("features").resolve.root.render(ConfigRenderOptions.concise())).as(jsonMimeType))
+
   def features: Action[AnyContent] = sil.UserAwareAction {
-    addNoCacheHeaderFallback(
-      Ok(conf.raw.underlying.getConfig("features").resolve.root.render(ConfigRenderOptions.concise())).as(jsonMimeType))
+    cachedFeaturesResult
   }
 
   def health: Action[AnyContent] = Action {

diff --git a/app/controllers/AuthenticationController.scala b/app/controllers/AuthenticationController.scala
@@ -277,7 +277,8 @@ class AuthenticationController @Inject()(
             case None => Future.successful(NotFound(Messages("error.noUser")))
             case Some(user) =>
               for {
-                token <- bearerTokenAuthenticatorService.createAndInit(user.loginInfo, TokenType.ResetPassword)
+                token <- bearerTokenAuthenticatorService
+                  .createAndInit(user.loginInfo, TokenType.ResetPassword, deleteOld = true)
               } yield {
                 Mailer ! Send(defaultMails.resetPasswordMail(user.name, email.toLowerCase, token))
                 Ok

diff --git a/app/controllers/LegacyApiController.scala b/app/controllers/LegacyApiController.scala
@@ -127,20 +127,6 @@ class LegacyApiController @Inject()(annotationController: AnnotationController,
       } yield adaptedResult
     }
 
-  def updateTaskV8(taskId: String): Action[LegacyTaskParameters] =
-    sil.SecuredAction.async(validateJson[LegacyTaskParameters]) { implicit request =>
-      val params = request.body
-      for {
-        dataset <- datasetDAO.findOneByIdOrNameAndOrganization(params.datasetId,
-                                                               params.dataSet,
-                                                               request.identity._organization)
-        paramsWithDatasetId = TaskParameters.fromLegacyTaskParameters(params, dataset._id)
-        requestWithUpdatedBody = request.withBody(paramsWithDatasetId)
-        result <- taskController.update(taskId)(requestWithUpdatedBody)
-        adaptedResult <- replaceInResult(addLegacyDataSetFieldToTask)(result)
-      } yield adaptedResult
-    }
-
   def tasksForProjectV8(id: String,
                         limit: Option[Int] = None,
                         pageNumber: Option[Int] = None,
@@ -153,23 +139,6 @@ class LegacyApiController @Inject()(annotationController: AnnotationController,
       } yield replacedResults
     }
 
-  def annotationInfoV8(id: String, timestamp: Long): Action[AnyContent] = sil.SecuredAction.async { implicit request =>
-    for {
-      _ <- Fox.successful(logVersioned(request))
-      result <- annotationController.infoWithoutType(id, timestamp)(request)
-      adaptedResult <- replaceInResult(addDataSetToTaskInAnnotation)(result)
-    } yield adaptedResult
-  }
-
-  def annotationsForTaskV8(taskId: String): Action[AnyContent] =
-    sil.SecuredAction.async { implicit request =>
-      for {
-        _ <- Fox.successful(logVersioned(request))
-        result <- annotationController.annotationsForTask(taskId)(request)
-        adaptedResult <- replaceInResult(addDataSetToTaskInAnnotation)(result)
-      } yield adaptedResult
-    }
-
   /* provide v7 */
 
   def listDatasetsV7(isActive: Option[Boolean],
@@ -256,17 +225,6 @@ class LegacyApiController @Inject()(annotationController: AnnotationController,
     }
   }
 
-  private def addDataSetToTaskInAnnotation(jsResult: JsObject): Fox[JsObject] = {
-    val taskObjectOpt = (jsResult \ "task").asOpt[JsObject]
-    taskObjectOpt
-      .map(task =>
-        for {
-          adaptedTask <- addLegacyDataSetFieldToTask(task)
-          adaptedJsResult <- tryo(jsResult - "task" + ("task" -> adaptedTask)).toFox
-        } yield adaptedJsResult)
-      .getOrElse(Fox.successful(jsResult))
-  }
-
   private def addLegacyDataSetFieldToTaskCreationResult(jsResult: JsObject) =
     for {
       tasksResults <- tryo((jsResult \ "tasks").as[List[JsObject]]).toFox

diff --git a/app/controllers/OrganizationController.scala b/app/controllers/OrganizationController.scala
@@ -41,9 +41,9 @@ class OrganizationController @Inject()(
 
   def organizationsIsEmpty: Action[AnyContent] = Action.async { implicit request =>
     for {
-      allOrgs <- organizationDAO.findAll(GlobalAccessContext) ?~> "organization.list.failed"
+      orgaTableIsEmpty <- organizationDAO.isEmpty ?~> "organization.list.failed"
     } yield {
-      Ok(Json.toJson(allOrgs.isEmpty))
+      Ok(Json.toJson(orgaTableIsEmpty))
     }
   }
 

diff --git a/app/models/dataset/DataStore.scala b/app/models/dataset/DataStore.scala
@@ -71,7 +71,6 @@ class DataStoreService @Inject()(dataStoreDAO: DataStoreDAO, jobService: JobServ
       Json.obj(
         "name" -> dataStore.name,
         "url" -> dataStore.publicUrl,
-        "isScratch" -> dataStore.isScratch,
         "allowsUpload" -> dataStore.allowsUpload,
         "jobsSupportedByAvailableWorkers" -> Json.toJson(jobsSupportedByAvailableWorkers),
         "jobsEnabled" -> jobsEnabled

diff --git a/app/models/organization/Organization.scala b/app/models/organization/Organization.scala
@@ -80,6 +80,12 @@ class OrganizationDAO @Inject()(sqlClient: SqlClient)(implicit ec: ExecutionCont
       parsed <- parseAll(r)
     } yield parsed
 
+  def isEmpty: Fox[Boolean] =
+    for {
+      rows <- run(q"SELECT COUNT(*) FROM $existingCollectionName".as[Int])
+      value <- rows.headOption
+    } yield value == 0
+
   @deprecated("use findOne with string type instead", since = "")
   override def findOne(id: ObjectId)(implicit ctx: DBAccessContext): Fox[Organization] =
     Fox.failure("Cannot find organization by ObjectId. Use findOne with string type instead")

diff --git a/app/security/BearerTokenAuthenticatorRepository.scala b/app/security/BearerTokenAuthenticatorRepository.scala
@@ -56,16 +56,20 @@ class BearerTokenAuthenticatorRepository(tokenDAO: TokenDAO)(implicit ec: Execut
 
   def add(authenticator: BearerTokenAuthenticator,
           tokenType: TokenType,
-          deleteOld: Boolean = true): Future[BearerTokenAuthenticator] =
+          deleteOld: Boolean = true): Future[BearerTokenAuthenticator] = {
+    if (deleteOld) {
+      removeByLoginInfoIfPresent(authenticator.loginInfo, tokenType)
+    }
     for {
-      oldAuthenticatorOpt <- findOneByLoginInfo(authenticator.loginInfo, tokenType)
       _ <- insert(authenticator, tokenType).futureBox
-    } yield {
-      if (deleteOld) {
-        oldAuthenticatorOpt.map(a => remove(a.id))
-      }
-      authenticator
-    }
+    } yield authenticator
+  }
+
+  private def removeByLoginInfoIfPresent(loginInfo: LoginInfo, tokenType: TokenType): Unit =
+    for {
+      oldOpt <- findOneByLoginInfo(loginInfo, tokenType)
+      _ = oldOpt.foreach(old => remove(old.id))
+    } yield ()
 
   private def insert(authenticator: BearerTokenAuthenticator, tokenType: TokenType): Fox[Unit] =
     for {

diff --git a/app/security/CombinedAuthenticatorService.scala b/app/security/CombinedAuthenticatorService.scala
@@ -39,14 +39,14 @@ case class CombinedAuthenticatorService(cookieSettings: CookieAuthenticatorSetti
 
   private val cookieSigner = new JcaSigner(JcaSignerSettings(conf.Silhouette.CookieAuthenticator.signerSecret))
 
-  val cookieAuthenticatorService = new CookieAuthenticatorService(cookieSettings,
-                                                                  None,
-                                                                  cookieSigner,
-                                                                  cookieHeaderEncoding,
-                                                                  new Base64AuthenticatorEncoder,
-                                                                  fingerprintGenerator,
-                                                                  idGenerator,
-                                                                  clock)
+  private val cookieAuthenticatorService = new CookieAuthenticatorService(cookieSettings,
+                                                                          None,
+                                                                          cookieSigner,
+                                                                          cookieHeaderEncoding,
+                                                                          new Base64AuthenticatorEncoder,
+                                                                          fingerprintGenerator,
+                                                                          idGenerator,
+                                                                          clock)
 
   val tokenAuthenticatorService =
     new WebknossosBearerTokenAuthenticatorService(tokenSettings, tokenDao, idGenerator, clock, userService, conf)
@@ -55,9 +55,9 @@ case class CombinedAuthenticatorService(cookieSettings: CookieAuthenticatorSetti
   override def create(loginInfo: LoginInfo)(implicit request: RequestHeader): Future[CombinedAuthenticator] =
     cookieAuthenticatorService.create(loginInfo).map(CombinedAuthenticator(_))
 
-  def createToken(loginInfo: LoginInfo): Future[CombinedAuthenticator] = {
+  private def createToken(loginInfo: LoginInfo): Future[CombinedAuthenticator] = {
     val tokenAuthenticator = tokenAuthenticatorService.create(loginInfo, TokenType.Authentication)
-    tokenAuthenticator.map(tokenAuthenticatorService.init(_, TokenType.Authentication))
+    tokenAuthenticator.map(tokenAuthenticatorService.init(_, TokenType.Authentication, deleteOld = true))
     tokenAuthenticator.map(CombinedAuthenticator(_))
   }
 

diff --git a/app/security/WebknossosBearerTokenAuthenticatorService.scala b/app/security/WebknossosBearerTokenAuthenticatorService.scala
@@ -54,7 +54,7 @@ class WebknossosBearerTokenAuthenticatorService(settings: BearerTokenAuthenticat
     }
   }
 
-  def init(authenticator: BearerTokenAuthenticator, tokenType: TokenType, deleteOld: Boolean = true): Future[String] =
+  def init(authenticator: BearerTokenAuthenticator, tokenType: TokenType, deleteOld: Boolean): Future[String] =
     repository
       .add(authenticator, tokenType, deleteOld)
       .map { a =>
@@ -67,7 +67,7 @@ class WebknossosBearerTokenAuthenticatorService(settings: BearerTokenAuthenticat
   def createAndInitDataStoreTokenForUser(user: User): Fox[String] =
     createAndInit(user.loginInfo, TokenType.DataStore, deleteOld = false)
 
-  def createAndInit(loginInfo: LoginInfo, tokenType: TokenType, deleteOld: Boolean = true): Future[String] =
+  def createAndInit(loginInfo: LoginInfo, tokenType: TokenType, deleteOld: Boolean): Future[String] =
     for {
       tokenAuthenticator <- create(loginInfo, tokenType)
       tokenId <- init(tokenAuthenticator, tokenType, deleteOld)