MemoryArtifactStore for unit testing and ArtifactStore SPI Validation (…

…apache#3517)

Provides MemoryArtifactStore as an in-memory ArtifactStore implementation which can be used for unit testing and ArtifactStore SPI contract validation.

CouchDB views determine which fields are included in returned document as part of query results. For non CouchDB cases this field list is defined by fieldsRequiredForView. ArtifactStore implementations can use it for projecting which fields should be included.

CouchDB supports joins which is used for subject queries to fetch the limits. Most other NoSQL dbs do not support such joins. So for such cases transformViewResult can be used which would be responsible for performing the join. For it to work the ArtifactStore needs to provide an implementation of DocumentProvider which returns the raw json for a provided doc id.

The MemoryViewMapper is an ArtifactStore implementation specific abstraction which converts the query keys passed to query to underlying storage query syntax. Each store implementation needs to have similar logic implemented to cover all possible scenarios from all the active views. Here the test suite plays an important role by validating that all query cases are covered.

The added test suites need to be kept in sync with any change in view logic or addition of new views. Then only it can be ensured that other ArtifactStore implementation cover all the use cases as supported by default CouchDB. So going forward MemoryArtifactStore would become a canonical implementation of ArtifactStore contract.

common/scala/src/main/scala/whisk/core/database/ArtifactStoreExceptions.scala

@@ -28,3 +28,9 @@ case class DocumentTypeMismatchException(message: String) extends ArtifactStoreE
 case class DocumentUnreadable(message: String) extends ArtifactStoreException(message)
 
 case class PutException(message: String) extends ArtifactStoreException(message)
+
+sealed abstract class ArtifactStoreRuntimeException(message: String) extends RuntimeException(message)
+
+case class UnsupportedQueryKeys(message: String) extends ArtifactStoreRuntimeException(message)
+
+case class UnsupportedView(message: String) extends ArtifactStoreRuntimeException(message)

common/scala/src/main/scala/whisk/core/database/CouchDbRestStore.scala

@@ -28,10 +28,9 @@ import akka.stream.scaladsl._
 import akka.util.ByteString
 import spray.json._
 import whisk.common.{Logging, LoggingMarkers, MetricEmitter, TransactionId}
+import whisk.core.database.StoreUtils._
 import whisk.core.entity.BulkEntityResult
 import whisk.core.entity.DocInfo
-import whisk.core.entity.DocRevision
-import whisk.core.entity.WhiskDocument
 import whisk.http.Messages
 import whisk.core.entity.DocumentReader
 
@@ -224,27 +223,7 @@ class CouchDbRestStore[DocumentAbstraction <: DocumentSerializer](dbProtocol: St
       e match {
         case Right(response) =>
           transid.finished(this, start, s"[GET] '$dbName' completed: found document '$doc'")
-
-          val asFormat = try {
-            docReader.read(ma, response)
-          } catch {
-            case e: Exception => jsonFormat.read(response)
-          }
-
-          if (asFormat.getClass != ma.runtimeClass) {
-            throw DocumentTypeMismatchException(
-              s"document type ${asFormat.getClass} did not match expected type ${ma.runtimeClass}.")
-          }
-
-          val deserialized = asFormat.asInstanceOf[A]
-
-          val responseRev = response.fields("_rev").convertTo[String]
-          assert(doc.rev.rev == null || doc.rev.rev == responseRev, "Returned revision should match original argument")
-          // FIXME remove mutability from appropriate classes now that it is no longer required by GSON.
-          deserialized.asInstanceOf[WhiskDocument].revision(DocRevision(responseRev))
-
-          deserialized
-
+          deserialize[A, DocumentAbstraction](doc, response)
         case Left(StatusCodes.NotFound) =>
           transid.finished(this, start, s"[GET] '$dbName', document: '${doc}'; not found.")
           // for compatibility