groue · groue · Dec 15, 2019 · Dec 12, 2019 · Dec 12, 2019 · Dec 12, 2019
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -65,6 +65,7 @@ GRDB adheres to [Semantic Versioning](https://semver.org/), with one exception:
 - [#659](https://github.com/groue/GRDB.swift/pull/659): Database interruption
 - [#660](https://github.com/groue/GRDB.swift/pull/660): Database Lock Prevention
 - [#662](https://github.com/groue/GRDB.swift/pull/662): Upgrade custom SQLite builds to version 3.30.1 (thanks to [@swiftlyfalling](https://github.com/swiftlyfalling/SQLiteLib))
+- [#668](https://github.com/groue/GRDB.swift/pull/668): Database Suspension
 
 ### Breaking Changes
 
@@ -80,7 +81,39 @@ let dbQueue = try DatabaseQueue(path: ..., configuration: configuration)
 
 ### Documentation Diff
 
-A new [Interrupt a Database](README.md#interrupt-a-database) chapter documents the new `interrupt()` method.
+The new [Interrupt a Database](README.md#interrupt-a-database) chapter documents the new `interrupt()` method.
+
+The new [Sharing a Datatase in an App Group Container](Documentation/AppGroupContainers.md) guide explains how to setup GRDB when you share a database in an iOS App Group container.
+
+
+### API Diff
+
+```diff
+ struct Configuration {
++    var observesSuspensionNotifications: Bool // Experimental
++    var acceptsDoubleQuotedStringLiterals: Bool
+ }
+
+ class Database {
++    static let suspendNotification: Notification.Name // Experimental
++    static let resumeNotification: Notification.Name  // Experimental
+ }
+
+ extension DatabaseError {
++    var isInterruptionError: Bool { get }
+ }
+
+ protocol DatabaseReader {
++    func interrupt()
+ }
+
+ extension SQLSpecificExpressible {
++    #if GRDBCUSTOMSQLITE
++    var ascNullsLast: SQLOrderingTerm { get }
++    var descNullsFirst: SQLOrderingTerm { get }
++    #endif
+ }
+```
 
 
 ## 4.6.2

diff --git a/Documentation/AppGroupContainers.md b/Documentation/AppGroupContainers.md
@@ -0,0 +1,239 @@
+Sharing a Datatase in an App Group Container
+============================================
+
+On iOS, you can share database files between multiple processes by storing them in an [App Group Container](https://developer.apple.com/documentation/foundation/nsfilemanager/1412643-containerurlforsecurityapplicati).
+
+A shared database is accessed from several SQLite connections, from several processes. This creates challenges at various levels:
+
+1. **Database setup** may be attempted by multiple processes, concurrently.
+2. **SQLite** may throw `SQLITE_BUSY` errors, code 5, "database is locked".
+3. **iOS** may kill your application with a `0xDEAD10CC` exception.
+4. **GRDB** database observation misses changes performed by external processes.
+
+We'll address all of those challenges below.
+
+- [Use a Database Pool]
+- [How to limit the `SQLITE_BUSY` error]
+- [How to limit the `0xDEAD10CC` exception]
+- [How to perform cross-process database observation]
+
+
+## Use a Database Pool
+
+In order to access a shared database, use a [Database Pool]. It opens the database in the [WAL mode](https://www.sqlite.org/wal.html), which helps sharing a database.
+
+Since several processes may open the database at the same time, protect the creation of the database pool with an [NSFileCoordinator].
+
+- In a process that can create and write in the database, use this sample code:
+
+    ```swift
+    /// Returns an initialized database pool at the shared location databaseURL
+    func openSharedDatabase(at databaseURL: URL) throws -> DatabasePool {
+        let coordinator = NSFileCoordinator(filePresenter: nil)
+        var coordinatorError: NSError?
+        var dbPool: DatabasePool?
+        var dbError: Error?
+        coordinator.coordinate(writingItemAt: databaseURL, options: .forMerging, error: &coordinatorError, byAccessor: { url in
+            do {
+                dbPool = try openDatabase(at: url)
+            } catch {
+                dbError = error
+            }
+        })
+        if let error = dbError ?? coordinatorError {
+            throw error
+        }
+        return dbPool!
+    }
+
+    private func openDatabase(at databaseURL: URL) throws -> DatabasePool {
+        let dbPool = try DatabasePool(path: databaseURL.path)
+        // Perform here other database setups, such as defining 
+        // the database schema with a DatabaseMigrator.
+        return dbPool
+    }
+    ```
+
+- In a process that only reads in the database, use this sample code:
+
+    ```swift
+    /// Returns an initialized database pool at the shared location databaseURL,
+    /// or nil if the database was not created yet.
+    func openSharedReadOnlyDatabase(at databaseURL: URL) throws -> DatabasePool? {
+        let coordinator = NSFileCoordinator(filePresenter: nil)
+        var coordinatorError: NSError?
+        var dbPool: DatabasePool?
+        var dbError: Error?
+        coordinator.coordinate(readingItemAt: databaseURL, options: .withoutChanges, error: &coordinatorError, byAccessor: { url in
+            do {
+                dbPool = try openReadOnlyDatabase(at: url)
+            } catch {
+                dbError = error
+            }
+        })
+        if let error = dbError ?? coordinatorError {
+            throw error
+        }
+        return dbPool
+    }
+
+    private func openReadOnlyDatabase(at databaseURL: URL) throws -> DatabasePool? {
+        do {
+            var configuration = Configuration()
+            configuration.readonly = true
+            return try DatabasePool(path: databaseURL.path, configuration: configuration)
+        } catch {
+            if FileManager.default.fileExists(atPath: databaseURL.path) {
+                throw error
+            } else {
+                return nil
+            }
+        }
+    }
+    ```
+
+
+## How to limit the `SQLITE_BUSY` error
+
+> The SQLITE_BUSY result code indicates that the database file could not be written (or in some cases read) because of concurrent activity by some other database connection, usually a database connection in a separate process.
+
+See https://www.sqlite.org/rescode.html#busy for more information about this error.
+
+If several processes want to write in the database, configure the database pool of each process that wants to write:
+
+```swift
+var configuration = Configuration()
+configuration.busyMode = .timeout(/* a TimeInterval */)
+configuration.defaultTransactionKind = .immediate
+let dbPool = try DatabasePool(path: ..., configuration: configuration)
+```
+
+With such a setup, you may still get `SQLITE_BUSY` (5, "database is locked") errors from all write operations. They will occur if the database remains locked by another process for longer than the specified timeout.
+
+```swift
+do {
+    try dbPool.write { db in ... }
+} catch let error as DatabaseError where error.resultCode == .SQLITE_BUSY {
+    // Another process won't let you write. Deal with it.
+}
+```
+
+> :bulb: **Tip**: In order to be nice to other processes, measure the duration of your longest writes, and attempt at optimizing the ones that last for too long.
+
+
+## How to limit the `0xDEAD10CC` exception
+
+> The exception code 0xDEAD10CC indicates that an application has been terminated by the OS because it held on to a file lock or sqlite database lock during suspension.
+
+See https://developer.apple.com/library/archive/technotes/tn2151/_index.html for more information about this exception.
+
+1. If you use SQLCipher, use SQLCipher 4+, and call the `cipher_plaintext_header_size` pragma from your database preparation function:
+
+    ```swift
+    var configuration = Configuration()
+    configuration.prepareDatabase = { (db: Database) in
+        try db.usePassphrase("secret")
+        try db.execute(sql: "PRAGMA cipher_plaintext_header_size = 32")
+    }
+    let dbPool = try DatabasePool(path: ..., configuration: configuration)
+    ```
+
+    This will avoid https://github.com/sqlcipher/sqlcipher/issues/255.
+
+2. [**:fire: EXPERIMENTAL**](README.md#what-are-experimental-features) In each process that wants to write in the database:
+
+    Set the `observesSuspensionNotifications` configuration flag:
+
+    ```swift
+    var configuration = Configuration()
+    configuration.suspendsOnBackgroundTimeExpiration = true
+    let dbPool = try DatabasePool(path: ..., configuration: configuration)
+    ```
+
+    Post `Database.suspendNotification` when the application is about to be [suspended](https://developer.apple.com/documentation/uikit/app_and_environment/managing_your_app_s_life_cycle). You can for example post this notification from `UIApplicationDelegate.applicationDidEnterBackground(_:)`, or in the expiration handler of a [background task](https://forums.developer.apple.com/thread/85066).
+
+    ```swift
+    @UIApplicationMain
+    class AppDelegate: UIResponder, UIApplicationDelegate {
+        func applicationDidEnterBackground(_ application: UIApplication) {
+            // Suspend databases
+            NotificationCenter.default.post(name: Database.suspendNotification, object: self)
+        }
+    }
+    ```
+
+    Post `Database.resumeNotification` from `UIApplicationDelegate.applicationWillEnterForeground(_:)` (or `SceneDelegate.sceneWillEnterForeground(_:)` for scene-based applications):
+
+    ```swift
+    @UIApplicationMain
+    class AppDelegate: UIResponder, UIApplicationDelegate {
+        func applicationWillEnterForeground(_ application: UIApplication) {
+            // Resume databases
+            NotificationCenter.default.post(name: Database.resumeNotification, object: self)
+        }
+    }
+    ```
+
+    If the application uses the background modes supported by iOS, post `Database.resumeNotification` method from each and every background mode callback that may use the database. For example, if your application supports background fetches:
+
+    ```swift
+    @UIApplicationMain
+    class AppDelegate: UIResponder, UIApplicationDelegate {
+        func application(_ application: UIApplication, performFetchWithCompletionHandler completionHandler: @escaping (UIBackgroundFetchResult) -> Void) {
+            // Resume databases
+            NotificationCenter.default.post(name: Database.resumeNotification, object: self)
+            // Proceed with background fetch
+            ...
+        }
+    }
+    ```
+
+    Suspended databases greatly reduce the odds of `0xDEAD10CC` exception are greatly reduced. If you see one in your crash logs, please open an issue!
+
+    In exchange, you will get `SQLITE_INTERRUPT` (9) or `SQLITE_ABORT` (4) errors, with messages "Database is suspended", "Transaction was aborted", or "interrupted", for any attempt at writing in the database when it is suspended.
+
+    You can catch those errors:
+
+    ```swift
+    do {
+        try dbPool.write { db in ... }
+    } catch let error as DatabaseError where error.isInterruptionError {
+        // Oops, the database is suspended.
+        // Maybe try again after database is resumed?
+    }
+    ```
+
+
+## How to perform cross-process database observation
+
+GRDB [Database Observation] features, as well as [GRDBCombine] and [RxGRDB], are not able to notify database changes performed by other processes.
+
+Whenever you need to notify other processes that the database has been changed, you will have to use a cross-process notification mechanism such as [NSFileCoordinator] or [CFNotificationCenterGetDarwinNotifyCenter].
+
+You can trigger those notifications automatically with [DatabaseRegionObservation]:
+
+```swift
+// Notify all changes made to the "player" and "team" database tables
+let observation = DatabaseRegionObservation(tracking: Player.all(), Team.all())
+let observer = try observation.start(in: dbPool) { (db: Database) in
+    // Notify other processes
+}
+
+// Notify all changes made to the database
+let observation = DatabaseRegionObservation(tracking: DatabaseRegion.fullDatabase)
+let observer = try observation.start(in: dbPool) { (db: Database) in
+    // Notify other processes
+}
+```
+
+[Use a Database Pool]: #use-a-database-pool
+[How to limit the `SQLITE_BUSY` error]: #how-to-limit-the-sqlite_busy-error
+[How to limit the `0xDEAD10CC` exception]: #how-to-limit-the-0xdead10cc-exception
+[How to perform cross-process database observation]: #how-to-perform-cross-process-database-observation
+[Database Pool]: ../README.md#database-pools
+[Database Observation]: ../README.md#database-changes-observation
+[GRDBCombine]: http://github.com/groue/GRDBCombine
+[RxGRDB]: https://github.com/RxSwiftCommunity/RxGRDB
+[NSFileCoordinator]: https://developer.apple.com/documentation/foundation/nsfilecoordinator
+[CFNotificationCenterGetDarwinNotifyCenter]: https://developer.apple.com/documentation/corefoundation/1542572-cfnotificationcentergetdarwinnot
+[DatabaseRegionObservation]: ../README.md#databaseregionobservation
diff --git a/GRDB.xcodeproj/project.pbxproj b/GRDB.xcodeproj/project.pbxproj
@@ -494,9 +494,9 @@
 		567F45AC1F888B2600030B59 /* TruncateOptimizationTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 567F45A71F888B2600030B59 /* TruncateOptimizationTests.swift */; };
 		568068311EBBA26100EFB8AA /* SQLRequestTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 568068301EBBA26100EFB8AA /* SQLRequestTests.swift */; };
 		568068351EBBA26100EFB8AA /* SQLRequestTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 568068301EBBA26100EFB8AA /* SQLRequestTests.swift */; };
-		5682D721239582AA004B58C4 /* DatabaseLockPreventionTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 5682D71A239582AA004B58C4 /* DatabaseLockPreventionTests.swift */; };
-		5682D722239582AA004B58C4 /* DatabaseLockPreventionTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 5682D71A239582AA004B58C4 /* DatabaseLockPreventionTests.swift */; };
-		5682D723239582AA004B58C4 /* DatabaseLockPreventionTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 5682D71A239582AA004B58C4 /* DatabaseLockPreventionTests.swift */; };
+		5682D721239582AA004B58C4 /* DatabaseSuspensionTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 5682D71A239582AA004B58C4 /* DatabaseSuspensionTests.swift */; };
+		5682D722239582AA004B58C4 /* DatabaseSuspensionTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 5682D71A239582AA004B58C4 /* DatabaseSuspensionTests.swift */; };
+		5682D723239582AA004B58C4 /* DatabaseSuspensionTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 5682D71A239582AA004B58C4 /* DatabaseSuspensionTests.swift */; };
 		568735A21CEDE16C009B9116 /* Betty.jpeg in Resources */ = {isa = PBXBuildFile; fileRef = 5687359E1CEDE16C009B9116 /* Betty.jpeg */; };
 		568D131F2207213E00674B58 /* SQLQueryGenerator.swift in Sources */ = {isa = PBXBuildFile; fileRef = 568D13182207213E00674B58 /* SQLQueryGenerator.swift */; };
 		568D13202207213E00674B58 /* SQLQueryGenerator.swift in Sources */ = {isa = PBXBuildFile; fileRef = 568D13182207213E00674B58 /* SQLQueryGenerator.swift */; };
@@ -1500,7 +1500,7 @@
 		567F0B2C220F0E2E00D111FB /* SQLInterpolationTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SQLInterpolationTests.swift; sourceTree = "<group>"; };
 		567F45A71F888B2600030B59 /* TruncateOptimizationTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = TruncateOptimizationTests.swift; sourceTree = "<group>"; };
 		568068301EBBA26100EFB8AA /* SQLRequestTests.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = SQLRequestTests.swift; sourceTree = "<group>"; };
-		5682D71A239582AA004B58C4 /* DatabaseLockPreventionTests.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = DatabaseLockPreventionTests.swift; sourceTree = "<group>"; };
+		5682D71A239582AA004B58C4 /* DatabaseSuspensionTests.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = DatabaseSuspensionTests.swift; sourceTree = "<group>"; };
 		5687359E1CEDE16C009B9116 /* Betty.jpeg */ = {isa = PBXFileReference; lastKnownFileType = image.jpeg; path = Betty.jpeg; sourceTree = "<group>"; };
 		568D13182207213E00674B58 /* SQLQueryGenerator.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = SQLQueryGenerator.swift; sourceTree = "<group>"; };
 		5690AFD72120589A001530EA /* InsertRecordCodableTests.swift */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.swift; path = InsertRecordCodableTests.swift; sourceTree = "<group>"; };
@@ -2233,14 +2233,14 @@
 				563DE4EC231A91E2005081B7 /* DatabaseConfigurationTests.swift */,
 				56A238161B9C74A90082EB20 /* DatabaseErrorTests.swift */,
 				560C97C61BFD0B8400BF8471 /* DatabaseFunctionTests.swift */,
-				5682D71A239582AA004B58C4 /* DatabaseLockPreventionTests.swift */,
 				567DAF341EAB789800FC0928 /* DatabaseLogErrorTests.swift */,
 				560A37A91C90084600949E71 /* DatabasePool */,
 				563363BB1C93FD32000BE133 /* DatabaseQueue */,
 				56EA86931C91DFE7002BB4DF /* DatabaseReaderTests.swift */,
 				564CE5BD21B8FFA300652B19 /* DatabaseRegionObservationTests.swift */,
 				56C3F7521CF9F12400F6A361 /* DatabaseSavepointTests.swift */,
 				566A8424204120B700E50BFD /* DatabaseSnapshotTests.swift */,
+				5682D71A239582AA004B58C4 /* DatabaseSuspensionTests.swift */,
 				56A238131B9C74A90082EB20 /* DatabaseTests.swift */,
 				564FCE5D20F7E11A00202B90 /* DatabaseValueConversionErrorTests.swift */,
 				56A2381B1B9C74A90082EB20 /* DatabaseValueConversionTests.swift */,
@@ -3491,7 +3491,7 @@
 				56EA869F1C932597002BB4DF /* DatabasePoolReadOnlyTests.swift in Sources */,
 				5615B288222B17C000061C1C /* AssociationHasOneThroughDecodableRecordTests.swift in Sources */,
 				567F45AC1F888B2600030B59 /* TruncateOptimizationTests.swift in Sources */,
-				5682D722239582AA004B58C4 /* DatabaseLockPreventionTests.swift in Sources */,
+				5682D722239582AA004B58C4 /* DatabaseSuspensionTests.swift in Sources */,
 				5657AB621D108BA9006283EF /* FoundationNSURLTests.swift in Sources */,
 				5657AB6A1D108BA9006283EF /* FoundationURLTests.swift in Sources */,
 				563B06C42185D29F00B38F35 /* ValueObservationExtentTests.swift in Sources */,
@@ -3698,7 +3698,7 @@
 				56176C591EACCCC7000F3F2B /* FTS5CustomTokenizerTests.swift in Sources */,
 				5615B289222B17C000061C1C /* AssociationHasOneThroughDecodableRecordTests.swift in Sources */,
 				56D4968A1D81316E008276D7 /* DatabaseValueConvertibleSubclassTests.swift in Sources */,
-				5682D721239582AA004B58C4 /* DatabaseLockPreventionTests.swift in Sources */,
+				5682D721239582AA004B58C4 /* DatabaseSuspensionTests.swift in Sources */,
 				56D496601D81304E008276D7 /* FoundationNSUUIDTests.swift in Sources */,
 				567F45A81F888B2600030B59 /* TruncateOptimizationTests.swift in Sources */,
 				563B06C32185D29F00B38F35 /* ValueObservationExtentTests.swift in Sources */,
@@ -4038,7 +4038,7 @@
 				AAA4DD4D230F262000C74B15 /* DatabasePoolReadOnlyTests.swift in Sources */,
 				AAA4DD4E230F262000C74B15 /* AssociationHasOneThroughDecodableRecordTests.swift in Sources */,
 				AAA4DD4F230F262000C74B15 /* TruncateOptimizationTests.swift in Sources */,
-				5682D723239582AA004B58C4 /* DatabaseLockPreventionTests.swift in Sources */,
+				5682D723239582AA004B58C4 /* DatabaseSuspensionTests.swift in Sources */,
 				AAA4DD50230F262000C74B15 /* FoundationNSURLTests.swift in Sources */,
 				AAA4DD51230F262000C74B15 /* FoundationURLTests.swift in Sources */,
 				AAA4DD52230F262000C74B15 /* ValueObservationExtentTests.swift in Sources */,

diff --git a/GRDB/Core/Configuration.swift b/GRDB/Core/Configuration.swift
@@ -98,6 +98,19 @@ public struct Configuration {
     /// - Default value: false
     public var acceptsDoubleQuotedStringLiterals = false
 
+    /// When true, the `Database.suspendNotification` and
+    /// `Database.resumeNotification` suspend and resume the database. Database
+    /// suspension helps avoiding the [`0xdead10cc`
+    /// exception](https://developer.apple.com/library/archive/technotes/tn2151/_index.html).
+    ///
+    /// During suspension, all database accesses but reads in WAL mode may throw
+    /// a DatabaseError of code `SQLITE_INTERRUPT`, or `SQLITE_ABORT`. You can
+    /// check for those error codes with the
+    /// `DatabaseError.isInterruptionError` property.
+    ///
+    /// [**Experimental**](http://github.com/groue/GRDB.swift#what-are-experimental-features)
+    public var observesSuspensionNotifications = false
+
     // MARK: - Encryption
 
     #if SQLITE_HAS_CODEC