Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GRDB 4 and SQLCipher 4 #480

Closed
groue opened this issue Feb 17, 2019 · 8 comments
Closed

GRDB 4 and SQLCipher 4 #480

groue opened this issue Feb 17, 2019 · 8 comments
Labels
enhancement help wanted Extra attention is needed
Milestone

Comments

@groue
Copy link
Owner

groue commented Feb 17, 2019

The work on next major version GRDB 4 has started. It is an opportunity to upgrade SQLCipher.

See Upgrading to SQLCipher 4 for more information about the breaking changes introduced by SQLCipher 4.

The SQLCipher 4 repository no longer contains any Xcode project (sqlcipher/sqlcipher#292). This has a big impact on the techniques documented by GRDB for embedding SQLCipher.

  • CocoaPods: OK. Upgrading the GRDBCipher podspec so that it depends on SQLCipher 4 should be straightforward.

  • Manual installation: NOT OK. The GRDBCipher.xcodeproj project contained in the GRDB repository can no longer embed the non-existing SQLCipher Xcode project.

  • Carthage: NOT OK. Even if GRDB does not "officially" support Carthage, some users fork GRDB and set up their customized Carthage support. Those setups will break because the GRDBCipher.xcodeproj project contained in the GRDB repository can no longer embed the non-existing SQLCipher Xcode project.

A few possible options:

  1. Go full CocoaPods

    The GRDBCipher.xcodeproj project contained in the GRDB repository would only exist for tests, not for user consumption. It would embed SQLCipher through CocoaPods.

    The only documented and supported way to embed GRDBCipher in an application would be through CocoaPods. Manual installation and Carthage would no longer be supported.

    User damage: atomic, because some options become simply unavailable.

    Setup cost: medium, because we'll be forced to upgrade GRDB 3.6.2 for Swift 5, in order to provide an Xcode upgrade path which does not break GRDB. I was hoping we could avoid this step which is painful because GRDB 3 must support SWIFT_VERSION=4 (details).

    Long run cost: zero.

  2. Build a clone of the excellent http://github.com/swiftlyfalling/SQLiteLib for SQLCipher. This would allow GRDB to keep on supporting manual installation, just like custom SQLite builds. But someone has to build and maintain a whole new repository.

    User damage: high because things will break anyway.

    Setup Cost: unknown.

    Long run cost: not zero.

  3. Do not upgrade SQLCipher.

    User damage: I don't quite know how to evaluate how much SQLCipher 4 is an improvement over SQLCipher 3.

    Setup Cost: zero.

    Long run cost: zero.

cc @swiftlyfalling and @darrenclark because you may help figuring out the best option.

cc @developernotes and @sjlombardo from SQLCipher, for information.

@groue groue added this to the GRDB 4.0.0 milestone Feb 17, 2019
@groue groue added the help wanted Extra attention is needed label Feb 17, 2019
This was referenced Feb 17, 2019
darrenclark added a commit to darrenclark/GRDB.swift that referenced this issue Feb 17, 2019
Proof of concept related to:
groue#480

Not working:

- Various failing tests with getting back `SQLITE_ERROR` instead of
  `SQLITE_MISUSE`.  Not sure about this..

- Doesn't work with new Xcode build system.  Since the
  `sqlcipher-amalgamation` target builds `sqlite3.c`, I don't think the
  new build syste

  Might need to use a custom run script build phase that explicitly sets
  the outputs.
darrenclark added a commit to darrenclark/GRDB.swift that referenced this issue Feb 17, 2019
Proof of concept related to:
groue#480

Not working:

- Various failing tests with getting back `SQLITE_ERROR` instead of
  `SQLITE_MISUSE`.  Not sure about this..

- Doesn't work with new Xcode build system.  Since the
  `sqlcipher-amalgamation` target builds `sqlite3.c`, I don't think the
  new build system knows about this, so it fails the build of
  the `sqlcipher-ios` target immediately say that "sqlite3.c is missing"

  Might need to use a custom run script build phase that explicitly sets
  the outputs.
@darrenclark
Copy link
Contributor

Sorta related to #2, another option might be to move the targets from the old sqlcipher.xcodeproj into GRDBCipher.xcodeproj. Would still have the sqlcipher source code pulled in via the submodule, just the Xcode targets to actually build it would live in groue/GRDB.swift now.

Still is some maintenance cost, but it is primarily just copy/pasting settings from the sqlchipher podspec

I opened a proof of concept PR here: #481

@groue
Copy link
Owner Author

groue commented Feb 18, 2019

Sorta related to #2, another option might be to move the targets from the old sqlcipher.xcodeproj into GRDBCipher.xcodeproj. Would still have the sqlcipher source code pulled in via the submodule, just the Xcode targets to actually build it would live in groue/GRDB.swift now.

That's true!

This solution would not be as flexible as @swiftlyfalling's SQLiteLib. SQLiteLib allows the client app to customize building options without any modification of GRDB and its dependencies, and this is where it really shines.

Yet the current support for SQLCipher does not allow such customization: your suggestion is just as good. I can support it! 👍

Still is some maintenance cost, but it is primarily just copy/pasting settings from the sqlchipher podspec

I'm sure we could handle that in the long run. Once the initial setup is done, I only foresee the updates of SQLCipher sources, and, as you say, keeping an eye on the usual build options.

I opened a proof of concept PR here: #481

That's exactly the kind of bold move I was hoping, you know! Thank you very much, @darrenclark!

I really wish this proof of concept can become production-grade. Let's talk over there!

@charlesmchen-signal
Copy link
Contributor

User damage: I don't quite know how to evaluate how much SQLCipher 4 is an improvement over SQLCipher 3.

One advantage of SQLCipher 4 over 3 is that it is based on a more recent version of SQLite.

Another advantage is that it includes a fix for this issue:

sqlcipher/sqlcipher#255

The issue affects iOS apps using SQLCipher 3 where the database file resides in the iOS "shared container". If an app has an app extension (e.g. a share extension), placing the database file in the "shared container" is the only way for the main app and app extension to share access to a single database. But iOS kills apps that maintain locks on files in the "shared container" while in the background. So iOS would kill your main app every time it enters the background.

@groue
Copy link
Owner Author

groue commented Mar 27, 2019

Thanks @charlesmchen-signal, for reminding this important issue.

@darrenclark, Xcode 10.2 is out now: it would be great to be able to have a working SQLCipher integration soon. Do you think #481 is close to a finish? @Marus would then be able to work on helping users migrate from SQLCipher 3 to SQLCipher 4. And meanwhile I could finish #479 (the main GRDB 4 PR) so that we could ship all this goodness in a nice release :-)

@groue
Copy link
Owner Author

groue commented Apr 10, 2019

Hi folks, here is a report of the current state of affairs regarding SQLCipher 4 and GRDB 4.

  1. Allow Database Connection Configuration #508 by @michaelkirk-signal is merged: it makes it possible to open existing SQLCipher 3 databases with GRDB 4 and SQLCipher 4.
  2. Move sqlcipher.xcodeproj targets into GRDBCipher.xcodeproj #481 by @darrenclark has been discarded, which means that nobody is currently working on an integration of SQLCipher 4 which looks like the one we had with SQLCipher 3.
  3. Because of the previous point, I plan to ship GRDB 4 with CocoaPods as the unique way to use SQLCipher.
  4. We still need a SQLCipher 4 transition guide that would explain how to open an existing SQLCipher 3 database with SQLCipher 4, and how to dump the content of an existing SQLCipher 3 database into a new SQLCipher 4 database (guidance).

The points 2 and 3 may create pain for current users of SQLCipher who don't use CocoaPods. It will be possible to address this later, though, if anybody happens to continue the work started in #481. Applications are open!

Contributions for the fourth point, documentation, will also be warmly welcome.

The faster will be the better. Xcode 10.2 is out, and GRDB 4 is late.

@charlesmchen-signal
Copy link
Contributor

@groue Sounds good. Thanks for the update.

@groue
Copy link
Owner Author

groue commented Apr 21, 2019

Superseded by #517

@groue groue closed this as completed Apr 21, 2019
@groue
Copy link
Owner Author

groue commented Apr 22, 2019

#517 is merged into the GRDB-4.0 branch. We lack a detailed transition guide from SQLCipher 3 to SQLCipher 4, but this is not a blocker. The SQLCipher 4 topic is now behind us, as far as I am concerned.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

3 participants