Carthage and xcode 10: build failed

[freddy5566]

What did you do?

carthage update --platform osx --no-use-binaries

What did you expect to happen?

it should build GRDBCustomSQLiteOSX successful

What happened instead?

I built GRDBOSX, GRDBCipherOSX successful, but when GRDBCustomSQLiteOSX exit code 65 just came out

i've tried reinstall carthage via brew/pkg, also delete the cache of carthage, but none of these work

Environment

**GRDB flavor(s): GRDBCustom
**GRDB version: 3.3.1
**Installation method: Carthage
**Xcode version: Xcode 10
**Swift version: Apple Swift version 4.2 (swiftlang-1000.11.37.1 clang-1000.11.45.1)
**Platform(s) running GRDB: macOS
**macOS version running Xcode: 10.14 GM

Demo Project

[groue]

Hello @jamfly,

As told in the main README.md (Installation chapter), Carthage is not supported. This means that you may not get any support here.

I'll leave this issue open for a week, so that other GRDB users have the time to chime in and help. I'll close it afterwards, because I don't like issues to remain open when nobody is working on it.

Meanwhile. I suggest you choose a supported installation method instead. If you want to keep Carthage, summon your local Xcode guru, ask on StackOverflow, or directly in the Carthage repo.

If you are surprised by this answer, please look for "Carthage" in other issues and pull requests.

[groue]

cc @KyleLeneau and @darrenclark, in case you'd have any idea what's going on.

[freddy5566]

Hello @groue

I am so sorry it bothers you, if you don't like it, i can remove it.

but the reason that I came here is because

error: /Users/Freddy/developer/macos/myProject/Carthage/Checkouts/GRDB.swift/SQLiteCustom/src/SQLiteLib.xcconfig:8: could not find included file 'SQLiteLib-USER.xcconfig' in search paths (in target 'amalgamation')

Build system information
error: /Users/Freddy/developer/macos/myProject/Carthage/Checkouts/GRDB.swift/SQLiteCustom/src/SQLiteLib.xcconfig:8: could not find included file 'SQLiteLib-USER.xcconfig' in search paths (in target 'amalgamation')

Build system information
error: /Users/Freddy/developer/macos/myProject/Carthage/Checkouts/GRDB.swift/SQLiteCustom/src/SQLiteLib.xcconfig:8: could not find included file 'SQLiteLib-USER.xcconfig' in search paths (in target 'amalgamation')

Build system information
error: /Users/Freddy/developer/macos/myProject/Carthage/Checkouts/GRDB.swift/SQLiteCustom/src/SQLiteLib.xcconfig:8: could not find included file 'SQLiteLib-USER.xcconfig' in search paths (in target 'sqlitecustom')

Build system information
error: /Users/Freddy/developer/macos/myProject/Carthage/Checkouts/GRDB.swift/SQLiteCustom/GRDB.xcconfig:5: could not find included file 'GRDBCustomSQLite-USER.xcconfig' in search paths (in target 'GRDBCustomSQLiteOSX')

Build system information
error: /Users/Freddy/developer/macos/myProject/Carthage/Checkouts/GRDB.swift/SQLiteCustom/src/SQLiteLib.xcconfig:8: could not find included file 'SQLiteLib-USER.xcconfig' in search paths (in target 'sqlitecustom')

Build system information
error: /Users/Freddy/developer/macos/myProject/Carthage/Checkouts/GRDB.swift/SQLiteCustom/GRDB.xcconfig:6: could not find included file 'src/SQLiteLib-USER.xcconfig' in search paths (in target 'GRDBCustomSQLiteOSX')

Build system information
error: /Users/Freddy/developer/macos/myProject/Carthage/Checkouts/GRDB.swift/SQLiteCustom/src/SQLiteLib.xcconfig:8: could not find included file 'SQLiteLib-USER.xcconfig' in search paths (in target 'sqlitecustom')

Build system information
error: /Users/Freddy/developer/macos/myProject/Carthage/Checkouts/GRDB.swift/SQLiteCustom/GRDB.xcconfig:5: could not find included file 'GRDBCustomSQLite-USER.xcconfig' in search paths (in target 'GRDBCustomSQLiteOSX')

Build system information
error: /Users/Freddy/developer/macos/myProject/Carthage/Checkouts/GRDB.swift/SQLiteCustom/src/SQLiteLib.xcconfig:8: could not find included file 'SQLiteLib-USER.xcconfig' in search paths (in target 'sqlitecustom')

Build system information
error: /Users/Freddy/developer/macos/myProject/Carthage/Checkouts/GRDB.swift/SQLiteCustom/GRDB.xcconfig:6: could not find included file 'src/SQLiteLib-USER.xcconfig' in search paths (in target 'GRDBCustomSQLiteOSX')

Build system information
error: /Users/Freddy/developer/macos/myProject/Carthage/Checkouts/GRDB.swift/SQLiteCustom/GRDB.xcconfig:5: could not find included file 'GRDBCustomSQLite-USER.xcconfig' in search paths (in target 'GRDBCustomSQLiteOSX')

Build system information
error: /Users/Freddy/developer/macos/myProject/Carthage/Checkouts/GRDB.swift/SQLiteCustom/GRDB.xcconfig:6: could not find included file 'src/SQLiteLib-USER.xcconfig' in search paths (in target 'GRDBCustomSQLiteOSX')

** ARCHIVE FAILED **

even though i manually built the GRDBCustomSQLiteOSX , it still came out with the same error

Thank you for your respond.

[groue]

It does not bother me. If it bothers you, follow installation instructions. Success guaranteed.

[darrenclark]

@jamfly @groue Regarding the custom build, I actually wrapped my carthage update in a shell script that generated those missing files. This is what I used for iOS, it should be a good start for getting a working custom build for macOS:

dependencies.sh

#!/bin/sh

# Config

export GRDB_DIR="/tmp/GRDB_project_name_temp"
export GRDB_REF="development"   # commit/branch/etc. you want to use

export SQLITELIB_XCCONFIG='
CUSTOM_SQLLIBRARY_CFLAGS = -DSQLITE_ENABLE_FTS5 -DSQLITE_ENABLE_PREUPDATE_HOOK -DSQLITE_ENABLE_JSON1
'
export GRDBCUSTOM_XCCONFIG='
CUSTOM_OTHER_SWIFT_FLAGS = -D SQLITE_ENABLE_FTS5 -D SQLITE_ENABLE_PREUPDATE_HOOK -D SQLITE_ENABLE_JSON1
'
export GRDBCUSTOM_H='
#define SQLITE_ENABLE_FTS5
#define SQLITE_ENABLE_PREUPDATE_HOOK
#define SQLITE_ENABLE_JSON1
'

# Build setup

rm -rf "$GRDB_DIR"

git clone [email protected]:groue/GRDB.swift "$GRDB_DIR"
cd "$GRDB_DIR"
git checkout "$GRDB_REF"

git submodule update --init SQLiteCustom/src
git rm --cached -r SQLiteCustom/src
git config -f .gitmodules --remove-section submodule.SQLiteCustom/src
rm -rf SQLiteCustom/src/.git
git add SQLiteCustom/src

echo "$SQLITELIB_XCCONFIG" >> SQLiteCustom/src/SQLiteLib-USER.xcconfig
echo "$GRDBCUSTOM_XCCONFIG" >> SQLiteCustom/GRDBCustomSQLite-USER.xcconfig
echo "$GRDBCUSTOM_H" >> SQLiteCustom/GRDBCustomSQLite-USER.h

# these files are .gitignore'd but we need them checked in because Carthage is gonna clone this repo...
git add -f SQLiteCustom/GRDBCustomSQLite-USER.xcconfig SQLiteCustom/GRDBCustomSQLite-USER.h SQLiteCustom/src/SQLiteLib-USER.xcconfig

git commit -am "Patching GRDB source code for Carthage compatibility"
git checkout -b patched-for-carthage

# run carthage
cd -
carthage update --platform ios

and then in my Cartfile:

Cartfile

git "file:///tmp/GRDB_project_name_temp" "patched-for-carthage"

Basic gist of it is:

1. Clone the GRDB to a temporary directory
2. Generate those missing files
   • SQLiteCustom/GRDBCustomSQLite-USER.xcconfig
   • SQLiteCustom/GRDBCustomSQLite-USER.h
   • SQLiteCustom/src/SQLiteLib-USER.xcconfig
3. Check in those files to your local copy (as Carthage will clone the repo from your /tmp directory)
4. Reference the git repo in your /tmp directory in the Cartfile

[darrenclark]

I haven't had the chance yet to actually test that with Xcode 10, but hopefully it's a good starting point

[freddy5566]

Hi @darrenclark

thank you for your respond, and it is exactly what i did in these days.

but i had never written shell script before, so the quality is kinda shitty.
so, your respond save my days!!! thank you again.

[darrenclark]

@jamfly you're welcome, glad to hear it helped! Feel free to reach out if you have any questions

[groue]

Thank you @darrenclark for this inspiring technique! 👍

[KyleLeneau]

Did you try running carthage checkout --platform osx --no-use-binaries. The issue is that the project is using a Git submodule through standard git practices and Carthage does not see or know how to do anything with them. Thus you need the additional script to "build" the dependencies. In all my projects I usually have a makefile that wraps all these commands so I can simple run make update or make bootstrap and that will clone and build everything. BUT... Like you said doing and update in another repo is going to cause issues unless you have also run git submodule update --init --recursive at some point after the clone.

There are 2 ways to fix this:

1. One possible fix I can think of to support this is to have a build step (run phase script) for Carthage only (based on environment variable) to perform this submodule update prior to a Carthage build. I think this will work the more that I think about it and should have no impact to other installation patterns and uses.
2. In GRDB maintain the list of dependencies (even submodules) using Carthage and let Carthage try and check these out for you. Then do some path messing around to have it see the version in the Carthage/Checkouts/{platform} folder. On and Carthage update all project dependencies defined in Cartfile are also pulled down with the user of GRDB. This option seems a bit risky but it's also the cleanest, the only question is what does it do to the other install methods.

@jamfly can you provide the version you were on and the version you were trying to update to.

[freddy5566]

HI @KyleLeneau

thank you for your respond, i've fixed this problem via the script from @darrenclark's.

my version was 3.2.0 and i was trying to update to 3.3.1, but like i said i just failed by running carthage checkout --platform osx --no-use-binaries, however this issue is fixed now.

and i am sorry that i am still a beginner of Carthage , could you offer me some related reading, if it doesn't bother you, then i could get into it shortly.

and thank again of your respond, save my days!!!!

[groue]

It looks like the issue can be closed now. Thanks to all!

[KyleLeneau]

FYI, I have been trying to get my option 1 to work up above (I was up late last night building and rebuilding) but I have not had success yet on how to integrate the submodule and xcconfig file creation done BEFORE xcodebuild (via carthage) tries to build the targets.

I am proposing something that will fix it in Carthage main but feel like that might not result in anything "quick": Carthage/Carthage#1227 (comment)

[kroussevrb]

I researched quite a few SQL libraries for iOS development and GRDB was the clear winner but the lack of Carthage support was a deal breaker. We had to settle for another lib..

[groue]

@kroussevrb, I'm really sorry you had to settle with a clear loser. See #433 in order to understand the "Carthage tax" on OSS maintainers.