Reduce library dependencies

[QuintinWillison]

Which version of the Ably SDK are you using?

1.1.14

On which platform does the issue happen?

iOS13

Are you using Cocoapods?

1.8.4

Which version of Xcode are you using?

11.3

Problem

We have a customer who is integrating our SDK within their own SDK which is then offered out to their customers. The problem they have is that the five dependencies of our CocoaPod are proving to be a big concern to their customers. They would like to make their own SDK as lightweight as possible. Our transitive dependencies don't help!

┆Issue is synchronized with this Jira Story by Unito

[QuintinWillison]

I'm awaiting confirmation from the customer as to their build and distribution process for their SDK.

[QuintinWillison]

Some more information from the customer in respect of how they are distributing their SDK to their customers:

Our SDK can be distributed by both Cocoapods and Carthage, and will support Swift Package Manager in the future. Since Carthage doesn't support prebuilt framework with dependencies, our customers have to add all of our dependencies in their carthfile. After that, they also have to copy the dependencies path (including the dependencies of each dependency) into the build phase section. It's pretty inconvenient. Cocoapods is the most recommended way as they only need to add our SDK into the podfile. However, seeing a lot of dependencies are installed into their app is scary. That's why we have to make our dependencies as few as possible.

In terms of linking choices (e.g. static vs dynamic) the customer has put it very succinctly for us:

In the beginning, we tried to archive our SDK as an umbrella framework, but we faced the code-sign problem, and Apple also doesn't recommend the umbrella framework. Now we still include some static libraries in our SDK, that's ok because code-sign problem affects the dynamic library only. FYI.

Furthermore, when they build an app against their SDK (via CocoaPods), they are getting the following on launching the app:

error: module importing failed: invalid pathname
dyld: Library not loaded: @rpath/KSCrash.framework/KSCrash
  Referenced from:
      [REDACTED path to customer SDK within customer test app]
  Reason: image not found
Message from debugger: Terminated due to signal 6

With their succinct analysis being:

I think that’s because we link the KSCrash.framework in our SDK, but when the customer’s app installs the dependencies from CocoaPods, these framework would have different name (KSCrashAblyFork.framework). So the framework name is mismatched. I think this situation also happens on others ably-fork frameworks.

[ricardopereira]

About removing dependencies:

• SocketRocket: we can replace it using the native WebSocket support from NSURLSession but it's only available on iOS 13 (https://developer.apple.com/documentation/foundation/nsurlsession/3181171-websockettaskwithurl?language=objc). BTW, there's a related issue Compile SocketRocket with main library #805
• SAMKeychain: we can implement this in the library since it's not so difficult. We added this dep just for convenience and to win time. Related issue Replace SAMKeychain with a maintained library #897
• ULID: it's maintained by me. We can add the source with a submodule because it's something that won't change, I guess.
• MsgPack: it's not maintained anymore and that's why we have a fork.
• KSCrash: this one is tricky.

[QuintinWillison]

Thanks, @ricardopereira ... We may have to take a slightly more pragmatic approach in the short term in respect of SocketRocket - perhaps bringing it in as a Git submodule. Same perhaps even for KSCrash. To make this play with downstream linkers (app developers) there should be an opportunity hide 'transitively built' symbols in our build output or something like that but that'll need R&D and plenty of testing (i.e. -fvisibility=hidden, etc..).

[lawrence-forooghian]

It sounds like, if our immediate goal is to reduce the number of dependencies that are forced upon users of our library, then we could continue to use SocketRocket by compiling it directly into the SDK. Is this perhaps what you mean when you talk about "bringing it in as a Git submodule", @QuintinWillison? I think that for me the path of least friction looks like:

1. Bring the code from our SocketRocket fork into this repo, either by adding it as a submodule or just copying the code directly.
2. Find and replace SR prefix in SocketRocket with an Ably-specific prefix, e.g. ARTSR.
3. Add the SocketRocket source files into the Ably-* framework targets in the Ably Xcode project (adding all of the SocketRocket headers as private headers).
4. Remove SocketRocket from CocoaPods specification etc.

The net result of this will be that:

• users of our library will no longer explicitly need to include SocketRocket
• users of our library who have their own dependency on SocketRocket will not notice the extra copy of SocketRocket that we bundle and will not be affected by its presence (since we've namespaced it)

We'd need to make sure that all of the existing code in SocketRocket is properly namespaced, but at a glance it looks like it is. We'd also need to make sure that we satisfy the conditions of SocketRocket’s LICENSE.

My vague memory of the -fvisibility=hidden flag is that it allows us to remove symbols from the symbol table of our compiled framework, is that right? If so, I'm not sure that it'd be necessary. Yes, we'd end up with a bunch of ARTSR… symbols, which are only used internally by the library, being visible in the compiled framework, but this is no different to the way in which our current “private” classes (e.g. ARTRestChannelInternal) are visible in the symbol table of the compiled framework. I think that as long as we're not cluttering the public interface (as expressed through header files), and not stomping on other people's symbols, then we don't need to worry about it. Furthermore, even if we changed the symbols' visibility, we'd still need to namespace them too, since Objective-C classes share a global namespace within the Obj-C runtime, independent of linker visibility.

But I guess the other question is whether we do have a broader interest in replacing SocketRocket? #805 suggests that we do, but it seems like it's served us well until now. Might it be something that we can live with until we're eventually able to just rely on iOS 13+'s built-in support? As far as I can tell, the alternatives to SocketRocket right now would be either a different Objective-C library (which, as you've said there, all seem to be unsupported now), or to pick a lightweight C library and integrate that ourselves. At a brief look of the C WS client lib scene there seem to be a few options, but I'm not sure about Darwin support. Ideally we'd want a library that allows us to plug in to some native API for initiating a TLS connection, instead of having to bundle something like OpenSSL.

[QuintinWillison]

Thanks @lawrence-forooghian ... for the moment I've taken this discussion internal.

[maratal]

Since nobody among our clients is going to use delta-codec-cocoa and msgpack-objective-C directly ever, we can just move them into git submodules effectively hiding these dependencies from end users. WDYT? @ben-xD @QuintinWillison @lukasz-szyszkowski

[lukasz-szyszkowski]

Since nobody among our clients is going use delta-codec-cocoa and msgpack-objective-C directly ever, we can just move them into git submodules effectively hiding these dependencies from end users. WDYT? @ben-xD @QuintinWillison @lukasz-szyszkowski

Didn't this require users to install these dependencies via git submodule update --init --recursive?

[maratal]

I was inspired byxdelta which is submodule of delta-codec-cocoa. But we manually pull sources in podspec for it via s.source_files. 🙃

[maratal]

And nothing prevents us to use the same trick for delta-codec-cocoa in ably-cocoa podspec.

[ben-xD]

The problem they have is that the five dependencies of our CocoaPod are proving to be a big concern to their customers.

The customer complaint started off with 5 dependencies, but with msgpack and AblyDeltaCodec, this shouldn't be a problem, should it?

I think we should not use submodules and keep those 2 dependencies, they are core functionality for Ably: msgpack for message structure, and AblyDeltaCodec for delta compression.

---

A bit more detail:

After using submodules, the dependency is still there. Using submodules, we are using less common ways of integrating dependencies (submodules are known to be a pain, and their integration with other tools will need to be tested, such as Cocoapods, Carthage and SPM). Now we will have recursive submodules 😅 .

What is the point of hiding a dependency? I think this is different to reducing the number of dependencies. Some dependencies are still good :). Hiding dependencies makes it more complicated to evaluate the number of dependencies, and doesn't bring actual benefits to the size. If AblyDeltaCodec and Msgpack are used, they should be kept as dependencies.

[maratal]

BTW, @ben-xD @lukasz-szyszkowski we can use spec.source = { ... :submodules => true } for CocoaPods to automatically resolve submodules, but you have a strong point there. Nothing wrong in having just two essential dependencies. So, we are good on this issue then, right? @QuintinWillison

[lukasz-szyszkowski]

I agree with @ben-xD . Let's keep these two dependencies as they are.

[maratal]

@QuintinWillison could you please give us your refreshed opinion on this?

[QuintinWillison]

@ben-xD was indeed spot on with:

What is the point of hiding a dependency? I think this is different to reducing the number of dependencies.

So, yes, I agree. Thanks for checking, @maratal. 😄