Support external target dependencies via subprojects #701

evandcoleman · 2019-10-30T19:36:07Z

Resolves #570

This builds upon the work done in #655 by allowing target dependencies to reference targets in other Xcode projects.

Motivation

At my company, we have several internal dependencies that we include via submodules. We'd like the ability to include these dependencies as subprojects and reference their targets in our main Xcode project.

Changes

As introduced in #655, subprojects are declared at the top level.

projectReferences:
  FooLib:
    path: path/to/FooLib.xcodeproj
targets:
  MyApp:
    type: application
    platform: iOS
    deploymentTarget: "11.0"
    sources: [Sources]
    dependencies:
      - target: FooLib/MyTarget

liamnichols · 2019-11-04T17:25:05Z

Just wanted to say thanks for this by the way! I'm experimenting with adopting XcodeGen and this would have been one of the blockers for us however when running your fork it works great, so good timing 😄

I did however noticed something while testing... But I'm not sure if it relates to this change in particular or if it was to do with the initial introduction of projectReferences... I'm gonna go ahead and ask anyway...

If I declare my projectReferences from within an included spec in a different directory, the path isn't relative to the directory that the included spec was contained in and instead its relative to the main spec. For example:

ProjectDefinitions/base.yml

---
projectReferences:
  Network:
    path: Frameworks/Network/Network.xcodeproj
  Logger:
    path: Frameworks/Logger/Logger.xcodeproj
  Persistence:
    path: Frameworks/Persistence/Persistence.xcodeproj
options:
  xcodeVersion: "11.1"
  deploymentTarget:
    iOS: "11.4"
settings:
  base:
    CURRENT_PROJECT_VERSION: 9.58.0.0
    VERSIONING_SYSTEM: apple-generic
    SWIFT_VERSION: 5.0
    INFOPLIST_FILE: ${TARGET_NAME}/SupportingFiles/Info.plist
targetTemplates:
  CoreModule:
    platform: iOS
    type: framework
    sources:
      - ../Frameworks/${frameworkName}/${frameworkName}

project.yml

name: Global
include:
  - ProjectDefinitions/base.yml
...

This example produces the xcodeproj correctly however the ProjectDefinitions/base.yml looks wrong since I have my projectReferences paths relative to a different directory from the targetTemplates sources path.

I first tried implementing the projectReferences paths relative to the file they're defined in but I was then hit with the following while generating the project:

⚙️  Generating project...
The project cannot be found at ../Frameworks/Network/Network.xcodeproj

I just wanted to flag it here incase it's related to this change specifically... If it's not i can rase it separately and even have a go at correcting it. Thanks!

yonaskolb · 2019-11-07T11:52:50Z

Thank you for the great PR @evandcoleman! You've correctly identified the planned next step in harnessing the new project references 👍
I don't have time to review this right now, but will do so in the next couple of days. As an overview things looks pretty good, there's just some code duplication we can remove

evandcoleman · 2019-11-07T15:55:17Z

@yonaskolb Thanks! I'm really looking forward to using XcodeGen and this is the last step for us! I'd also like your opinion on @liamnichols' comment. I wasn't really sure if I should be referencing subprojects via their relative path, or to prepend the project basepath so I think there's some room for improvement and consistency there.

acecilia · 2019-11-15T11:12:17Z

just adding some more context here. I managed to depend on targets from external projects without using the new projectReferences option by:

Adding the external project to the main one using the fileGroups option.
Using implicit framework dependencies.

An example project.yml:

name: Example

options:
  minimumXcodeGenVersion: 2.10.1

fileGroups: [
  FooLib/FooLib.xcodeproj
]

targets:
  Bar:
    platform: iOS
    type: application
    dependencies:
      - framework: FooLib.framework
        implicit: true

Find a working example here (under the Workaround folder): https://github.com/acecilia/XcodeGenProjectReference

evandcoleman · 2019-11-18T16:17:20Z

@acecilia I don't believe that will actually add a target dependency to your target. That was one of the driving factors here since there's no requirement that target product names be unique.

acecilia · 2019-11-20T16:43:00Z

@evandcoleman It adds a framework dependency that Xcode is able to implicitly understand that corresponds to the target in the external project (due to the "Find Implicit Dependencies" option of the scheme), and thus it will build both correctly. If you do not believe it works you can have a look at the example project.

That said, it does not turn this PR invalid. My goal was just to add my experience and some extra context to this discussion :)

evandcoleman · 2019-12-18T15:14:40Z

@yonaskolb Is this PR still being considered? The solutions posted here may work for some, but we include several dependencies that have multiple targets with the same product name (for different platforms). So implicit dependencies will not work.

mathieutozer · 2020-01-11T16:37:42Z

I'm wrestling with a similar issue. Unfortunately some of my sub dependencies have cocoapod dependencies too so the fileGroups workaround that @acecilia demonstrated doesn't work. Some of them support Carthage so I could move to that and dump cocoa pods for everything below my root applications, unless anyone can think of another work around. I understand trying to marry the cocoa pods approach and this one is not really feasible or desirable.

For context I am trying to modularize a codebase into a tree of modules that compile into cross platform apps.

yonaskolb

Sorry for the late review @evandcoleman. Great work on this one! I've left some comments above. This PR will need to have master merged in as well (preferable to a rebase at this point, so we have visibility of the changes, in regards to conflict resolution)

Sources/XcodeGenKit/PBXProjGenerator.swift

yonaskolb · 2020-01-12T11:02:31Z

Sources/XcodeGenKit/PBXProjGenerator.swift

+
+                    let link = dependency.link ??
+                        ((dependecyLinkage == .dynamic && target.type != .staticLibrary) ||
+                        (dependecyLinkage == .static && target.type.isExecutable))


Can we pull common stuff like this out and above the switch

Tests/Fixtures/external_target_test/test_project.yml

yonaskolb · 2020-01-12T11:10:13Z

Sources/XcodeGenKit/PBXProjGenerator.swift

+                PBXGroup(
+                    children: subprojectFileReferences,
+                    sourceTree: .group,
+                    name: "Projects"


Is adding all projects into a Projects group sufficient for most people? I could imagine people wanting to add their projects to a specific group.

For my use case, it doesn't really matter. But I agree, I could see a use for it. How would you think that would work best? A new parameter under projectReferences?

For now it's ok 👍

yonaskolb · 2020-01-12T11:11:22Z

Sources/XcodeGenKit/PBXProjGenerator.swift

+
+        guard let index = projectFileReferenceIndex,
+            let projectFileReference = self.pbxProj.rootObject?.projects[index]["ProjectRef"] as? PBXFileReference,
+            let productsGroup = self.pbxProj.rootObject?.projects[index]["ProductGroup"] as? PBXGroup else {


This file is getting too large and complicated. Will have to look at splitting this up into things like TargetGenerator and the like, after this

yonaskolb · 2020-01-12T11:23:05Z

@liamnichols the relative project reference paths in included files should be fixed with #740 and #751

evandcoleman · 2020-01-13T19:08:32Z

@yonaskolb Just pushed up a bunch of changes and rebased with master. Let me know what you think of the dependency target generation now. I opted to create a "fake" Target using the subproject dependency's PBXTarget. This allowed me to reuse almost all of the generation code for both local and external targets.

yonaskolb

Thanks for updating @evandcoleman. Left some comments above

yonaskolb · 2020-01-15T11:29:27Z

Sources/XcodeGenKit/PBXProjGenerator.swift

+        if let cachedProject = projects[reference] {
+            return cachedProject
+        }
+        let pbxproj = try XcodeProj(pathString: (project.basePath + Path(reference.path).normalize()).string).pbxproj


You're right about not being able to import GenerationError. We can just leave this as it is for now

yonaskolb · 2020-01-15T11:52:26Z

Sources/XcodeGenKit/PBXProjGenerator.swift

+                PBXGroup(
+                    children: subprojectFileReferences,
+                    sourceTree: .group,
+                    name: "Projects"


For now it's ok 👍

yonaskolb · 2020-01-15T12:02:14Z

Tests/Fixtures/TestProject/Project.xcodeproj/project.pbxproj

@@ -78,6 +79,7 @@
 		66C3C5E3C13325F351A3008F /* module.modulemap in CopyFiles */ = {isa = PBXBuildFile; fileRef = F2950763C4C568CC85021D18 /* module.modulemap */; };
 		6E8F8303759824631C8D9DA3 /* Localizable.strings in Resources */ = {isa = PBXBuildFile; fileRef = 9E17D598D98065767A04740F /* Localizable.strings */; };
 		7148A4172BFA1CC22E6ED5DB /* MainInterface.storyboard in Resources */ = {isa = PBXBuildFile; fileRef = 753001CDCEAA4C4E1AFF8E87 /* MainInterface.storyboard */; };
+		715DD7C4C194A04A5C4049EF /* ExternalTarget.framework in Embed Frameworks */ = {isa = PBXBuildFile; fileRef = 3DCDCD083AABD1D30EA34576 /* ExternalTarget.framework */; settings = {ATTRIBUTES = (CodeSignOnCopy, RemoveHeadersOnCopy, ); }; };


Are there supposed to be 2 build files added? As these are referencing the same PBXFileReference, XcodeProj can't give them a deterministic id hence the failing diff tests.
If they both need to be there then XcodeProj needs to be updated to take into account something that is different about them like the settings. That has to be done here https://github.com/tuist/XcodeProj/blob/master/Sources/XcodeProj/Utils/ReferenceGenerator.swift#L290

Yeah, one is for linking and the other is to embed it. I notice this is also the case for Framework2.framework in that test project, but it does not cause a failing diff test. Any idea why?

This will be fixed in tuist/XcodeProj#518. Will update this PR once that is merged.

yonaskolb · 2020-01-15T12:23:22Z

Sources/ProjectSpec/Linkage.swift

+
+extension PBXTarget {
+
+    public var defaultLinkage: Linkage {


This is the same logic as above. Let's combine to an extension on PBXProductType in this file

yonaskolb · 2020-01-15T12:25:39Z

Sources/XcodeGenKit/PBXProjGenerator.swift

-
-            let embed = dependency.embed ?? target.shouldEmbedDependencies
+        func processTargetDependency(_ dependency: Dependency, dependencyTarget: Target, embedFileReference: PBXFileElement) {
+            let dependencyLinkage = dependencyTarget.defaultLinkage


We can just use the following once defaultLinkage is extracted to PBXProductType in Linkage.swift:

let dependencyLinkage = dependencyTarget.productType.defaultLinkage ?? .none

evandcoleman · 2020-01-29T16:57:51Z

@yonaskolb Updated to the latest XcodeProj and tests are passing now 🎉

mobileben · 2020-02-01T01:47:34Z

I've started to look at projectReferences. Our use case is to dynamically to add sub-projects versus prebuilt binaries for debugging. I'm experimenting with this. What I have found though is a couple of issues. Not sure if this latest merge will address.

When you include a project reference, it puts it in a folder group based on path. I ideally want my sub-projects at the root so they are easier to see. I would imagine this is something others would want as well, because it makes it easier to work with subprojects.
I added a scheme which would also be the scheme for the main target which is the app. To this scheme I added the project reference target. This resulted in the creation of an identical schema (but a framework).

"Application"

name: Project1
options:
  bundleIdPrefix:  com.company.proj1
fileGroups:
  - ../proj2/Proj2Framework.xcodeproj
targets:
  Project1:
    type: application
    platform: iOS
    deploymentTarget: "11.0"
    sources:
      - Classes
projectReferences: 
  Proj2Framework:
    path: ../proj2/Proj2Framework.xcodeproj
schemes:
  Project1:
    build:
      targets:
        Proj2Framework/Project2: ["build"]

"Framework"

name: Proj2Framework
options:
  bundleIdPrefix: com.company.proj2
targets:
  Project2:
      type: framework
      platform: iOS
      deploymentTarget: "11.0"
      sources: Classes

I did use evandcoleman's branch here: https://github.com/evandcoleman/XcodeGen/tree/edc-external-target-dependency

The issue with the scheme exists still (could be user error of course). The difference is a group called Projects is created with the subproject. Definitely better, but would be helpful to be able to define where to place it.

evandcoleman · 2020-02-01T17:55:28Z

@mobileben I believe the issue you are seeing is not related to this PR. This PR relates to depending on targets from subprojects. The feature to create schemes based on external targets was done in #655.

yonaskolb

Great work @evandcoleman!
I've just fixed some small conflicts and added a note in the docs

evandcoleman · 2020-02-02T16:27:14Z

Thanks @yonaskolb!

evandcoleman force-pushed the edc-external-target-dependency branch from 08f7443 to 253cbab Compare November 1, 2019 14:42

acecilia mentioned this pull request Nov 25, 2019

How to use projectReferences? #717

Open

kateinoigakukun mentioned this pull request Dec 5, 2019

Add bundle dependency type #723

Merged

yonaskolb requested changes Jan 12, 2020

View reviewed changes

evandcoleman force-pushed the edc-external-target-dependency branch from 453f5f2 to 75b426d Compare January 13, 2020 17:12

yonaskolb reviewed Jan 15, 2020

View reviewed changes

evandcoleman mentioned this pull request Jan 27, 2020

Generate project references before targets tuist/XcodeProj#518

Merged

evandcoleman added 11 commits January 29, 2020 11:34

Allow external target dependencies via subprojects

d830ffb

Update CHANGELOG

f725e1f

Update ProjectSpec

20c2dab

Fix test

1a5caa0

Use existing fixture for test

2735845

Sort subprojects by name

e855183

Throw subproject generation error instead of crashing

25d46de

Cleanup target dependency generation

af6a358

Update test fixture project

a02e060

Combine extentions in Linkage

de34f3e

Update XcodeProj to 7.7.0

8cb1f5a

evandcoleman force-pushed the edc-external-target-dependency branch from ab0cc74 to 8cb1f5a Compare January 29, 2020 16:37

evandcoleman requested a review from yonaskolb January 29, 2020 16:58

liamnichols mentioned this pull request Jan 30, 2020

[RFC] Workspace generation #168

Open

yonaskolb added 3 commits February 2, 2020 09:36

Merge branch 'master' into edc-external-target-dependency

0113d9b

Update CHANGELOG.md

2071b01

Update ProjectSpec.md

3d8c4a3

yonaskolb approved these changes Feb 1, 2020

View reviewed changes

yonaskolb merged commit 6bfd620 into yonaskolb:master Feb 1, 2020

evandcoleman deleted the edc-external-target-dependency branch February 2, 2020 16:27

evandcoleman mentioned this pull request Mar 3, 2020

Allow external dependencies in aggregate targets #797

Open

liamnichols mentioned this pull request Jun 14, 2020

Idea - Chaining spec generation using projectReferences #891

Draft

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support external target dependencies via subprojects #701

Support external target dependencies via subprojects #701

evandcoleman commented Oct 30, 2019 •

edited

Loading

liamnichols commented Nov 4, 2019

yonaskolb commented Nov 7, 2019

evandcoleman commented Nov 7, 2019 •

edited

Loading

acecilia commented Nov 15, 2019

evandcoleman commented Nov 18, 2019

acecilia commented Nov 20, 2019

evandcoleman commented Dec 18, 2019

mathieutozer commented Jan 11, 2020 •

edited

Loading

yonaskolb left a comment

yonaskolb Jan 12, 2020

yonaskolb Jan 12, 2020

evandcoleman Jan 13, 2020

yonaskolb Jan 15, 2020

yonaskolb Jan 12, 2020

yonaskolb commented Jan 12, 2020

evandcoleman commented Jan 13, 2020

yonaskolb left a comment

yonaskolb Jan 15, 2020

yonaskolb Jan 15, 2020

yonaskolb Jan 15, 2020

evandcoleman Jan 15, 2020

evandcoleman Jan 27, 2020

yonaskolb Jan 15, 2020 •

edited

Loading

yonaskolb Jan 15, 2020

evandcoleman commented Jan 29, 2020

mobileben commented Feb 1, 2020 •

edited

Loading

evandcoleman commented Feb 1, 2020

yonaskolb left a comment

evandcoleman commented Feb 2, 2020

Support external target dependencies via subprojects #701

Support external target dependencies via subprojects #701

Conversation

evandcoleman commented Oct 30, 2019 • edited Loading

Motivation

Changes

liamnichols commented Nov 4, 2019

yonaskolb commented Nov 7, 2019

evandcoleman commented Nov 7, 2019 • edited Loading

acecilia commented Nov 15, 2019

evandcoleman commented Nov 18, 2019

acecilia commented Nov 20, 2019

evandcoleman commented Dec 18, 2019

mathieutozer commented Jan 11, 2020 • edited Loading

yonaskolb left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

yonaskolb commented Jan 12, 2020

evandcoleman commented Jan 13, 2020

yonaskolb left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

yonaskolb Jan 15, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

evandcoleman commented Jan 29, 2020

mobileben commented Feb 1, 2020 • edited Loading

evandcoleman commented Feb 1, 2020

yonaskolb left a comment

Choose a reason for hiding this comment

evandcoleman commented Feb 2, 2020

evandcoleman commented Oct 30, 2019 •

edited

Loading

evandcoleman commented Nov 7, 2019 •

edited

Loading

mathieutozer commented Jan 11, 2020 •

edited

Loading

yonaskolb Jan 15, 2020 •

edited

Loading

mobileben commented Feb 1, 2020 •

edited

Loading