Skip to content

Commit

Permalink
[build] Expunge use of CONFIGURATIONS (#2325)
Browse files Browse the repository at this point in the history 
The xamarin-android Jenkins builds take awhile --
[over 5h 30min recently][0] -- in part because everything is built
*twice*: for the Debug and Release configurations; see a16673d.
(Note that this is roughly the fastest things can go, as this
build didn't need to rebuild mono.  Mono rebuilds are longer.)

An unanswered question in a16673d is *why* `make jenkins` builds
both Debug and Release configurations, *especially* considering that
in 3821e82 the `$(CONFIGURATIONS)` variable was introduced so that
the Debug configuration could be *excluded*.

Why build both Debug and Release from `make jenkins`?  Primarily
because if we're going to support both configurations, we should
ensure that they both *work*.  The Debug configuration has the nice
quality that native libraries are built with full debug information
and optimizations *disabled*, which is useful for *debugging* and
related development activities.  The Release configuration is what
the commercial products *actually ship with*.

(Additionally, `make jenkins` builds both because it just seemed like
a good idea at the time.  One target to build all!)

In retrospect, "one target to build all" is a bit of a mistake, as
"all" isn't always needed (again, 3821e82).

Additionally, building everything means builds take twice as long!

Thus, an idea: what if `make jenkins` built only *one* configuration,
and then we built *both* configurations *on separate machines*?
Then each machine could build in (roughly) half the time.

Expunge the `$(CONFIGURATIONS)` make variable from the repo.
Instead, the Jenkins configuration will *separately* build each
configuration as part of the build process, e.g.

	MSBUILD_AUTOPROVISION_ARGS="/p:AutoProvision=True /p:AutoProvisionUsesSudo=True /p:IgnoreMaxMonoVersion=False"
	
	make prepare-deps                 CONFIGURATION=Debug   V=1   MSBUILD_ARGS="$MSBUILD_AUTOPROVISION_ARGS"
	make prepare-image-dependencies   CONFIGURATION=Debug   V=1   MSBUILD=msbuild || true
	make jenkins                      CONFIGURATION=Debug   V=1   MSBUILD_ARGS="$MSBUILD_AUTOPROVISION_ARGS"
	
	make prepare-deps                 CONFIGURATION=Release V=1   MSBUILD_ARGS="$MSBUILD_AUTOPROVISION_ARGS"
	make jenkins                      CONFIGURATION=Release V=1   MSBUILD_ARGS="$MSBUILD_AUTOPROVISION_ARGS"
	
	make create-vsix                  CONFIGURATION=Debug   V=1 || true
	make create-vsix                  CONFIGURATION=Release V=1 || true
	
	make all-tests                    CONFIGURATION=Debug   V=1
	make all-tests                    CONFIGURATION=Release V=1

We can then introduce separate Jenkins jobs which build *solely* the
Debug or Release configurations, without further altering the
`make jenkins` semantics.

[0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/1229/
  • Loading branch information
@luhenry @jonpryor
luhenry authored and jonpryor committed Oct 25, 2018
1 parent 18becd2 commit ae093bf
Show file tree
Hide file tree
Showing 4 changed files with 49 additions and 71 deletions.
6 changes: 0 additions & 6 deletions Documentation/building/unix-instructions.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -190,12 +190,6 @@ There are a few ways to do it:
* Build all the platforms with:

make framework-assemblies

Note that `make framework-assemblies` builds `Mono.Android.dll`
for *both* Debug and Release configurations. To build only a
single configuration, set the `$(CONFIGURATIONS)` make variable:

make framework-assemblies CONFIGURATIONS=Debug

* Build several platforms other than the default

Expand Down
13 changes: 5 additions & 8 deletions Makefile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -102,16 +102,13 @@ prepare-external:
git submodule update --init --recursive
nuget restore $(SOLUTION)
nuget restore Xamarin.Android-Tests.sln
$(foreach conf, $(CONFIGURATIONS), \
(cd external/xamarin-android-tools && make prepare CONFIGURATION=$(conf)) && \
(cd $(call GetPath,JavaInterop) && make prepare CONFIGURATION=$(conf) JI_MAX_JDK=8) && \
(cd $(call GetPath,JavaInterop) && make bin/Build$(conf)/JdkInfo.props CONFIGURATION=$(conf) JI_MAX_JDK=8) && ) \
true
(cd external/xamarin-android-tools && make prepare CONFIGURATION=$(CONFIGURATION))
(cd $(call GetPath,JavaInterop) && make prepare CONFIGURATION=$(CONFIGURATION) JI_MAX_JDK=8)
(cd $(call GetPath,JavaInterop) && make bin/Build$(CONFIGURATION)/JdkInfo.props CONFIGURATION=$(CONFIGURATION) JI_MAX_JDK=8)

prepare-deps: prepare-external
./build-tools/scripts/generate-os-info Configuration.OperatingSystem.props
$(foreach conf, $(CONFIGURATIONS), \
mkdir -p bin/Build$(conf) ; )
mkdir -p bin/Build$(CONFIGURATION)
$(call MSBUILD_BINLOG,prepare-deps) build-tools/dependencies/dependencies.csproj
$(call MSBUILD_BINLOG,prepare-bundle) build-tools/download-bundle/download-bundle.csproj

Expand Down Expand Up @@ -142,7 +139,7 @@ include tests/api-compatibility/api-compatibility.mk
topdir := $(shell pwd)


XA_BUILD_PATHS_OUT = $(CONFIGURATIONS:%=bin/Test%/XABuildPaths.cs)
XA_BUILD_PATHS_OUT = bin/Test$(CONFIGURATION)/XABuildPaths.cs

prepare-paths: $(XA_BUILD_PATHS_OUT)

Expand Down
69 changes: 30 additions & 39 deletions build-tools/scripts/BuildEverything.mk
View file
Original file line number Diff line number Diff line change
Expand Up @@ -80,8 +80,6 @@ _MSBUILD_ARGS = \
/p:AndroidSupportedHostJitAbis=$(call join-with,:,$(ALL_HOST_ABIS)) \
/p:AndroidSupportedTargetAotAbis=$(call join-with,:,$(ALL_AOT_ABIS))

CONFIGURATIONS ?= Debug Release

.PHONY: leeroy jenkins leeroy-all opentk-jcw framework-assemblies
.PHONY: create-vsix

Expand All @@ -90,50 +88,43 @@ jenkins: prepare leeroy $(ZIP_OUTPUT)
leeroy: leeroy-all framework-assemblies opentk-jcw

leeroy-all:
$(foreach conf, $(CONFIGURATIONS), \
$(call MSBUILD_BINLOG,leeroy-all,$(_SLN_BUILD),$(conf)) $(SOLUTION) /p:Configuration=$(conf) $(_MSBUILD_ARGS) && \
$(call CREATE_THIRD_PARTY_NOTICES,$(conf),bin/$(conf)/lib/xamarin.android/ThirdPartyNotices.txt,$(THIRD_PARTY_NOTICE_LICENSE_TYPE),True,False) && ) \
true
$(call MSBUILD_BINLOG,leeroy-all,$(_SLN_BUILD),$(CONFIGURATION)) $(SOLUTION) /p:Configuration=$(CONFIGURATION) $(_MSBUILD_ARGS) && \
$(call CREATE_THIRD_PARTY_NOTICES,$(CONFIGURATION),bin/$(CONFIGURATION)/lib/xamarin.android/ThirdPartyNotices.txt,$(THIRD_PARTY_NOTICE_LICENSE_TYPE),True,False)

framework-assemblies:
PREV_VERSION="v1.0"; \
$(foreach a, $(API_LEVELS), \
CUR_VERSION=`echo "$(ALL_FRAMEWORKS)"|tr -s " "|cut -d " " -s -f $(a)`; \
$(foreach conf, $(CONFIGURATIONS), \
REDIST_FILE=bin/$(conf)/lib/xamarin.android/xbuild-frameworks/MonoAndroid/$${CUR_VERSION}/RedistList/FrameworkList.xml; \
grep -q $${PREV_VERSION} $${REDIST_FILE}; \
if [ $$? -ne 0 ] ; then \
rm -f bin/$(conf)/lib/xamarin.android/xbuild-frameworks/MonoAndroid/$${CUR_VERSION}/RedistList/FrameworkList.xml; \
fi; \
$(call MSBUILD_BINLOG,Mono.Android,$(_SLN_BUILD),$(conf)) src/Mono.Android/Mono.Android.csproj \
/p:Configuration=$(conf) $(_MSBUILD_ARGS) \
/p:AndroidApiLevel=$(a) /p:AndroidPlatformId=$(word $(a), $(ALL_PLATFORM_IDS)) /p:AndroidFrameworkVersion=$${CUR_VERSION} \
/p:AndroidPreviousFrameworkVersion=$${PREV_VERSION} || exit 1; ) \
REDIST_FILE=bin/$(CONFIGURATION)/lib/xamarin.android/xbuild-frameworks/MonoAndroid/$${CUR_VERSION}/RedistList/FrameworkList.xml; \
grep -q $${PREV_VERSION} $${REDIST_FILE}; \
if [ $$? -ne 0 ] ; then \
rm -f bin/$(CONFIGURATION)/lib/xamarin.android/xbuild-frameworks/MonoAndroid/$${CUR_VERSION}/RedistList/FrameworkList.xml; \
fi; \
$(call MSBUILD_BINLOG,Mono.Android,$(_SLN_BUILD),$(CONFIGURATION)) src/Mono.Android/Mono.Android.csproj \
/p:Configuration=$(CONFIGURATION) $(_MSBUILD_ARGS) \
/p:AndroidApiLevel=$(a) /p:AndroidPlatformId=$(word $(a), $(ALL_PLATFORM_IDS)) /p:AndroidFrameworkVersion=$${CUR_VERSION} \
/p:AndroidPreviousFrameworkVersion=$${PREV_VERSION} || exit 1; \
PREV_VERSION=$${CUR_VERSION}; )
$(foreach conf, $(CONFIGURATIONS), \
rm -f bin/$(conf)/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/Xamarin.Android.NUnitLite.dll; \
$(call MSBUILD_BINLOG,NUnitLite,$(_SLN_BUILD),$(conf)) $(MSBUILD_FLAGS) src/Xamarin.Android.NUnitLite/Xamarin.Android.NUnitLite.csproj \
/p:Configuration=$(conf) $(_MSBUILD_ARGS) \
/p:AndroidApiLevel=$(firstword $(API_LEVELS)) /p:AndroidPlatformId=$(word $(firstword $(API_LEVELS)), $(ALL_PLATFORM_IDS)) \
/p:AndroidFrameworkVersion=$(firstword $(FRAMEWORKS)) || exit 1; )
rm -f bin/$(CONFIGURATION)/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/Xamarin.Android.NUnitLite.dll; \
$(call MSBUILD_BINLOG,NUnitLite,$(_SLN_BUILD),$(CONFIGURATION)) $(MSBUILD_FLAGS) src/Xamarin.Android.NUnitLite/Xamarin.Android.NUnitLite.csproj \
/p:Configuration=$(CONFIGURATION) $(_MSBUILD_ARGS) \
/p:AndroidApiLevel=$(firstword $(API_LEVELS)) /p:AndroidPlatformId=$(word $(firstword $(API_LEVELS)), $(ALL_PLATFORM_IDS)) \
/p:AndroidFrameworkVersion=$(firstword $(FRAMEWORKS)) || exit 1;
_latest_stable_framework=$$($(MSBUILD) /p:DoNotLoadOSProperties=True /nologo /v:minimal /t:GetAndroidLatestStableFrameworkVersion build-tools/scripts/Info.targets | tr -d '[[:space:]]') ; \
$(foreach conf, $(CONFIGURATIONS), \
rm -f "bin/$(conf)/lib/xamarin.android/xbuild-frameworks/MonoAndroid/$$_latest_stable_framework"/Mono.Android.Export.* ; \
$(call MSBUILD_BINLOG,Mono.Android.Export,$(_SLN_BUILD),$(conf)) $(MSBUILD_FLAGS) src/Mono.Android.Export/Mono.Android.Export.csproj \
/p:Configuration=$(conf) $(_MSBUILD_ARGS) \
/p:AndroidApiLevel=$(firstword $(API_LEVELS)) /p:AndroidPlatformId=$(word $(firstword $(API_LEVELS)), $(ALL_PLATFORM_IDS)) \
/p:AndroidFrameworkVersion=$(firstword $(FRAMEWORKS)) || exit 1; ) \
$(foreach conf, $(CONFIGURATIONS), \
rm -f "bin/$(conf)/lib/xamarin.android/xbuild-frameworks/MonoAndroid/$$_latest_stable_framework"/OpenTK-1.0.* ; \
$(call MSBUILD_BINLOG,OpenTK,$(_SLN_BUILD),$(conf)) $(MSBUILD_FLAGS) src/OpenTK-1.0/OpenTK.csproj \
/p:Configuration=$(conf) $(_MSBUILD_ARGS) \
/p:AndroidApiLevel=$(firstword $(API_LEVELS)) /p:AndroidPlatformId=$(word $(firstword $(API_LEVELS)), $(ALL_PLATFORM_IDS)) \
/p:AndroidFrameworkVersion=$(firstword $(FRAMEWORKS)) || exit 1; )
rm -f "bin/$(CONFIGURATION)/lib/xamarin.android/xbuild-frameworks/MonoAndroid/$$_latest_stable_framework"/Mono.Android.Export.* ; \
$(call MSBUILD_BINLOG,Mono.Android.Export,$(_SLN_BUILD),$(CONFIGURATION)) $(MSBUILD_FLAGS) src/Mono.Android.Export/Mono.Android.Export.csproj \
/p:Configuration=$(CONFIGURATION) $(_MSBUILD_ARGS) \
/p:AndroidApiLevel=$(firstword $(API_LEVELS)) /p:AndroidPlatformId=$(word $(firstword $(API_LEVELS)), $(ALL_PLATFORM_IDS)) \
/p:AndroidFrameworkVersion=$(firstword $(FRAMEWORKS)) || exit 1; \
rm -f "bin/$(CONFIGURATION)/lib/xamarin.android/xbuild-frameworks/MonoAndroid/$$_latest_stable_framework"/OpenTK-1.0.* ; \
$(call MSBUILD_BINLOG,OpenTK,$(_SLN_BUILD),$(CONFIGURATION)) $(MSBUILD_FLAGS) src/OpenTK-1.0/OpenTK.csproj \
/p:Configuration=$(CONFIGURATION) $(_MSBUILD_ARGS) \
/p:AndroidApiLevel=$(firstword $(API_LEVELS)) /p:AndroidPlatformId=$(word $(firstword $(API_LEVELS)), $(ALL_PLATFORM_IDS)) \
/p:AndroidFrameworkVersion=$(firstword $(FRAMEWORKS)) || exit 1;

opentk-jcw:
$(foreach a, $(API_LEVELS), \
$(foreach conf, $(CONFIGURATIONS), \
touch bin/$(conf)/lib/xamarin.android/xbuild-frameworks/MonoAndroid/*/OpenTK-1.0.dll; \
$(call MSBUILD_BINLOG,OpenTK-JCW,$(_SLN_BUILD),$(conf)) $(MSBUILD_FLAGS) src/OpenTK-1.0/OpenTK.csproj \
/t:GenerateJavaCallableWrappers /p:Configuration=$(conf) $(_MSBUILD_ARGS) \
/p:AndroidApiLevel=$(a) /p:AndroidPlatformId=$(word $(a), $(ALL_PLATFORM_IDS)) /p:AndroidFrameworkVersion=$(word $(a), $(ALL_FRAMEWORKS)) || exit 1; ))
touch bin/$(CONFIGURATION)/lib/xamarin.android/xbuild-frameworks/MonoAndroid/*/OpenTK-1.0.dll; \
$(call MSBUILD_BINLOG,OpenTK-JCW,$(_SLN_BUILD),$(CONFIGURATION)) $(MSBUILD_FLAGS) src/OpenTK-1.0/OpenTK.csproj \
/t:GenerateJavaCallableWrappers /p:Configuration=$(CONFIGURATION) $(_MSBUILD_ARGS) \
/p:AndroidApiLevel=$(a) /p:AndroidPlatformId=$(word $(a), $(ALL_PLATFORM_IDS)) /p:AndroidFrameworkVersion=$(word $(a), $(ALL_FRAMEWORKS)) || exit 1; )
32 changes: 14 additions & 18 deletions build-tools/scripts/Packaging.mk
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,18 +7,16 @@ _BUNDLE_ZIPS_EXCLUDE = \
$(ZIP_OUTPUT_BASENAME)/bin/*/bundle-*.zip

create-vsix:
$(foreach conf, $(CONFIGURATIONS), \
MONO_IOMAP=all MONO_OPTIONS="$(MONO_OPTIONS)" $(call MSBUILD_BINLOG,create-vsix) /p:Configuration=$(conf) /p:CreateVsixContainer=True \
build-tools/create-vsix/create-vsix.csproj \
$(if $(VSIX),"/p:VsixPath=$(VSIX)") \
$(if $(EXPERIMENTAL),/p:IsExperimental="$(EXPERIMENTAL)") \
$(if $(PRODUCT_COMPONENT),/p:IsProductComponent="$(PRODUCT_COMPONENT)") \
$(if $(PACKAGE_VERSION),/p:ProductVersion="$(PACKAGE_VERSION)") \
$(if $(REPO_NAME),/p:XARepositoryName="$(REPO_NAME)") \
$(if $(PACKAGE_HEAD_BRANCH),/p:XAVersionBranch="$(PACKAGE_HEAD_BRANCH)") \
$(if $(PACKAGE_VERSION_REV),/p:XAVersionCommitCount="$(PACKAGE_VERSION_REV)") \
$(if $(COMMIT),/p:XAVersionHash="$(COMMIT)") && ) \
true
MONO_IOMAP=all MONO_OPTIONS="$(MONO_OPTIONS)" $(call MSBUILD_BINLOG,create-vsix) /p:Configuration=$(CONFIGURATION) /p:CreateVsixContainer=True \
build-tools/create-vsix/create-vsix.csproj \
$(if $(VSIX),"/p:VsixPath=$(VSIX)") \
$(if $(EXPERIMENTAL),/p:IsExperimental="$(EXPERIMENTAL)") \
$(if $(PRODUCT_COMPONENT),/p:IsProductComponent="$(PRODUCT_COMPONENT)") \
$(if $(PACKAGE_VERSION),/p:ProductVersion="$(PACKAGE_VERSION)") \
$(if $(REPO_NAME),/p:XARepositoryName="$(REPO_NAME)") \
$(if $(PACKAGE_HEAD_BRANCH),/p:XAVersionBranch="$(PACKAGE_HEAD_BRANCH)") \
$(if $(PACKAGE_VERSION_REV),/p:XAVersionCommitCount="$(PACKAGE_VERSION_REV)") \
$(if $(COMMIT),/p:XAVersionHash="$(COMMIT)")

package-oss-name:
@echo ZIP_OUTPUT=$(ZIP_OUTPUT)
Expand All @@ -35,12 +33,10 @@ package-oss $(ZIP_OUTPUT):
fi
_exclude_list=".__exclude_list.txt"; \
ls -1d $(_BUNDLE_ZIPS_EXCLUDE) > "$$_exclude_list" 2>/dev/null ; \
for c in $(CONFIGURATIONS) ; do \
_sl="$(ZIP_OUTPUT_BASENAME)/bin/$$c/lib/xamarin.android/xbuild/.__sys_links.txt"; \
if [ ! -f "$$_sl" ]; then continue; fi; \
for f in `cat $$_sl` ; do \
echo "$(ZIP_OUTPUT_BASENAME)/bin/$$c/lib/xamarin.android/xbuild/$$f" >> "$$_exclude_list"; \
done; \
_sl="$(ZIP_OUTPUT_BASENAME)/bin/$(CONFIGURATION)/lib/xamarin.android/xbuild/.__sys_links.txt"; \
if [ ! -f "$$_sl" ]; then continue; fi; \
for f in `cat $$_sl` ; do \
echo "$(ZIP_OUTPUT_BASENAME)/bin/$CONFIGURATION/lib/xamarin.android/xbuild/$$f" >> "$$_exclude_list"; \
done
echo "Exclude List:"
cat ".__exclude_list.txt"
Expand Down

0 comments on commit ae093bf

Please sign in to comment.