Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding Git binary to core rp image #5568

Merged
merged 5 commits into from
May 17, 2023
Merged

Adding Git binary to core rp image #5568

merged 5 commits into from
May 17, 2023

Conversation

kachawla
Copy link
Contributor

@kachawla kachawla commented May 17, 2023

Description

Terraform uses Git to download modules from TF registry, so we need Git as a part of core rp since Terraform recipe execution will happen inside core RP for now.

To achieve this, switching CoreRP to use Alpine image instead of distroless. We can revert back to distroless when we move Terraform execution to separate container.

Issue reference

Fixes: https://dev.azure.com/azure-octo/Incubations/_workitems/edit/7919

Checklist

Please make sure you've completed the relevant tasks for this PR, out of the following list:

  • Code compiles correctly
  • Adds necessary unit tests for change - Not applicable
  • Adds necessary E2E tests for change - Covered by existing tests
  • Unit tests passing
  • Extended the documentation / Created issue for it - Not applicable

@kachawla kachawla requested review from rynowak and youngbupark May 17, 2023 04:15
@github-actions
Copy link

github-actions bot commented May 17, 2023

Radius functional test overview

🔍 Go to test action run

Name Value
Repository project-radius/radius
Commit ref 01f594d
Unique ID 1385ce3bc2
Image tag pr-1385ce3bc2
Click here to see the list of tools in the current test run
  • KinD: v0.18.0
  • Dapr: 1.10.0
  • Azure KeyVault CSI driver: 1.4.2
  • Azure Workload identity webhook: 1.0.0
  • recipe location radiusdev.azurecr.io/test/functional/corerp/recipes/<name>:pr-1385ce3bc2
  • appcore-rp test image location: radiusdev.azurecr.io/appcore-rp:pr-1385ce3bc2
  • ucp test image location: radiusdev.azurecr.io/ucpd:pr-1385ce3bc2

Test Status

⌛ Building Radius and pushing container images for functional tests...
❌ Container images build failed
❌ Test recipe publishing failed

build/git.mk Outdated Show resolved Hide resolved
build/git.mk Outdated
sudo make install install-doc install-html install-info
mkdir -p $(OUT_DIR)/gitbinary
cp /tmp/gitbinary/bin/git $(OUT_DIR)/gitbinary/
chmod +x $(OUT_DIR)/gitbinary/git
Copy link
Contributor Author

@kachawla kachawla May 17, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Git installation goes through, but looks like the binary isn't executable based on the error (install directory permissions seemed ok)

Error: Failed to download module\n\nCould not download module \"\" (main.tf.json:25) source code from\n\"git::https://github.com/Azure/terraform-azurerm-cosmosdb?ref=v1.0.0\": error\ndownloading 'https://github.com/Azure/terraform-azurerm-cosmosdb?ref=v1.0.0':\nerror running /usr/local/bin/git: .\n\n\nError: Failed to download module\n\nCould not download module \"\" (main.tf.json:25) source code from\n\"git::https://github.com/Azure/terraform-azurerm-cosmosdb?ref=v1.0.0\": error\ndownloading 'https://github.com/Azure/terraform-azurerm-cosmosdb?ref=v1.0.0':\nerror running /usr/local/bin/git: .\n\n\nError: Failed to update module manifest\n\nUnable to write the module manifest file: open\n.terraform/modules/modules.json: no such file or directory

@github-actions
Copy link

github-actions bot commented May 17, 2023

Test Results

2 549 tests  ±0   2 542 ✔️ ±0   1m 57s ⏱️ -1s
   229 suites ±0          7 💤 ±0 
       1 files   ±0          0 ±0 

Results for commit 32321fb. ± Comparison against base commit 3300c29.

This pull request removes 2 and adds 2 tests. Note that renamed tests count towards both.
github.com/project-radius/radius/pkg/armrpc/frontend/controller ‑ TestValidateEtag_IfMatch/61f3f268-282b-4378-bf38-ed66889fa7f6
github.com/project-radius/radius/pkg/armrpc/frontend/controller ‑ TestValidateEtag_IfMatch/61f3f268-282b-4378-bf38-ed66889fa7f6#01
github.com/project-radius/radius/pkg/armrpc/frontend/controller ‑ TestValidateEtag_IfMatch/fdd583e3-cde4-4696-ad66-1e22d050a136
github.com/project-radius/radius/pkg/armrpc/frontend/controller ‑ TestValidateEtag_IfMatch/fdd583e3-cde4-4696-ad66-1e22d050a136#01

♻️ This comment has been updated with latest results.

@github-actions
Copy link

62.6

For the detailed report, please go to Checks tab, click Build and Test, and then download unit_test_coverage artifact at the bottom of build page.

  • Your PR branch coverage: 62.6 %
  • main branch coverage: 62.6 %
  • diff coverage: 0 %

The coverage result does not include the functional test coverage.

@kachawla kachawla force-pushed the kachawla/git-binary branch from 01f594d to 4fc6031 Compare May 17, 2023 04:30
@github-actions
Copy link

github-actions bot commented May 17, 2023

Radius functional test overview

🔍 Go to test action run

Name Value
Repository project-radius/radius
Commit ref 4fc6031
Unique ID 6ead9e3982
Image tag pr-6ead9e3982
Click here to see the list of tools in the current test run
  • KinD: v0.18.0
  • Dapr: 1.10.0
  • Azure KeyVault CSI driver: 1.4.2
  • Azure Workload identity webhook: 1.0.0
  • recipe location radiusdev.azurecr.io/test/functional/corerp/recipes/<name>:pr-6ead9e3982
  • appcore-rp test image location: radiusdev.azurecr.io/appcore-rp:pr-6ead9e3982
  • ucp test image location: radiusdev.azurecr.io/ucpd:pr-6ead9e3982

Test Status

⌛ Building Radius and pushing container images for functional tests...
❌ Container images build failed
❌ Test recipe publishing failed

@github-actions
Copy link

62.6

For the detailed report, please go to Checks tab, click Build and Test, and then download unit_test_coverage artifact at the bottom of build page.

  • Your PR branch coverage: 62.6 %
  • main branch coverage: 62.6 %
  • diff coverage: 0 %

The coverage result does not include the functional test coverage.

@kachawla kachawla force-pushed the kachawla/git-binary branch from 4fc6031 to 07a3847 Compare May 17, 2023 04:55
@github-actions
Copy link

github-actions bot commented May 17, 2023

Radius functional test overview

🔍 Go to test action run

Name Value
Repository project-radius/radius
Commit ref 07a3847
Unique ID cab1038ad8
Image tag pr-cab1038ad8
Click here to see the list of tools in the current test run
  • KinD: v0.18.0
  • Dapr: 1.10.0
  • Azure KeyVault CSI driver: 1.4.2
  • Azure Workload identity webhook: 1.0.0
  • recipe location radiusdev.azurecr.io/test/functional/corerp/recipes/<name>:pr-cab1038ad8
  • appcore-rp test image location: radiusdev.azurecr.io/appcore-rp:pr-cab1038ad8
  • ucp test image location: radiusdev.azurecr.io/ucpd:pr-cab1038ad8

Test Status

⌛ Building Radius and pushing container images for functional tests...
❌ Container images build failed
❌ Test recipe publishing failed

@kachawla kachawla force-pushed the kachawla/git-binary branch from 07a3847 to 779d954 Compare May 17, 2023 05:05
@github-actions
Copy link

github-actions bot commented May 17, 2023

Radius functional test overview

🔍 Go to test action run

Name Value
Repository project-radius/radius
Commit ref 779d954
Unique ID 886e65ce4d
Image tag pr-886e65ce4d
Click here to see the list of tools in the current test run
  • KinD: v0.18.0
  • Dapr: 1.10.0
  • Azure KeyVault CSI driver: 1.4.2
  • Azure Workload identity webhook: 1.0.0
  • recipe location radiusdev.azurecr.io/test/functional/corerp/recipes/<name>:pr-886e65ce4d
  • appcore-rp test image location: radiusdev.azurecr.io/appcore-rp:pr-886e65ce4d
  • ucp test image location: radiusdev.azurecr.io/ucpd:pr-886e65ce4d

Test Status

⌛ Building Radius and pushing container images for functional tests...
❌ Container images build failed
❌ Test recipe publishing failed

@kachawla kachawla force-pushed the kachawla/git-binary branch from 779d954 to cfcfb64 Compare May 17, 2023 05:12
@github-actions
Copy link

github-actions bot commented May 17, 2023

Radius functional test overview

🔍 Go to test action run

Name Value
Repository project-radius/radius
Commit ref cfcfb64
Unique ID 012af489a4
Image tag pr-012af489a4
Click here to see the list of tools in the current test run
  • KinD: v0.18.0
  • Dapr: 1.10.0
  • Azure KeyVault CSI driver: 1.4.2
  • Azure Workload identity webhook: 1.0.0
  • recipe location radiusdev.azurecr.io/test/functional/corerp/recipes/<name>:pr-012af489a4
  • appcore-rp test image location: radiusdev.azurecr.io/appcore-rp:pr-012af489a4
  • ucp test image location: radiusdev.azurecr.io/ucpd:pr-012af489a4

Test Status

⌛ Building Radius and pushing container images for functional tests...
❌ Container images build failed
❌ Test recipe publishing failed

@kachawla kachawla force-pushed the kachawla/git-binary branch from cfcfb64 to 6998519 Compare May 17, 2023 05:19
@github-actions
Copy link

github-actions bot commented May 17, 2023

Radius functional test overview

🔍 Go to test action run

Name Value
Repository project-radius/radius
Commit ref 6998519
Unique ID 4ca23f8802
Image tag pr-4ca23f8802
Click here to see the list of tools in the current test run
  • KinD: v0.18.0
  • Dapr: 1.10.0
  • Azure KeyVault CSI driver: 1.4.2
  • Azure Workload identity webhook: 1.0.0
  • recipe location radiusdev.azurecr.io/test/functional/corerp/recipes/<name>:pr-4ca23f8802
  • appcore-rp test image location: radiusdev.azurecr.io/appcore-rp:pr-4ca23f8802
  • ucp test image location: radiusdev.azurecr.io/ucpd:pr-4ca23f8802

Test Status

⌛ Building Radius and pushing container images for functional tests...
❌ Container images build failed
❌ Test recipe publishing failed

@kachawla kachawla force-pushed the kachawla/git-binary branch from 6998519 to 322cb07 Compare May 17, 2023 05:28
@github-actions
Copy link

github-actions bot commented May 17, 2023

Radius functional test overview

🔍 Go to test action run

Name Value
Repository project-radius/radius
Commit ref 322cb07
Unique ID 291173a91a
Image tag pr-291173a91a
Click here to see the list of tools in the current test run
  • KinD: v0.18.0
  • Dapr: 1.10.0
  • Azure KeyVault CSI driver: 1.4.2
  • Azure Workload identity webhook: 1.0.0
  • recipe location radiusdev.azurecr.io/test/functional/corerp/recipes/<name>:pr-291173a91a
  • appcore-rp test image location: radiusdev.azurecr.io/appcore-rp:pr-291173a91a
  • ucp test image location: radiusdev.azurecr.io/ucpd:pr-291173a91a

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting corerp functional tests...
⌛ Starting ucp functional tests...
✅ samples functional tests succeeded
✅ ucp functional tests succeeded
✅ corerp functional tests succeeded

@kachawla kachawla force-pushed the kachawla/git-binary branch from 322cb07 to 0856378 Compare May 17, 2023 05:45
@github-actions
Copy link

github-actions bot commented May 17, 2023

Radius functional test overview

🔍 Go to test action run

Name Value
Repository project-radius/radius
Commit ref 0856378
Unique ID c737a7e3a2
Image tag pr-c737a7e3a2
Click here to see the list of tools in the current test run
  • KinD: v0.18.0
  • Dapr: 1.10.0
  • Azure KeyVault CSI driver: 1.4.2
  • Azure Workload identity webhook: 1.0.0
  • recipe location radiusdev.azurecr.io/test/functional/corerp/recipes/<name>:pr-c737a7e3a2
  • appcore-rp test image location: radiusdev.azurecr.io/appcore-rp:pr-c737a7e3a2
  • ucp test image location: radiusdev.azurecr.io/ucpd:pr-c737a7e3a2

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting ucp functional tests...
⌛ Starting samples functional tests...
⌛ Starting corerp functional tests...
❌ Test tool installation for samples failed. Please check the logs for more details
❌ Failed to install Radius for samples functional test. Please check the logs for more details
❌ samples functional test failed. Please check the logs for more details

@github-actions
Copy link

62.6

For the detailed report, please go to Checks tab, click Build and Test, and then download unit_test_coverage artifact at the bottom of build page.

  • Your PR branch coverage: 62.6 %
  • main branch coverage: 62.6 %
  • diff coverage: 0 %

The coverage result does not include the functional test coverage.

@@ -7,6 +7,7 @@ ARG TARGETARCH
WORKDIR /

COPY ./linux_${TARGETARCH:-amd64}/release/appcore-rp /
COPY ./gitbinary/git /usr/local/bin/
Copy link
Contributor

@lakshmimsft lakshmimsft May 17, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An option may be to include directly in Docker File:

RUN apt-get update && apt-get install -y git

not sure if it is ideal approach but copy-pasting from chatgpt:
This command will update the package index and install Git in the container. The -y flag is used to automatically answer "yes" to any prompts that may appear during the installation process. Note that you should also include the apt-get update command to ensure that the package index is up to date before installing Git.

Copy link
Contributor Author

@kachawla kachawla May 17, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tried this as the first thing, but the base image is distroless so package managers are not available, hence the need to build it from the source. Let me know if you meant something else.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI we discussed offline and decided to pivot back to alpine.

@github-actions
Copy link

github-actions bot commented May 17, 2023

Radius functional test overview

🔍 Go to test action run

Name Value
Repository project-radius/radius
Commit ref 11d0f22
Unique ID 9e754072a1
Image tag pr-9e754072a1
Click here to see the list of tools in the current test run
  • KinD: v0.18.0
  • Dapr: 1.10.0
  • Azure KeyVault CSI driver: 1.4.2
  • Azure Workload identity webhook: 1.0.0
  • recipe location radiusdev.azurecr.io/test/functional/corerp/recipes/<name>:pr-9e754072a1
  • appcore-rp test image location: radiusdev.azurecr.io/appcore-rp:pr-9e754072a1
  • ucp test image location: radiusdev.azurecr.io/ucpd:pr-9e754072a1

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting ucp functional tests...
⌛ Starting corerp functional tests...
⌛ Starting samples functional tests...
✅ samples functional tests succeeded
✅ ucp functional tests succeeded
✅ corerp functional tests succeeded

@github-actions
Copy link

62.6

For the detailed report, please go to Checks tab, click Build and Test, and then download unit_test_coverage artifact at the bottom of build page.

  • Your PR branch coverage: 62.6 %
  • main branch coverage: 62.6 %
  • diff coverage: 0 %

The coverage result does not include the functional test coverage.

build/git.mk Outdated
mkdir -p $(OUT_DIR)/gitbinary
cp /tmp/gitbinary/bin/git $(OUT_DIR)/gitbinary/
chmod +x $(OUT_DIR)/gitbinary/git
sudo rm -rf /tmp/gitbinary
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can undo this now right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, removing it :) thanks for your help with the options here

RUN apk --no-cache add ca-certificates

# Install Git (required for Terraform module downloads)
RUN apk --no-cache add git
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If possible, you really want to combine these two steps (line 7-10). It will result in fewer layers

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm glad you found --no-cache. That was the thing I suspected I would have to give feedback about 😆

@@ -1,14 +1,23 @@
# Note: distroless already includes ca-certificates
FROM --platform=${TARGETPLATFORM:-linux/amd64} gcr.io/distroless/static:nonroot
FROM --platform=${TARGETPLATFORM:-linux/amd64} alpine:latest
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, updated

@github-actions
Copy link

github-actions bot commented May 17, 2023

Radius functional test overview

🔍 Go to test action run

Name Value
Repository project-radius/radius
Commit ref efb5a4b
Unique ID e395b24b26
Image tag pr-e395b24b26
Click here to see the list of tools in the current test run
  • KinD: v0.18.0
  • Dapr: 1.10.0
  • Azure KeyVault CSI driver: 1.4.2
  • Azure Workload identity webhook: 1.0.0
  • recipe location radiusdev.azurecr.io/test/functional/corerp/recipes/<name>:pr-e395b24b26
  • appcore-rp test image location: radiusdev.azurecr.io/appcore-rp:pr-e395b24b26
  • ucp test image location: radiusdev.azurecr.io/ucpd:pr-e395b24b26

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting ucp functional tests...
⌛ Starting corerp functional tests...
✅ ucp functional tests succeeded
⌛ Starting samples functional tests...
✅ samples functional tests succeeded
✅ corerp functional tests succeeded

@github-actions
Copy link

62.6

For the detailed report, please go to Checks tab, click Build and Test, and then download unit_test_coverage artifact at the bottom of build page.

  • Your PR branch coverage: 62.6 %
  • main branch coverage: 62.6 %
  • diff coverage: 0 %

The coverage result does not include the functional test coverage.

@kachawla kachawla force-pushed the kachawla/git-binary branch from 0c75594 to b297999 Compare May 17, 2023 21:50
@github-actions
Copy link

github-actions bot commented May 17, 2023

Radius functional test overview

🔍 Go to test action run

Name Value
Repository project-radius/radius
Commit ref 0c75594
Unique ID a64b854311
Image tag pr-a64b854311
Click here to see the list of tools in the current test run
  • KinD: v0.18.0
  • Dapr: 1.10.0
  • Azure KeyVault CSI driver: 1.4.2
  • Azure Workload identity webhook: 1.0.0
  • recipe location radiusdev.azurecr.io/test/functional/corerp/recipes/<name>:pr-a64b854311
  • appcore-rp test image location: radiusdev.azurecr.io/appcore-rp:pr-a64b854311
  • ucp test image location: radiusdev.azurecr.io/ucpd:pr-a64b854311

Test Status

⌛ Building Radius and pushing container images for functional tests...
❌ Container images build failed
❌ Test recipe publishing failed

@kachawla kachawla marked this pull request as ready for review May 17, 2023 21:51
@kachawla kachawla requested a review from a team as a code owner May 17, 2023 21:51
@kachawla kachawla force-pushed the kachawla/git-binary branch from b297999 to 6e2baf3 Compare May 17, 2023 21:52
@github-actions
Copy link

github-actions bot commented May 17, 2023

Radius functional test overview

🔍 Go to test action run

Name Value
Repository project-radius/radius
Commit ref b297999
Unique ID 189516c2be
Image tag pr-189516c2be
Click here to see the list of tools in the current test run
  • KinD: v0.18.0
  • Dapr: 1.10.0
  • Azure KeyVault CSI driver: 1.4.2
  • Azure Workload identity webhook: 1.0.0
  • recipe location radiusdev.azurecr.io/test/functional/corerp/recipes/<name>:pr-189516c2be
  • appcore-rp test image location: radiusdev.azurecr.io/appcore-rp:pr-189516c2be
  • ucp test image location: radiusdev.azurecr.io/ucpd:pr-189516c2be

Test Status

⌛ Building Radius and pushing container images for functional tests...
❌ Container images build failed
❌ Test recipe publishing failed

@github-actions
Copy link

github-actions bot commented May 17, 2023

Radius functional test overview

🔍 Go to test action run

Name Value
Repository project-radius/radius
Commit ref 6e2baf3
Unique ID 8051c3d679
Image tag pr-8051c3d679
Click here to see the list of tools in the current test run
  • KinD: v0.18.0
  • Dapr: 1.10.0
  • Azure KeyVault CSI driver: 1.4.2
  • Azure Workload identity webhook: 1.0.0
  • recipe location radiusdev.azurecr.io/test/functional/corerp/recipes/<name>:pr-8051c3d679
  • appcore-rp test image location: radiusdev.azurecr.io/appcore-rp:pr-8051c3d679
  • ucp test image location: radiusdev.azurecr.io/ucpd:pr-8051c3d679

Test Status

⌛ Building Radius and pushing container images for functional tests...
❌ Container images build failed
❌ Test recipe publishing failed

@github-actions
Copy link

62.6

For the detailed report, please go to Checks tab, click Build and Test, and then download unit_test_coverage artifact at the bottom of build page.

  • Your PR branch coverage: 62.6 %
  • main branch coverage: 62.6 %
  • diff coverage: 0 %

The coverage result does not include the functional test coverage.

@kachawla kachawla force-pushed the kachawla/git-binary branch from 6e2baf3 to 32321fb Compare May 17, 2023 22:02
@github-actions
Copy link

github-actions bot commented May 17, 2023

Radius functional test overview

🔍 Go to test action run

Name Value
Repository project-radius/radius
Commit ref 32321fb
Unique ID d986b98cac
Image tag pr-d986b98cac
Click here to see the list of tools in the current test run
  • KinD: v0.18.0
  • Dapr: 1.10.0
  • Azure KeyVault CSI driver: 1.4.2
  • Azure Workload identity webhook: 1.0.0
  • recipe location radiusdev.azurecr.io/test/functional/corerp/recipes/<name>:pr-d986b98cac
  • appcore-rp test image location: radiusdev.azurecr.io/appcore-rp:pr-d986b98cac
  • ucp test image location: radiusdev.azurecr.io/ucpd:pr-d986b98cac

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting samples functional tests...
⌛ Starting corerp functional tests...
⌛ Starting ucp functional tests...
✅ samples functional tests succeeded
✅ ucp functional tests succeeded
✅ corerp functional tests succeeded

@github-actions
Copy link

62.6

For the detailed report, please go to Checks tab, click Build and Test, and then download unit_test_coverage artifact at the bottom of build page.

  • Your PR branch coverage: 62.6 %
  • main branch coverage: 62.6 %
  • diff coverage: 0 %

The coverage result does not include the functional test coverage.

@kachawla kachawla merged commit 47b5b8f into main May 17, 2023
@kachawla kachawla deleted the kachawla/git-binary branch May 17, 2023 22:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants