Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Need docker installed on the helix images for Windows, to allow individual tests to create containers #2067

Open
3 tasks
radical opened this issue Feb 22, 2024 · 10 comments
Open
3 tasks

Need docker installed on the helix images for Windows, to allow individual tests to create containers #2067

radical opened this issue Feb 22, 2024 · 10 comments
Assignees
@garath
Labels
Ops - Service Maintenance Used to track issues related to maintaining the services .NET Eng Supports Ops - Spike Work items to be included in our Ops Spike

Comments

@radical
Copy link
Member

radical commented Feb 22, 2024

We have some EndToEnd tests in dotnet/aspire that use docker to start some containers, and then interact with them. The tests are sent as helix work items. And the individual tests then indirectly use docker to create containers.

build machine
    -> send-helix-workitems
        -> each workitem:
             -> `dotnet test ...`
                -> individual tests can use `docker.exe` to create multiple containers, and interact with them

This works on helix/linux with Ubuntu.2204.Amd64[.Open] queue. But not with the windows queues (tried Windows.10.Amd64.Client[.Open], Windows.10.Amd64.ServerRS5[.Open], and Windows.Amd64.Server2022[.Open]) as docker is not installed.

This is important for running integration/end-to-end tests for dotnet/aspire - which starts a bunch of containers for things like redis/mysql/etc.

This is a test run from helix/linux .

cc @dougbu @eerhardt @joperezr

Release Note Category

  • Feature changes/additions
  • Bug fixes
  • Internal Infrastructure Improvements

Release Note Description
@radical
Copy link
Member Author

radical commented Feb 24, 2024

This is probably relevant - Run Docker Desktop for Windows in a VM or VDI environment.

@radical radical mentioned this issue Feb 29, 2024
Merged
@radical
Copy link
Member Author

radical commented Feb 29, 2024

@dougbu Is there any other info that I can provide to help with this?

@dougbu
Copy link
Member

dougbu commented Feb 29, 2024

I'm not sure that helps because I don't know if we are Docker Business customers. In any case, this issue has yet to be triaged or assigned and we're strapped at the moment.

radical added a commit to dotnet/aspire that referenced this issue Mar 7, 2024
@radical
[tests] Add support for running EndToEnd tests on helix with the `asp…
ea3e182 
…ire` workload (#2534)

# Problem:

1. We want to run some EndToEnd tests on CI, which can `dotnet run` an aspire project,
and allow individual tests to interact with the services.
This requires:

	- Ability to build, and run an aspire project - IOW, a sdk with the `aspire` workload installed.
	- `docker`

2. Also, allow using `TestProject.*` in `tests/testproject`, in two modes:
    - `in-tree` test run which directly reference aspire projects, and repo targets
    - `out-of-tree` test runs which uses a SDK+workload based on local build output

# Solution:

### SDK+workload

- SDK with version from `global.json` is installed into `artifacts/bin/dotnet-{none,latest}`
    - The `aspire` workload is installed into the sdk using the nuget packages in `artifacts/packages` with a version like `8.0.0-{dev,ci}`
- this also allows running tests that do `dotnet new aspire-starter; dotnet run` same as a user
- this utilizes an internal nuget from `dotnet/runtime`

### TestProject

- This can switch between the two test run modes using the msbuild property `$(TestsRunningOutsideRepo)`
    - when running `in-tree` the test project directly references hosting targets, and aspire projects via `ProjectReference`
    - when running `out-of-tree` the `ProjectReferences` and imports are replaced with `PackageReferences` to the Aspire nugets
- Default is to run `in-tree`

### Helix

- The tests are built on the build machine
- The testproject, and the sdk+workload is sent to helix
  - where the tests run using `dotnet` from the sdk+workload path
- Since `docker` is needed to helix, this is enabled only for `Linux` in this PR. Blocked on dotnet/dnceng#2067 for windows support.

## TL;dr or How do I use this?

### Steps to prepare for `out-of-tree` runs

1. `.\build.cmd -pack` - to build all the nugets
2. `dotnet build tests/Aspire.EndToEnd.Tests/Aspire.EndToEnd.csproj /t:InstallWorkloadUsingArtifacts /p:Configuration=<config>`
    - this will install the sdk, and the `aspire` workload using the nugets from `artifacts/packages`

- at this point the sdk is usable outside the repo too. Note: you need to add `artifacts/packages/$(Configuration)/Shipping` as a nuget source so the locally built packages can be picked up.

### Using it from VS

- No difference when running the EndToEnd tests in `in-tree` mode
- For switching to `out-of-tree` add `<TestsRunningOutsideRepo>true</TestsRunningOutsideRepo>` to `tests/Aspire.EndToEnd.Tests/Directory.props` *before* any imports.
    - tests cannot be run at this point as they will fail complaining about `artifacts/bin/dotnet-latest` being missing
    - Install the sdk+workload following the steps above
    - Run/debug the tests normally now, and they will be using the sdk
    - Also note that in this case the testproject is run from the bindir for `Aspire.EndToEnd.Tests`, so a path like `artifacts/bin/Aspire.EndToEnd.Tests/Debug/net8.0/testassets/testproject/`

### Using it from command line

- No difference when running the EndToEnd tests in `in-tree` mode
- Install the sdk+workload following the steps above
- When running the tests you can either:
    - set `<TestsRunningOutsideRepo>true</TestsRunningOutsideRepo>` to `tests/Aspire.EndToEnd.Tests/Directory.props` before any imports
    - or set the environment variable `TestsRunningOutsideRepo=true`

### Other details

- `tests/Shared/WorkloadTesting` - has sources essentially from `dotnet/runtime`'s `Wasm.Build.Tests`
- TestProject supports skipping individual resources like `--skip-resources oracledatabase,cosmos`
- `.runsettings` with 15min timeout
- `xunit.runner.json` - to get diagnostic messages
- The core infrastructure is not specific for `Aspire.EndToEnd.Tests`, and will get used by other test projects in the future

 ###### Microsoft Reviewers: [Open in CodeFlow](https://microsoft.github.io/open-pr/?codeflow=https://github.com/dotnet/aspire/pull/2534)
@dougbu dougbu added the Ops - Service Maintenance Used to track issues related to maintaining the services .NET Eng Supports label Mar 21, 2024
@dougbu
Copy link
Member

dougbu commented Mar 21, 2024

Adding to Ops - Service Maintenance /cc @garath and @ilyas1974.

Conversation about licensing issues and how we might do this work seems to have stalled. I'll nudge it…

@dougbu
Copy link
Member

dougbu commented Mar 21, 2024

@radical you aren't part of that conversation. We'll report back once we have more information.

@garath garath added the Ops - P2 Operations task, priority 2 label Mar 25, 2024
@dougbu dougbu mentioned this issue Mar 26, 2024
Open
3 tasks
@ilyas1974
Copy link
Contributor

While the two are related - mostly because we only have a single docker artifact that is used in both our build queues and our test queues, the images being referenced are different. You've made a link above between the two and I will make sure that when this is addressed, the other issue is also taken into account. I will let the Dev doing the work decide how he wants to address the github issue(s).

@garath garath removed the Ops - P2 Operations task, priority 2 label Apr 9, 2024
@riarenas riarenas self-assigned this Jun 3, 2024
@riarenas
Copy link
Member

riarenas commented Jun 3, 2024

@dougbu Can you point me to where the licensing conversations are happening? Is this something we can actually do, or we still don't know?

@riarenas riarenas removed their assignment Jun 3, 2024
@dougbu
Copy link
Member

dougbu commented Jun 3, 2024

There was an email thread on the licensing subject. I was going to at least look at the site-licensed installer and see if it Just worked ™️ on a DTL machine. Never got around to it and this whole subject has been on the backlog since.

I'll add you to the thread if you're not there already.

@riarenas
Copy link
Member

riarenas commented Jun 3, 2024 •
edited
Loading

The answer is we don't know how this will work, and we need to reach out to procurement for guidance.

I'll work on updating our docker artifact first, as that will be a prerequisite anyways. I don't think I'll get far enough to figure this out, as it's a completely new scenario where we don't have a clear path forward.

@riarenas
Copy link
Member

riarenas commented Jun 4, 2024

Recommended next steps from the thread: reach out to ProcureWeb - Find a Procurement contact (sharepoint.com)

@ilyas1974 ilyas1974 added Ops - P1 Operations task, priority 1 (highest priority) Ops - Spike Work items to be included in our Ops Spike and removed Ops - P1 Operations task, priority 1 (highest priority) labels Jul 24, 2024
@garath garath self-assigned this Aug 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@garath garath

Labels
Ops - Service Maintenance Used to track issues related to maintaining the services .NET Eng Supports Ops - Spike Work items to be included in our Ops Spike
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@radical @garath @dougbu @ilyas1974 @riarenas