rad env init for self-hosted is taking too long to initialize (~3/4 minutes in average)

[asilverman]

Observed behavior
rad env init for self-hosted is taking too long to initialize (~3/4 minutes in average), we should improve this to less than 1 minute to meet a good user experience aligned with our OKR: Developers love using azure

Desired behavior
A self-hosted rad environment should start within at most 60 sec
OS/rad version
Any

Screenshots

Additional context

AB#6945

[rynowak]

I super agree with the feedback. We should reassess this when we get rid of the old CustomRP code. Right now the CustomRP runs Mongo in the cluster, which has a really big image size and slow startup.

[rynowak]

I just tested this locally AFTER the removal of the Radius CustomRP + Mongo. It took 5m36s.

This numbers were all taken on my machine with my slow home internet. Pull times will have less of an impact in the cloud, so consider these a worst-case.

Image	Pull Time (seconds)	Size (mb)
ucpd	12	44
appcore-rp	56	245
bicep-extensibility	59	223
deployments	57	243

Reciepts

appcore-rp

ucpd

bicep-extensibility

deployments

[rynowak]

Radius needs a diet 🥗 🥗 🥗 🥗 🥗 🥗 🥗 🥗 🥗 🥗

[rynowak]

I think it's also worth looking into why the ucpd binary is 44mb https://www.cockroachlabs.com/blog/go-file-size/

[jkotalik]

I think we should be able to do some trimming (hopefully) for the .NET images.

[rynowak]

We should be able to get to ~60mb for .NET-based images with just assembly-level trimming.

[emily-potyraj]

triage decision: We need to make improvements here for v0.12. If the work ends up taking longer than we expect, we might decide not to hold the release for it.

Comparison: Dapr installation takes ~45 sec and installs more containers than we do.

Let's aim to get radius install time down to 90 sec by v0.12.

@rynowak - has some ideas here for low hanging fruit.

[jkotalik]

@emily-potyraj @rynowak can we just not install dapr as a starting point?

[rynowak]

That's already planned. The time taken (because of download size) just for the radius part is unacceptable even if we remove Dapr.

[rynowak]

FYI in our CI runs it looks like this is taking about 1 minute from end to end. So this confirms that download size is the biggest factor.

[emily-potyraj]

@rynowak - can you please add your ideas for low-hanging next steps here? sorry @jkotalik I forgot what we had in mind when we also assigned you to this issue.

[rynowak]

For AppCoreRP we should be able to get down to ~40mb with distroless and avoiding redundant copies. We should use the way the UCP and CustomRP images are built as the starting point

For the two .NET projects we need to move them to alpine as self-contained apps. We can then add assembly-level trimming which will get them down to ~50mb or less. I'm not sure if we build the bicep-extensibility image today or if we reuse that from elsewhere. We'd probably need to build it ourselves.

These steps will remove ~ 550mb of deadweight.

[rynowak]

Here's an update after getting all of the low-hanging fruit:

	Install Time	Image Size (Radius Only)
Before	5m36s	755mb
After	3m52s	307mb

[rynowak]

If we remove Dapr and execute Contour and Dapr in parallel:

Just Radius (no Contour or Dapr)

[emily-potyraj]

Great investigation. let's call it done for v0.12.
For v0.12, Radius installation may take up to ~3 min, faster with good internet: 30 sec in AKS.

filed #2952 for possibly removing Dapr installation in the future.

[jkotalik]

I think removing contour will be a no-go for now, but great work here Ry dawg.

[rynowak]

Great, removed this from the board for now.

[rynowak]

With the removal of Dapr we're down to ~2 mins on my terrible home internet.