AKS Edge Essentials single node deployment failure on connect to Arc

[clin4]

Is your issue related to a Jumpstart scenario, ArcBox, HCIBox, or Agora?
NO

Describe the issue or the bug

I am following the steps from https://learn.microsoft.com/en-us/azure/aks/hybrid/aks-edge-quickstart to deploy a single node AKS Edge Essentials on my window 10 machine. but failed to connect Arc.

To Reproduce

1. I follow this link to setup the mechine. [https://learn.microsoft.com/en-us/azure/aks/hybrid/aks-edge-howto-setup-machine]
2. I follow this link to setup the single linux node k3s cluster [https://learn.microsoft.com/en-us/azure/aks/hybrid/aks-edge-howto-single-node-deployment]
3. The cluster itself works, I can connect to it and deploy the azure vote app and test it use the NodePort on the vm
4. I failed to follow this link to connect the k3s to azure arc [https://learn.microsoft.com/en-us/azure/aks/hybrid/aks-edge-howto-connect-to-arc]
   a. I created the resource group in my subscription
   b. I register all the required provider
   Microsoft.HybridCompute
   Microsoft.GuestConfiguration
   Microsoft.HybridConnectivity
   Microsoft.Kubernetes
   Microsoft.ExtendedLocation
   Microsoft.KubernetesConfiguration
   c. I have a service principle with the role of "Microsoft.Kubernetes connected cluster"
   d. I update the aksedge-config.json with all required value for the keys:
   ClusterName
   Location
   ResourceGroupName
   SubscriptionId
   TenantId
   ClientId
   ClientSecret
   e. I run the script [Connect-AksEdgeArc -JsonConfigFilePath .\aksedge-config.json], and get expected log-

[12/12/2023 11:06:49] AksEdge - Connecting cluster to Azure Arc

• Checking Az PS module dependencies
• Checking for NuGet
• NuGet found
• Az.Resources module with version 6.4.1 is found
• Az.Accounts module with version 2.11.2 is found
• Az.ConnectedKubernetes module with version 0.9.0 is found
• Connecting to Azure Account
• Verifying Azure Account connection ...
• Verifying the Azure resource providers Microsoft.Kubernetes, Microsoft.KubernetesConfiguration, Microsoft.ExtendedLocation are registered
• Resource provider Microsoft.Kubernetes is registered.
• Resource provider Microsoft.KubernetesConfiguration is registered.
• Resource provider Microsoft.ExtendedLocation is registered.
• Checking whether cluster 'default' is connected to Azure Arc...
• All checks succeeded. Connecting cluster to Azure Arc.
• Populating tags for AKS-EE Cluster
  OK

f. But I cannot get anything from the resource group

g. When I check the pods states in the azure-arc namespace, the kube-aad-proxy pod cannot start..
kube-aad-proxy-c4f67f59d-t7dv5 0/2 ContainerCreating 0 24m
when I inspect the pod: I saw the following error:
Warning FailedMount 2m32s (x19 over 25m) kubelet MountVolume.SetUp failed for volume "kube-aad-proxy-tls" : secret "kube-aad-proxy-certificate" not found

h. when I tried to list the secrets under the azure-arc namespaces, I cannot find it
kubectl -n azure-arc get secret
NAME TYPE DATA AGE
azure-arc-connect-privatekey Opaque 1 27m

i. I uninstalled the arc component via this command and repeat the step e, the same issue again.
az connectedk8s delete --name "cluster-name"--resource-group "my rg"

Expected behavior
I am expecting to get the arc connect described from this link [https://learn.microsoft.com/en-us/azure/aks/hybrid/aks-edge-howto-connect-to-arc]

Environment summary
windows 10 Exterprise, 22H2, 19045.3693, W10X64 1803
the machine is behind a company gateway.

Screenshots

Additional context

[github-actions]

Hey friend! Thanks for opening this issue. We appreciate your contribution and welcome you to our community! We are glad to have you here and to have your input on the Azure Arc Jumpstart.

[clin4]

Some updates, I use
K3s installer (1.26.6) | aka.ms/aks-edge/k3s-msi

I tried on both my local computer and also on an Azure VM.
for the Azure VM, I choose a Standard D4ads v5 (4 vcpus, 16 GiB memory), which have nested virtualizatoin enabled . with Windows Server 2022

I follow the same steps, the k3s created but not able to connect to ACR due to the same reason.

[fcabrera23]

Hi @clin4,

Thanks for reaching out. This issue is related to AKS Edge Essentials and impacts the automation created by the Arc Jumpstart team. Please file a new issue to AKS-EE Repo. We will work with the AKS-EE team to try to get it prioritized.

Thanks,
Francisco

[clin4]

sorry, yes, I guess I should post it on the otherside. The only reason for me to reach here was because I search for the reason, google guide me to this place :). I just submit it on AKS EE. Azure/AKS-Edge#164. If you want, I can close this issue from this side.