AKS Edge Essentials single node deployment failure on connect to Arc[BUG]

[clin4]

Describe the bug
I am following the steps from https://learn.microsoft.com/en-us/azure/aks/hybrid/aks-edge-quickstart to deploy a single node AKS Edge Essentials on my window 10 machine also from an Azure VM running windows 10 server. but failed to connect Arc.

To Reproduce
Steps to reproduce the behavior:

1. I follow this link to setup the mechine. [https://learn.microsoft.com/en-us/azure/aks/hybrid/aks-edge-howto-setup-machine]
2. I follow this link to setup the single linux node k3s cluster [https://learn.microsoft.com/en-us/azure/aks/hybrid/aks-edge-howto-single-node-deployment]
3. The cluster itself works, I can connect to it and deploy the azure vote app and test it use the NodePort on the vm
4. I failed to follow this link to connect the k3s to azure arc [https://learn.microsoft.com/en-us/azure/aks/hybrid/aks-edge-howto-connect-to-arc]
   a. I created the resource group in my subscription
   b. I register all the required provider
   Microsoft.HybridCompute
   Microsoft.GuestConfiguration
   Microsoft.HybridConnectivity
   Microsoft.Kubernetes
   Microsoft.ExtendedLocation
   Microsoft.KubernetesConfiguration
   c. I have a service principle with the role of "Microsoft.Kubernetes connected cluster"
   d. I update the aksedge-config.json with all required value for the keys:
   ClusterName
   Location
   ResourceGroupName
   SubscriptionId
   TenantId
   ClientId
   ClientSecret
   e. I run the script [Connect-AksEdgeArc -JsonConfigFilePath .\aksedge-config.json], and get expected log-

[12/12/2023 11:06:49] AksEdge - Connecting cluster to Azure Arc

• Checking Az PS module dependencies
• Checking for NuGet
• NuGet found
• Az.Resources module with version 6.4.1 is found
• Az.Accounts module with version 2.11.2 is found
• Az.ConnectedKubernetes module with version 0.9.0 is found
• Connecting to Azure Account
• Verifying Azure Account connection ...
• Verifying the Azure resource providers Microsoft.Kubernetes, Microsoft.KubernetesConfiguration, Microsoft.ExtendedLocation are registered
• Resource provider Microsoft.Kubernetes is registered.
• Resource provider Microsoft.KubernetesConfiguration is registered.
• Resource provider Microsoft.ExtendedLocation is registered.
• Checking whether cluster 'default' is connected to Azure Arc...
• All checks succeeded. Connecting cluster to Azure Arc.
• Populating tags for AKS-EE Cluster
  OK

f. But I cannot get anything from the resource group

g. When I check the pods states in the azure-arc namespace, the kube-aad-proxy pod cannot start..
kube-aad-proxy-c4f67f59d-t7dv5 0/2 ContainerCreating 0 24m
when I inspect the pod: I saw the following error:
Warning FailedMount 2m32s (x19 over 25m) kubelet MountVolume.SetUp failed for volume "kube-aad-proxy-tls" : secret "kube-aad-proxy-certificate" not found

h. when I tried to list the secrets under the azure-arc namespaces, I cannot find it
kubectl -n azure-arc get secret
NAME TYPE DATA AGE
azure-arc-connect-privatekey Opaque 1 27m

i. I uninstalled the arc component via this command and repeat the step e, the same issue again.
az connectedk8s delete --name "cluster-name"--resource-group "my rg"

Expected behavior
I am expecting to get the arc connect described from this link [https://learn.microsoft.com/en-us/azure/aks/hybrid/aks-edge-howto-connect-to-arc]

Screenshots
If applicable, add screenshots to help explain your problem.

Environment (please complete the following information):

• AKS Edge Essentials Version [e.g. 3.22]
  K3s installer (1.26.6)
• Kubernetes version [e.g. 1.24.3]
  K3S

Try 1:

• Windows Host OS (please complete the following information):
  • Edition: Exterprise
  • Version: windows 10 Exterprise, 22H2, 19045.3693, W10X64 1803
  • Virtual Machine: Laptop
    Try 2
• Windows Host OS (please complete the following information):
  • Edition: Server
  • Version: 2022 (2022-datacenter-azure-edition-hotpatch)
  • Virtual Machine: Azure VM

[gshiva]

@clin4 - did you increase the Linux Node size to have at least 4.5GB RAM and 20GB disk space?

I had a similar issue here #156

AKS-Edge team, can we make the scripts fail if the Azure Arc connection setup is attempted and the allocated memory is not at least 4.5GB?

As you can see, we don't read the doc very well :)

[barnstee]

I have the same issue and my single node cluster has 10GB of RAM and 40GB of storage...

[barnstee]

Switching back to Version 1.2.414.0 (K8s) resolves the error for me.