Intermittent slowness in ACL API, observed race condition between tailscale_acl and tailscale_tailnet_key?

[kris-standardai]

Throughout 10/23/2024 and 10/24/2024, I observed intermittent errors creating tailscale_tailnet_key resources using tags that had been added to a tailscale_acl resource in the same terraform run. The tailscale_tailnet_key resource was failing because the tag did not exist yet.

requested tags [tag:example1] are invalid or not permitted (400).

The ACL was updated properly each time. The failed tailscale_tailnet_keys would apply properly on the next run. I observed about 50-80% first run failures over the dates mentioned and added a time_sleep resource to workaround the issue.

I am returning to this issue today , 10/28/2024 and am no longer able to reproduce the tailscale_tailnet_key failures. Was anything reported on 10/23/24 or 10/24/24? Did something change? I am concerned the errors will return.

Thanks!

[mpminardi]

Hey @kris-standardai! I was able to reproduce this on my end. There is indeed a race condition between the resources in this scenario as Terraform does not know that there is a dependency between the ACL and the tailscale_tailnet_key (due to the latter needing the tag to exist in the former) and attempts to provision both in parallel.

Adding a depends_on argument to the tailscale_tailnet_key that references the tailscale_acl will inform Terraform of this dependency and prevent this race condition (was unable to reproduce a single time locally after adding this in).

Potentially this was worsened by some sort of slowness in the API on those days you were testing, but I have not found anything obvious on our end yet.