pulumi refresh fail to handle resource deleted outside of pulumi management (manual delete)

[Yhanli]

What happened?

Issue Description

If resources are deleted outside of pulumi, say, via the Azure portal, and then pulumi refresh is run, it throws these errors:

Previewing refresh (organization/test-ts-iac/dev):
  pulumi:pulumi:Stack: (same)
    [urn=urn:pulumi:dev::test-ts-iac::pulumi:pulumi:Stack::test-ts-iac-dev]
debug: 2024/08/19 11:32:24 Testing if Service Principal / Client Certificate is applicable for Authentication..
debug: 2024/08/19 11:32:24 Testing if Multi Tenant Service Principal / Client Secret is applicable for Authentication..
debug: 2024/08/19 11:32:24 Testing if Service Principal / Client Secret is applicable for Authentication..
debug: 2024/08/19 11:32:24 Testing if OIDC is applicable for Authentication..
debug: 2024/08/19 11:32:24 Testing if Managed Service Identity is applicable for Authentication..
debug: 2024/08/19 11:32:24 Testing if Obtaining a Multi-tenant token from the Azure CLI is applicable for Authentication..
debug: 2024/08/19 11:32:24 Testing if Obtaining a token from the Azure CLI is applicable for Authentication..
debug: 2024/08/19 11:32:24 Using Obtaining a token from the Azure CLI for Authentication
debug: 2024/08/19 11:32:24 Getting OAuth config for endpoint https://login.microsoftonline.com/ with  tenant <tenant_id>
error: Preview failed: autorest/azure: Service returned an error. Status=404 Code="ResourceNotFound" Message="The Resource 'Microsoft.Web/sites/new-app-test' under resource group 'NZ-SBX-ARG-TEST' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"
error: preview failed
Resources:
    3 unchanged

Work Around

delete the resource from the state json

Example

1. create resource via pulumi up
2. delete resource manually on azure portal
3. run pulumi refresh command

Output of pulumi about

CLI
Version 3.129.0
Go Version go1.22.6
Go Compiler gc

Plugins
KIND NAME VERSION
resource azure 5.86.0
resource azure-native 2.56.0
language nodejs unknown

Host
OS darwin
Version 14.5
Arch arm64

This project is written in nodejs: executable='/Users/hanli/Library/Caches/fnm_multishells/6085_1724016448699/bin/node' version='v18.20.3'

Current Stack: organization/test-ts-iac/dev

TYPE URN
pulumi:pulumi:Stack urn:pulumi:dev::test-ts-iac::pulumi:pulumi:Stack::test-ts-iac-dev
pulumi:providers:azure-native urn:pulumi:dev::test-ts-iac::pulumi:providers:azure-native::default_2_56_0
azure-native:resources:ResourceGroup urn:pulumi:dev::test-ts-iac::azure-native:resources:ResourceGroup::NZ-SBX-ARG-TEST
azure-native:web:AppServicePlan urn:pulumi:dev::test-ts-iac::azure-native:web:AppServicePlan::plan
azure-native:web:WebApp urn:pulumi:dev::test-ts-iac::azure-native:web:WebApp::new-app-test

Found no pending operations associated with dev

Backend
Name <>
URL file://~
User <>
Organizations
Token type personal

Dependencies:
NAME VERSION
@pulumi/azure 5.86.0
@pulumi/azure-native 2.56.0
@pulumi/pulumi 3.129.0
dotenv 16.4.5
cross-env 7.0.3
zod 3.23.8
@types/node 18.19.43
@typescript-eslint/eslint-plugin 8.0.1
@typescript-eslint/parser 8.0.1
eslint 8.57.0
eslint-config-prettier 9.1.0
eslint-plugin-prettier 5.2.1
prettier 3.3.3
typescript 5.5.4

Pulumi locates its logs in /var/folders/sf/2q6k2cx56j9dr5_cd6rvb8b00000gn/T/ by default

Additional context

is this the kind of expected behaviour for the pulumi refresh command? would've imagine it should update the state if to align with the resources in the resource group?

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[jeevanions]

👍

[justinvp]

This looks like a possible issue in Azure Native. During a refresh, if the resource is not found, it should be removed from the state -- it shouldn't result in a failed operation. Transferring to the Azure Native repo and someone will take a look.

[thomas11]

That is indeed a provider bug. I just opened a PR to fix it. Thank you for reporting this, @Yhanli .

[pulumi-bot]

This issue has been addressed in PR #3533 and shipped in release v2.57.1.