Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RPD-279] [BUG] Automatically deal with stale states #170

Conversation

Christopher-Norman
Copy link
Member

@Christopher-Norman Christopher-Norman commented Jul 21, 2023

This bug is recreated by allowing the state storage to be provisioned and stopping the program before reaching the AKS and other resources provisioning. This meant that the matcha.state file looks similar to:

{
    "cloud": {
        "prefix": "test",
        "location": "uksouth",
    }
}

Which does not include resource-group-name hence when we try to access this value to check if the matcha state is stale
we get an error that the value does not exist.
Instead the matcha.state file needs to be:

{
    "cloud": {
        "prefix": "test",
        "location": "uksouth",
        "resource-group-name": "test-resources"
    }
}

After the resource group and state storage is provisioned.

This bug can happen if there is something wrong with the resources Terraform and the Terraform init/apply fails causing matcha to halt halfway through provisioning.

To fix this we add the resource group name to matcha.state file after provisioning the remote state rg and storage.

Checklist

Please ensure you have done the following:

  • I have read the CONTRIBUTING guide.
  • I have updated the documentation if required.
  • I have added tests which cover my changes.

Type of change

Tick all those that apply:

  • Bug Fix (non-breaking change, fixing an issue)
  • New feature (non-breaking change to add functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Other (add details above)

@Christopher-Norman Christopher-Norman self-assigned this Jul 21, 2023
@Christopher-Norman Christopher-Norman added the bug Something isn't working label Jul 21, 2023
Copy link
Member

@JonoCX JonoCX left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! 🐛

@Christopher-Norman Christopher-Norman merged commit 74423b5 into feature/bug-fixes-july-2023 Jul 21, 2023
@Christopher-Norman Christopher-Norman deleted the RPD-279-fix-resource-group-name-could-not-be-found-error branch July 21, 2023 14:16
JonoCX added a commit that referenced this pull request Jul 28, 2023
* [RPD-261] [BUG] destroy leads to missing matcha config file error (#165)

* Update context managers to avoid erroring on destroy after remote bucket is removed

* Update docstrings

* Add tests for context managers new destroy parameter

* [RPD-272] Updates to documentation (#166)

* RPD-272 updates to pages based on ticket description

* RPD-272 updated permissions

* [RPD-276] [BUG] Turning analytics on causes hanging in the analytics service (#168)

* [RPD-271] [BUG] Fix orphaned NetworkWatcherRG resource group (#167)

* [RPD-279] [BUG] Automatically deal with stale states #170

* [RPD-273] Improvements to the `provision` user experience (#173)

* RPD-273 added a terraform state update function and minor improvements to terraform service

* RPD-273 updated typing and fixed pre-commit issues

* RPD-273 undoing a change to the gitignore

* RPD-273 updated docstrings + grammar

* RPD-272 updated typing on context manager for Spinner

* [RPD-274] [BUG] Fix inaccurate provisioning messages (#177)

* Update message on Terraform apply for clarity

* Update test with new message

* Update base_runner.py

* Fix test

* Revert message parameter to empty string

* [RPD-263] add inference of zenml version from environment (#178)

* function checking local zenml version.

* function checking local zenml version.

* function checking local zenml version.

* remove return from ds as there is none

---------

Co-authored-by: Jonathan Carlton <[email protected]>
Co-authored-by: KirsoppJ <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants