Upgrade CosmosDB SDK to 2019-08-01 from 2015-04-08 #6253

jackbatzner · 2020-03-25T18:33:36Z

Upgrade CosmosDB SDK to 2019-08-01 from 2015-04-08 to address @tombuildsstuff comments in PR 6189 about upgrading the remaining resources.

Below is the basic acceptance test results for each resource that was modified:
--- PASS: TestAccAzureRMCosmosDBAccount_basic_global_boundedStaleness (1043.38s)
--- PASS: TestAccAzureRMCosmosDbCassandraKeyspace_basic (1301.73s)
--- PASS: TestAccAzureRMCosmosGremlinDatabase_basic (1254.89s)
--- PASS: TestAccAzureRMCosmosDbGremlinGraph_basic (1371.48s)
--- PASS: TestAccAzureRMCosmosDbMongoCollection_basic (1338.83s)
--- PASS: TestAccAzureRMCosmosDbMongoDatabase_basic (1285.84s)
--- PASS: TestAccAzureRMCosmosDbSqlContainer_basic (1382.01s)
--- PASS: TestAccAzureRMCosmosDbSqlDatabase_basic (1284.77s)
--- PASS: TestAccAzureRMCosmosDbTable_basic (1314.77s)

jackbatzner · 2020-03-25T18:59:44Z

azurerm/internal/services/cosmos/resource_arm_cosmosdb_gremlin_graph.go

@@ -386,14 +388,18 @@ func resourceArmCosmosDbGremlinGraphRead(d *schema.ResourceData, meta interface{
 			d.Set("throughput", nil)
 		}
 	} else {
-		d.Set("throughput", throughputResp.Throughput)
+		if props := throughputResp.ThroughputSettingsGetProperties; props != nil {


Since this is used in almost every resource I am wondering if we should pull this out to a helper function? Thoughts would be appreciated

I could see the benefit of a helper here, returning a default value for this field 👍

jackbatzner · 2020-03-25T19:06:56Z

I will move all of the functions from ./azurerm/helpers/azure/cosmos.go to the respective ./azurerm/internal/services/cosmos/parse/**.go folder, most of the tests above are failing.

katbyte · 2020-03-25T21:57:03Z

Looks like the parse functions need to be updated as the resource IDs have changed:

------ Stdout: -------
=== RUN   TestAccAzureRMCosmosDbCassandraKeyspace_complete
=== PAUSE TestAccAzureRMCosmosDbCassandraKeyspace_complete
=== CONT  TestAccAzureRMCosmosDbCassandraKeyspace_complete
--- FAIL: TestAccAzureRMCosmosDbCassandraKeyspace_complete (836.88s)
    testing.go:640: Step 0 error: errors during apply:
        
        Error: Error: Unable to parse Cosmos Keyspace Resource ID: keyspaces is missing from: /subscriptions/1a6092a6-137e-4025-9a7c-ef77f76f2c02/resourceGroups/acctestRG-cosmos-200325212702140546/providers/Microsoft.DocumentDB/databaseAccounts/acctest-ca-200325212702140546/cassandraKeyspaces/acctest-200325212702140546

i think we'll have to handle the IDs changing for existing resources 🤔

- Move parse functionality from /helpers/azure/cosmos.go to individual helper functions.

jackbatzner · 2020-03-26T01:44:13Z

i think we'll have to handle the IDs changing for existing resources 🤔

I've added handling for the new and existing ID parsing so that we can still support parsing existing resources. Not sure if there is something else we should be changing in addition to this?

TestAccAzureRMCosmosDbCassandraKeyspace_basic test is passing with latest parse changes. I've kicked off the remainder basic tests locally as well. I am updating their status in the top level PR Comment

=== RUN TestAccAzureRMCosmosDbCassandraKeyspace_basic
=== PAUSE TestAccAzureRMCosmosDbCassandraKeyspace_basic
=== CONT TestAccAzureRMCosmosDbCassandraKeyspace_basic
--- PASS: TestAccAzureRMCosmosDbCassandraKeyspace_basic (1301.73s)
PASS
ok github.com/terraform-providers/terraform-provider-azurerm/azurerm/internal/services/cosmos/tests 1301.964s

tombuildsstuff

hey @Brunhil

Thanks for this splitting this out 😃

I've taken a look through and left some comments inline, but this is off to a good start; if we can add the state migrations to update the ID (which'll update the ID in the Statefile for existing users, meaning we can assume everyone's using the updated ID format going forward, as is shown in the documentation) then this should otherwise be good 👍

Thanks!

azurerm/internal/services/cosmos/parse/cassandra_keyspace.go

azurerm/internal/services/cosmos/parse/cassandra_keyspace_test.go

azurerm/internal/services/cosmos/parse/gremlin_database.go

azurerm/internal/services/cosmos/parse/gremlin_database_test.go

azurerm/internal/services/cosmos/parse/gremlin_graph.go

azurerm/internal/services/cosmos/resource_arm_cosmosdb_mongo_collection.go

azurerm/internal/services/cosmos/resource_arm_cosmosdb_mongo_database.go

azurerm/internal/services/cosmos/resource_arm_cosmosdb_sql_container.go

azurerm/internal/services/cosmos/resource_arm_cosmosdb_sql_database.go

azurerm/internal/services/cosmos/resource_arm_cosmosdb_table.go

jackbatzner · 2020-03-26T17:26:43Z

@tombuildsstuff , I've added the state migration changes, but I do believe something is wrong with them. The acceptance test for Cassandra is now failing after the latest changes. I can't seem to find where I've done something wrong, I'm hoping a second set of eyes will make it clear!

=== RUN   TestAccAzureRMCosmosDbCassandraKeyspace_complete
=== PAUSE TestAccAzureRMCosmosDbCassandraKeyspace_complete
=== CONT  TestAccAzureRMCosmosDbCassandraKeyspace_complete
--- FAIL: TestAccAzureRMCosmosDbCassandraKeyspace_complete (1152.96s)
    testing.go:640: Step 0 error: After applying this step, the plan was not empty:

        DIFF:

        DESTROY/CREATE: azurerm_cosmosdb_account.test
          capabilities.#:                               "1" => "1"
          capabilities.0.name:                          "EnableCassandra" => "EnableCassandra"
          connection_strings:                           "" => "<computed>"
          connection_strings.#:                         "8" => ""
          connection_strings.0:                         "AccountEndpoint=https://acctest-ca-200326104554447378.documents.azure.com:443/;AccountKey=********;" => 
""
          connection_strings.1:                         "AccountEndpoint=https://acctest-ca-200326104554447378.documents.azure.com:443/;AccountKey=********;" => 
""
          connection_strings.2:                         "AccountEndpoint=https://acctest-ca-200326104554447378.documents.azure.com:443/;AccountKey=********;" => 
""
          connection_strings.3:                         "AccountEndpoint=https://acctest-ca-200326104554447378.documents.azure.com:443/;AccountKey=********;" => 
""
          connection_strings.4:                         "HostName=acctest-ca-200326104554447378.cassandra.cosmos.azure.com;Username=acctest-ca-200326104554447378;Password=********;Port=10350" => ""
          connection_strings.5:                         "HostName=acctest-ca-200326104554447378.cassandra.cosmos.azure.com;Username=acctest-ca-200326104554447378;Password=********;Port=10350" => ""
          connection_strings.6:                         "HostName=acctest-ca-200326104554447378.cassandra.cosmos.azure.com;Username=acctest-ca-200326104554447378;Password=********;Port=10350" => ""
          connection_strings.7:                         "HostName=acctest-ca-200326104554447378.cassandra.cosmos.azure.com;Username=acctest-ca-200326104554447378;Password=********;Port=10350" => ""
          consistency_policy.#:                         "1" => "1"
          consistency_policy.0.consistency_level:       "Strong" => "Strong"
          consistency_policy.0.max_interval_in_seconds: "5" => "5"
          consistency_policy.0.max_staleness_prefix:    "100" => "100"
          enable_automatic_failover:                    "false" => "false"
          enable_multiple_write_locations:              "false" => "false"
          endpoint:                                     "https://acctest-ca-200326104554447378.documents.azure.com:443/" => "<computed>"
          geo_location.#:                               "1" => "1" (forces new resource)
          geo_location.0.failover_priority:             "0" => "0" (forces new resource)
          geo_location.0.id:                            "acctest-ca-200326104554447378-eastus" => "<computed>" (forces new resource)
          geo_location.0.location:                      "eastus" => "eastus" (forces new resource)
          geo_location.0.prefix:                        "acctest-ca-200326104554447378-eastus" => "" (forces new resource)
          id:                                           "/subscriptions/********/resourceGroups/acctestRG-cosmos-200326104554447378/providers/Microsoft.DocumentDB/databaseAccounts/acctest-ca-200326104554447378" => "<computed>"
          ip_range_filter:                              "" => ""
          is_virtual_network_filter_enabled:            "false" => "false"
          kind:                                         "GlobalDocumentDB" => "GlobalDocumentDB"
          location:                                     "eastus" => "eastus"
          name:                                         "acctest-ca-200326104554447378" => "acctest-ca-200326104554447378"
          offer_type:                                   "Standard" => "Standard"
          primary_master_key:                           "********" => "<computed>"
          primary_readonly_master_key:                  "********" => "<computed>"
          read_endpoints:                               "" => "<computed>"
          read_endpoints.#:                             "1" => ""
          read_endpoints.0:                             "https://acctest-ca-200326104554447378-eastus.documents.azure.com:443/" => ""
          resource_group_name:                          "acctestRG-cosmos-200326104554447378" => "acctestRG-cosmos-200326104554447378"
          secondary_master_key:                         "********" => "<computed>"
          secondary_readonly_master_key:                "********" => "<computed>"
          virtual_network_rule.#:                       "0" => "0"
          write_endpoints:                              "" => "<computed>"
          write_endpoints.#:                            "1" => ""
          write_endpoints.0:                            "https://acctest-ca-200326104554447378-eastus.documents.azure.com:443/" => ""



        STATE:

        azurerm_cosmosdb_account.test:
          ID = /subscriptions/********/resourceGroups/acctestRG-cosmos-200326104554447378/providers/Microsoft.DocumentDB/databaseAccounts/acctest-ca-200326104554447378
          provider = provider.azurerm
          capabilities.# = 1
          capabilities.0.name = EnableCassandra
          connection_strings.# = 8
          connection_strings.0 = AccountEndpoint=https://acctest-ca-200326104554447378.documents.azure.com:443/;AccountKey=********;
          connection_strings.1 = AccountEndpoint=https://acctest-ca-200326104554447378.documents.azure.com:443/;AccountKey=********;
          connection_strings.2 = AccountEndpoint=https://acctest-ca-200326104554447378.documents.azure.com:443/;AccountKey=********;
          connection_strings.3 = AccountEndpoint=https://acctest-ca-200326104554447378.documents.azure.com:443/;AccountKey=********;
          connection_strings.4 = HostName=acctest-ca-200326104554447378.cassandra.cosmos.azure.com;Username=acctest-ca-200326104554447378;Password=********;Port=10350
          connection_strings.5 = HostName=acctest-ca-200326104554447378.cassandra.cosmos.azure.com;Username=acctest-ca-200326104554447378;Password=********;Port=10350
          connection_strings.6 = HostName=acctest-ca-200326104554447378.cassandra.cosmos.azure.com;Username=acctest-ca-200326104554447378;Password=********;Port=10350
          connection_strings.7 = HostName=acctest-ca-200326104554447378.cassandra.cosmos.azure.com;Username=acctest-ca-200326104554447378;Password=********;Port=10350
          consistency_policy.# = 1
          consistency_policy.0.consistency_level = Strong
          consistency_policy.0.max_interval_in_seconds = 5
          consistency_policy.0.max_staleness_prefix = 100
          enable_automatic_failover = false
          enable_multiple_write_locations = false
          endpoint = https://acctest-ca-200326104554447378.documents.azure.com:443/
          geo_location.# = 1
          geo_location.0.failover_priority = 0
          geo_location.0.id = acctest-ca-200326104554447378-eastus
          geo_location.0.location = eastus
          geo_location.0.prefix = acctest-ca-200326104554447378-eastus
          ip_range_filter =
          is_virtual_network_filter_enabled = false
          kind = GlobalDocumentDB
          location = eastus
          name = acctest-ca-200326104554447378
          offer_type = Standard
          primary_master_key = ********
          primary_readonly_master_key = ********
          read_endpoints.# = 1
          read_endpoints.0 = https://acctest-ca-200326104554447378-eastus.documents.azure.com:443/
          resource_group_name = acctestRG-cosmos-200326104554447378
          secondary_master_key = ********
          secondary_readonly_master_key = ********
          write_endpoints.# = 1
          write_endpoints.0 = https://acctest-ca-200326104554447378-eastus.documents.azure.com:443/

          Dependencies:
            azurerm_resource_group.test
        azurerm_cosmosdb_cassandra_keyspace.test:
          ID = /subscriptions/********/resourceGroups/acctestRG-cosmos-200326104554447378/providers/Microsoft.DocumentDB/databaseAccounts/acctest-ca-200326104554447378/cassandraKeyspaces/acctest-200326104554447378
          provider = provider.azurerm
          account_name = acctest-ca-200326104554447378
          name = acctest-200326104554447378
          resource_group_name = acctestRG-cosmos-200326104554447378
          throughput = 700

          Dependencies:
            azurerm_cosmosdb_account.test
        azurerm_resource_group.test:
          ID = /subscriptions/********/resourceGroups/acctestRG-cosmos-200326104554447378
          provider = provider.azurerm
          location = eastus
          name = acctestRG-cosmos-200326104554447378
FAIL
FAIL    github.com/terraform-providers/terraform-provider-azurerm/azurerm/internal/services/cosmos/tests        1166.916s
FAIL

antempus · 2020-05-29T17:00:54Z

I'm going to look at it over the course of the day as it takes 20min to run a test

I've noticed issues with normal provision behavior, creating DB accounts can take > 10 mins in many cases; this may be the source of test lengths.

katbyte · 2020-07-09T19:50:42Z

hey @Brunhil - i'm going to close this in favour of #7597 which is both a new api version and actively being worked on

ghost · 2020-07-10T01:24:20Z

This has been released in version 2.18.0 of the provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading. As an example:

provider "azurerm" {
    version = "~> 2.18.0"
}
# ... other configuration ...

Forked from #6253 and brings the Cosmos DB API up to 2020-04-01 Closes #6253

ghost · 2020-08-09T14:53:06Z

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 [email protected]. Thanks!

Upgrade CosmosDB SDK to 2019-08-01 from 2015-04-08

0c2d734

ghost added size/XXL dependencies labels Mar 25, 2020

jackbatzner mentioned this pull request Mar 25, 2020

New Resource azurerm_cosmosdb_sql_storedprocedure #6189

Merged

jackbatzner commented Mar 25, 2020

View reviewed changes

katbyte added the service/cosmosdb label Mar 25, 2020

Parse new and old SDK IDs.

20ccef0

- Move parse functionality from /helpers/azure/cosmos.go to individual helper functions.

jackbatzner added 2 commits March 25, 2020 21:27

Update import statements to reflect new API Resource ID

60ad873

Properly reference old SDK Resource IDs

ad0631b

tombuildsstuff suggested changes Mar 26, 2020

View reviewed changes

tombuildsstuff added enhancement waiting-response labels Mar 26, 2020

tombuildsstuff added this to the v2.4.0 milestone Mar 26, 2020

jackbatzner added 5 commits March 26, 2020 08:19

Properly retrieve Gremlin Graph

97b6cf3

Create StateMigration for old SDK to new SDK

e2e63a0

Format error messages

2b3ed49

Create common helper for getting throughput value

24f9e7b

Properly generate error when importing ID

867386e

ghost removed the waiting-response label Mar 26, 2020

jackbatzner requested a review from tombuildsstuff March 26, 2020 17:38

tombuildsstuff modified the milestones: v2.4.0, v2.5.0, v2.6.0 Apr 2, 2020

katbyte added 2 commits April 14, 2020 20:53

merge master

1296bbc

fix migration logic

619d5aa

katbyte added this to the v2.9.0 milestone Apr 30, 2020

tombuildsstuff modified the milestones: v2.9.0, v2.10.0 May 7, 2020

katbyte modified the milestones: v2.10.0, v2.11.0 May 14, 2020

katbyte modified the milestones: v2.11.0, v2.12.0, v2.13.0 May 22, 2020

tombuildsstuff mentioned this pull request May 29, 2020

Support for Free Tier on CosmosDB #6030

Closed

tombuildsstuff modified the milestones: v2.13.0, v2.14.0, v2.15.0 Jun 4, 2020

katbyte modified the milestones: v2.15.0, v2.16.0 Jun 18, 2020

tombuildsstuff modified the milestones: v2.16.0, v2.17.0, v2.18.0 Jun 25, 2020

neil-yechenwei mentioned this pull request Jul 7, 2020

Deprecate the prefix property of azurerm_cosmosdb_account since API's behavior has been changed #7588

Closed

marc-sensenich mentioned this pull request Jul 7, 2020

Update Azure Cosmos DB API to to 2020-04-01 from 2015-04-08 #7597

Merged

katbyte closed this Jul 9, 2020

katbyte pushed a commit that referenced this pull request Jul 20, 2020

Update Azure Cosmos DB API to to 2020-04-01 from 2015-04-08 (#7597)

c5e6f59

Forked from #6253 and brings the Cosmos DB API up to 2020-04-01 Closes #6253

ghost locked and limited conversation to collaborators Aug 9, 2020

jackbatzner deleted the upgrade_cosmos_sdk branch March 2, 2021 02:27

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Upgrade CosmosDB SDK to 2019-08-01 from 2015-04-08 #6253

Upgrade CosmosDB SDK to 2019-08-01 from 2015-04-08 #6253

jackbatzner commented Mar 25, 2020 •

edited

Loading

jackbatzner Mar 25, 2020

tombuildsstuff Mar 26, 2020

jackbatzner commented Mar 25, 2020 •

edited

Loading

katbyte commented Mar 25, 2020

jackbatzner commented Mar 26, 2020 •

edited

Loading

tombuildsstuff left a comment

jackbatzner commented Mar 26, 2020 •

edited

Loading

antempus commented May 29, 2020

katbyte commented Jul 9, 2020

ghost commented Jul 10, 2020

ghost commented Aug 9, 2020

Upgrade CosmosDB SDK to 2019-08-01 from 2015-04-08 #6253

Upgrade CosmosDB SDK to 2019-08-01 from 2015-04-08 #6253

Conversation

jackbatzner commented Mar 25, 2020 • edited Loading

jackbatzner Mar 25, 2020

Choose a reason for hiding this comment

tombuildsstuff Mar 26, 2020

Choose a reason for hiding this comment

jackbatzner commented Mar 25, 2020 • edited Loading

katbyte commented Mar 25, 2020

jackbatzner commented Mar 26, 2020 • edited Loading

tombuildsstuff left a comment

Choose a reason for hiding this comment

jackbatzner commented Mar 26, 2020 • edited Loading

antempus commented May 29, 2020

katbyte commented Jul 9, 2020

ghost commented Jul 10, 2020

ghost commented Aug 9, 2020

jackbatzner commented Mar 25, 2020 •

edited

Loading

jackbatzner commented Mar 25, 2020 •

edited

Loading

jackbatzner commented Mar 26, 2020 •

edited

Loading

jackbatzner commented Mar 26, 2020 •

edited

Loading