New Data Source: cloudflare_zones #168

ewilde · 2018-11-25T12:42:43Z

This PR adds support for a new data source cloudflare_zones. To look up zones based on a filter

cloudflare_zones

Use this data source to look up [Zone][1] records.

Example Usage

The example below matches all active zones that begin with example. and are not paused. The matched zones are then
locked down using the cloudflare_zone_lockdown resource.

data "cloudflare_zones" "test" {
 filter {
   zone   = "example.*"
   status = "active"
   paused = false
 }
}

resource "cloudflare_zone_lockdown" "endpoint_lockdown" {
 count       = "${length(data.cloudflare_zones.test.zones)}"
 zone        = "${element(data.cloudflare_zones.test.zones, count.index)}"
 paused      = "false"
 description = "Restrict access to these endpoints to requests from a known IP address"
 urls = [
   "api.mysite.com/some/endpoint*",
 ]
 configurations = [
   {
     "target" = "ip"
     "value" = "198.51.100.4"
   },
 ]
}

Argument Reference

filter - (Required) filter - (Optional) One or more values filter off of. It more than one value is given all values
must match, see below for full list.
filter
zone - (Optional) A regular expression matching the zone to lookup.
status - (Optional) Status of the zone to lookup. Valid values: active, pending, initializing, moved, deleted, deactivated and read only.
paused - (Optional) Paused status of the zone to lookup. Valid values are true or false.

Attributes Reference

zones - A list of zone names
[1]: https://api.cloudflare.com/#zone-properties

Testing

Tested website manually

Tested website with make website-test

make website-test 
==> Testing cloudflare provider website in Docker...
Error response from daemon: No such container: tf-website-cloudflare-temp
make[1]: [website-provider-test] Error 1 (ignored)
272b7c8daa186403f547689a2640a84a81129db7c7751089cc4cb7cc9131a226
until curl -sS http://localhost:4567/ > /dev/null; do sleep 1; done
curl: (52) Empty reply from server
curl: (52) Empty reply from server
curl: (52) Empty reply from server
curl: (52) Empty reply from server
curl: (52) Empty reply from server
curl: (52) Empty reply from server
curl: (52) Empty reply from server
curl: (52) Empty reply from server
curl: (52) Empty reply from server
curl: (52) Empty reply from server
curl: (52) Empty reply from server
curl: (52) Empty reply from server
curl: (52) Empty reply from server
curl: (52) Empty reply from server
curl: (52) Empty reply from server
curl: (52) Empty reply from server
curl: (52) Empty reply from server
curl: (52) Empty reply from server
curl: (52) Empty reply from server
curl: (52) Empty reply from server
curl: (52) Empty reply from server
curl: (52) Empty reply from server
/Users/ewilde/dev/src/github.com/hashicorp/terraform-website/content/scripts/check-links.sh "http://127.0.0.1:4567/docs/providers/cloudflare/"
2018-11-25 17:57:02 URL:http://127.0.0.1:4567/docs/providers/cloudflare/ [30256/30256] -> "index.html.tmp.tmp" [1]
2018-11-25 17:57:02 URL:http://127.0.0.1:4567/robots.txt [44/44] -> "robots.txt.tmp" [1]
2018-11-25 17:57:02 URL:http://127.0.0.1:4567/docs/providers/cloudflare/index.html [30256/30256] -> "index.html.tmp.tmp" [1]
2018-11-25 17:57:02 URL:http://127.0.0.1:4567/docs/providers/cloudflare/d/ip_ranges.html [27911/27911] -> "ip_ranges.html.tmp.tmp" [1]
2018-11-25 17:57:02 URL:http://127.0.0.1:4567/docs/providers/cloudflare/d/zones.html [29438/29438] -> "zones.html.tmp.tmp" [1]
2018-11-25 17:57:03 URL:http://127.0.0.1:4567/docs/providers/cloudflare/r/access_application.html [28448/28448] -> "access_application.html.tmp.tmp" [1]
2018-11-25 17:57:03 URL:http://127.0.0.1:4567/docs/providers/cloudflare/r/access_policy.html [32207/32207] -> "access_policy.html.tmp.tmp" [1]
2018-11-25 17:57:03 URL:http://127.0.0.1:4567/docs/providers/cloudflare/r/access_rule.html [32943/32943] -> "access_rule.html.tmp.tmp" [1]
2018-11-25 17:57:03 URL:http://127.0.0.1:4567/docs/providers/cloudflare/r/account_member.html [28428/28428] -> "account_member.html.tmp.tmp" [1]
2018-11-25 17:57:03 URL:http://127.0.0.1:4567/docs/providers/cloudflare/r/custom_pages.html [29474/29474] -> "custom_pages.html.tmp.tmp" [1]
2018-11-25 17:57:03 URL:http://127.0.0.1:4567/docs/providers/cloudflare/r/filter.html [29622/29622] -> "filter.html.tmp.tmp" [1]
2018-11-25 17:57:03 URL:http://127.0.0.1:4567/docs/providers/cloudflare/r/firewall_rule.html [30784/30784] -> "firewall_rule.html.tmp.tmp" [1]
2018-11-25 17:57:03 URL:http://127.0.0.1:4567/docs/providers/cloudflare/r/load_balancer.html [33725/33725] -> "load_balancer.html.tmp.tmp" [1]
2018-11-25 17:57:03 URL:http://127.0.0.1:4567/docs/providers/cloudflare/r/load_balancer_monitor.html [30883/30883] -> "load_balancer_monitor.html.tmp.tmp" [1]
2018-11-25 17:57:03 URL:http://127.0.0.1:4567/docs/providers/cloudflare/r/load_balancer_pool.html [31769/31769] -> "load_balancer_pool.html.tmp.tmp" [1]
2018-11-25 17:57:03 URL:http://127.0.0.1:4567/docs/providers/cloudflare/r/page_rule.html [34448/34448] -> "page_rule.html.tmp.tmp" [1]
2018-11-25 17:57:03 URL:http://127.0.0.1:4567/docs/providers/cloudflare/r/rate_limit.html [35431/35431] -> "rate_limit.html.tmp.tmp" [1]
2018-11-25 17:57:03 URL:http://127.0.0.1:4567/docs/providers/cloudflare/r/record.html [30127/30127] -> "record.html.tmp.tmp" [1]
2018-11-25 17:57:04 URL:http://127.0.0.1:4567/docs/providers/cloudflare/r/waf_rule.html [28296/28296] -> "waf_rule.html.tmp.tmp" [1]
2018-11-25 17:57:04 URL:http://127.0.0.1:4567/docs/providers/cloudflare/r/worker_route.html [31039/31039] -> "worker_route.html.tmp.tmp" [1]
2018-11-25 17:57:04 URL:http://127.0.0.1:4567/docs/providers/cloudflare/r/worker_script.html [30028/30028] -> "worker_script.html.tmp.tmp" [1]
2018-11-25 17:57:04 URL:http://127.0.0.1:4567/docs/providers/cloudflare/r/zone.html [29371/29371] -> "zone.html.tmp.tmp" [1]
2018-11-25 17:57:04 URL:http://127.0.0.1:4567/docs/providers/cloudflare/r/zone_lockdown.html [30840/30840] -> "zone_lockdown.html.tmp.tmp" [1]
2018-11-25 17:57:04 URL:http://127.0.0.1:4567/docs/providers/cloudflare/r/zone_settings_override.html [37557/37557] -> "zone_settings_override.html.tmp.tmp" [1]
Found no broken links.

FINISHED --2018-11-25 17:57:04--
Total wall clock time: 1.8s
Downloaded: 24 files, 697K in 0.05s (13.2 MB/s)
tf-website-cloudflare-temp

Tested new data source with new integration tests

=== RUN   TestAccCloudflareZonesMatchName
--- PASS: TestAccCloudflareZonesMatchName (40.03s)
=== RUN   TestAccCloudflareZonesMatchPaused
--- PASS: TestAccCloudflareZonesMatchPaused (47.11s)
=== RUN   TestAccCloudflareZonesMatchStatus
--- PASS: TestAccCloudflareZonesMatchStatus (17.50s)
PASS

Process finished with exit code 0

Signed-off-by: Edward Wilde [email protected]

1. Tested website manually 2. Tested website with `make website-test` 3. Tested new data source with new integration tests Signed-off-by: Edward Wilde <[email protected]>

ewilde · 2018-11-25T18:07:08Z

cloudflare/data_source_zones_test.go

+data "cloudflare_zones" "examples_domains" {
+  filter {
+    zone   = "baa.*"
+    paused = "${cloudflare_zone.foo_net.paused}" // true


Hack This might look a bit weird, but it's necessary to reference a resource here so it gets created before the data source look up executes.

I did try referencing the test zones from the data source using depends_on attribute, but this caused a test failure during the check plan diff is empty phase.

patryk · 2018-12-04T19:26:03Z

cloudflare/data_source_zones.go

+				MaxItems: 1,
+				Elem: &schema.Resource{
+					Schema: map[string]*schema.Schema{
+						"zone": {


Can we rename it to name, in consistency with the API?

patryk · 2019-01-11T15:41:56Z

Thanks @ewilde !

The initial implementation of the datasource had the provider request all the available zones and then we would perform some logic within the provider to filter down to the match the attributes. This is slightly inefficient for a couple of reasons: - We are requesting *all* zones (even if we only want a small subset of them); and - We are duplicating the zone filtering logic in the provider that is already available in the API This change replaces that functionality by pushing the responsibility of filtering back to the API. It maintains a reasonable amount of backwards compatibility outlined in the initial PR (cloudflare#168) where the "name" had to accept wildcards but under the hood accepted a full regex. Instead, we use the `contains:` modifier on the name search parameter which will instead fuzzy match. The rest remains the same but is performed via the API. Fixes cloudflare#707

ghost added the size/XL label Nov 25, 2018

ewilde changed the title ~~New Data Source: cloudflare_zones~~ [WIP] New Data Source: cloudflare_zones Nov 25, 2018

ewilde force-pushed the ewilde-datasource-zones branch 2 times, most recently from 63107b6 to 22f3ec2 Compare November 25, 2018 17:47

ewilde changed the title ~~[WIP] New Data Source: cloudflare_zones~~ New Data Source: cloudflare_zones Nov 25, 2018

New Data Source: cloudflare_zones

ac8b484

1. Tested website manually 2. Tested website with `make website-test` 3. Tested new data source with new integration tests Signed-off-by: Edward Wilde <[email protected]>

ewilde force-pushed the ewilde-datasource-zones branch from 22f3ec2 to ac8b484 Compare November 25, 2018 18:02

ewilde commented Nov 25, 2018

View reviewed changes

ewilde mentioned this pull request Nov 26, 2018

New Data Source: cloudflare_zone #167

Closed

patryk requested changes Dec 17, 2018

View reviewed changes

patryk mentioned this pull request Jan 7, 2019

Add new zone data source #163

Closed

jacobbednarz mentioned this pull request Jan 9, 2019

Data Provider For Zone #190

Closed

Rename zone to name in filter definition

c065737

ghost added the kind/documentation Categorizes issue or PR as related to documentation. label Jan 11, 2019

deuill approved these changes Jan 11, 2019

View reviewed changes

patryk approved these changes Jan 11, 2019

View reviewed changes

patryk merged commit 8d52193 into cloudflare:master Jan 11, 2019

jacobbednarz mentioned this pull request Jan 4, 2020

cloudflare_zones returns a list instead of a map #569

Closed

jacobbednarz mentioned this pull request Jun 12, 2020

datasource/zones: Perform filtering on server side #708

Merged

ewilde deleted the ewilde-datasource-zones branch June 12, 2020 11:36

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New Data Source: cloudflare_zones #168

New Data Source: cloudflare_zones #168

ewilde commented Nov 25, 2018 •

edited

Loading

ewilde Nov 25, 2018 •

edited

Loading

patryk Dec 4, 2018

patryk commented Jan 11, 2019

New Data Source: cloudflare_zones #168

New Data Source: cloudflare_zones #168

Conversation

ewilde commented Nov 25, 2018 • edited Loading

cloudflare_zones

Example Usage

Argument Reference

Attributes Reference

Testing

ewilde Nov 25, 2018 • edited Loading

Choose a reason for hiding this comment

patryk Dec 4, 2018

Choose a reason for hiding this comment

patryk commented Jan 11, 2019

ewilde commented Nov 25, 2018 •

edited

Loading

ewilde Nov 25, 2018 •

edited

Loading