From 8b3862087cd0755a84d6229916bd9e2635ed0276 Mon Sep 17 00:00:00 2001 From: Iristyle Date: Fri, 14 Sep 2018 13:42:13 -0700 Subject: [PATCH] (CPR-645) Setup Windows compose tests - Add Azure Pipelines invocation to prep for specs, to run `bundle exec rspec spec` for basic smoke tests, to collect the Junit output using an Azure Pipelines task, and to issue cleanup commands to purge dangling containers / images regardless of the success / failure of prior steps. This prevents aborted jobs from cluttering. - Add a default docker-compose.override.yml which includes Linux specific configuration for 2 things: * the image of puppet/puppetdb-postgres * the container path for the local ./volumes/puppetdb-postgres/data With `docker-compose up` called, these settings are consumed by default. - Windows specific changes to the default config reside in docker-compose.windows.yml and as part of the Azure pipeline job, are copied *over* docker-compose.override.yml to take effect. Note that there are no builds of the puppetdb-postgres container for Windows as the VOLUME support required for the base postgres container does not currently work on Windows. Instead, use a Windows based postgres container that mimics the official container / supports all of the configuration ENV vars. - The minor changes for postgres support in docker-compose.windows.yml also includes mounting a local directory containting SQL scripts to run when postgres starts (using \docker-entrypoint-initdb.d). Further, it sets POSTGRES_DB=puppetdb to create the default database. The extensions.sql script contains the SQL necessary to add 2 extensions to the puppetdb database that are currently handled in the puppetdb-postgres container. This sets the stage for moving from that container to the official postgres container. - Lastly, the Windows specific docker-compose.windows.yml sets up a custom alias in the "default" network so that an extra DNS name for puppetserver can be set based on the FQDN that Facter determines. Without this additional DNS reservation, the `puppetserver ca` command will be unable to connect to the REST endpoint. A better long-term solution is making sure puppetserver is setup to point to `puppet` as the host instead of an FQDN. - Fix health check in specs on Windows based on how docker inspect works for Windows - Fix spec that requests a --tty since that doesn't work on Windows. - Update README accordingly --- commercial/README.md | 27 +++++------ commercial/azure-pipelines.yml | 51 +++++++++++++++++++++ commercial/docker-compose.override.yml | 7 +++ commercial/docker-compose.windows.yml | 24 +++++----- commercial/docker-compose.yml | 8 ++-- commercial/postgres-custom/extensions.sql | 2 + commercial/spec/examples/running_cluster.rb | 8 ++-- 7 files changed, 93 insertions(+), 34 deletions(-) create mode 100644 commercial/docker-compose.override.yml create mode 100644 commercial/postgres-custom/extensions.sql diff --git a/commercial/README.md b/commercial/README.md index b63f6566..1b9d8a95 100644 --- a/commercial/README.md +++ b/commercial/README.md @@ -7,7 +7,7 @@ To get started, you will need an installation of [Docker Compose](https://docs.docker.com/compose/install/) on the host on which you will run your Puppet Infrastructure. -Once you have Docker Compose installed, you can start the stack with +Once you have Docker Compose installed, you can start the stack on Linux with: ``` DNS_ALT_NAMES=puppet,host.exmple.com docker-compose up -d ``` @@ -42,28 +42,24 @@ PuppetDB `Preferences>File Sharing` in order for these directories to be created and volume-mounted automatically. There is no need to add each sub directory. -## Docker for Windows +## Pupperware on Windows (using LCOW) Complete instructions for provisiong a server with LCOW support are in [README-windows.md](./README-windows.md) -Due to [permissions issues with Postgres](https://forums.docker.com/t/trying-to-get-postgres-to-work-on-persistent-windows-mount-two-issues/12456/4) on Docker for Windows, an external volume and a slightly different configuration is required. +Due to [permissions issues with Postgres](https://forums.docker.com/t/trying-to-get-postgres-to-work-on-persistent-windows-mount-two-issues/12456/4) on Docker for Windows, to run under the LCOW environment, the Windows stack relies on the [`stellirin/postgres-windows`](https://hub.docker.com/r/stellirin/postgres-windows/) Windows variant of the upstream [`postgres`](https://hub.docker.com/_/postgres/) container instead. To create the stack: ``` powershell -PS> docker volume create --name puppetdb-postgres-volume -d local -puppetdb-postgres-volume - PS> $ENV:DNS_ALT_NAMES = 'puppet,host.exmple.com' PS> docker-compose -f .\docker-compose.yml -f .\docker-compose.windows.yml up -Creating pupperware_postgres_1 ... done -Creating pupperware_puppet_1 ... done -Creating pupperware_puppetdb_1 ... done -Attaching to pupperware_postgres_1, pupperware_puppet_1, pupperware_puppetdb_1 -postgres_1 | The files belonging to this database system will be owned by user "postgres". -postgres_1 | This user must also own the server process. -postgres_1 | +Creating network "pupperware_default" with the default driver +Creating pupperware_puppet_1_4be38bcee346 ... done +Creating pupperware_postgres_1_c82bfeb597f5 ... done +Creating pupperware_puppetdb_1_bcd7e5f54a3f ... done +Attaching to pupperware_postgres_1_cf9a935a098e, pupperware_puppet_1_79b6ff064b91, pupperware_puppetdb_1_70edf5d8cd1e + ... ``` @@ -73,11 +69,10 @@ To delete the stack: PS> docker-compose down Removing network pupperware_default ... - -PS> docker volume rm puppetdb-postgres-volume -puppetdb-postgres-volume ``` +Note that `docker-compose down` may perform slowly on Windows - see [docker/for-win 629](https://github.com/docker/for-win/issues/629) and [docker/compose](https://github.com/docker/compose/issues/3419) for further information. + ## Managing the stack The script `bin/puppet` (or `bin\puppet.ps1` on Windows) makes it easy to run `puppet` commands on the diff --git a/commercial/azure-pipelines.yml b/commercial/azure-pipelines.yml index f1a47dd3..d95099e4 100644 --- a/commercial/azure-pipelines.yml +++ b/commercial/azure-pipelines.yml @@ -4,6 +4,9 @@ pool: # includes Ruby 2.5, Go 1.10, Node.js 10.10, hadolint name: Default +variables: + COMPOSE_PROJECT_NAME: pupperware + steps: - checkout: self # self represents the repo where the initial Pipelines YAML file was found clean: true # whether to fetch clean each time @@ -28,6 +31,7 @@ steps: docker version docker images docker info + docker-compose version sc.exe qc docker # # Ruby @@ -43,3 +47,50 @@ steps: Get-ChildItem Env: | % { Write-Host "$($_.Key): $($_.Value)" } displayName: Diagnostic Host Information name: hostinfo + +- powershell: | + bundle install --with test --path '.bundle/gems' + # make sure windows yml slots in as the default override + Copy-Item ./docker-compose.windows.yml ./docker-compose.override.yml -Force + Write-Host 'Forcibly removing previous cluster config / data' + Remove-Item 'volumes' -Force -Recurse -ErrorAction SilentlyContinue; + New-Item 'volumes' -Type Directory + Push-Location 'volumes' + 'code', 'puppet', 'serverdata', 'puppetdb\ssl', 'puppetdb-postgres\data' | + % { New-Item $_ -Type Directory } + Pop-Location + displayName: Prepare Test Environment + name: test_prepare + +- powershell: | + $domain = Get-WmiObject -Class Win32_NetworkAdapterConfiguration | + Select -ExpandProperty DNSDomain | + Select -First 1 + Write-Host 'Writing compose config to disk' + $content = @" + AZURE_DOMAIN=$domain + "@ + $Utf8NoBomEncoding = New-Object System.Text.UTF8Encoding $False + [System.IO.File]::WriteAllLines(".env", $content, $Utf8NoBomEncoding) + Get-Content -Path '.env' + Write-Host 'Executing Pupperware specs' + bundle exec rspec spec --fail-fast + displayName: Test pupperware + name: test_pupperware + +- task: PublishTestResults@2 + displayName: Publish pupperware test results + inputs: + testResultsFormat: 'JUnit' + testResultsFiles: 'TEST-rspec.xml' + testRunTitle: pupperware Test Results + condition: always() + +- powershell: | + Write-Host 'Pruning containers' + docker container prune --force + Write-Host 'Pruning images' + docker image prune --filter "dangling=true" --force + displayName: Container Cleanup + timeoutInMinutes: 3 + condition: always() diff --git a/commercial/docker-compose.override.yml b/commercial/docker-compose.override.yml new file mode 100644 index 00000000..a08d80af --- /dev/null +++ b/commercial/docker-compose.override.yml @@ -0,0 +1,7 @@ +version: '3' + +services: + postgres: + image: puppet/puppetdb-postgres + volumes: + - ./volumes/puppetdb-postgres/data:/var/lib/postgresql/data/ diff --git a/commercial/docker-compose.windows.yml b/commercial/docker-compose.windows.yml index 22f385ab..000a128e 100644 --- a/commercial/docker-compose.windows.yml +++ b/commercial/docker-compose.windows.yml @@ -1,16 +1,16 @@ -version: '2' +version: '3' services: + puppet: + networks: + default: + aliases: + - puppet.${AZURE_DOMAIN} + postgres: + image: stellirin/postgres-windows:9.6 + environment: + - PGDATA=c:\data volumes: - - puppetdb-postgres-volume:/var/lib/postgresql/data/ - -# An external volume is needed for postgres as when under Docker For Windows the files have a different -# owner that the starting process. -# ref: https://forums.docker.com/t/trying-to-get-postgres-to-work-on-persistent-windows-mount-two-issues/12456/4 -# -# To create the volume use; docker volume create --name puppetdb-postgres-volume -d local -# To delete the volume use; docker volume rm puppetdb-postgres-volume -volumes: - puppetdb-postgres-volume: - external: true + - ./volumes/puppetdb-postgres/data:c:\data + - ./postgres-custom:c:\docker-entrypoint-initdb.d diff --git a/commercial/docker-compose.yml b/commercial/docker-compose.yml index 24a9301f..96aac246 100644 --- a/commercial/docker-compose.yml +++ b/commercial/docker-compose.yml @@ -21,14 +21,16 @@ services: default: postgres: - image: puppet/puppetdb-postgres environment: - POSTGRES_PASSWORD=puppetdb - POSTGRES_USER=puppetdb + - POSTGRES_DB=puppetdb expose: - 5432 - volumes: - - ./volumes/puppetdb-postgres/data:/var/lib/postgresql/data/ + # volumes can only be overriden by *target*, not *source* + # so define them in .override.yml and .windows.yml since target varies + # either /var/lib/postgresql/data/ on Linux or c:\data on Windows + # https://docs.docker.com/compose/extends/#adding-and-overriding-configuration networks: default: diff --git a/commercial/postgres-custom/extensions.sql b/commercial/postgres-custom/extensions.sql new file mode 100644 index 00000000..093f31cb --- /dev/null +++ b/commercial/postgres-custom/extensions.sql @@ -0,0 +1,2 @@ +CREATE EXTENSION IF NOT EXISTS pg_trgm; +CREATE EXTENSION IF NOT EXISTS pgcrypto; diff --git a/commercial/spec/examples/running_cluster.rb b/commercial/spec/examples/running_cluster.rb index 61e900d5..a867dd37 100644 --- a/commercial/spec/examples/running_cluster.rb +++ b/commercial/spec/examples/running_cluster.rb @@ -51,7 +51,7 @@ def start_puppetserver container = get_service_container('puppet') status = get_container_status(container) # puppetserver has a healthcheck, we can let that deal with timeouts - while status == 'starting' + while (status == 'starting' || status == "'starting'") sleep(1) status = get_container_status(container) end @@ -63,7 +63,9 @@ def start_puppetserver end def run_agent(agent_name) - %x(docker run --rm --interactive --tty --network pupperware_default --name #{agent_name} --hostname #{agent_name} puppet/puppet-agent-alpine) + # lack of TTY here causes Windows to not show output of container download / start :( + tty = File::ALT_SEPARATOR.nil? ? '--tty' : '' + %x(docker run --rm --interactive #{tty} --network pupperware_default --name #{agent_name} --hostname #{agent_name} puppet/puppet-agent-alpine) return $? end @@ -133,7 +135,7 @@ def start_puppetdb it 'should start puppetserver' do status = start_puppetserver - expect(status).to eq('healthy') + expect(status).to match(/\'?healthy\'?/) end it 'should start puppetdb' do