Giv din stemme

Local development

This project contains a docker compose setup build around https://github.com/itk-dev/devops_itkdev-docker which is a simple wrapper script around docker compose using traefik as a revers proxy to get FQDN to access the site.

If you do not want to use this wrapper, you can substitute the itkdev-docer-compose command with normal docker compose remembering to load the right composer file using the -f <file> option.

This setup also assumes that you have a docker network shared with treafik, if you are not using the wrapper, use this command to create the network first.

docker network create --driver=bridge --attachable --internal=false frontend

Building assets for the frontend

Run the following to install dependencies with yarn.

docker compose run --rm node yarn install

Run the following to continuously build assets upon file changes.

docker compose run --rm node yarn watch

Run the following to build assets once.

docker compose run --rm node yarn build

Build assets for Giv Din Stemme module

Run the following to install dependencies with yarn.

docker compose run --rm node yarn --cwd web/modules/custom/giv_din_stemme/ install

Run the following to continuously build assets upon file changes.

docker compose run --rm node yarn --cwd web/modules/custom/giv_din_stemme/ watch

Run the following to build assets once.

docker compose run --rm node yarn --cwd web/modules/custom/giv_din_stemme/ build

Site installation

Run the following commands to set up the site. This will run a normal Drupal site installation with the existing configuration that comes with this project.

itkdev-docker-compose up --detach
itkdev-docker-compose composer install
itkdev-docker-compose drush site-install --existing-config --yes

When the installation is completed, that admin user is created and the password for logging in the outputted. If you forget the password, use drush uli command to get a one-time-login link (not the uri here only works if you are using trafik).

itkdev-docker-compose drush --uri="https://givdinstemme.local.itkdev.dk/" user:login

Access the site

If you are using out itkdev-docker-compose simple use the command below to åbne the site in you default browser.

itkdev-docker-compose open

Alternatively you can find the port number that is mapped nginx container that server the site at http://0.0.0.0:PORT by using this command:

open "http://$(docker compose port nginx 8080)"

Additional microphone permissions

Some browsers on some platforms require additional microphone permissions to allow use of the microphone on all pages of a site.

We use a regular expression to detect Safari on iOS based on the user agent string (cf. Drupal\giv_din_stemme\Controller\GivDinStemmeController::test()).

During testing and development the regular expression can easily be changed in settings.local.php, e.g.:

# settings.local.php
// The default value matching iPhone and Safari (in any order and ignoring case)
$settings['giv_din_stemme']['requires_additional_microphone_permissions_pattern'] = '/^(?=.*\biPhone\b)(?=.*\bSafari\b).*$/i';

// Match any user agent string
$settings['giv_din_stemme']['requires_additional_microphone_permissions_pattern'] = '/./';

The actual help page for details on what actually must be done to grant the additional microphone permissions is set under "References" on /admin/site-setup/general.

Browsers requiring additional microphone permissions

	Safari	Chrome	Firefox
iOS	✓
Android
macOS

(The table reflects the checks currently implemented)

Drupal config

This project uses Drupal's configuration import and export to handle configuration changes and uses the config ignore module to protect some of the site settings form being overridden. For local and production configuration settings that you do not want to export, please use settings.local.php to override default configuration values.

Export config created from drupal:

itkdev-docker-compose drush config:export

Import config from config files:

itkdev-docker-compose drush config:import

Production setup

@todo Write this section.

OpenID Connect

We use the OpenID Connect for OpenID Connect.

Development

During development, we use OpenId Connect Server Mock to test OpenID Connect authentication, and this is reflected in the default OIDC configuration.

Run

itkdev-docker-compose --profile oidc up --detach

to start the OIDC mock along with the other stuff.

The OIDC mock uses a selfsigned pfx certificate for HTTPS, and to make everything work during development a little patch must be applied to Guzzle:

docker compose exec phpfpm bash -c 'patch --strip=1 < patches/guzzle-with-self-signed-certificate.patch'

Updating the self-signed certificate

Note: This section is only kept as an internal note on how the self-signed certificate, .docker/oidc-server-mock/cert/docker.pfx, is generated from our self-signed development Traefik certificates. The certificate is committed to Git and should only be updated if our Traefik certificates are ever updated.

cert_path="$(dirname $(dirname $(which itkdev-docker-compose)))/traefik/ssl"
openssl pkcs12 -export -out .docker/oidc-server-mock/cert/docker.pfx -inkey $cert_path/docker.key -in $cert_path/docker.crt -passout pass:mock

openssl pkcs12 -in .docker/oidc-server-mock/cert/docker.pfx -passin pass:mock -passout pass: -info

Production

For production, we override (some) OpenID Connect configuration (rather than ignoring config) in settings.local.php:

// settings.local.php
// …

// https://idp-citizen.givdinstemme.srvitkstgweb01.itkdev.dk/.well-known/openid-configuration
$config['openid_connect.client.generic']['settings']['client_id'] = '…';
$config['openid_connect.client.generic']['settings']['client_secret'] = '…';

// Get these from your OIDC Discovery document.
$config['openid_connect.client.generic']['settings']['authorization_endpoint'] = '…';
$config['openid_connect.client.generic']['settings']['token_endpoint'] = '…';
$config['openid_connect.client.generic']['settings']['end_session_endpoint'] = '…';

Whisper

The custom Giv din stemme module adds commands using Whisper for qualifying donations.

Qualifying is done by asking Whisper to transcribe the donation and then comparing it to the original text, using one of three metrics.

Similar text score

similar_text gives similarity between two sentences as a percentage. That is, a high percentage means the sentences are similar. See ADR-002 for more details.

Note: to align with the WER and CER scores beneath, we report the complimentary, i.e. the dissimilarity, on the list of donations.

WER and CER

Word error rate(WER) and character error rate(CER). For more details on these, see itk-dev/sentence-similarity-metrics. This is a number between 0 and 1 A low score means the transcribed sentence is similar to the original one.

Configuration

Before using the qualifying command you must configure

• Whisper API endpoint
• Whisper API key
• Similar text score threshold for when donations should be validated (int or null/unset to disable).
• WER threshold for when donations should be validated (float or null/unset to disable).
• CER threshold for when donations should be validated (float or null/unset to disable).

// settings.local.php
// …

$settings['itkdev_whisper_api_endpoint'] = '…';
$settings['itkdev_whisper_api_key'] = '…';
$settings['itkdev_automatic_validation_threshold_similar_text_score'] = 80;
$settings['itkdev_automatic_validation_threshold_wer'] = 0.20;
$settings['itkdev_automatic_validation_threshold_cer'] = 0.20;

See 1Password for both api endpoint and key.

Commands

Qualify all unqualified donations with

itkdev-docker-compose drush giv_din_stemme:qualify:transcribe

or re-qualify donations by adding the --re-qualify flag.

Qualify a specific donation with

itkdev-docker-compose drush giv_din_stemme:qualify:transcribe:id DONATION_ID

Note the similar_text qualifying command will validate donations if they result in a score that surpasses the configured threshold level. The wer and cer will validate the donation if the score does not surpass the configured threshold levels. The commands will never invalidate donations.

Calculate similar text score with

itkdev-docker-compose drush giv-din-stemme:qualify:similar-text-score

or re-calculate rates by adding the --re-calculate flag.

Calculate WER with

itkdev-docker-compose drush giv-din-stemme:qualify:wer

or re-calculate rates by adding the --re-calculate flag.

Calculate CER with

itkdev-docker-compose drush giv-din-stemme:qualify:cer

or re-calculate rates by adding the --re-calculate flag.

To continuously qualify donations consider running the qualifying commands via a cronjobs.

Coding standards

docker compose run --rm phpfpm composer install
docker compose run --rm phpfpm composer normalize

docker compose run --rm phpfpm composer install
docker compose run --rm phpfpm composer coding-standards-apply/phpcs
docker compose run --rm phpfpm composer coding-standards-check/phpcs

docker compose run --rm phpfpm composer install
docker compose run --rm phpfpm composer coding-standards-apply/twig-cs-fixer
docker compose run --rm phpfpm composer coding-standards-check/twig-cs-fixer

docker run --rm --volume "$PWD:/md" itkdev/markdownlint $(git ls-files *.md) --fix
docker run --rm --volume "$PWD:/md" itkdev/markdownlint $(git ls-files *.md)

docker compose run --rm node yarn install
docker compose run --rm node yarn coding-standards-apply
docker compose run --rm node yarn coding-standards-check