Could a parallelism/multi-thread feature be introduced?

[eugeneromero]

Feature Request

Ideally, a flag which, when passed, allowed terraform-compliance to run with some sort of multithreading. This would mostly be for perfomance reasons.

Feature description :
The way I envision it, terraform-compliance could first see the total amount of tests to run, split them evenly between the amount of "instances" specified, run all instances in parallel, then "reconcile" the results at the end to present a single, unified list of results as it does now.

The reason for this request is that terraform-compliance seems to run very slow with certain tests, and being able to run multiple instances might be a good way overcome this limitation. As an example, we have one set of tests which is not particularly huge, but terraform-compliance takes a good 3 minutes to parse through all of the tests:

30 features (30 passed)
361 scenarios (361 passed)
1588 steps (1588 passed)
Run 1615460162 finished within 3 minutes

The terraform plan json file its iterating upon, is ~12,500 lines total. So, big, but not huge. When our average pipeline duration is ~4 minutes, this test almost doubles that. Splitting the job into multiple instances might be a simple way of speeding this up.

Suggested Solution description ( if you have any ) :
Something along the lines of terraform-compliance -j / --jobs or -t / --threads which would take an integer as input.

Describe alternatives you've considered :
I considered running multiple instances of terraform-compliance manually, but this would not work since there is no way of splitting the "job" between them.

[eerkunt]

Thanks for reporting this Euegene. May I please learn the version of terraform-compliance you are trying with ? Thanks!

[eugeneromero]

Absolutely! This is on the latest docker version, v1.3.13.

[eerkunt]

Hi @eugeneromero,

4 minutes for running a whole test is simply too much. In order to understand the root cause, is it possible to transfer that plan.out.json privately somehow with us ? I have a plan.out.json file around 20k lines, it just runs everything in seconds.

There must be a small detail somewhere where we allocate unnecessary resources, but hard to guess without having a debugging session.

[eugeneromero]

Hey @eerkunt, yes, that could work. Here's a Google Drive link:
https://drive.google.com/file/d/1MvkjxJI1busul9UgR1YiO_zjlYnS7ZEC/view?usp=sharing

Thanks again!

[eerkunt]

Hi @eugeneromero,

Tried your plan against all features in https://github.com/terraform-compliance/user-friendly-features.git, everything ran in 4 seconds. Maybe the problem is somewhere within the features, is it possible to share the features ?

[eugeneromero]

For sure! here is a link to those: https://drive.google.com/file/d/1Qcb1Yo4sVRR7HUn-hjS23aSTj0qClyg_/view?usp=sharing

[eerkunt]

Thanks I can reproduce the problem right now :)

[eerkunt]

Thanks, will release the fix upon merge with master branch.

I observed 20x performance upgrade. Initial run was around 240 seconds based on your plan & features. Now its around 10-12 seconds.

[eugeneromero]

That is wonderful! I look forward to testing out the new version. Thank you for your support with this issue @eerkunt!

[eerkunt]

@eugeneromero with the change happened in 1.3.20 (#516 & #192) when you update terraform-compliance, it will possibly run slower again.

Please install it via pip install terraform-compliance[faster_parsing].

[eugeneromero]

Thank you for the warning @eerkunt!. We use terraform-compliance from the Docker image. What would be the correct command in that case? Currently we use it like this in our pipelines:

docker run --rm -v $(pwd):/target -t eerkunt/terraform-compliance --features unit/ --planfile release.json

[eerkunt]

oh if you are using the docker image, then nothing is required. Its already embedded there :)

[eugeneromero]

Even better :D Thank you!