feat(terraform_docs): Add support for custom markers to better support other formats than Markdown

[tarfu]

Put an x into the box if that apply:

• This PR introduces breaking change.
• This PR fixes a bug.
• This PR adds new functionality.
• This PR enhances existing functionality.

Description of your changes

This adds support to pass custom markers.
This helps to improve support for other formats like asciidoc which you can use if you use a config for terraform-doc.

How can we test changes

Add your custom marker to a README and run the hook with the options. You may also want to change the template used by terraform-doc

terraform_docs.sh --hook-config=--path-to-file=README.adoc --hook-config="--custom-marker-begin=// BEGIN_TF_DOCS" --hook-config="--custom-marker-end=// END_TF_DOCS" --args=--config=.terraform-docs.yml ./path/to/file.tf

[yermulnik]

Thanks for your contribution. Please have a look into comments I added.
I'd also think of adding this as a single configuration parameter to avoid use of mix of standard and custom markers. Something like --custom-markers="// BEGIN;// END". This may decrease clarity and readability and separator needs a more thorough think though.

[yermulnik]

Looks good. @MaxymVlasov What do you think?

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 96.38%. Comparing base (92fbec7) to head (9c6dc16).
Report is 2 commits behind head on master.

✅ All tests successful. No failed tests found.

Additional details and impacted files

@@           Coverage Diff           @@
##           master     #752   +/-   ##
=======================================
  Coverage   96.38%   96.38%           
=======================================
  Files          10       10           
  Lines         249      249           
  Branches        7        7           
=======================================
  Hits          240      240           
  Misses          9        9           

Flag	Coverage Δ
CI-GHA	96.38% <ø> (ø)
MyPy	91.05% <ø> (ø)
OS-Linux	96.38% <ø> (ø)
OS-Windows	100.00% <ø> (ø)
OS-macOS	100.00% <ø> (ø)
Py-3.10.11	100.00% <ø> (ø)
Py-3.10.16	100.00% <ø> (ø)
Py-3.11.11	100.00% <ø> (ø)
Py-3.11.9	100.00% <ø> (ø)
Py-3.12.8	100.00% <ø> (ø)
Py-3.13.1	96.38% <ø> (ø)
Py-3.9.13	100.00% <ø> (ø)
Py-3.9.21	100.00% <ø> (ø)
VM-macos-13	100.00% <ø> (ø)
VM-macos-14	100.00% <ø> (ø)
VM-ubuntu-24.04	100.00% <ø> (ø)
VM-ubuntu-latest	91.05% <ø> (ø)
VM-windows-2025	100.00% <ø> (ø)
pytest	100.00% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[MaxymVlasov]

Hmm, you probably had // BEGINNING OF PRE-COMMIT-TERRAFORM DOCS HOOK and // END OF PRE-COMMIT-TERRAFORM DOCS HOOK?

trace: replace_old_markers terraform_docs main main
       terraform_docs.sh:45: sed -i -e 's/^<!-- BEGINNING OF PRE-COMMIT-TERRAFORM DOCS HOOK -->$/\/\/ BEGIN_TF_DOCS/' README.adoc

trace: replace_old_markers terraform_docs main main
       terraform_docs.sh:46: sed -i -e 's/^<!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK -->$/\/\/ END_TF_DOCS/' README.adoc

[MaxymVlasov]

Also, I found that

formatter: asciidoc table
output:
  file: README.adoc
  mode: inject
  template: |-
    // BEGIN_TF_DOCS
    {{ .Content }}
    // END_TF_DOCS

repos:
- repo: https://github.com/antonbabenko/pre-commit-terraform
  rev: fc53d0a10b7846fa8571ddd504660cbe5c6110d6
    - id: terraform_docs
      args:
      - --hook-config=--add-to-existing-file=true     # Boolean. true or false
      #- --hook-config=--custom-marker-begin=// BEGIN_TF_DOCS
      #- --hook-config=--custom-marker-end=// END_TF_DOCS

When you have empty README.adoc, works fine without custom-marker-begin and custom-marker-end options.

And it produces identical result as for rev: 1.96.3

Can you please elaborate on what exactly you tried to fix/add by this PR?

[tarfu]

If you got a non-empty file, it won't work for me without the custom markers.

= Terraform Module
This is a short description of the module and below this should be the terraform docs

// BEGIN_TF_DOCS

// END_TF_DOCS

With this snippet in an existing file, I won't get any output with the latest release/master.
If you add the standard markers below (which are then shown in the rendered asciidoc as it is not parsing them as comments), you'll get the output.

= Terraform Module
This is a short description of the module and below this should be the terraform docs

// BEGIN_TF_DOCS

// END_TF_DOCS

<!-- BEGIN_TF_DOCS -->
<!-- END_TF_DOCS -->

This will result in:

= Hub Management

This folder contains the Terraform code for deployment of the Hub resources, including Virtual WAN and Intermediate Hubs.

// BEGIN_TF_DOCS
=== Requirements

[cols="a,a",options="header,autowidth"]
|===
|Name |Version
|[[requirement_terraform]] <<requirement_terraform,terraform>> |~> 1.7
|===

=== Providers

No providers.

=== Modules

No modules.

=== Resources

No resources.

=== Inputs

No inputs.

=== Outputs

No outputs.
// END_TF_DOCS

<!-- BEGIN_TF_DOCS -->
<!-- END_TF_DOCS -->

This also works if you only have a begin marker, as it is the only thing the grep looks for.
That is the reason for the custom marker. You could also try to parse the config YAML and interpret what the markers are, but this would be a bit too much, I think.

[MaxymVlasov]

I see what you're talking about

Interesting, that specified by you example of

= Terraform Module
This is a short description of the module and below this should be the terraform docs

// BEGIN_TF_DOCS

// END_TF_DOCS

work well with

formatter: asciidoc table
output:
  file: README.adoc
  mode: inject
  template: |-
    // BEGIN_TF_DOCS
    {{ .Content }}
    // END_TF_DOCS

repos:
- repo: https://github.com/antonbabenko/pre-commit-terraform
  rev: v1.96.3
  hooks:
    - id: terraform_docs
      args:
      - --hook-config=--add-to-existing-file=true     # Boolean. true or false
      - --args=--config=.terraform-docs.yml

but not with - --hook-config=--add-to-existing-file=false which is default

and

      args:
      - --hook-config=--create-file-if-not-exist=true # Boolean. true or false
      - --args=--config=.terraform-docs.yml

produce

# dir-name

<!-- BEGIN_TF_DOCS -->
<!-- END_TF_DOCS -->

// BEGIN_TF_DOCS
== Requirements
...
// END_TF_DOCS

---

So if --hook-config=--add-to-existing-file=true able to use right templating in 1.96.3 from .terraform-docs.yml, I'm curious why default can't do the same without extra configs to hook? There could be a bug somewhere 🤔 => there should be a better way than proposed by this PR.

If you have time, please investigate what happens, using PCT_LOG=trace pre-comit run command and let us know if you'll find anything weird and try to fix it if possible. If not, I'll check later by my own

[tarfu]

--hook-config=--add-to-existing-file=true skips the marker check, so it will always run tf-docs if it is set to false, it will check if the marker we expect is present and only if it is we will run tf-docs.
As we don't parse the config and try to extract the marker from there, I think the behavior is expected. If it is desired like that is another question.

You could try to parse the template in the config, grab the first line of it, check if it is not {{ .Content }} and hope everyone is using the YAML string style we expect. Which leads to other issues.

[MaxymVlasov]

Am I right that you anyway need to specify template: in .terraform-docs.yml if you not use <!-- BEGIN_TF_DOCS --> markers, which looks like are set by default?
https://terraform-docs.io/user-guide/configuration/

So, we actually need to find all formatter: options that not support HTML comments, and add check that if such formater provided - run terraform-docs same way as for --hook-config=--add-to-existing-file=true

[tarfu]

Yeah, it is set up defaulting to Markdown and HTML comments, but there would be other options in MD as well.

If you run it like it was --hook-config=--add-to-existing-file=true you would lose the ability to skip files without the marker. Which is funnily enough the behavior I want/need :D

[MaxymVlasov]

You would lose the ability to skip files without the marker

Ah, yes. And parse it from template will be problematic.
Your PR looks perfectly fine then.

Just let make file creation header (#) be customizable too then for - --hook-config=--create-file-if-not-exist=true, in

pre-commit-terraform/hooks/terraform_docs.sh

Lines 199 to 204 in e01b4de

	# Use of insertion markers, where there is no existing README file
	{
	echo -e "# ${PWD##*/}\n"
	echo "$insertion_marker_begin"
	echo "$insertion_marker_end"
	} >> "$output_file"

as

# dir-name-here

// BEGIN_TF_DOCS
== Requirements
...
// END_TF_DOCS

will be annoying to fix.
There could be sense to make ${PWD##*/} customizable as well, but it out of scope for sure (and maybe not needed at all).

So, let's reiterate on same question but for different use-case - can we just get header data from formatter like doc_header_prefix[${formatter}] or we need to hardcode it as well via --hook-config=--custom-doc-header?

[tarfu]

I think --hook-config=--custom-doc-header would be the way to go, as different formatters have multiple options to go with and would be the easiest way to be customized.

Another way could be just, letting terraform-docs create the file would be an option as well, which should work if there is a tf file.
This would enable us to throw away one marker and only test against the beginning marker.

[MaxymVlasov]

Please add --hook-config=--custom-doc-header then, with default to #
Don't forget docs section as well.

Another way could be just, letting terraform-docs create the file would be an option as well, which should work if there is a tf file. This would enable us to throw away one marker and only test against the beginning marker.

Interesting solution, but let someone comes with feature request, as I don't see who could use it :)

[MaxymVlasov]

Amazing job, thank you.
Btw, as we are here, just a "quick interview": What would be easier for you to contribute: hooks on bash or on Python? If there are any preferences, ofc.
Structure for hooks will be the same, in terms that all hook-specific code will be located in the same file, similar to what we have in bash right now.

[tarfu]

I think python would be more comfortable and would enable to parse yaml for example to have less redundant things in config and arguments of tools.

[antonbabenko]

This PR is included in version 1.97.0 🎉