Upgrade ktlint (0.40.0 ➜ 0.42.1)

[bittner]

Describe the bug
The Kotlin linter ktlint is outdated. A few substantial rules were added since the 0.40.0 release.

To Reproduce
Steps to reproduce the behavior:

1. See generated linter-versions.json.

Expected behavior
The latest stable release of klint is available with Mega-Linter.

Additional context
I'd be ready contribute the upgrade via a PR. Can you advise which file(s) to update before running bash build.sh for this change?

[bittner]

Is this issue obsolete as of #860 (comment)?

If yes, is a new mega-linter release needed to push the change out to the public?

[lars-reimann]

If I understand the auto-update correctly, it works by building a new Docker image, reading the version of each installed linter, and comparing those versions with the previous versions (see this function). By default, versions are retrieved using a --version subcommand, which is fine for ktlint. If any differences are found, it updates the changelog and creates a pull request. It also pushes the Docker image to Docker Hub if requested.

The Docker images mentioned above are created by Dockerfiles, but these in turn are automatically created by build.py from descriptors, like this one for Kotlin linters. In there we find the installation instruction for ktlint:

    install:
      dockerfile:
        - |
          RUN curl --retry 5 --retry-delay 5 -sSLO https://github.com/pinterest/ktlint/releases/download/0.40.0/ktlint && \
              chmod a+x ktlint && \
              mv "ktlint" /usr/bin/

As we can see, the version is fixed to 0.40.0 so the automatic update is effectively disabled. To enable automatic updates again, the line https://github.com/pinterest/ktlint/releases/download/0.40.0/ktlint needs to be changed to https://github.com/pinterest/ktlint/releases/latest/download/ktlint. For this to work, the downloadable file always needs to be called ktlint and the remaining installation instructions have to stay the same. But this seems to be the case for now.

Personally, I'd also like ktlint to be updated, since version 0.4.0 is unable to handle value classes. That said, there's probably a reason it was fixed to this version. If the auto-update is not possible for some reason, maybe ktlint could at least be bumped to a higher fixed version, though.

[nvuillam]

@lars-reimann I see you get very well how MegaLinter auto-updates itself :)

I don't see why we would freeze the version of ktlint, so let's see whtat happens with ##976 :)

[nvuillam]

I see that some time ago I downgraded on purpose... sometimes new versions are crashing, let's see what CI & test classes will say :)

[nvuillam]

I now remember why I downgraded ktlint...

ktlinters, any chance you can help me about this error ?

https://github.com/megalinter/megalinter/runs/4181048292?check_suite_focus=true

+----MATCHING LINTERS-+----------+----------------+------------+
| Descriptor | Linter | Criteria | Matching files | Format/Fix |
+------------+--------+----------+----------------+------------+
| KOTLIN     | ktlint | .kt|.kts | 1              | no         |
+------------+--------+----------+----------------+------------+

MegaLinter flavor is "all", no need to check match with linters
[ktlint] command: ['ktlint', '/tmp/lint/.automation/test/kotlin/kotlint_good_1.kt']
[ktlint] CWD: /
[ktlint] result: 1 Exception in thread "main" java.util.concurrent.ExecutionException: java.lang.ExceptionInInitializerError
	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
	at java.util.concurrent.FutureTask.get(FutureTask.java:192)
	at com.pinterest.ktlint.KtlintCommandLine.parallel(Main.kt:576)
	at com.pinterest.ktlint.KtlintCommandLine.parallel$default(Main.kt:548)
	at com.pinterest.ktlint.KtlintCommandLine.lintFiles(Main.kt:298)
	at com.pinterest.ktlint.KtlintCommandLine.run(Main.kt:262)
	at com.pinterest.ktlint.Main.main(Main.kt:70)
Caused by: java.lang.ExceptionInInitializerError
	at com.pinterest.ktlint.internal.FileUtilsKt.lintFile(FileUtils.kt:172)
	at com.pinterest.ktlint.KtlintCommandLine.process(Main.kt:395)
	at com.pinterest.ktlint.KtlintCommandLine.access$process(Main.kt:89)
	at com.pinterest.ktlint.KtlintCommandLine$lintFiles$3.invoke$lambda-0(Main.kt:289)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: java.util.zip.ZipException: invalid distance too far back
	at org.jetbrains.kotlin.com.intellij.util.ExceptionUtil.rethrow(ExceptionUtil.java:116)
	at org.jetbrains.kotlin.com.intellij.ide.plugins.PluginDescriptorLoader.loadDescriptorFromJar(PluginDescriptorLoader.java:107)
	at org.jetbrains.kotlin.com.intellij.ide.plugins.PluginManagerCore.registerExtensionPointAndExtensions(PluginManagerCore.java:1401)
	at org.jetbrains.kotlin.com.intellij.core.CoreApplicationEnvironment.registerExtensionPointAndExtensions(CoreApplicationEnvironment.java:266)
	at org.jetbrains.kotlin.cli.jvm.compiler.KotlinCoreEnvironment$Companion.registerApplicationExtensionPointsAndExtensionsFrom(KotlinCoreEnvironment.kt:575)
	at org.jetbrains.kotlin.cli.jvm.compiler.KotlinCoreEnvironment$Companion.createApplicationEnvironment(KotlinCoreEnvironment.kt:540)
	at org.jetbrains.kotlin.cli.jvm.compiler.KotlinCoreEnvironment$Companion.getOrCreateApplicationEnvironment(KotlinCoreEnvironment.kt:497)
	at org.jetbrains.kotlin.cli.jvm.compiler.KotlinCoreEnvironment$Companion.getOrCreateApplicationEnvironmentForProduction(KotlinCoreEnvironment.kt:485)
	at org.jetbrains.kotlin.cli.jvm.compiler.KotlinCoreEnvironment$Companion.createForProduction(KotlinCoreEnvironment.kt:429)
	at com.pinterest.ktlint.core.internal.KotlinPsiFileFactoryKt.initPsiFileFactory(KotlinPsiFileFactory.kt:32)
	at com.pinterest.ktlint.core.KtLint.<clinit>(KtLint.kt:42)
	... 8 more
Caused by: java.util.zip.ZipException: invalid distance too far back
	at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:164)
	at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
	at java.io.BufferedInputStream.read(BufferedInputStream.java:265)
	at org.jetbrains.kotlin.com.intellij.openapi.vfs.CharsetToolkit.inputStreamSkippingBOM(CharsetToolkit.java:116)
	at org.jetbrains.kotlin.com.intellij.openapi.util.JDOMUtil.load(JDOMUtil.java:351)
	at org.jetbrains.kotlin.com.intellij.ide.plugins.PluginDescriptorLoader.loadDescriptorFromJar(PluginDescriptorLoader.java:87)
	... 17 more

[lars-reimann]

Seems to be this issue: pinterest/ktlint#1271.

[nvuillam]

it does not seems there is a fix... as I have several linters depending on java, there is too much impacts if I change the java version, so I thin we'll wait for a ktlint fix, and if really it never comes, maybe install another java version and use it only to call ktlint

[lars-reimann]

v0.43.1 fixed the issue with JDK 1.8.

[bittner]

Is this included in the megalinter/megalinter-java:v5 Docker image already?

[lars-reimann]

No, but it might be worthwhile to run the tests for #976 again to see if this can now be integrated.

[bittner]

Cool! – Is this planned to integrate for v5, or is this better expected for v6?

[nvuillam]

v6 will have major changes and may be released in several months, but v5 will continue to live meanwhile :)

I just tried again to use latest ktlint, let's see what CI says :)

#976