Coursier Refresh

[er1c]

• Moved to GitHub Actions (this is also in Remove travis, add GitHub Actions #67)
• Fix region DNS lookup/add test (this is also in Fix default lookup of s3 region based upon dns lookup and add tests #68)
• Add scripted tests-backed by localstack s3 implementation
• Add coursier custom protocol handler.
• Fix some coursier settings

[er1c]

@tpunder I updated this again with additional documentation on the coursier changes and running tests.

This should be a good for a "1.0.0" release.

[tpunder]

This is really hard to follow. It looks like you have refactored the entire project and maybe fixed some bugs along the way (e.g. dns and coursier).

• What is the DNS bug?
• Are the Coursier changes trying to fix s3CredentialsProvider is no longer respected with SBT 1.3.x/Coursier #59? Something else? Does it really take 2,250 lines of new code to fix it?

You also have commits that are stepping over each other. You add .github/workflows/ci.yml in one commit and then change it around a bunch in 2 other commits (including fixing what looks like a copy/paste error)?

[er1c]

@tpunder The primary motivation is #63 which requires generating a maven-metadata.xml with version info for s3 artifacts.

It also tackles:

• https://github.com/coursier/coursier/blob/master/doc/docs/extra.md#extra-protocols
• sbt 1.6.x updates: coursier/sbt-coursier@92e40c2 which is pulled in through the lm-coursier

After sbt 1.3.x, this plugin "sort of works" with coursier, by just falling back to the ivy resolver. This doesn't really take advantage of the coursier benefits when starting up sbt and resolution speed.

There is also significant work to verify sbt plugin compatibility with unit tests - some which are broken in the later sbt 1.6.x releases.

I can recreate the branches for easier review, the start is: #67 and #68

DNS Bug:

[info] [info] S3URLHandler - Looking up AWS Credentials for bucket: maven.custom ...
[info] [info] S3URLHandler - Using AWS Access Key Id: test for bucket: maven.custom
[error] Jan 03, 2022 5:20:05 PM com.amazonaws.util.EC2MetadataUtils getItems
[error] WARNING: Unable to retrieve the requested metadata (/latest/dynamic/instance-identity/document). The requested metadata is not found at http://169.254.169.254/latest/dynamic/instance-identity/document
[error] com.amazonaws.SdkClientException: The requested metadata is not found at http://169.254.169.254/latest/dynamic/instance-identity/document
[error] 	at com.amazonaws.internal.EC2ResourceFetcher.doReadResource(EC2ResourceFetcher.java:89)
[error] 	at com.amazonaws.internal.EC2ResourceFetcher.doReadResource(EC2ResourceFetcher.java:70)
[error] 	at com.amazonaws.internal.InstanceMetadataServiceResourceFetcher.readResource(InstanceMetadataServiceResourceFetcher.java:75)
[error] 	at com.amazonaws.internal.EC2ResourceFetcher.readResource(EC2ResourceFetcher.java:66)
[error] 	at com.amazonaws.util.EC2MetadataUtils.getItems(EC2MetadataUtils.java:407)
[error] 	at com.amazonaws.util.EC2MetadataUtils.getData(EC2MetadataUtils.java:376)
[error] 	at com.amazonaws.util.EC2MetadataUtils.getData(EC2MetadataUtils.java:372)
[error] 	at com.amazonaws.util.EC2MetadataUtils.getEC2InstanceRegion(EC2MetadataUtils.java:287)
[error] 	at com.amazonaws.regions.Regions.getCurrentRegion(Regions.java:109)

Which is rooted in this pre-existing behavior changing: https://github.com/tpunder/fm-sbt-s3-resolver/blob/master/src/main/scala/fm/sbt/S3URLHandler.scala#L414-L416

Which I've fixed and added unit tests for.

[tpunder]

This is ready for a rebase.

Also, your Coursier coursier.cache.protocol.S3Handler (like mine from 2019: 112db50) does not seem to do anything. I changed the code to System.exit(-1) and all of your tests still pass:

package coursier.cache.protocol

import java.net.{URLStreamHandler, URLStreamHandlerFactory}

/**
 * This class name must be coursier.cache.protocol.{protocol.capitalize}Handler
 */
class S3Handler extends URLStreamHandlerFactory {
  def createURLStreamHandler(protocol: String): URLStreamHandler = {
    System.exit(-1)
    throw new AssertionError("S3Handler")
  }
}

[er1c]

This is ready for a rebase.

Also, your Coursier coursier.cache.protocol.S3Handler (like mine from 2019: 112db50) does not seem to do anything. I changed the code to System.exit(-1) and all of your tests still pass:

package coursier.cache.protocol

import java.net.{URLStreamHandler, URLStreamHandlerFactory}

/**
 * This class name must be coursier.cache.protocol.{protocol.capitalize}Handler
 */
class S3Handler extends URLStreamHandlerFactory {
  def createURLStreamHandler(protocol: String): URLStreamHandler = {
    System.exit(-1)
    throw new AssertionError("S3Handler")
  }
}

Hmm not sure, but I definitely get:

[info] [info] - getVersions: library *** FAILED ***
[info] [info]   java.lang.Throwable: download error: Caught java.net.MalformedURLException (unknown protocol: s3) while downloading s3://fm-sbt-s3-resolver-example-bucket/cb851287-f8f0-4722-8e95-2a1cb8356358/com/example/example-lib/maven-metadata.xml. Visit https://get-coursier.io/docs/extra.html#extra-protocols to learn how to handle custom protocols.
[info] [info]   at org.scalasteward.core.coursier.CoursierAlg$$anon$1.$anonfun$getVersions$5(CoursierAlg.scala:95)
[info] [info]   at cats.syntax.FlatMapOps$.$anonfun$$greater$greater$1(flatMap.scala:33)
[info] [info]   at debug @ org.scalasteward.core.coursier.CoursierAlg$$anon$1.$anonfun$getVersions$3(CoursierAlg.scala:95)
[info] [info]   at debug @ org.scalasteward.core.coursier.CoursierAlg$$anon$1.$anonfun$getVersions$3(CoursierAlg.scala:95)
[info] [info]   at >>$extension @ org.scalasteward.core.coursier.CoursierAlg$$anon$1.$anonfun$getVersions$3(CoursierAlg.scala:95)
[info] [info]   at blocking @ org.scalasteward.core.coursier.package$$anon$1.schedule(package.scala:49)
[info] [info]   at blocking @ org.scalasteward.core.coursier.package$$anon$1.schedule(package.scala:49)
[info] [info]   at delay @ org.scalasteward.core.coursier.package$$anon$1.delay(package.scala:31)
[info] [info]   at flatMap @ org.scalasteward.core.coursier.package$$anon$1.bind(package.scala:46)
[info] [info]   at flatMap @ org.scalasteward.core.coursier.package$$anon$1.bind(package.scala:46)

If the protocol isn't added, I'll see about flushing out the specific tests in isolation as well

[er1c]

replacing in piece-meal