[Anatomy] Add configlet documentation

[ErikSchierboom]

No description provided.

[iHiD]

Maybe move this to anatomy/tracks/configlet.md?

[ErikSchierboom]

Maybe move this to anatomy/tracks/configlet.md?

It should have been. My bad

[ee7]

I think we should mention that the UUIDs should be valid version 4 UUIDs, and probably with the additional constraint of:

- the UUID does not contain an uppercase character

We've had a clear preference towards lowercase-only in the past.

[ErikSchierboom]

@ee7 Updated!

[ee7]

Nice.

Another thing about UUIDs: I think we should be more specific than saying a UUID must be "unique".

In particular:

1. Should configlet lint indicate failure for a config.json file that contains a UUID that exists on another track?
2. Should configlet lint indicate failure for a config.json file that contains a UUID that exists in problem-specifications?

If the answer to 1 is yes, then we'll have to fix the already-existing duplicate UUIDs so that configlet lint is useful; it would currently fail on otherwise flawless config.json files for at least the following tracks:

bash
crystal
dart
elixir
gleam
groovy
java
javascript
kotlin
python
ruby
tcl

See this gist for a list of UUIDs that are shared between current v2 track repos.

And if the answer to 1 is "yes", but we also allow skipping the checks for Exercism-wide uniqueness, then I guess I lean towards "make it hard to misuse". That is, make configlet lint include the online checks, and allow using --offline (rather than making it work offline by default, and adding --online). Then it's consistent with the existing configlet sync --offline.

[ee7]

More questions:
3. Did we decide against the rule of "each JSON object must contain only the specified keys"?
4. Is the answer to 3 the same for every .json file that we lint?

See: exercism/v3#2771 (comment)

[iHiD]

Another thing about UUIDs: I think we should be more specific than saying a UUID must be "unique".

I think there is scope here for us to check that at the database level at sync time, rather than at configlet time. If configlet checks the scope (track config) for uniqueness, the website can check the rest. I think that will keep configlet much lighter (rather than needing to check 52+ different repositories for uuids).

---

We will fix any duplicate uuids during ETL.

[ErikSchierboom]

3. Did we decide against the rule of "each JSON object must contain only the specified keys"?

I'm fine with tracks adding track-specific information to the config.json file (or other JSON files), as long as the required keys are there.

4. Is the answer to 3 the same for every .json file that we lint?

Probably.

[ee7]

check that at the database level at sync time, rather than at configlet time.

Does this mean the best we can do is "don't sync after a bad PR is merged, and automatically open an issue in the offending repo"?

If so, I think it's better to catch problems at CI-time when possible. And requiring every track to add a CI check outside configlet seems painful, even if we added a new action to exercism/github-actions. So my thinking is that it should be in configlet, even if we always need to have the sync-time check anyway (to prevent races).

The naive "check every repo" approach is indeed wasteful, but it doesn't have to be slow for the end-user. We can do some combination of:

• Just download the latest config.json from each track, rather than doing a git (shallow) clone
• Make requests concurrently
• Make configlet lint run the Exercism-wide check only when it detects that it is running in CI (adding an extra flag so that users can still run it locally if they want).

A more sophisticated long-term option is to build an API endpoint that receives a list of UUIDs and returns a list of all that are bad.

[ee7]

Another question:
5. What should configlet lint do if it sees a uuid key with the value null? There are some of these in the v3 repo, in the config.json files for common-lisp and go. Currently we have the wording:

The "concepts[].uuid" value must be a unique, lowercased v4 UUID string

[ErikSchierboom]

5. What should configlet lint do if it sees a uuid key with the value null? There are some of these in the v3 repo, in the config.json files for common-lisp and go. Currently we have the wording:

Personally I think the null cases are only really useful while we're building v3, and that we should not allow these once we've exploded the v3 repo.

[ee7]

And a question prompted by @iHiD's comment in exercism/problem-specifications#1735 (comment)

I'd quite like to have ${PREFIX}-${v4 UUID} everywhere. Where the prefix signifies the context (e.g. this would be PSTC (Problem specs test case) or similar)

6. Should we prefix UUIDs with their context?

I think I lean towards "no".

It's tempting to think that it's nice to look at a UUID and know where it comes from. But is that actually useful after we've fixed all the bad UUIDs and added checks so that more can't be added?

We'd have to:

• decide whether we want a different prefix for each track (and between concept/practice exercises), and if so, what those prefixes would be.
• check in CI that the correct prefix is used everywhere (otherwise it's guaranteed that people will typo a track name, or write e.g. PTSC).
• decide whether configlet uuid should include a prefix.

Furthermore, I think it doesn't really solve the duplicated UUIDs problem - I'd argue that we shouldn't merge a PR that adds e.g.
python-68a4a063-a4d0-4af6-8957-b98d90639689
when there already exists
PTSC-68a4a063-a4d0-4af6-8957-b98d90639689
because that's confusing, and makes it harder to search Exercism-wide by UUID (e.g. to find tracks that implement a test case).

[iHiD]

Approving this so Erik can merge when y'all ready. I'm not reading the issue, but let me know if I need to :)

[ErikSchierboom]

I'm gonna merge this and we can update things later.