Document special collations

[bdarnell]

Our collation docs link to a list of languages that we support, but there's no mention of other modifiers. One useful modified locale is en_u_ks_level2, which is case-insensitive english (I think the _u_ks_level2 modifier can be added to other languages too to use their case-sensitivity rules). I'm not sure if there are other useful modifiers that are supported by our collation package. These modifiers are standardized, but the references I've found don't seem to agree with each other. Unicode TR 35 looks like the authority on this, although it doesn't define the level2 syntax. collate/option.go is where these modifiers get parsed.

[jseldess]

@sploiselle, since you documented collations originally, I suspect this would be a very quick update for you. Feel free to unassign if you feel it's more involved and needs to wait until later.

[jseldess]

Helpful supplementary thread on the forum: https://forum.cockroachlabs.com/t/case-insensitive-collations/926/4

[jseldess]

From @justinj, copied over from #2430:

I actually have no idea how this stuff works, but there's some magical collations that just seem to exist, like en_u_ks_level1, which is English text ignoring case. I can't find anything in our docs on this sub-language of how to specify additional rules for collations, but it seems WAY too useful to not have documented on the collations page (even just for the special case of only case-insensitivity).

This page seems to have some info on it.

cc @mjibson
cc @knz because it seems you reviewed some of the PRs for collations.

[jseldess]

This goes back to 1.1, but moving it to the 2.1 milestone.

[bdarnell]

PostgreSQL 12 just added support for these collations, which they call "non-deterministic collations": https://www.postgresql.org/docs/12/collation.html

Their syntax for this is different from ours and they just point out to the unicode docs for more detail. We should be sure to include the phrase "non-deterministic collations" when we document this for SEO purposes.

[ericharmeling]

@bdarnell

Their syntax for this is different from ours and they just point out to the unicode docs for more detail. We should be sure to include the phrase "non-deterministic collations" when we document this for SEO purposes.

Do we support both deterministic and non-deterministic collations?

[bdarnell]

Yes, we have both deterministic and non-deterministic collations. But looking back at this, I think these terms are fairly obscure - users don't need to care whether a collation is deterministic or not. The only real reason to use these terms is for a user going from "postgres 12 introduces support for non-deterministic collations" to searching for whether cockroachdb supports non-deterministic collations.

[ericharmeling]

The PG docs talk about nondeterministic collations in the context of CREATE COLLATION (deterministic=false), which I don't believe we support.

So I'm still a little confused about how best to document this. For now, I can go ahead and just add a generic note that we support both deterministic and nondeterministic collations. (see #8523)