Case insensitive filesystems

[rene-d]

The directory tree of the crate mirror uses package names, which can mix lower and uppercase characters.

In the Cargo Book, it says that file names should be in lowercase, with a lowercase prefix (reference). So in the Git crates.io-index also, filenames are lowercase.

Case-insensitive file systems (macOS, Windows...) break Panamax and prevent it from delivering crates.

I will certainly modify to create a lowercase directory tree, but the question of transition arises: should I move files if both exist (example: crates/An/ and crates/an/)? If a script is acceptable, it would be much easier.

[k3d3]

Oh shoot, I never noticed it downloaded crates case-sensitively. Since crates.io-index stores everything lowercase, I think it's fine to modify the creates directory to make everything lowercase.

As far as a non-breaking transition, that might be a bit tougher. The two options I can think of are:

1. Run a one-time function to lowercase everything, and change panamax serve and the nginx config to lowercase the filename,
2. Only make future syncs download to lowercase files, and change panamax serve to try accessing normally, then as lowercase (since otherwise Linux would break, I think). I don't believe there's an nginx equivalent, but I'd be okay dropping that as a requirement.

Any thoughts on either of those two solutions?

[rene-d]

Well, it's a little more complicated than I thought... And finally, I'm not sure that it really prevents panamax from serving crates, if the mirror has been made on a case insensitive filesystem. My problem occurred when I rsync from Linux to a Windows formatted external disk: crates/an has been replaced by crates/An.

cargo seems to use a URI with lowercase and uppercase characters. For example:

GET /crates/An/xi/Anxiety/0.0.0/Anxiety-0.0.0.crate
GET /crates/bv/e-/bve-native/0.0.0-Placeholder/bve-native-0.0.0-Placeholder.crate 

e.g. in "http://mirror/crates/{prefix}/{crate}/{version}/{crate}-{version}.crate", prefix, crate and version use the lower and upper case characters found in the index entry. However, the entry is in an/xi/anxiety file.

No easy solutions... Probably the best one is to change {prefix} to {lowerprefix} which seems to be added two years ago (ref) and to make the prefix part of the crates/ tree in lowercase. The crate name and its version will remain unchanged. It will work because package names are unique regardless of the case of the letters.

Dealing with case-insensitive filenames/filesystems is a pain. With Go modules, it's very complicated too: in the download uri, uppercase letters are replaced with an exclamation mark followed by the letter's lowercase equivalent: github.com/BurntSushi → github.com/!burnt!sushi 😐

[k3d3]

So it looks like in crates.io-index, all crate names are lowercase, but they have different casing in the names within the JSON. Additionally, as far as I can tell, there's no way to register a crate of the same name with differing case.

In that case (pun not intended), it should be just a matter of downloading every crate as lowercase, and modifying warp to lowercase all URLs.

As an aside, it's strange that {lowerprefix} exists, but not {lowercrate}.