User-Agent string lists system before product, contrary to RFC 7231 recommendation

[lognaturel]

As I looked at addressing #3252, I noticed that the User-Agent string currently used in requests does not follow the order recommended by RFC 7231:

By convention, the product identifiers are listed in decreasing order of their significance for identifying the user agent software.

As described by #800 and shown in tests, Collect's User-Agent string looks like:
Dalvik/2.1.0 (Linux; U; Android 5.1.1; hi6210sft Build/LMY47X) org.odk.collect.android/v1.5.1-10-ge20fa334-dirty

The RFC only makes a recommendation so this is not a big deal but I wanted to at least make the decision explicit.

I'm guessing that it was done this way because the application ID looks odd at the head.

Using the application ID is also unorthodox, I think. It has the nice property of ensuring that forks by default provide an identifying User-Agent string. But we could also easily put a string in a config file that forks don't get and have forks be identified as something like "UnidentifiedODKCollectFork". @zestyping discovered in #3253 (comment) that Mapbox uses the application ID as well. Thinking about it more, I think it's the right way to go.

Is it worth changing either the order or the product name or both? I think this is unlikely to be disruptive so I would tend to change it but I may not be thinking of ways this could break downstream tools. Naturally, it would reset quotas based on User-Agent strings.

CC @zestyping @yanokwa

[zestyping]

Putting it first (before Dalvik) seems reasonable. I don't think we need to replace the app ID; I agree it's unusual but it also works well. Just my two cents. 😄

[zestyping]

Side note: I confirmed that the tile requests from OSMDroid do not use the same HTTP infrastructure. Those requests look like this, and would need to have their headers set separately:

GET /2/3/2.png HTTP/1.1
User-Agent: osmdroid
Host: 192.168.1.67:9999
Connection: Keep-Alive
Accept-Encoding: gzip

[lognaturel]

@zestyping I'm hoping you'll enjoy #3254 (writing the description up right now).

[zestyping]

I just tried the same experiment with Mapbox, and discovered that it makes its own User-Agent string as well:

GET /17/21017/50624.png HTTP/1.1
User-Agent: org.odk.collect.android/v1.23.0-beta.4-15-g4a4742c-dirty (3397) Mapbox/7.3.0 (4260644) Android/28 (armeabi-v7a)
Host: 192.168.1.67:9898
Connection: Keep-Alive
Accept-Encoding: gzip

[zestyping]

We can't change the Mapbox one, though. It's statically defined:

https://github.com/mapbox/mapbox-gl-native/blob/24b7b413b5232121ba1599b613152dea124314bf/platform/android/MapboxGLAndroidSDK/src/main/java/com/mapbox/mapboxsdk/module/http/HttpRequestImpl.java

https://github.com/mapbox/mapbox-gl-native/blob/master/platform/android/MapboxGLAndroidSDK/src/main/java/com/mapbox/mapboxsdk/http/HttpIdentifier.java

[lognaturel]

Great to see that Mapbox uses the application ID as well. Upon further reflection, I think we should keep that part.

Seems totally fine for Mapbox to set its own User-Agent string since it's just communicating with its own servers, anyway.

[zestyping]

If/when we give it tile URL templates (e.g. if we use it to replace OSMDroid), it'll fetch those tiles from whatever other servers using that User-Agent. I like their User-Agent string though; it looks totally reasonable to me.

[lognaturel]

Oh, right, I hadn't thought about that! But yes, I agree that it's reasonable even in that context. If servers are doing throttling based on User-Agent string, they probably have some heuristics for identifying the application part which hopefully wouldn't get fooled by that switch. Though I think we're fine on terms of service other than openstreetmap.org

[lognaturel]

The goal is to get to something that looks like Dalvik/2.1.0 org.odk.collect.android/v1.5.1-10-ge20fa334-dirty (Linux; U; Android 5.1.1; hi6210sft Build/LMY47X)

[SaumiaSinghal]

@opendatakit-bot claim

[lognaturel]

I wrote above that "The goal is to get to something that looks like Dalvik/2.1.0 org.odk.collect.android/v1.5.1-10-ge20fa334-dirty (Linux; U; Android 5.1.1; hi6210sft Build/LMY47X)" but I now don't understand why. It seems that if we're going by decreasing order of usefulness for identifying the agent, it should be org.odk.collect.android/v1.5.1-10-ge20fa334-dirty Dalvik/2.1.0 (Linux; U; Android 5.1.1; hi6210sft Build/LMY47X).

@SaumiaSinghal, @seadowg, can either of you think of a reason of why I may have split the host identifier? I wonder whether I was just going too quickly and made a mistake.

[seadowg]

No actually! Your new example makes more sense to me.

[SaumiaSinghal]

Yes, also the tests don't fail like that. :)