HTTP: consider quoted charset

[asterite]

Fixes #6353

[jhass]

Do we also want to allow single quotes (') here?

[asterite]

I'm not sure single quotes are allowed, but if others agree I can add it to this PR.

[straight-shoota]

No, single quotes should not be interpreted as string delimiters, see definition of quoted-string in RFC 723 §3.2.6. Most other implementations I've looked at don't interpret single quotes. For example: Go test line 196

[RX14]

Don't a lot of HTTP headers have the standard value; optional-key="value" format? Is there an RFC or standard of these headers - or even an unofficial standard - to implement a parser for them?

[straight-shoota]

@RX14 There are a ton of RFCs about mime type syntax. The most recent one is RFC 7321.

As already mentioned, I have a working parser implemented, it just needs #5765 and a little bit of finishing 👍

[RX14]

I'm not talking about mime types - i'm talking about the generic HTTP header value syntax. We already need this for here and thats nothing to do with mime types. The syntax dates from the email header days.

[straight-shoota]

Yeah, I know what this is about. It's not specific to any protocol, neither HTTP nor mail. I'm not sure if there's a proper name, but this is commonly described as mime type (examples: .NET, Java, Rust, Go)

[straight-shoota]

And despite any of this, it's still true that Crystal needs a proper parser implementation.

[RX14]

My bad, that was mime types; i'm a bit tired. I meant that mime types, set-cookie, and content-disposition have the same approximate header syntax with one or more initial values then the same primaryvalue; optionalkey=value; optionalkey=value format. Does that syntax have a name and should we have a generic parser for HTTP headers of this style?

[straight-shoota]

As far as I know, this is called mime type, despite the fact that applications like Set-Cookie have nothing to to with mime. And yes, we should have a generic parser for this.

[RX14]

I've never seen it named as such and I can't imagine it being called that since it would be so confusing. "mime type" already means a specific concrete RFC and concept.

[straight-shoota]

Sry, I meant media type. Doesn't make any difference, though. I guess it's just that this concept initially comes from mime type declarations and was adopted for other uses as well, without there being a proper name for this syntax.

Implementation wise, it is typically named a MIME media type parser. This implementation can be applied to non-MIME contexts such as Set-Cookie or they can use a more specific, custom parser. Go for example uses ParseMimeType for Content-Type, Content-Disposition but a custom implementation which only recognizes a fixed set of attributes for Set-Cookie.

[RX14]

I'd never find such a parser as a user if I had to think "mime". So I think we shouldn't name it that way. It should be in the generic HTTP module.

[ysbaddaden]

Merging the fix. Discussion feels unrelated or that it could come later.

[RX14]

Yep, discussion was unrelated, sorry.