Close codes 1012 and 1013 are valid

[jchampio]

TestSuite currently fails my library because it does not reject the following codes:

• 1004
• 1012
• 1013
• 1014
• 1016
• 1100
• 2000
• 2999

The rationale, I assume, is that anything reserved and not defined by RFC 6455 is considered "invalid". This puts me in an interesting position, though: if my library hardcodes these undefined opcodes to reject them, no one will be able to use it to design new extensions that use new opcodes.

I could always have a "developer mode" for the library that lets undefined opcodes through, and switch it off when running TestSuite. But that's further muddied by the fact that codes 1012 and 1013 are, in fact, defined as of May 2012.

Does Autobahn have a policy for evolving with new opcode definitions, and/or has there been any discussion on allowing a more relaxed test mode?

[oberstet]

1012 and 1013 are not in RFC6455, which is what the test suite tests. But these 2: yeah, one could argue it's nevertheless registered. So I would call it a bug.

design new extensions

If it is not an official extension, you can't use those reserved codes: "1000-2999 Standards Action, Specification Required or IESG Review"

If you are developing an unofficial extension, there are number ranges to use: "4000-4999 Reserved for Private Use"

I think the test suite does probe these .. not sure though right now.

[jchampio]

If it is not an official extension, you can't use those reserved codes

Right, I agree -- but as a library maintainer I can't very well know who is using my module in an "official" vs. "unofficial" capacity.

[oberstet]

Yes, and as a test suite developer, we can't provide test cases for unknown extensions;)

Seriously: e.g. say you introduce new opcodes, complete new semantics etc .. a general testsuite can't do that. The testsuite has stuff for the now official WebSocket compression extension, and that's it.

In any case: when your library is NOT used for writing an extension, but for plain WS, you want (?) to have it conform to the strict letter. And if you want to support your library being used like this, it should have a mode "simple user mode" for that? If you would provide that, you could publish reports for that mode only. Which would make sense IMO.

[jchampio]

In any case: when your library is NOT used for writing an extension, but for plain WS, you want (?) to have it conform to the strict letter. And if you want to support your library being used like this, it should have a mode "simple user mode" for that? If you would provide that, you could publish reports for that mode only. Which would make sense IMO.

Yes, I think so too. I'll go down that route, in which case this issue is now only for the new(er) codes 1012 and 1013. Thanks for the input!

[joakime]

RFC6455 - Section 11.7 - WebSocket Close Code Number Registry

http://tools.ietf.org/html/rfc6455#section-11.7

That references the close code number registry.
Which can be found at
https://www.iana.org/assignments/websocket/websocket.xml#close-code-number-rules

Which was proposed in the hybi mailing list on May 16, 2012 (which was 5 months after RFC6455)
http://www.ietf.org/mail-archive/web/hybi/current/msg09670.html

Most implementations in the wild already know what these 2 close codes are and consider them part of the standard implementation for websocket (even chrome and firefox!)

[oberstet]

fixed in the new release: https://pypi.python.org/pypi/autobahntestsuite/0.7.5

[jchampio]

Works great; thanks!

As an aside, that patch changed the test numbers (e.g. 7.9.8 used to be the close code 1014 test; it is now 1016). Anyone who was using a cases or exclude-cases option to refer to those specific numbers in their test spec will have to update it. I don't know what your compatibility policy is for 0.x, but it might be something to consider after you release a 1.0.

[oberstet]

Regarding test case numbers: I think the unfortunate turth might be that we never really talked about what kind of stability in the produced reports the test suite claims to provide=( Which of course is an issue.

In the particular case, simply removing the tested values (and forgetting to add them on the "positive" list of tested values;) made the test numbers shift. An alternative would be: skip the originally tested values - the test case will appear as "missing". Essentially, we could have a list of "deprecated test numbers" somewhere in the code which are then skipped automatically.

[jchampio]

Essentially, we could have a list of "deprecated test numbers" somewhere in the code which are then skipped automatically.

I think I would (eventually) prefer this approach, once you decide on a "stable" test interface. I was not affected by it for this patch, though, so no worries!

[karthik55]

I am getting ProtocolError With close Code 1000 and Message is Empty, any one come across issue. If so could you tell me the reason for the issue.