port Zinc-WebSocket packages to GemStone

[dalehenrich]

No description provided.

[dalehenrich]

Paul (@pdebruic),

Just this week, Sebastian (@HeSe) has blown the dust off the Cryptography project[1] to support his port of Web-Sockets to GemStone[2].

Since my inclination is to move projects off of GemSource and onto GitHub when we start active work, I started poking around the current state of Cryptography, since the last time that project had been touched was 4 years ago.

On SqueakSource there is a Cryptography project[3] that appears to have a newer versions of the Cryptography and PasswordHashingFFI packages. this project has a number of interestingly named packages that may or may not be useful to have ported to GemStone. The Blowfish package is interesting because there's a pretty recent version not to mention the fact that you've got a blowfish project up on github[4].

In the pharo universe it seems that there is some recent activity with ApplicationSecurity[5] and Nacl[6] projects.

I am a little baffled by the lay of the land as there seem to be quite a variety of apis available and I'm not sure which ones apply to which problems ...

Soo ... I am curious about your thoughts on these various projects and the idea of consolidating the current work on GsDevKit ... part of the trick here is to define the "current work" and also look at where we "should be" ... Of course, I want to support existing packages that are used in existing projects and I'd like to start addressing performance issues ...

I would like to support packages that are available on Pharo and Squeak and get them ported to GemStone... If GemStone needs to add fundamental capabilities at the image level then now is a good time as we start work on the 3.3 release ...

Dale

[1] http://seaside.gemtalksystems.com/ss/Cryptography.html
[2] #58
[3] http://www.squeaksource.com/Cryptography.html
[4] https://github.com/pdebruic/blowfish-smalltalk
[5] http://www.smalltalkhub.com/mc/hernan/ApplicationSecurity/main
[6] http://www.smalltalkhub.com/mc/tonyg/Crypto-Nacl/main

[jbrichau]

FYI: In the summer of 2013, I (urgently) needed an implementation of SHA256 in GemStone. I tried to blow the dust of the Cryptography package for GS but I had to limit myself to the one or two packages (or parts thereof) of the project I needed to run SHA256.

So, not all is in ruins but I did notice that a lot of the things were not working. Also in Pharo, the project does not load anymore as various parts have been copy/pasted into the image.

[HeSe]

Hi Johan,

Well, I just had a look.
It seems the hash functions are still doing fine.
Many errors are due to a different hex implementation in both Gemstone
portability layers and Squeak/Pharo

Whereas in gemstone hex is delivered in lowercase it is uppercase in Pharo.
The values are right though.
We might need to clarify if there is a need for Gemstone to have hex in
class ByteArray handling lowercase representation of a hex.

Would it make sense to at least do obvious changes and add Cryptography
as prerequisite to Zinc-Websockets for Gemstone?

Sebastian

On 2014-11-02 7:36 AM, Johan Brichau wrote:

FYI: In the summer of 2013, I (urgently) needed an implementation of
SHA256 in GemStone. I tried to blow the dust of the Cryptography
package for GS but I had to limit myself to the one or two packages
(or parts thereof) of the project I needed to run SHA256.

So, not all is in ruins but I did notice that a lot of the things were
not working. Also in Pharo, the project does not load anymore as
various parts have been copy/pasted into the image.

—
Reply to this email directly or view it on GitHub
#58 (comment).

[pdebruic]

Hi Dale,

We should probably hop on the cryptography list: http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/cryptography I've cc'd it for now.

I still use the PasswordHashingFFI code on squeak/pharo and GemStone. Its pretty basic, but useful. With libxcrypt installed, & on linux, it lets a user hash passwords using bcrypt or SHA-512 (but the password hashing version). A primer on why you'd want to do that is here: http://codahale.com/how-to-safely-store-a-password/ . Another good, slow, password hashing algorithm is scrypt (https://en.wikipedia.org/wiki/Scrypt) but we don't have an FFI library for it yet. In addtiion to Hernan's Application Security project Pierce Ng has some projects he mentions on his blog: http://www.samadhiweb.com/tags/cryptography

Blowfish is a 2 way encryption algorithm which allows someone to encrypt something and then decrypt it later. The version I wrote only encrypts an 8 byte block at a time. Anything longer looks encrypted but isn't. To do longer things someone would need to add code that does cipher block chaining (CBC) (e.g. https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation#Cipher-block_chaining_.28CBC.29) I never added the CBC code because I wrote the Blowfish code while investigating having a Smalltalk bcrypt password hashing implementation. Bcrypt uses blowfish while hashing passwords for long term storage. The Smalltalk implementation ended up being more than 5,000x slower than the FFI version and not usable in practice. From my point of view right now the Blowfish code shouldn't be used for anything without adding the CBC code and even then a person may be better off using a newer algorithm than Blowfish for 2 way encryption because of both hardware and algorithm advances (e.g. the NaCl library you mention). That being said the Smalltalk Blowfish implementation does work for 8 byte blocks and is only Smalltalk, no FFI.

I think its a good idea for some tidying in the Cryptography ecosystem, but don't have a strong idea of what the end state (packages and their location on the internet) should be.

Hope this helps

Paul

On Nov 1, 2014, at 11:20 AM, Dale Henrichs [email protected] wrote:

Paul (@pdebruic),

Just this week, Sebastian (@HeSe) has blown the dust off the Cryptography project[1] to support his port of Web-Sockets to GemStone[2].

Since my inclination is to move projects off of GemSource and onto GitHub when we start active work, I started poking around the current state of Cryptography, since the last time that project had been touched was 4 years ago.

On SqueakSource there is a Cryptography project[3] that appears to have a newer versions of the Cryptography and PasswordHashingFFI packages. this project has a number of interestingly named packages that may or may not be useful to have ported to GemStone. The Blowfish package is interesting because there's a pretty recent version not to mention the fact that you've got a blowfish project up on github[4].

In the pharo universe it seems that there is some recent activity with ApplicationSecurity[5] and Nacl[6] projects.

I am a little baffled by the lay of the land as there seem to be quite a variety of apis available and I'm not sure which ones apply to which problems ...

Soo ... I am curious about your thoughts on these various projects and the idea of consolidating the current work on GsDevKit ... part of the trick here is to define the "current work" and also look at where we "should be" ... Of course, I want to support existing packages that are used in existing projects and I'd like to start addressing performance issues ...

I would like to support packages that are available on Pharo and Squeak and get them ported to GemStone... If GemStone needs to add fundamental capabilities at the image level then now is a good time as we start work on the 3.3 release ...

Dale

[1] http://seaside.gemtalksystems.com/ss/Cryptography.html
[2] #58
[3] http://www.squeaksource.com/Cryptography.html
[4] https://github.com/pdebruic/blowfish-smalltalk
[5] http://www.smalltalkhub.com/mc/hernan/ApplicationSecurity/main
[6] http://www.smalltalkhub.com/mc/tonyg/Crypto-Nacl/main

—
Reply to this email directly or view it on GitHub.

[dalehenrich]

Thanks @pdebruic, @HeSe and @jbrichau ... I've subscribed to the cryptography list so that I can follow any responses there, but I I am inclined to think that we in the GemStone community are the only ones with a vested interest in a cross-platform cryptography library so it seems that for us the best solution would be to create a Cryptography project on GsDevKit and try to maintain the subset of the library that is being actively used by the GemStone community (with tests on Pharo and Squeak) and expand the subset as needed ... we should try to mirror the new packages on the SqueakSource list as we do with zinc, seaside, etc.

If the discussion on the cryptography list heats up, I'm inclined to follow their liead ...

What do you guys think?

[jbrichau]

Sound like a plan.
I’m going to have to focus on Seaside and XMLParser for now, but I can definitely already check out the status of the packages of Cryptography that we are using.

Johan

On 03 Nov 2014, at 19:07, Dale Henrichs [email protected] wrote:

Thanks @pdebruic https://github.com/pdebruic, @HeSe https://github.com/HeSe and @jbrichau https://github.com/jbrichau ... I've subscribed to the cryptography list so that I can follow any responses there, but I I am inclined to think that we in the GemStone community are the only ones with a vested interest in a cross-platform cryptography library so it seems that for us the best solution would be to create a Cryptography project on GsDevKit and try to maintain the subset of the library that is being actively used by the GemStone community (with tests on Pharo and Squeak) and expand the subset as needed ... we should try to mirror the new packages on the SqueakSource list as we do with zinc, seaside, etc.

If the discussion on the cryptography list heats up, I'm inclined to follow their liead ...

What do you guys think?

—
Reply to this email directly or view it on GitHub #58 (comment).

[dalehenrich]

For starters we'll be referencing the new Cryptography project from Zinc on
the issue_58 branch and then only from the Web-Socket package ... get all
the tests green and then move from there ...

On Mon, Nov 3, 2014 at 12:41 PM, Johan Brichau [email protected]
wrote:

Sound like a plan.
I’m going to have to focus on Seaside and XMLParser for now, but I can
definitely already check out the status of the packages of Cryptography
that we are using.

Johan

On 03 Nov 2014, at 19:07, Dale Henrichs [email protected]
wrote:

Thanks @pdebruic https://github.com/pdebruic, @HeSe <
https://github.com/HeSe> and @jbrichau https://github.com/jbrichau ...
I've subscribed to the cryptography list so that I can follow any responses
there, but I I am inclined to think that we in the GemStone community are
the only ones with a vested interest in a cross-platform cryptography
library so it seems that for us the best solution would be to create a
Cryptography project on GsDevKit and try to maintain the subset of the
library that is being actively used by the GemStone community (with tests
on Pharo and Squeak) and expand the subset as needed ... we should try to
mirror the new packages on the SqueakSource list as we do with zinc,
seaside, etc.

If the discussion on the cryptography list heats up, I'm inclined to
follow their liead ...

What do you guys think?

—
Reply to this email directly or view it on GitHub <
https://github.com/GsDevKit/zinc/issues/58#issuecomment-61520755>.

—
Reply to this email directly or view it on GitHub
#58 (comment).

[dalehenrich]

@pdebruic, @HeSe and @jbrichau ... Cryptography project added for now it is just serving the two packages that have been ported to GemStone, slthough I did copy all of the packages from the SqueakSource project for good measure. Didn't add them to baseline because at this moment don't know which ones run in Squeak or Pharo or nowhere ...

On the issue_58 branch I've updated the baseline to reference the GsDevKit/Cryptography project, but we have a number of test failures ...

Hooked up the NeoJSON project as well, and it has some failing tests, too ...

I'm setting up the Cryptography project on travis so I'll be looking into getting things to pass there ...

[dalehenrich]

@pdebruic, @HeSe and @jbrichau ... Cryptography tests are green across the range of GemStone version

[dalehenrich]

Switched to running all unit tests in image to pick up errors that may be introduced (like SentButNotImplemented and UndefinedSymbols) and that may be failing from required projects and here's the tale of tape... mainly websocket test failures:

**************************************************************************************
    Results for TestCase
1061 run, 1041 passes, 3 expected defects, 2 failures, 15 errors, 0 unexpected passes
**************************************************************************************
*** FAILURES *******************
    SentButNotImplementedTest debug: #'testSentButNotImplemented'.
    UndefinedSymbolsTest debug: #'testUndefinedSymbols'.
*** ERRORS *******************
    ZnWebSocketFrameTests debug: #'testReading256ByteSingleFrameUnmaskedBinaryMessage'.
    ZnWebSocketFrameTests debug: #'testReading256KiBSingleFrameUnmaskedBinaryMessage'.
    ZnWebSocketFrameTests debug: #'testReadingFragmentedUnmaskedTextMessage'.
    ZnWebSocketFrameTests debug: #'testReadingPing'.
    ZnWebSocketFrameTests debug: #'testReadingPong'.
    ZnWebSocketFrameTests debug: #'testReadingSingleFrameMaskedTextMessage'.
    ZnWebSocketFrameTests debug: #'testReadingSingleFrameUnmaskedTextMessage'.
    ZnWebSocketFrameTests debug: #'testWriting256ByteSingleFrameUnmaskedBinaryMessage'.
    ZnWebSocketFrameTests debug: #'testWriting256KiBSingleFrameUnmaskedBinaryMessage'.
    ZnWebSocketFrameTests debug: #'testWritingPing'.
    ZnWebSocketFrameTests debug: #'testWritingSingleFrameUnmaskedTextMessage'.
    ZnWebSocketTests debug: #'testChatroom'.
    ZnWebSocketTests debug: #'testEcho'.
    ZnWebSocketTests debug: #'testEchoSecureWebSocketsDotOrg'.
    ZnWebSocketTests debug: #'testStatus'.
**************************************************************************************

[dalehenrich]

slug of sentButNotImplemented messages (not sure where they are all coming from)

[dalehenrich]

Note that the class ZnServerStarter is needed for the ZnWebSocketTests and that class will need a custom bash script for starting a separate server for the tests (thanks @HeSe ). So we may have to wait until I change buiderCI to use gsDevKitHome for installing GLASS/GsDevKit before getting the tests to run clean...

[dalehenrich]

It looks like some of the websocket code depends upon some methods that are in STON ... so will need to migrate those methods into GsSqueakCommon or GsPharo ....

[dalehenrich]

The class ZNServerStarter is over in https://github.com/HeSe/gsApplicationTools

[dalehenrich]

clean sheet for Cryptography tests