Improve OpenPGP #89

the-djmaze · 2021-05-19T13:44:04Z

Describe the bug
The RainLoop OpenPGP implementation is incorrect.

Encrypt message should be with 'Recipient' public key (not your own key)
Sign a message should be with 'From' private key
Received signed "Content-Type: multipart/alternative" message can't be verified yet (only plain)
etc. etc.

Some limitations are caused by the current implementation in JavaScript.
Either the whole message body should be rendered as-is in JavaScript or handled in PHP with https://php.net/gnupg or others.

Reported issues at RainLoop:
https://github.com/RainLoop/rainloop-webmail/issues?q=is%3Aissue+is%3Aopen+pgp

I made a Wiki page that explains the rules.
https://github.com/the-djmaze/snappymail/wiki/OpenPGP

TODO:

Keys

Import public/private keys

GnuPG
Mailvelope
OpenPGP.js

Note: as of v2.34 you can search public key servers to find them, and import all keys from server into OpenPGP.js

View public/private keys

GnuPG
Mailvelope
OpenPGP.js

Delete public/private keys

GnuPG
Mailvelope
OpenPGP.js

Allow private keys without password

Mailvelope
❌ GnuPG not secure/too risky when allowed
OpenPGP.js

Decrypt / Sign when multiple keys exist

Detect which key to use
or let user select

Received messages

PGP/Inline (cleartext)

Decrypt

Mailvelope
GnuPG
OpenPGP.js

Verify signature

❌ Mailvelope API: Enabled signature verification mailvelope/mailvelope#434
GnuPG
OpenPGP.js

Decrypt then verify signature

Mailvelope
GnuPG
OpenPGP.js

PGP/MIME (multipart)

Decrypt

Mailvelope
GnuPG
OpenPGP.js

Verify signature

❌ Mailvelope API: Enabled signature verification mailvelope/mailvelope#434
GnuPG
OpenPGP.js

Decrypt then verify signature

Mailvelope
GnuPG
OpenPGP.js

Sending messages

PGP/Inline (cleartext)

❌ no, everything is PGP/MIME

PGP/MIME (multipart)

Encrypt message text

Mailvelope (no html, only plain)
GnuPG
OpenPGP.js

Encrypt attachments

Mailvelope
GnuPG
OpenPGP.js

Sign

❌ Mailvelope Sign only mode. mailvelope/mailvelope#750
GnuPG
OpenPGP.js (no attachments yet)

Sign then Encrypt

❌ Mailvelope (no html, only plain, can't select sign key)
GnuPG (also encrypts attachments)
OpenPGP.js (no attachments yet)

Autocrypt

As requested in issue #342 for https://autocrypt.org/

Import public keys from MIME header
Send public keys in MIME header

NOTE

Although we properly support PGP/MIME
I've discovered some systems don't, including Mailvelope:
roundcube/roundcubemail#8417 (comment)

Also there's the crypto refresh https://datatracker.ietf.org/doc/draft-ietf-openpgp-crypto-refresh/
And see https://fosdem.org/2024/schedule/event/fosdem-2024-2669--security-modernizing-email-encryption-the-crypto-refresh-of-openpgp/

ervee · 2021-08-10T14:30:44Z

I remember a discussion back on RainLoop Git. If I recall correctly the keys (so also your private key!) are stored server side. In my case, I'm the server admin so I kind of trust this. But I would not trust anyone else with my private key. Even if I used my own server, I would not trust it if it was a VPS or some other shared service.

So if this is currently broken, I would consider this a good candidate to shrink the SnappyMail code :)

the-djmaze · 2021-08-10T15:04:35Z

If I recall correctly the keys (so also your private key!) are stored server side.

No, private keys are in your browser localStorage (which also sucks).
An "encrypted" copy could be stored on the server (with symmetric encryption and passphrase) but that defeats the use of serverside gnupg.

RoundCube: uses gnupg
ProtonMail: does not use PGP (they say they do, but they send an e-mail with a link and you must open the link)
etc.

There are only 2 cases where we need private keys:

Sign a message
Decrypt received message

Public keys is no issue and we can still use for:

Verify received signature
Encrypt message to send

Solving the public keys issue would be the first step to solve.
This should be easy for webmail without security issues.

GregThib · 2021-12-16T20:49:53Z

ProtonMail: does not use PGP (they say they do, but they send an e-mail with a link and you must open the link)
etc.

Plaît-il ?
Some people send messages to me from protonmail, and I received them PGP-encrypted (as I have WKD configured in my domain, and then my public keys are easily discoverable).

Maybe you receive the message in this form when the user force encryption to a recipient who don't have a public key known or discoverable via WKD?
Cause, with protonmail we are able to encrypt to recipient whithout PGP-enabled encryption, with something like a password.

GregThib · 2021-12-16T21:00:25Z

Just my two cents: in the RainLoop issue feed, there is a lot of questions about PGP implementation, e.g. the use of WKD to discover keys, the opportunistic encryption, and a few things more.

Today, I think it's important to reach a core-base PGP encryption (encrypt, sign, verify) without getting too much pressure about the remaining (trust model, key discover, etc.).

As we can see with Thunderbird, since they have dropped the Enigmail support, their implementation is suffering too.

the-djmaze · 2021-12-22T19:37:57Z

@GregThib oh yes, i've added OpenPGP.js 5.0.1 for comparison and development.
Problem is that the library is not 100% compiled for web, it also (still) has Node.js code.
So i'm cleaning up the 1.5 MiB code to get more in line with the old 636 KiB
openpgp-sm.js

But still, i should look at GnuPG as well

the-djmaze · 2022-01-16T12:21:02Z

Working on this issue revealed that RainLoop never correctly verified signed messages.
A signature must be done on the whole multipart/signed but it never does.
It only checks if the pgp sgnature is valid.

GregThib · 2022-01-16T13:03:46Z

Oo
This is a pretty serious issue.

Now it does not fetch the PGP signature, because validation was broken anyway. Instead it validates multipart/signed according to RFC 3156 section 5 and returns details for the signed part: * BodyPartId * SigPartId * MicAlg So in the future several implementations (GnuPG, OpenPGP.js, etc.) can use the correct data for verification.

This part fetches the required message parts for pgp verification.

Bugfix SnappyMail\PGP\Keyservers Renamed DoPgpVerify to DoMessagePgpVerify

the-djmaze · 2022-01-18T14:10:55Z

Oo This is a pretty serious issue.

Yep, but as you can see all the commits, there is work in progress :)

For now it uses GnuPG to verify PGP messages.
This method is tested and now works properly here.

uses php pecl gnupg
store public keys in: …/_data_/_default_/storage/[example.com]/[account]/.gnupg (no UI yet)
JavaScript click on "🔒OpenPGP signed message (click to verify)" will call the new MessagePgpVerify (result not in UI yet)

Still todo regarding "verify":

Improve UI /#/settings/openpgp with public keys section for verifying received messages
Show verify results in UI
Alternatives to GnuPG extension: Crypt_GPG, OpenPGP.js, Mailvelope, openpgp-php + phpseclib

Todo after that:

Encrypt using public keys
Manage private keys
Decrypt messages with private keys
Sign messages with private keys

the-djmaze · 2022-01-19T19:33:01Z

Update: latest changes now properly load keyrings of:

Mailvelope
OpenPGP.js
GnuPG

Composer window is revamped and has no PopupsComposeOpenPgp.
Instead the dropdown menu now has two options:

Sign
Encrypt

Sign

Only enabled when chosen identity (from) has a private key.

Encrypt

Only enabled when all recipients (to, cc, bcc) have a public key.
Also all recipients must be either in Mailvelope or OpenPGP.js or GnuPG and can't be mixed.
TODO: should be extended to lookup HKP servers and others to find the keys.

The system is still defunct and does not Sign nor Encrypt yet.
First i need to get all keyring systems to work properly.

GregThib · 2022-01-19T20:46:23Z

Impressive!

I wanted to congratulate you very much for your work on this tool. I am impressed by the speed and efficiency with which you deal with this problem, and thank you for the attention given to your users.

Gratefully!

the-djmaze · 2022-01-20T15:49:37Z

@GregThib thank you!

Working on this, i noticed many problems that have to be dealt with:

PHP PECL GnuPG doesn't import private keys. I need to call gpg --import --homedir=/home/snappymail/__data__/,,,,/.gnupg secret_key.asc
gnupg_keyinfo() v1.5+ should have hidden param to list private keys, else only public keys are returned
OpenPGP.js and Mailvelope can't sign/encrypt attachments or the whole message (only cleartext)
etc.

So basically it is a PITA to sign/encrypt properly.
But hopefully i get there eventually.

Looking at the Enigma source of RoundCube it seems that it also should have issues with it.

For our issue #89

Improment for #89

#89

the-djmaze · 2022-01-24T20:27:05Z

Dropped support for PEAR Crypt_GPG because it is missing features.
For best memory usage i want to stream data in/out the mailso imap stream with GnuPG in between.
That way it is easy to encrypt/decrypt/sign/verify large messages (20MiB+).
Crypt_GPG does have encryptFile but not an encryptStream.

Also it can't generate ECC keys.

So i'm building a new gpg.php class that shall solve it using gnupg and Crypt_GPG as examples.

luluwebmaster · 2023-02-11T10:17:31Z

Hello !

Note : My English is not very good, sorry for that.

I'm having a problem on my docker installation, based on @kouinkouin image ( #44 ).

Very simply, I can't decipher the encrypted emails.

Whether with OpenPGP or GnuPG, the decryption is done well (I have the message "Message encrypted by OpenPGP" after decryption), but the encrypted message is not replaced in the interface.

I note an interesting point, with GnuPG, when I look at the Ajax requests and I launch the decryption, the response contains the decrypted message, but the JSON does not seem correct ( See the screenshots ).

Note that I don't know if this is a problem specific to the Docker image, or if the problem also exists in the context of a classic installation.

Is this issue known / related to this issue?

Thanks in advance !

After click on "Decrypt" button :

Reponse of request when use GnuPG ( Response is decrypted, I can read message ) :

the-djmaze · 2023-02-11T15:12:28Z

@luluwebmaster Who encrypted the message?
(Hopefully Thunderbird, because that has bugs and should only be used in PGP/MIME mode).

Looking at it, it is quoted-printable encoded HTML and that is wrong.

There are 2 types of PGP encryption:

PGP/Inline which is plain text (not HTML)
PGP/MIME which can be plain and/or HTML with optional attachments and/or inline images

luluwebmaster · 2023-02-13T10:07:40Z

The message received has been encrypted by Thunderbird.

Could this be a problem with Thunderbird ?

the-djmaze · 2023-02-13T15:22:39Z

Correct!
https://superuser.com/questions/794959/what-is-introducing-quoted-printables-into-my-pgp-encrypted-emails

Force Thunderbird to use PGP/MIME and try again

luluwebmaster · 2023-02-14T08:40:09Z

Does that mean that all my correspondents must activate this option .. ? It's not possible to "fix" this on Snappy interface ?

the-djmaze · 2023-02-14T09:42:26Z

If we make a workaround, how would other people with different applications see the broken Thunderbird?

gmail
kmail
fairemail
apple mail
outlook
microsoft mail
etc. etc.

the-djmaze · 2023-02-14T10:14:39Z

Looking at it, the issue is different these days and i will look into it.

luluwebmaster · 2023-02-14T10:17:11Z

Ok, not problem, I wait your feedback !

luluwebmaster · 2023-02-14T16:37:54Z

Hello,

After update of docker image ( 2.26.1 ), now that works correctly !

Thank you.

LeifAndersen · 2023-02-28T21:39:26Z

@the-djmaze FWIW, I seem to have the same problem that @luluwebmaster was describing. But the message was encrypted using mailvalope instead of thunderbird.

the-djmaze mentioned this issue Dec 16, 2021

PGP with ECDSA or curve25519 #172

Closed

the-djmaze pushed a commit that referenced this issue Dec 22, 2021

#89

b6db336

the-djmaze pushed a commit that referenced this issue Jan 17, 2022

Changes for #89

fde44c0

This part fetches the required message parts for pgp verification.

the-djmaze pushed a commit that referenced this issue Jan 17, 2022

Added GnuPG->verify() for testing #89

bf84684

Bugfix SnappyMail\PGP\Keyservers Renamed DoPgpVerify to DoMessagePgpVerify

the-djmaze pushed a commit that referenced this issue Jan 18, 2022

#89 DoMessagePgpVerify() succeeds with GnuPG

d354738

the-djmaze pushed a commit that referenced this issue Jan 18, 2022

#89 Detect and verify PGP cleartext/clearsigned messages

b8d898b

the-djmaze pushed a commit that referenced this issue Jan 20, 2022

Many more changes for #89

a7eeeb4

the-djmaze mentioned this issue Jan 20, 2022

private keys issues php-gnupg/php-gnupg#44

Closed

the-djmaze pushed a commit that referenced this issue Jan 21, 2022

WIP: revamp Crypt_GPG for #89

cff6620

the-djmaze pushed a commit that referenced this issue Jan 21, 2022

Solution regarding php-gnupg/php-gnupg#44

6df7b76

For our issue #89

the-djmaze pushed a commit that referenced this issue Jan 22, 2022

Better solution for gpg 'socket name' error

51fedab

Improment for #89

the-djmaze pushed a commit that referenced this issue Jan 24, 2022

Improved and bugfix handling of GnuPG for #89

1d3673f

the-djmaze pushed a commit that referenced this issue Jan 24, 2022

Get PGP verify message working using Promise.

ce971de

#89

the-djmaze pushed a commit that referenced this issue Jan 24, 2022

Drop support for PEAR Crypt_GPG as it is lacking features we want in #89

f26c35c

the-djmaze pushed a commit that referenced this issue Jan 25, 2022

Support generating ed25519/cv25519 keys for #89 and #172

151708a

the-djmaze mentioned this issue Jan 25, 2022

plugins/enigma Crypt_GPG broken roundcube/roundcubemail#8417

Closed

the-djmaze pushed a commit that referenced this issue Jan 25, 2022

Added sign, decrypt and encrypt for #89

54feb03

the-djmaze mentioned this issue Oct 21, 2022

[Nextcloud] feature requests #569

Open

15 tasks

the-djmaze mentioned this issue Nov 18, 2022

PGP Keys gone after restart RainLoop/rainloop-webmail#349

Open

the-djmaze mentioned this issue Jan 31, 2023

OpenPGP key unsaved RainLoop/rainloop-webmail#2204

Closed

the-djmaze pushed a commit that referenced this issue Feb 14, 2023

Bugfix: OpenPGP decrypted attachments friendlySize not a function #89

1f86b73

luluwebmaster mentioned this issue Feb 14, 2023

Can't open PGP keys file. #958

Closed

the-djmaze mentioned this issue Feb 16, 2023

OpenPGP Keys lost on fresh session #973

Closed

the-djmaze removed the bug Something isn't working label Dec 1, 2023

the-djmaze pushed a commit that referenced this issue Feb 7, 2024

Added support to lookup public keys online #89

6c01db3

the-djmaze pushed a commit that referenced this issue Feb 7, 2024

Added button to import keys from server into OpenPGP.js #89

f800215

the-djmaze pushed a commit that referenced this issue Feb 7, 2024

Fix pgp import button style #89

a7f7645

the-djmaze pushed a commit that referenced this issue Feb 7, 2024

Deduplicate and sort OpenPGP.js key stores #89

89b6e84

the-djmaze pushed a commit that referenced this issue Feb 8, 2024

Also allow backup PGP keys on server on import #89

43cbbd5

the-djmaze pushed a commit that referenced this issue Feb 11, 2024

Added deprecated MIME PGP headers for #89 ?

e28085f

the-djmaze pushed a commit that referenced this issue Feb 11, 2024

Automaticaly add PGP key from Autocrypt #89

17cb093

the-djmaze pushed a commit that referenced this issue Feb 11, 2024

Added options to Identity for PGP sign and encrypt #89

c72c2db

the-djmaze pushed a commit that referenced this issue Feb 11, 2024

Trim white space in PGP armored key #89

a45cd59

the-djmaze mentioned this issue Feb 12, 2024

Autocrypt support #342

Closed

the-djmaze pushed a commit that referenced this issue Feb 12, 2024

Enable OpenPGP by default #89

9cceafc

the-djmaze mentioned this issue May 28, 2024

Native GPG implementation in Mail app nextcloud/mail#9600

Open

the-djmaze unpinned this issue Aug 27, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve OpenPGP #89

Improve OpenPGP #89

the-djmaze commented May 19, 2021 •

edited

Loading

ervee commented Aug 10, 2021

the-djmaze commented Aug 10, 2021

GregThib commented Dec 16, 2021

GregThib commented Dec 16, 2021

the-djmaze commented Dec 22, 2021

the-djmaze commented Jan 16, 2022

GregThib commented Jan 16, 2022

the-djmaze commented Jan 18, 2022 •

edited

Loading

the-djmaze commented Jan 19, 2022

GregThib commented Jan 19, 2022

the-djmaze commented Jan 20, 2022 •

edited

Loading

the-djmaze commented Jan 24, 2022 •

edited

Loading

luluwebmaster commented Feb 11, 2023

the-djmaze commented Feb 11, 2023 •

edited

Loading

luluwebmaster commented Feb 13, 2023

the-djmaze commented Feb 13, 2023 •

edited

Loading

luluwebmaster commented Feb 14, 2023

the-djmaze commented Feb 14, 2023

the-djmaze commented Feb 14, 2023

luluwebmaster commented Feb 14, 2023

luluwebmaster commented Feb 14, 2023 •

edited

Loading

LeifAndersen commented Feb 28, 2023

Improve OpenPGP #89

Improve OpenPGP #89

Comments

the-djmaze commented May 19, 2021 • edited Loading

TODO:

Keys

Received messages

PGP/Inline (cleartext)

PGP/MIME (multipart)

Sending messages

PGP/Inline (cleartext)

PGP/MIME (multipart)

Autocrypt

NOTE

ervee commented Aug 10, 2021

the-djmaze commented Aug 10, 2021

GregThib commented Dec 16, 2021

GregThib commented Dec 16, 2021

the-djmaze commented Dec 22, 2021

the-djmaze commented Jan 16, 2022

GregThib commented Jan 16, 2022

the-djmaze commented Jan 18, 2022 • edited Loading

the-djmaze commented Jan 19, 2022

Sign

Encrypt

GregThib commented Jan 19, 2022

the-djmaze commented Jan 20, 2022 • edited Loading

the-djmaze commented Jan 24, 2022 • edited Loading

luluwebmaster commented Feb 11, 2023

the-djmaze commented Feb 11, 2023 • edited Loading

luluwebmaster commented Feb 13, 2023

the-djmaze commented Feb 13, 2023 • edited Loading

luluwebmaster commented Feb 14, 2023

the-djmaze commented Feb 14, 2023

the-djmaze commented Feb 14, 2023

luluwebmaster commented Feb 14, 2023

luluwebmaster commented Feb 14, 2023 • edited Loading

LeifAndersen commented Feb 28, 2023

the-djmaze commented May 19, 2021 •

edited

Loading

the-djmaze commented Jan 18, 2022 •

edited

Loading

the-djmaze commented Jan 20, 2022 •

edited

Loading

the-djmaze commented Jan 24, 2022 •

edited

Loading

the-djmaze commented Feb 11, 2023 •

edited

Loading

the-djmaze commented Feb 13, 2023 •

edited

Loading

luluwebmaster commented Feb 14, 2023 •

edited

Loading