Public-RPS and proposal failure

[martijnlammers]

I've create a single node network using cchost with only one initial activated member containing the default constitution.

; config.ini
enclave-file = ./libjs_generic.enclave.so.signed
enclave-type = release
consensus = cft
rpc-address = 10.0.0.18:443
node-address = 10.0.0.18:6600
public-rpc-address = [public VM addr]:443
ledger-dir = ./legder/
node-cert-file = ./certificates/node_cert.pem

[start]
network-cert-file = ./certificates/network_cert.pem
member-info = "./certificates/Martijn_cert.pem,./certificates/Martijn_enc_pubk.pem"
constitution = "./constitution/actions.js"
constitution = "./constitution/validate.js"
constitution = "./constitution/resolve.js"
constitution = "./constitution/apply.js"

I'm currently attempting to change my JS application through proposal, however I'm running into two problems.
Firstly, I'm unable to access my node via my public address, only through my local address.
Attempting to activate a member using:
$ curl https://[public VM addr]:443/gov/ack/update_state_digest -X POST --cacert ../certificates/network_cert.pem --key ../certificates/Martijn_privk.pem --cert ../certificates/Martijn_cert.pem

Outputs:

curl: (60) SSL: no alternative certificate subject name matches target host name '[my pub addr]'
More details here: https://curl.haxx.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

Port 443 is open.

Secondly, I am unable to apply my proposal to the running network via my local address.

Is there anything visible that I am doing wrong?

[eddyashton]

For the first part: It looks like the certs that are generated don't include the correct Subject Alternative Name (SAN) field that curl is looking for when it established a TLS connection. This should be derived from the --public-rpc-address option, but can also be explicitly specified with --san (see the note below the diagram here). You can print the fields in the cert with
$ openssl x509 -text -in ./certificates/node_cert.pem

Some examples:

; config.ini
public-rpc-address = 8.8.8.8:443

$ openssl x509 -text -in ./certificates/node_cert.pem
...
            X509v3 Subject Alternative Name: 
                IP Address:8.8.8.8
...

; config.ini
public-rpc-address = ccf.example.com:443

$ openssl x509 -text -in ./certificates/node_cert.pem
...
            X509v3 Subject Alternative Name: 
                DNS:ccf.example.com
...

; config.ini
public-rpc-address = 8.8.8.8:443
san = dNSName:ccf2.example.com

$ openssl x509 -text -in ./certificates/node_cert.pem
...
            X509v3 Subject Alternative Name: 
                DNS:ccf2.example.com
...

Can you check what SAN is being inserted, and confirm it matches your public VM address (in particular, that it is correctly identified as a DNS/IP?)?

To rule out any other factors, you should first test that you can reach a simpler unauthenticated endpoint (ie curl https://[public VM addr]:443/node/commit --cacert ../certificates/network_cert.pem, which does not need user or member auth).

EDIT: This theory on the second issue is wrong, see below*
For the second issue, I think the problem is with how you specified the constitution in the .ini file. When we specify multiple files on the CLI, this is multiple instances to --constitution, but it looks like the TOML representation of that is a list, rather than multiple values. In other words, replace

constitution = "./constitution/actions.js"
constitution = "./constitution/validate.js"
constitution = "./constitution/resolve.js"
constitution = "./constitution/apply.js"

with

constitution=["./constitution/actions.js", "./constitution/validate.js", "./constitution/resolve.js", "./constitution/apply.js"]

(Note: Not sure about this one - that's how our arg parser writes this value when we ask it to generate a .ini file, but afaict it should also parse the same from multiple entries. I'll see if I can work out what's going on there)

* I've tested this locally, and having multiple instances of constitution isn't a problem - I was still able to propose and vote for (and Accept) a proposal. Your error message suggests there's something missing from the constitution. What proposal function are you trying to call? The error includes some line numbers in the constitution (528 and 929), but they correspond to the concatenated constitution so can't be trivially looked up in the source. You could use the read_ledger tool to dump the contents of the constitution table, unescape that string, and look at the source there, or just manually concatenate the constitution files (single newline between each) and see if there's anything obviously odd about the JS code at those lines?

[eddyashton]

I think the problem there is trying to bind to 443 as non-root. Any well-known port (0-1023) is generally managed by the OS, and not available to user applications. So either use an alternative port (a random large number), or run it as root if you need it to be HTTPS-standard.

[achamayou]

@mAlyanak 443 is a system port, you cannot bind to it if you aren't root. Either run the process as root, or choose a port above 1023.

[martijnlammers]

Oh yeah that's right, very sharp to notice. Totally forgot about that little detail haha.

I ended up getting the empty string issue again.

$ sudo ./ccf/bin/cchost --config=config.ini
021-10-21T17:18:09.800113Z 100 [info ] ../src/host/node_connections.h:179 | Listening for node-to-node on 10.0.0.18:6600
2021-10-21T17:18:09.800731Z 100 [info ] ../src/host/rpc_connections.h:75 | Listening for RPCs on 10.0.0.18:443
Could not open file

I tried putting the const. seperate again now, reattempting to apply a proposal again with little success.
#config

[start]
network-cert-file = ./certificates/network_cert.pem
member-info = "./certificates/Martijn_cert.pem,./certificates/Martijn_enc_pubk.pem"
constitution = "/home/mar/CCF/src/runtime_config/default/actions.js"
constitution = "/home/mar/CCF/src/runtime_config/default/validate.js"
constitution = "/home/mar/CCF/src/runtime_config/default/resolve.js"
constitution = "/home/mar/CCF/src/runtime_config/default/apply.js"

#Start network
$ sudo /opt/ccf/bin/cchost --config=config.ini

#In a seperate terminal:
#Activating initial member
$ curl https://10.0.0.18:443/gov/ack/update_state_digest -X POST --cacert ../certificates/network_cert.pem --key ../certificates/Martijn_privk.pem --cert ../certificates/Martijn_cert.pem

$ scurl.sh https://10.0.0.18:443/gov/ack --cacert ./../certificates/network_cert.pem --signing-key ./../certificates/Martijn_privk.pem --signing-cert ./../certificates/Martijn_cert.pem --header "Content-Type: application/json" --data-binary '{"state_digest":"a9021ac46265b4158fad13ff41b0f1bd7d6f6c088930468a28d402af32e901db"}'

#Send JS app proposal
$ scurl.sh https://10.0.0.18:443/gov/proposals --cacert ../certificates/network_cert.pem --signing-key ../certificates/Martijn_privk.pem --signing-cert ../certificates/Martijn_cert.pem --data-binary @set_js_app_proposal.json -H "content-type: application/json"

$ scurl.sh https://10.0.0.18:443/gov/proposals/f8c8f9c84cc38f87c1a08df9d365338cd3fb1b6c14cf8936d3028bc9bbcc68ae/ballots --cacert ../certificates/network_cert.pem --signing-key ../certificates/Martijn_privk.pem --signing-cert ../certificates/Martijn_cert.pem --data-binary @vote_accept.json -H "content-type: application/json"

Output:

{"ballot_count":1,"failure":{"reason":"Failed to apply(): TypeError: not a function","trace":" at (public:ccf.gov.constitution[0]:528)\n at apply (public:ccf.gov.constitution[0]:959)\n"},"proposal_id":"f8c8f9c84cc38f87c1a08df9d365338cd3fb1b6c14cf8936d3028bc9bbcc68ae","proposer_id":"0eb426bc7351c937b03e81d23de9a17ea333fc650c5020dd57cf7a5de6df42ba","state":"Failed","votes":{"0eb426bc7351c937b03e81d23de9a17ea333fc650c5020dd57cf7a5de6df42ba":true}}

[eddyashton]

I think what's happening here is that you're using the .js scripts from the main branch of the repo, against a CCF node running 1.0.13. I dumped the constitution from a local instance*, and sure enough line 528 is inside the set_js_app apply function, but it is calling modulesMap.clear() which doesn't exist in 1.0.13. The fix here is to use the governance scripts from 1.0.13, not from main.

* In case it's useful to someone in future, I did that with python ../python/ccf/read_ledger.py workspace/content_types_auth_0/0.ledger/ --uncommitted -t ".*constitution.*" | grep -A2 'table "public:ccf.gov.constitution" (1 write)' | tail -n1 | sed 's/\\n/\n/g; s/\\"/"/g' > ./constitution.js. That's "call read_ledger.py, reading just the constitution table, grep out the constitution line, sed to unescape its newlines and quotes".

[martijnlammers]

Ohh, yeah that did seem to be the case. Major thanks for the help.