Stripeli/bugfix/ssl fhe (#29)

* Cleaning up and integrating fhe after code refactoring.

* Almost there. Fixing homomorphic encryption key generation and interoperability.

* removed federation initializer

* moved proto factory into proto dir; removed old fed env parser and utils in encryption

* removed redunant opt pb methods; parsed opt params in defenv

* removed uncessary imports

* removed optmizers from fed_env, fixed env generator

* ixed ssl certs; removed ssl stream from proto; moved create server entity into fedenv

* removed pre-generated cert; simplified resource path

* added CamelCase support for optimizer params

* renamaded target for controller; fedenv and fed env schema changes

* renamaded target for controller; fedenv and fed env schema changes

* removed defaults from controller main

* simplified cryptoparams path; fixed buy in fed env init

* refactored server entity to remove ssl stream; restored controller.so target due to  pybind error

* refactored server entity to remove ssl stream; restored controller.so target due to  pybind error

* renamed learner servicer to server

* fixed vars in grpc controller client

* fixed he bug

* schema and template changes

* added fhe paths in config

* fixes in keras model ops

* added _setup_fhe in fedenv

* added get_fhe_dir in config

* added _setup_fhe and renamed public methods to encrypt/decrypt in homomorphic.py

* controller so target restore in setup.py

* import fix in scalability testing example

* Final fixes.

* Extending encryption schemes to support masking.

* Fixed test cases and demos.

* Fixed ssl and brought back ssl streams and ssl files protos. Each proto has its own unique reason of existence. This is not explained in the implementation. Added optimizer functionality without enforcing proto structure. Almost finalized the Encryption module API and refactoring encryption function calls from controller and model modules.

* Changed PWA to SecAgg. Defined EncryptionScheme API.

* Rearranged the testing use cases path.

---------

Co-authored-by: Panagiotis Kyriakis <[email protected]>

.gitignore

@@ -99,7 +99,7 @@ fabric.properties
 /production
 /.sass-cache
 # Bazelisk version file
-.bazelversion
+# .bazelversion
 # User-specific .bazelrc
 user.bazelrc
 
@@ -135,7 +135,7 @@ user.bazelrc
 /venv*
 /.venv/*
 /.venv
-__pycache__/
+**/__pycache__
 .build/*
 build/*
 **experiment.json

BUILD.bazel

@@ -87,7 +87,7 @@ py_wheel(
             ":py39": "cp39",
             ":py310": "cp310",
         }),
-        # TODO: Make sure to sync req with txt
+        # TODO(@panoskyriakis): Make sure to sync req with txt
         requires = [
             "cloudpickle>=2.2.1",
             "fabric>=3.1.0",
@@ -103,7 +103,7 @@ py_wheel(
             "torch>=2.0",
             "scikit-learn>=1.2.2",
             "scipy>=1.10.1",
-            "tensorflow<=2.11",
+            "tensorflow>=2.0,<2.13.0",
             "termcolor>=2.3.0",
             "tensorflow>=2.0",
         ],

build_docker.sh

@@ -1,7 +1,7 @@
 #!/bin/bash
 IMAGE_NAME="${1:-nevron/ubuntu_focal_x86_64_py38}"
 
- IMG_LS=$(docker images | awk '$1 ~ /"$IMAGE_NAME"/ { print $1 }')
+IMG_LS=$(docker images | awk '$1 ~ /"$IMAGE_NAME"/ { print $1 }')
 nevron/ubuntu_focal_x86_64_py38
 if [[ $IMG_LS == "" ]]; then
   echo "Image $IMAGE_NAME not found. Pulling.."