Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add instructions to connect examples to distributed deployment #392

Merged
merged 6 commits into from
Jun 15, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion examples/mnist-keras/.gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,5 @@ data
*.npz
*.tgz
*.tar.gz
.mnist-keras
.mnist-keras
client.yaml
26 changes: 23 additions & 3 deletions examples/mnist-keras/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,15 +5,16 @@ This classic example of hand-written text recognition is well suited both as a l
- [MNIST Example (Keras version)](#mnist-example-keras-version)
- [Table of Contents](#table-of-contents)
- [Prerequisites](#prerequisites)
- [Running the example](#running-the-example)
- [Running the example (pseudo-distributed)](#running-the-example-pseudo-distributed)
- [Clean up](#clean-up)
- [Connecting to a distributed deployment](#connecting-to-a-distributed-deployment)

## Prerequisites
- [Docker](https://docs.docker.com/get-docker)
- [Docker Compose](https://docs.docker.com/compose/install)
- [Python 3.8](https://www.python.org/downloads)

## Running the example
## Running the example (pseudo-distributed)
Clone FEDn and locate into this directory.
```sh
git clone https://github.com/scaleoutsystems/fedn.git
Expand Down Expand Up @@ -52,4 +53,23 @@ bin/build.sh
Finally, you can start the experiment from the "control" tab.

## Clean up
You can clean up by running `docker-compose down`.
You can clean up by running `docker-compose down`.

## Connecting to a distributed deployment
To start and remotely connect a client with the required dependencies for this example, start by downloading the `client.yaml` file. You can either navigate the reducer UI or run the following command.

```bash
curl -k https://<reducer-fqdn>:<reducer-port>/config/download > client.yaml
```
> **Note** make sure to replace `<reducer-fqdn>` and `<reducer-port>` with appropriate values.

Now you are ready to start the client via Docker by running the following command.

```bash
docker run -d \
-v $PWD/client.yaml:/app/client.yaml \
-v $PWD/data:/var/data \
-e ENTRYPOINT_OPTS=--data_path=/var/data/mnist.npz \
ghcr.io/scaleoutsystems/fedn/fedn:develop-mnist-keras run client -in client.yaml
```
> **Note** If reducer and combiner host names, as specfied in the configuration files, are not resolvable in the client host network you need to use the docker option `--add-hosts` to make them resolvable. Please refer to the Docker documentation for more detail.
4 changes: 2 additions & 2 deletions examples/mnist-keras/client/fedn.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
entry_points:
train:
command: /venv/bin/python entrypoint train
command: /venv/bin/python entrypoint train $ENTRYPOINT_OPTS
validate:
command: /venv/bin/python entrypoint validate
command: /venv/bin/python entrypoint validate $ENTRYPOINT_OPTS
3 changes: 2 additions & 1 deletion examples/mnist-pytorch/.gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,5 @@ data
*.npz
*.tgz
*.tar.gz
.mnist-pytorch
.mnist-pytorch
client.yaml
26 changes: 23 additions & 3 deletions examples/mnist-pytorch/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,15 +5,16 @@ This classic example of hand-written text recognition is well suited both as a l
- [MNIST Example (PyTorch version)](#mnist-example-pytorch-version)
- [Table of Contents](#table-of-contents)
- [Prerequisites](#prerequisites)
- [Running the example](#running-the-example)
- [Running the example (pseudo-distributed)](#running-the-example-pseudo-distributed)
- [Clean up](#clean-up)
- [Connecting to a distributed deployment](#connecting-to-a-distributed-deployment)

## Prerequisites
- [Docker](https://docs.docker.com/get-docker)
- [Docker Compose](https://docs.docker.com/compose/install)
- [Python 3.8](https://www.python.org/downloads)

## Running the example
## Running the example (pseudo-distributed)
Start by initializing a virtual enviroment with all of the required dependencies.
```
bin/init_venv.sh
Expand Down Expand Up @@ -46,4 +47,23 @@ bin/build.sh
Finally, you can start the experiment from the "control" tab.

## Clean up
You can clean up by running `docker-compose down`.
You can clean up by running `docker-compose down`.

## Connecting to a distributed deployment
To start and remotely connect a client with the required dependencies for this example, start by downloading the `client.yaml` file. You can either navigate the reducer UI or run the following command.

```bash
curl -k https://<reducer-fqdn>:<reducer-port>/config/download > client.yaml
```
> **Note** make sure to replace `<reducer-fqdn>` and `<reducer-port>` with appropriate values.

Now you are ready to start the client via Docker by running the following command.

```bash
docker run -d \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets run it interactively by deafult, so that the user gets a visual confirmation that it is working / can see the printouts.

-v $PWD/client.yaml:/app/client.yaml \
-v $PWD/data:/var/data \
-e ENTRYPOINT_OPTS=--data_path=/var/data/mnist.npz \
ghcr.io/scaleoutsystems/fedn/fedn:develop-mnist-pytorch run client -in client.yaml
```
> **Note** If reducer and combiner host names, as specfied in the configuration files, are not resolvable in the client host network you need to use the docker option `--add-hosts` to make them resolvable. Please refer to the Docker documentation for more detail.
4 changes: 2 additions & 2 deletions examples/mnist-pytorch/client/fedn.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
entry_points:
train:
command: /venv/bin/python entrypoint train
command: /venv/bin/python entrypoint train $ENTRYPOINT_OPTS
validate:
command: /venv/bin/python entrypoint validate
command: /venv/bin/python entrypoint validate $ENTRYPOINT_OPTS