Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add instructions to connect examples to distributed deployment #392

Merged
merged 6 commits into from
Jun 15, 2022
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion examples/mnist-keras/.gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,5 @@ data
*.npz
*.tgz
*.tar.gz
.mnist-keras
.mnist-keras
client.yaml
28 changes: 25 additions & 3 deletions examples/mnist-keras/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,15 +5,16 @@ This classic example of hand-written text recognition is well suited both as a l
- [MNIST Example (Keras version)](#mnist-example-keras-version)
- [Table of Contents](#table-of-contents)
- [Prerequisites](#prerequisites)
- [Running the example](#running-the-example)
- [Running the example (pseudo-distributed)](#running-the-example-pseudo-distributed)
- [Clean up](#clean-up)
- [Connecting to a distributed deployment](#connecting-to-a-distributed-deployment)

## Prerequisites
- [Docker](https://docs.docker.com/get-docker)
- [Docker Compose](https://docs.docker.com/compose/install)
- [Python 3.8](https://www.python.org/downloads)

## Running the example
## Running the example (pseudo-distributed)
Clone FEDn and locate into this directory.
```sh
git clone https://github.com/scaleoutsystems/fedn.git
Expand Down Expand Up @@ -52,4 +53,25 @@ bin/build.sh
Finally, you can start the experiment from the "control" tab.

## Clean up
You can clean up by running `docker-compose down`.
You can clean up by running `docker-compose down`.

## Connecting to a distributed deployment
To start and remotely connect a client with the required dependencies for this example, start by downloading the `client.yaml` file. You can either navigate the reducer UI or run the following command.

```bash
curl -k https://<reducer-fqdn>:<reducer-port>/config/download > client.yaml
```
> **Note** make sure to replace `<reducer-fqdn>` and `<reducer-port>` with appropriate values.

Now you are ready to start the client via Docker by running the following command.

```bash
docker run -d \
-v $PWD/client.yaml:/app/client.yaml \
-v $PWD/data:/var/data \
-e ENTRYPOINT_OPTS=--data_path=/var/data/mnist.npz \
--add-host=reducer:<reducer-ip> \
mcapuccini marked this conversation as resolved.
Show resolved Hide resolved
--add-host=combiner:<combiner-ip> \
ghcr.io/scaleoutsystems/fedn/fedn:develop-mnist-keras run client -in client.yaml
```
> **Note** make sure to replace `<reducer-ip>` and `<combiner-ip>` with appropriate values.
mcapuccini marked this conversation as resolved.
Show resolved Hide resolved
4 changes: 2 additions & 2 deletions examples/mnist-keras/client/fedn.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
entry_points:
train:
command: /venv/bin/python entrypoint train
command: /venv/bin/python entrypoint train $ENTRYPOINT_OPTS
validate:
command: /venv/bin/python entrypoint validate
command: /venv/bin/python entrypoint validate $ENTRYPOINT_OPTS
3 changes: 2 additions & 1 deletion examples/mnist-pytorch/.gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,5 @@ data
*.npz
*.tgz
*.tar.gz
.mnist-pytorch
.mnist-pytorch
client.yaml
26 changes: 24 additions & 2 deletions examples/mnist-pytorch/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,13 +7,14 @@ This classic example of hand-written text recognition is well suited both as a l
- [Prerequisites](#prerequisites)
- [Running the example](#running-the-example)
- [Clean up](#clean-up)
- [Connecting to a distributed deployment](#connecting-to-a-distributed-deployment)

## Prerequisites
- [Docker](https://docs.docker.com/get-docker)
- [Docker Compose](https://docs.docker.com/compose/install)
- [Python 3.8](https://www.python.org/downloads)

## Running the example
## Running the example (pseudo-distributed)
Start by initializing a virtual enviroment with all of the required dependencies.
```
bin/init_venv.sh
Expand Down Expand Up @@ -46,4 +47,25 @@ bin/build.sh
Finally, you can start the experiment from the "control" tab.

## Clean up
You can clean up by running `docker-compose down`.
You can clean up by running `docker-compose down`.

## Connecting to a distributed deployment
To start and remotely connect a client with the required dependencies for this example, start by downloading the `client.yaml` file. You can either navigate the reducer UI or run the following command.

```bash
curl -k https://<reducer-fqdn>:<reducer-port>/config/download > client.yaml
```
> **Note** make sure to replace `<reducer-fqdn>` and `<reducer-port>` with appropriate values.

Now you are ready to start the client via Docker by running the following command.

```bash
docker run -d \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets run it interactively by deafult, so that the user gets a visual confirmation that it is working / can see the printouts.

-v $PWD/client.yaml:/app/client.yaml \
-v $PWD/data:/var/data \
-e ENTRYPOINT_OPTS=--data_path=/var/data/mnist.npz \
--add-host=reducer:<reducer-ip> \
--add-host=combiner:<combiner-ip> \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's leave these out and instead make a note / warning box explaining in more detail in which situations they are needed and provide the full example there / below. .

ghcr.io/scaleoutsystems/fedn/fedn:develop-mnist-pytorch run client -in client.yaml
```
> **Note** make sure to replace `<reducer-ip>` and `<combiner-ip>` with appropriate values.
4 changes: 2 additions & 2 deletions examples/mnist-pytorch/client/fedn.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
entry_points:
train:
command: /venv/bin/python entrypoint train
command: /venv/bin/python entrypoint train $ENTRYPOINT_OPTS
validate:
command: /venv/bin/python entrypoint validate
command: /venv/bin/python entrypoint validate $ENTRYPOINT_OPTS