Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Discovery basic features #1

Merged
merged 4 commits into from
Aug 19, 2015
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
.idea
.vagrant
65 changes: 65 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
PUBLISH=publish_weavediscovery

.DEFAULT: all
.PHONY: all update tests publish $(PUBLISH) clean prerequisites build travis run-smoketests

# If you can use docker without being root, you can do "make SUDO="
SUDO=sudo

DOCKERHUB_USER=weaveworks
WEAVE_VERSION=git-$(shell git rev-parse --short=12 HEAD)

WEAVEDISCOVERY_EXE=prog/weavediscovery/weavediscovery
EXES=$(WEAVEDISCOVERY_EXE)
WEAVEDISCOVERY_UPTODATE=.weavediscovery.uptodate
IMAGES_UPTODATE=$(WEAVEDISCOVERY_UPTODATE)
WEAVEDISCOVERY_IMAGE=$(DOCKERHUB_USER)/weavediscovery
IMAGES=$(WEAVEDISCOVERY_IMAGE)
WEAVE_EXPORT=weavediscovery.tar

all: $(WEAVE_EXPORT)

travis: $(EXES)

update:
go get -u -f -v -tags -netgo $(addprefix ./,$(dir $(EXES)))

$(WEAVEDISCOVERY_EXE):
go get -tags netgo ./$(@D)
go build -ldflags "-extldflags \"-static\" -X main.version $(WEAVE_VERSION)" -tags netgo -o $@ ./$(@D)
@strings $@ | grep cgo_stub\\\.go >/dev/null || { \
rm $@; \
echo "\nYour go standard library was built without the 'netgo' build tag."; \
echo "To fix that, run"; \
echo " sudo go clean -i net"; \
echo " sudo go install -tags netgo std"; \
false; \
}

$(WEAVEDISCOVERY_EXE): prog/weavediscovery/*.go

$(WEAVEDISCOVERY_UPTODATE): prog/weavediscovery/Dockerfile $(WEAVEDISCOVERY_EXE)
$(SUDO) docker build -t $(WEAVEDISCOVERY_IMAGE) prog/weavediscovery
touch $@

$(WEAVE_EXPORT): $(IMAGES_UPTODATE)
$(SUDO) docker save $(addsuffix :latest,$(IMAGES)) > $@

$(DOCKER_DISTRIB):
curl -o $(DOCKER_DISTRIB) $(DOCKER_DISTRIB_URL)

$(PUBLISH): publish_%:
$(SUDO) docker tag -f $(DOCKERHUB_USER)/$* $(DOCKERHUB_USER)/$*:$(WEAVE_VERSION)
$(SUDO) docker push $(DOCKERHUB_USER)/$*:$(WEAVE_VERSION)
$(SUDO) docker push $(DOCKERHUB_USER)/$*:latest

publish: $(PUBLISH)

clean:
-$(SUDO) docker rmi $(IMAGES) 2>/dev/null
rm -f $(EXES) $(IMAGES_UPTODATE) $(WEAVE_EXPORT) test/tls/*.pem coverage.html profile.cov

build:
$(SUDO) go clean -i net
$(SUDO) go install -tags netgo std
$(MAKE)
159 changes: 158 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,162 @@
Discovery
=========

TODO
Weave Discovery retrieves and watches a list of peers
available in a Weave network from a discovery backend.
Discovery uses this information for telling a Weave Router
about new peers it must connect to, as well as peers
that are not available anymore and must be forgotten.
Discovery will also advertise the Weave router in the
same backend so other peers can join it.

Installation
============

Just download the `discovery` script from the repository,
or use this short URL:

```
curl -O http://git.io/vmW3z
chmod a+x discovery
```

This will download the `discovery` script to the current
directory.

Usage
=====

Once you have the script, joining a new discovery URL
is as simple as running:

```
$ ./discovery join <URL>
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't you need to chmod it first?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed. Fixed.

```

The endpoint URL can be something like

- `etcd://<etcd_ip>/<path>`
- `consul://<consul_addr>/<path>`
- `zk://<zookeeper_addr1>,<zookeeper_addr2>/<path>`
- `token://<swarm_cluster_id>`

Example:

```
$ ./discovery join etcd://192.168.9.1/mycluster
```

You can get more information about the supported backends
as well as the URL formats [here](https://github.com/docker/swarm/tree/master/discovery).

By default, Discovery will configure the Weave router in the
local machine. However, you can point Discovery to a
different router with the `--weave` flag.

For example, we can point Discovery to the Weave HTTP
API found at _192.168.9.3:6784_ with:

```
$ ./discovery join --weave=192.168.9.3:6784 etcd://192.168.9.1/mycluster
```

Discovery will not try to advertise the Router to other
peers by default. a reachable IP in the same discovery
endpoint, so other peers will be able to find it.
Discovery guesses the advertised IP by trying _1)_ the
router's adress (when using a non-local router) _2)_ the
interface that is used for the default route. However,
you should probably overide this and make it explicit
with the `--advertise` flag.

Example:

```
$ ./discovery join --advertise=193.144.60.100:6784 etcd://192.168.9.1/mycluster
```

Discovery can also advertise the guessed external IP with
the `--advertise-external` flag. However this will not perform
any NAT mapping, so make sure the router is accessible at this
IP _and_ port by the rest of the peers in the backend.


Example
=======

A Vagrant file has been included for testing the functionaly
of Discovery. The Vagrant will start three VMs with:

* a Weave router and proxy
* a Swarm agent that uses some _token_ and points to the Weave proxy
* a Discovery instance that uses the same _token_ and points to the Weave router

You can launch the Vagrant VMs with:

```
$ vagrant up --provision
```

Machines will be provisioned and a new Swarm token ID will be
automatically generated. You will see the ID in the console messages.
After a few seconds, you will be able to check the list of peers
that have successfully registered in this token with:

```
$ swarm list token://TOKEN
10.246.2.3:12375
10.246.2.4:12375
10.246.2.2:12375
```

In the host machine (not in the Vagrants), you can start the cluster
manager with:

```
$ docker run -d -p 22375:2375 swarm manage token://TOKEN"
```

This manager can be reachable at port 22375. Point your Docker client to
this port for getting the list of the machines in the cluster, with:

```
$ docker -H 127.0.0.1:22375 info

Containers: 17
Images: 26
Storage Driver:
Role: primary
Strategy: spread
Filters: affinity, health, constraint, port, dependency
Nodes: 3
minion-0: 10.246.2.2:12375
└ Containers: 6
└ Reserved CPUs: 0 / 2
└ Reserved Memory: 0 B / 2.052 GiB
└ Labels: executiondriver=native-0.2, kernelversion=3.19.0-21-generic, operatingsystem=Ubuntu 15.04, storagedriver=overlay
minion-1: 10.246.2.3:12375
└ Containers: 6
└ Reserved CPUs: 0 / 2
└ Reserved Memory: 0 B / 2.052 GiB
└ Labels: executiondriver=native-0.2, kernelversion=3.19.0-21-generic, operatingsystem=Ubuntu 15.04, storagedriver=overlay
minion-2: 10.246.2.4:12375
└ Containers: 5
└ Reserved CPUs: 0 / 2
└ Reserved Memory: 0 B / 2.052 GiB
└ Labels: executiondriver=native-0.2, kernelversion=3.19.0-21-generic, operatingsystem=Ubuntu 15.04, storagedriver=overlay
Execution Driver:
Kernel Version:
Operating System:
CPUs: 6
Total Memory: 6.155 GiB
Name:
ID:
```

Now you can run a Ubuntu shell in _some_ machine in the cluster, with:

```
$ docker -H 127.0.0.1:22375 run -ti ubuntu
root@7f1007c28c1c:/#
```

56 changes: 56 additions & 0 deletions Vagrantfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# -*- mode: ruby -*-
# vi: set ft=ruby :

require './vagrant/provision.rb'
require './vagrant/swarm.rb'

# Vagrantfile API/syntax version. Don't touch unless you know what you're doing!
VAGRANTFILE_API_VERSION = "2"

# Require a recent version of vagrant otherwise some have reported errors setting host names on boxes
Vagrant.require_version ">= 1.6.2"

# Determine whether vagrant should use nfs to sync folders
$use_nfs = ENV['VAGRANT_USE_NFS'] == 'true'

# The number of minions to provision
$num_minion = 3

# ip configuration
$minion_ip_base = "10.246.2."
$minion_ips = $num_minion.times.collect { |n| $minion_ip_base + "#{n+2}" }
$minion_ips_str = $minion_ips.join(",")

$swarm_token = create_swarm()

##################################################
# Main
##################################################

################
# Main
################
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|

if Vagrant.has_plugin?("vagrant-vbguest") then
config.vbguest.auto_update = false
end

################
# Minions
################
$num_minion.times do |minion_index|
config.vm.define "minion-#{minion_index}" do |minion|
minion_ip = $minion_ips[minion_index]
minion.vm.hostname = "minion-#{minion_index}"
minion.vm.network "private_network", ip: "#{minion_ip}"

configure(minion, "minion-#{minion_index}", "#{minion_ip}")
provision(minion, "minion-#{minion_index}", "#{minion_ip}")

minion.vm.provision "shell", inline: "/vagrant/vagrant/provision-minion.sh #{minion_index} #{minion_ip} #{$num_minion} #{$swarm_token}"
end
end

end

Loading