Proposed: save public and private IP of ec2 instances in config.json

[andrewwatson]

to make it possible to use "machine inspect" to get the private IP it would be helpful if it was saved in config.json. Some systems like consul need to be configured to advertise the private IP and join gossip pools on the private IP space so this would make it easier to script the creation of a consul cluster created with machine.

[andrewwatson]

Also, this could address #581 by making the IPAddress field be the private address if there was no public one.

I propose that we augment config.json to look like this:

{
    "DriverName": "amazonec2",
    "Driver": {
        "Id": "948ca7dedfed7168d2087210aa15caea",
        "AccessKey": "[REDACTED]",
        "SecretKey": "[REDACTED]",
        "SessionToken": "",
        "Region": "us-west-1",
        "AMI": "ami-d1180894",
        "SSHKeyID": 0,
        "KeyName": "shipyard-worker2",
        "InstanceId": "i-6db0e9ae",
        "InstanceType": "t2.micro",
        "IPAddress": "54.153.39.165",
        "MachineName": "shipyard-worker2",
        "SecurityGroupId": "sg-2fc6614a",
        "SecurityGroupName": "docker-machine",
        "ReservationId": "",
        "RootSize": 16,
        "VpcId": "vpc-43844226",
        "SubnetId": "",
        "Zone": "a",
        "CaCertPath": "/Users/andrew/.docker/machine/certs/ca.pem",
        "PrivateKeyPath": "/Users/andrew/.docker/machine/certs/ca-key.pem",
        "SwarmMaster": false,
        "SwarmHost": "tcp://0.0.0.0:3376",
        "SwarmDiscovery": "",
        "Addresses": {
            "Public": "54.153.39.165",
            "Private": "10.1.9.91"
        }
    },
    "CaCertPath": "/Users/andrew/.docker/machine/certs/ca.pem",
    "ServerCertPath": "",
    "ServerKeyPath": "",
    "PrivateKeyPath": "/Users/andrew/.docker/machine/certs/ca-key.pem",
    "ClientCertPath": "",
    "SwarmMaster": false,
    "SwarmHost": "tcp://0.0.0.0:3376",
    "SwarmDiscovery": ""
}

[ehazlett]

+1

[andrewwatson]

is it ok if only ec2 machines have this added structure or would that cause angst/confusion?

[sthulb]

it's probably ok.

[sthulb]

It just depends on how we make it visible.

[andrewwatson]

so this is where i'm at right now:

is-mbp-andrew:machine andrew$ ./docker-machine_darwin-amd64 inspect privateIPx
{
    "DriverName": "amazonec2",
    "Driver": {
        "Id": "6545aa4ef3e7da0cffb83b96714ebbe9",
        "AccessKey": "[REDACTED]",
        "SecretKey": "[REDACTED]",
        "SessionToken": "",
        "Region": "us-west-1",
        "AMI": "ami-d1180894",
        "SSHKeyID": 0,
        "KeyName": "privateIPx",
        "InstanceId": "i-75c098b6",
        "InstanceType": "t2.micro",
        "IPAddress": "54.153.30.1",
        "PrivateIPAddress": "172.31.13.225",
        "MachineName": "privateIPx",
        "SecurityGroupId": "sg-2fc6614a",
        "SecurityGroupName": "docker-machine",
        "ReservationId": "",
        "RootSize": 16,
        "VpcId": "vpc-43844226",
        "SubnetId": "",
        "Zone": "a",
        "CaCertPath": "/Users/andrew/.docker/machine/certs/ca.pem",
        "PrivateKeyPath": "/Users/andrew/.docker/machine/certs/ca-key.pem",
        "SwarmMaster": false,
        "SwarmHost": "tcp://0.0.0.0:3376",
        "SwarmDiscovery": ""
    },
    "CaCertPath": "/Users/andrew/.docker/machine/certs/ca.pem",
    "ServerCertPath": "",
    "ServerKeyPath": "",
    "PrivateKeyPath": "/Users/andrew/.docker/machine/certs/ca-key.pem",
    "ClientCertPath": "",
    "SwarmMaster": false,
    "SwarmHost": "tcp://0.0.0.0:3376",
    "SwarmDiscovery": ""
}

i think it's fine that PrivateIP is at the same level as IPAddress because it's EC2 specific but so is AccessKey!

[nathanleclaire]

SGTM

Ultimately this is something I'd like to enforce for all drivers (assuming they have private IPs available).

[ehazlett]

Should probably be part of #554

[rossbachp]

I dectect this after restart my aws instance with docker-machine 0.0.1:

$ docker-machine start amazonec2-03
$ docker-machine ip amazonec2-03
52.11.238.23
$ docker-machine env amazonec2-03
export DOCKER_TLS_VERIFY=yes
export DOCKER_CERT_PATH=/Users/peter/.docker/machine/machines/amazonec2-03
export DOCKER_HOST=tcp://52.11.238.23:2376
$ docker-machine ssh amazonec2-03
[email protected]'s password: 
$ cat ~/.docker/machine/machines/amazonec2-03/config.json | jq ".Driver.IPAddress"
"52.10.115.103"

ip and env use the runtime amazon ip, but ssh the older one from config.json!

It works at master branch... Nice, but now I need PR #770 right, to regenerate the cert?

$ ./docker-machine_darwin-amd64 ls
NAME           ACTIVE   DRIVER       STATE     URL                        SWARM
amazonec2-03   *        amazonec2    Running   tcp://52.10.115.103:2376   

Why machine doesn't update the config.json?

[ehazlett]

@rossbachp this has been fixed as part of #770. thanks!

[sthulb]

Rethinking this, I'm not sure we should store IPs for hosts. We should find a way to query them from the hypervisor.

[ehazlett]

@sthulb +1 -- that's exactly what GetIP does in the ec2 driver now.

[ehazlett]

@sthulb although some providers allocate a dedicated IP for the life cycle of the instance (i.e. digitalocean).

[sthulb]

Even still, I think we should query for them.

[nathanleclaire]

Even still, I think we should query for them.

This thread seems to have veered a bit from the original issue, but I wanted to chime in and say that my understanding / perspective is that one of the goals of this round of refactors is to make this the case. That way, the drivers don't have to poll for the IP address in Create like some do now (which widens the window of failure before a config save) and the results returned from GetIP will generally be consistent, barring the failure case.

[ehazlett]

@nathanleclaire +1 -- I think we are good with this as the follow-up comment has been addressed.