Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add force_pull to docker driver #2147

Merged
merged 3 commits into from
Jan 11, 2017
Merged

add force_pull to docker driver #2147

merged 3 commits into from
Jan 11, 2017

Conversation

camerondavison
Copy link
Contributor

closes #639

dadgar
dadgar reviewed Jan 9, 2017
View reviewed changes
client/driver/docker_test.go
task, _, _ := dockerTask()
task.Config["force_pull"] = "nothing"

driverCtx, execCtx := testDriverContexts(task)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you rebase, this function has changed

client/driver/docker.go
@@ -136,6 +136,7 @@ type DockerDriverConfig struct {
WorkDir string `mapstructure:"work_dir"` // Working directory inside the container
Logging []DockerLoggingOpts `mapstructure:"logging"` // Logging options for syslog server
Volumes []string `mapstructure:"volumes"` // Host-Volumes to mount in, syntax: /path/to/host/directory:/destination/path/in/container
ForcePull bool `mapstructure:"force_pull"` // Always force pull before running image, usefull if your tags are mutable
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

misspelled usefull

@camerondavison
Copy link
Contributor Author

I rebased, and fixed the changes for the method, but now when I try and run my tests I get the following.

vagrant@nomad-server01:/opt/gopath/src/github.com/hashicorp/nomad$ go test -v github.com/hashicorp/nomad/client/driver/ -run 'TestDockerDriver_ForcePull.*'
=== RUN   TestDockerDriver_ForcePull_IsInvalidConfig
--- FAIL: TestDockerDriver_ForcePull_IsInvalidConfig (0.00s)
	driver_test.go:108: TaskDir.Build(map[string]string{"/usr":"/usr", "/bin":"/bin", "/etc":"/etc", "/lib":"/lib", "/lib32":"/lib32", "/lib64":"/lib64", "/run/resolvconf":"/run/resolvconf", "/sbin":"/sbin"}, "image") failed: %!v(MISSING)
=== RUN   TestDockerDriver_ForcePull
--- FAIL: TestDockerDriver_ForcePull (0.00s)
	docker.go:23: Successfully connected to docker daemon running version 1.12.5
	driver_test.go:108: TaskDir.Build(map[string]string{"/lib":"/lib", "/lib32":"/lib32", "/lib64":"/lib64", "/run/resolvconf":"/run/resolvconf", "/sbin":"/sbin", "/usr":"/usr", "/bin":"/bin", "/etc":"/etc"}, "image") failed: %!v(MISSING)
FAIL
exit status 1
FAIL	github.com/hashicorp/nomad/client/driver	0.016s

I am trying to see if I can figure out what is going on.

@schmichael
Copy link
Member

Ack, sorry for the broken test failure message! I pushed a fix for that here if you care to rebase: 1bb4806

Can you try running the test with sudo? Due to some changes in how we build the directory used by tasks it now requires root.

@camerondavison
Copy link
Contributor Author

after rebase and using sudo tests pass

vagrant@nomad-server01:/opt/gopath/src/github.com/hashicorp/nomad$ sudo -E /opt/go/bin/go test -v github.com/hashicorp/nomad/client/driver/ -run 'TestDockerDriver_ForcePull.*'
=== RUN   TestDockerDriver_ForcePull_IsInvalidConfig
--- PASS: TestDockerDriver_ForcePull_IsInvalidConfig (0.03s)
=== RUN   TestDockerDriver_ForcePull
2017/01/11 03:08:05 [DEBUG] driver.docker: using client connection initialized from environment
2017/01/11 03:08:05 [DEBUG] driver.docker: force pull image 'busybox:latest' instead of inspecting local
2017/01/11 03:08:05 [DEBUG] driver.docker: loading image from: /tmp/626443fb-048a-f577-320f-e1780113c940/redis-demo/local/busybox.tar
2017/01/11 03:08:05 [DEBUG] driver.docker: identified image busybox as sha256:47bcc53f74dc94b1920f0b34f6036096526296767650f223433fe65c35f149eb
2017/01/11 03:08:05 [DEBUG] plugin: starting plugin: /opt/gopath/bin/nomad []string{"/opt/gopath/bin/nomad", "executor", "/tmp/626443fb-048a-f577-320f-e1780113c940/redis-demo/executor.out"}
2017/01/11 03:08:05 [DEBUG] plugin: waiting for RPC address for: /opt/gopath/bin/nomad
2017/01/11 03:08:05 [DEBUG] plugin: nomad: 2017/01/11 03:08:05 [DEBUG] plugin: plugin address: unix /tmp/plugin130707298
2017/01/11 03:08:05 [DEBUG] driver.docker: Setting default logging options to syslog and unix:///tmp/plugin715761497
2017/01/11 03:08:05 [DEBUG] driver.docker: Using config for logging: {Type:syslog ConfigRaw:[] Config:map[syslog-address:unix:///tmp/plugin715761497]}
2017/01/11 03:08:05 [DEBUG] driver.docker: using 268435456 bytes memory for redis-demo
2017/01/11 03:08:05 [DEBUG] driver.docker: using 512 cpu shares for redis-demo
2017/01/11 03:08:05 [DEBUG] driver.docker: binding directories []string{"/tmp/626443fb-048a-f577-320f-e1780113c940/alloc:/alloc", "/tmp/626443fb-048a-f577-320f-e1780113c940/redis-demo/local:/local", "/tmp/626443fb-048a-f577-320f-e1780113c940/redis-demo/secrets:/secrets"} for redis-demo
2017/01/11 03:08:05 [DEBUG] driver.docker: networking mode not specified; defaulting to bridge
2017/01/11 03:08:05 [DEBUG] driver.docker: allocated port 127.0.0.1:55851 -> 55851 (static)
2017/01/11 03:08:05 [DEBUG] driver.docker: exposed port 55851
2017/01/11 03:08:05 [DEBUG] driver.docker: allocated port 127.0.0.1:35657 -> 35657 (mapped)
2017/01/11 03:08:05 [DEBUG] driver.docker: exposed port 35657
2017/01/11 03:08:05 [DEBUG] driver.docker: setting container startup command to: /bin/nc -l 127.0.0.1 -p 0
2017/01/11 03:08:05 [DEBUG] driver.docker: setting container name to: redis-demo-e27091a5-4782-ecc3-6019-049c598f1a46
2017/01/11 03:08:05 [INFO] driver.docker: created container 569749fb9de7f947c0b5025cebebef73c1b74a19fe4acc0ac613c84691a2ce53
2017/01/11 03:08:05 [INFO] driver.docker: started container 569749fb9de7f947c0b5025cebebef73c1b74a19fe4acc0ac613c84691a2ce53
2017/01/11 03:08:07 [DEBUG] driver.docker: error collecting stats from container 569749fb9de7f947c0b5025cebebef73c1b74a19fe4acc0ac613c84691a2ce53: io: read/write on closed pipe
2017/01/11 03:08:07 [INFO] driver.docker: stopped container 569749fb9de7f947c0b5025cebebef73c1b74a19fe4acc0ac613c84691a2ce53
2017/01/11 03:08:07 [ERR] driver.docker: error deregistering services: 1 error(s) occurred:

* Get http://127.0.0.1:8500/v1/agent/services: dial tcp 127.0.0.1:8500: getsockopt: connection refused
2017/01/11 03:08:07 [DEBUG] plugin: /opt/gopath/bin/nomad: plugin process exited
--- PASS: TestDockerDriver_ForcePull (1.89s)
	docker.go:23: Successfully connected to docker daemon running version 1.12.5
PASS
ok  	github.com/hashicorp/nomad/client/driver	1.938s

@schmichael schmichael merged commit 2855090 into hashicorp:master Jan 11, 2017
@schmichael
Copy link
Member

Thanks for the PR and testing! I've filed #2178 for myself to hopefully remove the root requirement for docker before the next release.

schmichael added a commit that referenced this pull request Jan 11, 2017
@schmichael
Add entry for force_pull #2147
1b56b60
mtxrym added a commit to mtxrym/nomad that referenced this pull request Jan 20, 2017
@mtxrym
Merge remote-tracking branch 'hashicorp/master'
c87a292 
* hashicorp/master: (527 commits)
  Moved functions to helper from structs
  fix a copy/paste error on api.Regions() comment
  changelog
  Ensuring we can register service names with duplicate name but different ports
  s/acess/access
  Support setting class_path and class name.
  Put the right hostname in /etc/hosts
  Added the API for GC of allocations and nodes
  Added executorconfig
  Remove unused code
  Filter executor log messages
  Add local hostname resolution to Vagrantfile
  Changelog
  Random wait
  Review fixes
  Add entry for force_pull hashicorp#2147
  changelog
  using new ctx instead of getting both params back
  fixing typo in comment
  add force_pull to docker driver
  ...
@github-actions
Copy link

I'm going to lock this pull request because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active contributions.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Apr 12, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@dadgar dadgar dadgar left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Flag in jobfile.nomad to force pull docker images from registry
3 participants
@camerondavison @schmichael @dadgar