Very slow fsync() on shared container volume

[osheroff]

When re-creating a development database, mysql under docker will call fsync() a lot. these calls are taking between 25 and 30ms each. The container was booted as:

docker run -v /var/lib/mysql --name MYSQL_DATA_NEW busybox true
docker run --volumes-from MYSQL_DATA_NEW --net docker -d --name mysql --hostname mysql  -p 3306:3306 [private-repo]/mysql bash run.sh```

You can see the slowness under strace:

root@ea5b3c31c695:/# strace -p `pgrep -f mysqld` -ff -t -TT 2>&1 | grep fsync
[pid 27356] 04:38:57 fsync(9)           = 0 <0.024066>
[pid 27356] 04:38:57 fsync(9)           = 0 <0.018833>
[pid 27356] 04:38:57 fsync(9)           = 0 <0.024052>
[pid 27356] 04:38:57 fsync(9)           = 0 <0.028534>
[pid 27356] 04:38:57 fsync(9)           = 0 <0.025760>
[pid 27356] 04:38:57 fsync(9)           = 0 <0.022935>
[pid 27356] 04:38:57 fsync(9)           = 0 <0.020960>
[pid 27356] 04:38:57 fsync(9)           = 0 <0.025935>
[pid 27356] 04:38:57 fsync(9)           = 0 <0.023266>
[pid 27356] 04:38:57 fsync(9)           = 0 <0.025116>
[pid 27356] 04:38:57 fsync(9)           = 0 <0.023262>

Information

• Docker for Mac: version: 1.12.1-beta25 (158f2ee)
• OSX 10.11.6 (15G31)

Steps to reproduce the behavior

1. Perform lots of "create table" statements with mysql with its data directory shared from another container. Watch the slowness commence.

Note that tuning and tweaking mysql to do less fsyncs gets some mild speed win, but nowhere near native speeds.

[libo]

Yeps.. I can see the same behaviour. Thanks @osheroff for investigating :-)

[samoht]

fsync on OSX unfortunately slower than on Linux: see mirage/mirage-block-unix#49 for the initial issue and mirage/mirage-block-unix#52 for the "fix". Since beta27 we allow to turn on an "experimental" mode where fsync is faster but less safe: if your computer crashed in the middle of an fsync, we won't guarantee in that mode that you won't corrupt the qcow2 image ... so use it at your own risk. Closing as a duplicate of #668 (comment)

[dsheets]

This looks like it is related to bind mounts rather than block device syncs. I haven't reproduced it yet but I am reopening as the block device issue is not the likely cause.

[docker-robott]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale comment.
Stale issues will be closed after an additional 30d of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle stale

[docker-robott]

Closed issues are locked after 30 days of inactivity.
This helps our team focus on active issues.

If you have found a problem that seems similar to this, please open a new issue.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle locked