Run disk quota excess check

[dilshat]

We need to run checks if containers exceed their disk quotas and:

1. Notify Hub about this if usage is close to 90%
2. Stop container if limit is exceeded by factor 1.5

See https://github.com/subutai-io/agent/wiki/Switch-to-Soft-Quota

[dilshat]

Hint:

obtain subdirs usage: du --max-depth=1 -B1 /var/snap/subutai-sysnet/common/lxc/Container1-jkz-1-4/

obtain container's total usage: du -s -B1 /var/snap/subutai-sysnet/common/lxc/Container1-jkz-1-4/

obtain all containers' usage (need to exclude non containers): du --max-depth=1 -B1 /var/snap/subutai-sysnet/common/lxc/

[happyaron]

Talked with @lbthomsen and we both agree that using du directly may hurt performance badly, so that as a trade-off we can use btrfs API to get the space usage, using accuracy to trade performance.

Also we are able to do more frequent statistics when using the btrfs API, to make sure the container does not fill up disk space quickly between our long check intervals.

lth@ncpws04:~$ sudo btrfs filesystem df /var/snap/subutai-master/common/lxc/ubuntu16/
Data, single: total=2.01GiB, used=543.50MiB
System, DUP: total=8.00MiB, used=16.00KiB
Metadata, DUP: total=1.00GiB, used=48.19MiB
GlobalReserve, single: total=16.00MiB, used=0.00B

[dilshat]

Good hint. Yes better use btrfs api

[dilshat]

these commands show somewhat near values:

root@localhost:~# /snap/subutai-sysnet/current/bin/btrfs filesystem du -s /var/snap/subutai-sysnet/common/lxc/Container1-elb-2-6/ 
     Total   Exclusive  Set shared  Filename
 184.15MiB   828.00KiB   183.34MiB  /var/snap/subutai-sysnet/common/lxc/Container1-elb-2-6/


root@localhost:~# du -sh /var/snap/subutai-sysnet/common/lxc/Container1-elb-2-6
219M    /var/snap/subutai-sysnet/common/lxc/Container1-elb-2-6

whereas btrfs filesystem df has different output:

root@localhost:~# /snap/subutai-sysnet/current/bin/btrfs filesystem df /var/snap/subutai-sysnet/common/lxc/Container1-elb-2-6/
Data, single: total=5.01GiB, used=4.63GiB
System, DUP: total=8.00MiB, used=16.00KiB
Metadata, DUP: total=1.50GiB, used=757.92MiB

[dilshat]

https://btrfs.wiki.kernel.org/index.php/FAQ#How_much_free_space_do_I_have.3F

[dilshat]

https://superuser.com/questions/654119/btrfs-huge-metadata-allocated

[dilshat]

https://btrfs.wiki.kernel.org/index.php/Manpage/btrfs-filesystem