CRIU supports dirty memory tracking to take incremental checkpoints.
Incremental checkpoints are one way of reducing downtime during
migration. The first checkpoint dumps all the memory pages and the
second (and third, and fourth, ...) only dumps pages which have changed.

Most of the necessary code has already been implemented. This just adds
the existing functionality to lxc-checkpoint:

  -p, --pre-dump            Only pre-dump the memory of the container.
                            Container keeps on running and following
                            checkpoints will only dump the changes.
  --predump-dir=DIR         path to images from previous dump (relative to -D)

The following is an example from a container running CentOS 7 with psql
and tomcat:

 # lxc-checkpoint -n c7 -D /tmp/cp -p
Container keeps on running
 # du -h /tmp/cp
 229M	/tmp/cp
Sync initial checkpoint to destination
 # rsync -a /tmp/cp host2:/tmp/
Sync file-system
 # rsync -a /var/lib/lxc/c7 host2:/var/lib/lxc/
Final dump; container is stopped
 # lxc-checkpoint -n c7 -D /tmp/cp --predump-dir=../cp -s
 # du -h /tmp/cp2
 90M	/tmp/cp2

After transferring the second (incremental checkpoint) and the changes
to the container's file system the container can be restored on the
second host by pointing lxc-checkpoint to the second checkpoint
directory:

 # lxc-checkpoint -n c7 -D /tmp/cp2 -r
Signed-off-by: Adrian Reber <areber@redhat.com>