When you run a docker container (via marathon) The underlying docker log files are not rotated or truncated.
These files can become arbitrarily large (and they often go to a system disk rather than a storage disk)
On a server with limited space on the relevant disk this can lead to running out of disk space and we all know how well docker / mesos / dcos copes on a server that has run out of disk space (if you don't know what happens, everything on the box goes to hell and often it ends up "lost")
docker log files for a container can be found using:
docker inspect --format='.LogPath' <container-name-or-id>
n.b. I am fully aware that internally mesos/marathon is using logrotate (by default rotating log files every 2MB and keeping 10 files.
I have long running kafka consumers whose logs have grown to 10s of GB
cross-post from https://dcosjira.atlassian.net/browse/DCOS-371
it looks like marathon needs to pass some sensible defaults to
docker run --log-opt max-size=
docker run --log-opt max-files=