Just raising this because I dont believe there is a way to spread task instances evenly on a cluster of hosts.
Reading through the issues I think there is quite a good awareness that GROUP_BY doesnt work at all well - and you can end up with horribly skewed task-instances across a cluster because at the point of scheduling the new task, marathon is only aware of existing offers from mesos for a subsection of nodes in a cluster.
The recommended way seems to be to supply GROUP_BY constraint with a number, indicating the size of your pool of (e.g. hosts) to try and spread the task-instances evenly over only that pre-defined number.
However this also doesnt work in an autoscaling group, when you dont know the number of available hosts in advance, and even when you set it to the number of hosts at time t, given that may change, marathon wont actively reschedule the task-instances to allow the autoscaling nature of the hosts.
UNIQUE is also not available as a solution as this automatically precludes having multiple task-instances on a host.
To be clear, the solution I am looking for is:
- I have 8 hosts, and wish to run 14 instances of a task - marathon schedules as 6 hosts with 2 task-instances and 2 hosts with 1 task-instances. (6_2 + 2_1)
- I later lose a host because of system failure - marathon schedules as 7 hosts with 2 task-instances
- The host comes back up and marathon reschedules as 6 hosts with 2 task-instances and 2 hosts with 1 task-instance as originally.
- A further host is added to the cluster and marathon reschedules as 5 hosts with 2 task-instances and 4 hosts with 1 task-instance.
I would have thought this is the most common desired situation of all - "Please evenly distribute n instances of a task on m hosts at all times".
Is there no way to do this in marathon, or have I missed something obvious? Many thanks