public class ProportionalCapacityPreemptionPolicy extends Object implements SchedulingEditPolicy
SchedulingEditPolicy
that is designed to be
paired with the CapacityScheduler
. At every invocation of editSchedule()
it computes the ideal amount of resources assigned to each
queue (for each queue in the hierarchy), and determines whether preemption
is needed. Overcapacity is distributed among queues in a weighted fair manner,
where the weight is the amount of guaranteed capacity for the queue.
Based on this ideal assignment it determines whether preemption is required
and select a set of containers from each application that would be killed if
the corresponding amount of resources is not freed up by the application.
If not in observeOnly
mode, it triggers preemption requests via a
ContainerPreemptEvent
that the ResourceManager
will ensure
to deliver to the application (or to execute).
If the deficit of resources is persistent over a long enough period of time
this policy will trigger forced termination of containers (again by generating
ContainerPreemptEvent
).Modifier and Type | Class and Description |
---|---|
static class |
ProportionalCapacityPreemptionPolicy.IntraQueuePreemptionOrderPolicy
IntraQueuePreemptionOrder will be used to define various priority orders
which could be configured by admin.
|
Constructor and Description |
---|
ProportionalCapacityPreemptionPolicy() |
ProportionalCapacityPreemptionPolicy(RMContext context,
CapacityScheduler scheduler,
org.apache.hadoop.yarn.util.Clock clock) |
public ProportionalCapacityPreemptionPolicy()
public ProportionalCapacityPreemptionPolicy(RMContext context, CapacityScheduler scheduler, org.apache.hadoop.yarn.util.Clock clock)
public void init(org.apache.hadoop.conf.Configuration config, RMContext context, ResourceScheduler sched)
init
in interface SchedulingEditPolicy
public org.apache.hadoop.yarn.util.resource.ResourceCalculator getResourceCalculator()
public void editSchedule()
SchedulingEditPolicy
editSchedule
in interface SchedulingEditPolicy
public long getMonitoringInterval()
getMonitoringInterval
in interface SchedulingEditPolicy
public String getPolicyName()
getPolicyName
in interface SchedulingEditPolicy
public Map<RMContainer,Long> getToPreemptContainers()
public TempQueuePerPartition getQueueByPartition(String queueName, String partition)
public Collection<TempQueuePerPartition> getQueuePartitions(String queueName)
public CapacityScheduler getScheduler()
public RMContext getRMContext()
public boolean isObserveOnly()
public Set<org.apache.hadoop.yarn.api.records.ContainerId> getKillableContainers()
public double getMaxIgnoreOverCapacity()
public double getNaturalTerminationFactor()
public int getClusterMaxApplicationPriority()
public float getMaxAllowableLimitForIntraQueuePreemption()
public float getMinimumThresholdForIntraQueuePreemption()
public org.apache.hadoop.yarn.api.records.Resource getPartitionResource(String partition)
public LinkedHashSet<String> getUnderServedQueuesPerPartition(String partition)
public void addPartitionToUnderServedQueues(String queueName, String partition)
public ProportionalCapacityPreemptionPolicy.IntraQueuePreemptionOrderPolicy getIntraQueuePreemptionOrderPolicy()
Copyright © 2020 Apache Software Foundation. All rights reserved.