org.apache.flink.mesos.runtime.clusterframework

Class MesosResourceManager

java.lang.Object

org.apache.flink.runtime.rpc.RpcEndpoint

org.apache.flink.runtime.rpc.FencedRpcEndpoint<org.apache.flink.runtime.resourcemanager.ResourceManagerId>

org.apache.flink.runtime.resourcemanager.ResourceManager<RegisteredMesosWorkerNode>

org.apache.flink.mesos.runtime.clusterframework.MesosResourceManager

All Implemented Interfaces:

org.apache.flink.runtime.leaderelection.LeaderContender, org.apache.flink.runtime.resourcemanager.ResourceManagerGateway, org.apache.flink.runtime.rpc.FencedRpcGateway<org.apache.flink.runtime.resourcemanager.ResourceManagerId>, org.apache.flink.runtime.rpc.RpcGateway

public class MesosResourceManager
extends org.apache.flink.runtime.resourcemanager.ResourceManager<RegisteredMesosWorkerNode>

The Mesos implementation of the resource manager.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.flink.runtime.rpc.RpcEndpoint

org.apache.flink.runtime.rpc.RpcEndpoint.MainThreadExecutor

Field Summary

Fields

Modifier and Type	Field and Description
protected static org.slf4j.Logger	LOG

Fields inherited from class org.apache.flink.runtime.resourcemanager.ResourceManager

RESOURCE_MANAGER_NAME

Fields inherited from class org.apache.flink.runtime.rpc.RpcEndpoint

log, rpcServer

Constructor Summary

Constructors

Constructor and Description

MesosResourceManager(org.apache.flink.runtime.rpc.RpcService rpcService, String resourceManagerEndpointId, org.apache.flink.runtime.clusterframework.types.ResourceID resourceId, org.apache.flink.runtime.highavailability.HighAvailabilityServices highAvailabilityServices, org.apache.flink.runtime.heartbeat.HeartbeatServices heartbeatServices, org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager slotManager, org.apache.flink.runtime.metrics.MetricRegistry metricRegistry, org.apache.flink.runtime.resourcemanager.JobLeaderIdService jobLeaderIdService, org.apache.flink.runtime.entrypoint.ClusterInformation clusterInformation, org.apache.flink.runtime.rpc.FatalErrorHandler fatalErrorHandler, org.apache.flink.configuration.Configuration flinkConfig, MesosServices mesosServices, MesosConfiguration mesosConfig, MesosTaskManagerParameters taskManagerParameters, org.apache.flink.runtime.clusterframework.ContainerSpecification taskManagerContainerSpec, String webUiUrl, org.apache.flink.runtime.metrics.groups.JobManagerMetricGroup jobManagerMetricGroup)

Method Summary

Modifier and Type	Method and Description
void	acceptOffers(AcceptOffers msg) Accept offers as advised by the launch coordinator.
protected CompletableFuture<Void>	clearStateAsync()
protected akka.actor.ActorRef	createConnectionMonitor()
protected akka.actor.ActorRef	createLaunchCoordinator(org.apache.mesos.SchedulerDriver schedulerDriver, akka.actor.ActorRef selfActor)
protected akka.actor.ActorRef	createReconciliationCoordinator(org.apache.mesos.SchedulerDriver schedulerDriver)
protected akka.actor.ActorRef	createSelfActor()
protected akka.actor.ActorRef	createTaskMonitor(org.apache.mesos.SchedulerDriver schedulerDriver)
protected void	disconnected(Disconnected message) Called when disconnected from Mesos.
protected void	executorLost(ExecutorLost message)
protected void	frameworkMessage(FrameworkMessage message)
protected void	initialize()
protected void	internalDeregisterApplication(org.apache.flink.runtime.clusterframework.ApplicationStatus finalStatus, String diagnostics)
protected void	offerRescinded(OfferRescinded message) Called when resource offers are rescinded.
CompletableFuture<Void>	postStop()
protected CompletableFuture<Void>	prepareLeadershipAsync()
void	reconcile(org.apache.flink.mesos.scheduler.ReconciliationCoordinator.Reconcile message) Handles a reconciliation request from a task monitor.
protected void	registered(Registered message)
protected void	reregistered(ReRegistered message) Called when reconnected to Mesos following a failover event.
protected void	resourceOffers(ResourceOffers message) Called when resource offers are made to the framework.
protected void	slaveLost(SlaveLost message)
Collection<org.apache.flink.runtime.clusterframework.types.ResourceProfile>	startNewWorker(org.apache.flink.runtime.clusterframework.types.ResourceProfile resourceProfile)
protected void	statusUpdate(StatusUpdate message) Handles a task status update from Mesos.
boolean	stopWorker(RegisteredMesosWorkerNode workerNode)
void	taskTerminated(org.apache.flink.mesos.scheduler.TaskMonitor.TaskTerminated message) Handles a termination notification from a task monitor.
protected RegisteredMesosWorkerNode	workerStarted(org.apache.flink.runtime.clusterframework.types.ResourceID resourceID) Callback when a worker was started.

Methods inherited from class org.apache.flink.runtime.resourcemanager.ResourceManager

cancelSlotRequest, closeJobManagerConnection, closeTaskManagerConnection, createSlotsPerWorker, deregisterApplication, disconnectJobManager, disconnectTaskManager, getNumberOfRegisteredTaskManagers, getNumberRequiredTaskManagerSlots, grantLeadership, handleError, heartbeatFromJobManager, heartbeatFromTaskManager, jobLeaderLostLeadership, notifySlotAvailable, onFatalError, registerInfoMessageListener, registerJobManager, registerTaskExecutor, releaseResource, removeJob, requestResourceOverview, requestSlot, requestTaskManagerFileUpload, requestTaskManagerInfo, requestTaskManagerInfo, requestTaskManagerMetricQueryServicePaths, revokeLeadership, sendInfoMessage, sendSlotReport, start, unRegisterInfoMessageListener

Methods inherited from class org.apache.flink.runtime.rpc.FencedRpcEndpoint

callAsyncWithoutFencing, getFencingToken, getMainThreadExecutor, getUnfencedMainThreadExecutor, runAsyncWithoutFencing, setFencingToken

Methods inherited from class org.apache.flink.runtime.rpc.RpcEndpoint

callAsync, getAddress, getEndpointId, getHostname, getRpcService, getSelfGateway, getTerminationFuture, runAsync, scheduleRunAsync, scheduleRunAsync, shutDown, stop, validateRunsInMainThread

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.runtime.rpc.FencedRpcGateway

getFencingToken

Methods inherited from interface org.apache.flink.runtime.rpc.RpcGateway

getAddress, getHostname

Methods inherited from interface org.apache.flink.runtime.leaderelection.LeaderContender

getAddress

Field Detail

LOG

protected static final org.slf4j.Logger LOG

Constructor Detail

MesosResourceManager

public MesosResourceManager(org.apache.flink.runtime.rpc.RpcService rpcService,
                            String resourceManagerEndpointId,
                            org.apache.flink.runtime.clusterframework.types.ResourceID resourceId,
                            org.apache.flink.runtime.highavailability.HighAvailabilityServices highAvailabilityServices,
                            org.apache.flink.runtime.heartbeat.HeartbeatServices heartbeatServices,
                            org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager slotManager,
                            org.apache.flink.runtime.metrics.MetricRegistry metricRegistry,
                            org.apache.flink.runtime.resourcemanager.JobLeaderIdService jobLeaderIdService,
                            org.apache.flink.runtime.entrypoint.ClusterInformation clusterInformation,
                            org.apache.flink.runtime.rpc.FatalErrorHandler fatalErrorHandler,
                            org.apache.flink.configuration.Configuration flinkConfig,
                            MesosServices mesosServices,
                            MesosConfiguration mesosConfig,
                            MesosTaskManagerParameters taskManagerParameters,
                            org.apache.flink.runtime.clusterframework.ContainerSpecification taskManagerContainerSpec,
                            @Nullable
                            String webUiUrl,
                            org.apache.flink.runtime.metrics.groups.JobManagerMetricGroup jobManagerMetricGroup)

Method Detail

createSelfActor

protected akka.actor.ActorRef createSelfActor()

createConnectionMonitor

protected akka.actor.ActorRef createConnectionMonitor()

createTaskMonitor

protected akka.actor.ActorRef createTaskMonitor(org.apache.mesos.SchedulerDriver schedulerDriver)

createLaunchCoordinator

protected akka.actor.ActorRef createLaunchCoordinator(org.apache.mesos.SchedulerDriver schedulerDriver,
                                                      akka.actor.ActorRef selfActor)

createReconciliationCoordinator

protected akka.actor.ActorRef createReconciliationCoordinator(org.apache.mesos.SchedulerDriver schedulerDriver)

initialize

protected void initialize()
                   throws org.apache.flink.runtime.resourcemanager.exceptions.ResourceManagerException

Specified by:

initialize in class org.apache.flink.runtime.resourcemanager.ResourceManager<RegisteredMesosWorkerNode>

Throws:

org.apache.flink.runtime.resourcemanager.exceptions.ResourceManagerException

prepareLeadershipAsync

protected CompletableFuture<Void> prepareLeadershipAsync()

Overrides:

prepareLeadershipAsync in class org.apache.flink.runtime.resourcemanager.ResourceManager<RegisteredMesosWorkerNode>

clearStateAsync

protected CompletableFuture<Void> clearStateAsync()

Overrides:

clearStateAsync in class org.apache.flink.runtime.resourcemanager.ResourceManager<RegisteredMesosWorkerNode>

postStop

public CompletableFuture<Void> postStop()

Overrides:

postStop in class org.apache.flink.runtime.resourcemanager.ResourceManager<RegisteredMesosWorkerNode>

internalDeregisterApplication

protected void internalDeregisterApplication(org.apache.flink.runtime.clusterframework.ApplicationStatus finalStatus,
                                             @Nullable
                                             String diagnostics)
                                      throws org.apache.flink.runtime.resourcemanager.exceptions.ResourceManagerException

Specified by:

internalDeregisterApplication in class org.apache.flink.runtime.resourcemanager.ResourceManager<RegisteredMesosWorkerNode>

Throws:

org.apache.flink.runtime.resourcemanager.exceptions.ResourceManagerException

startNewWorker

public Collection<org.apache.flink.runtime.clusterframework.types.ResourceProfile> startNewWorker(org.apache.flink.runtime.clusterframework.types.ResourceProfile resourceProfile)

Specified by:

startNewWorker in class org.apache.flink.runtime.resourcemanager.ResourceManager<RegisteredMesosWorkerNode>

stopWorker

public boolean stopWorker(RegisteredMesosWorkerNode workerNode)

Specified by:

stopWorker in class org.apache.flink.runtime.resourcemanager.ResourceManager<RegisteredMesosWorkerNode>

workerStarted

protected RegisteredMesosWorkerNode workerStarted(org.apache.flink.runtime.clusterframework.types.ResourceID resourceID)

Callback when a worker was started.

Specified by:

workerStarted in class org.apache.flink.runtime.resourcemanager.ResourceManager<RegisteredMesosWorkerNode>

Parameters:

resourceID - The worker resource id (as provided by the TaskExecutor)

registered

protected void registered(Registered message)

reregistered

protected void reregistered(ReRegistered message)

Called when reconnected to Mesos following a failover event.

disconnected

protected void disconnected(Disconnected message)

Called when disconnected from Mesos.

resourceOffers

protected void resourceOffers(ResourceOffers message)

Called when resource offers are made to the framework.

offerRescinded

protected void offerRescinded(OfferRescinded message)

Called when resource offers are rescinded.

statusUpdate

protected void statusUpdate(StatusUpdate message)

Handles a task status update from Mesos.

frameworkMessage

protected void frameworkMessage(FrameworkMessage message)

slaveLost

protected void slaveLost(SlaveLost message)

executorLost

protected void executorLost(ExecutorLost message)

acceptOffers

public void acceptOffers(AcceptOffers msg)

Accept offers as advised by the launch coordinator.

Acceptance is routed through the RM to update the persistent state before forwarding the message to Mesos.

reconcile

public void reconcile(org.apache.flink.mesos.scheduler.ReconciliationCoordinator.Reconcile message)

Handles a reconciliation request from a task monitor.

taskTerminated

public void taskTerminated(org.apache.flink.mesos.scheduler.TaskMonitor.TaskTerminated message)

Handles a termination notification from a task monitor.