org.apache.flink.yarn

Class YarnClusterClient

java.lang.Object

org.apache.flink.client.program.ClusterClient<org.apache.hadoop.yarn.api.records.ApplicationId>

org.apache.flink.yarn.YarnClusterClient

public class YarnClusterClient
extends org.apache.flink.client.program.ClusterClient<org.apache.hadoop.yarn.api.records.ApplicationId>

Java representation of a running Flink cluster within YARN.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.flink.client.program.ClusterClient

org.apache.flink.client.program.ClusterClient.LazyActorSystemLoader

Field Summary

Fields inherited from class org.apache.flink.client.program.ClusterClient

actorSystemLoader, flinkConfig, highAvailabilityServices, lastJobExecutionResult, log, MAX_SLOTS_UNKNOWN, timeout

Constructor Summary

Constructors

Constructor and Description
YarnClusterClient(AbstractYarnClusterDescriptor clusterDescriptor, int numberTaskManagers, int slotsPerTaskManager, org.apache.hadoop.yarn.api.records.ApplicationReport appReport, org.apache.flink.configuration.Configuration flinkConfig, boolean newlyCreatedCluster) Create a new Flink on YARN cluster.

Method Summary

Modifier and Type	Method and Description
org.apache.hadoop.yarn.api.records.ApplicationId	getApplicationId()
org.apache.hadoop.yarn.api.records.ApplicationId	getClusterId()
org.apache.flink.runtime.clusterframework.messages.GetClusterStatusResponse	getClusterStatus() This method is only available if the cluster hasn't been started in detached mode.
org.apache.flink.configuration.Configuration	getFlinkConfiguration()
int	getMaxSlots()
List<String>	getNewMessages()
String	getWebInterfaceURL()
boolean	hasUserJarsInClassPath(List<URL> userJarFiles)
boolean	isDetached()
void	shutDownCluster()
org.apache.flink.api.common.JobSubmissionResult	submitJob(org.apache.flink.runtime.jobgraph.JobGraph jobGraph, ClassLoader classLoader)
void	waitForClusterToBeReady() Blocks until all TaskManagers are connected to the JobManager.

Methods inherited from class org.apache.flink.client.program.ClusterClient

cancel, cancelWithSavepoint, connectToJob, disposeSavepoint, endSession, endSessions, getAccumulators, getAccumulators, getClusterConnectionInfo, getJobGraph, getJobGraph, getJobManagerGateway, getJobStatus, getOptimizedPlan, getOptimizedPlan, getOptimizedPlanAsJson, getPrintStatusDuringExecution, listJobs, logAndSysout, rescaleJob, retrieveJob, run, run, run, run, run, run, runDetached, setDetached, setPrintStatusDuringExecution, shutdown, stop, triggerSavepoint

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

YarnClusterClient

public YarnClusterClient(AbstractYarnClusterDescriptor clusterDescriptor,
                         int numberTaskManagers,
                         int slotsPerTaskManager,
                         org.apache.hadoop.yarn.api.records.ApplicationReport appReport,
                         org.apache.flink.configuration.Configuration flinkConfig,
                         boolean newlyCreatedCluster)
                  throws Exception

Create a new Flink on YARN cluster.

Parameters:

clusterDescriptor - The descriptor used at cluster creation

numberTaskManagers - The number of task managers, -1 if unknown

slotsPerTaskManager - Slots per task manager, -1 if unknown

appReport - the YARN application ID

flinkConfig - Flink configuration

newlyCreatedCluster - Indicator whether this cluster has just been created

Throws:

IOException

org.apache.hadoop.yarn.exceptions.YarnException

Exception

Method Detail

getFlinkConfiguration

public org.apache.flink.configuration.Configuration getFlinkConfiguration()

Overrides:

getFlinkConfiguration in class org.apache.flink.client.program.ClusterClient<org.apache.hadoop.yarn.api.records.ApplicationId>

getMaxSlots

public int getMaxSlots()

Specified by:

getMaxSlots in class org.apache.flink.client.program.ClusterClient<org.apache.hadoop.yarn.api.records.ApplicationId>

hasUserJarsInClassPath

public boolean hasUserJarsInClassPath(List<URL> userJarFiles)

Specified by:

hasUserJarsInClassPath in class org.apache.flink.client.program.ClusterClient<org.apache.hadoop.yarn.api.records.ApplicationId>

submitJob

public org.apache.flink.api.common.JobSubmissionResult submitJob(org.apache.flink.runtime.jobgraph.JobGraph jobGraph,
                                                                 ClassLoader classLoader)
                                                          throws org.apache.flink.client.program.ProgramInvocationException

Specified by:

submitJob in class org.apache.flink.client.program.ClusterClient<org.apache.hadoop.yarn.api.records.ApplicationId>

Throws:

org.apache.flink.client.program.ProgramInvocationException

getWebInterfaceURL

public String getWebInterfaceURL()

Specified by:

getWebInterfaceURL in class org.apache.flink.client.program.ClusterClient<org.apache.hadoop.yarn.api.records.ApplicationId>

getClusterStatus

public org.apache.flink.runtime.clusterframework.messages.GetClusterStatusResponse getClusterStatus()

This method is only available if the cluster hasn't been started in detached mode.

Specified by:

getClusterStatus in class org.apache.flink.client.program.ClusterClient<org.apache.hadoop.yarn.api.records.ApplicationId>

getNewMessages

public List<String> getNewMessages()

Specified by:

getNewMessages in class org.apache.flink.client.program.ClusterClient<org.apache.hadoop.yarn.api.records.ApplicationId>

getClusterId

public org.apache.hadoop.yarn.api.records.ApplicationId getClusterId()

Specified by:

getClusterId in class org.apache.flink.client.program.ClusterClient<org.apache.hadoop.yarn.api.records.ApplicationId>

isDetached

public boolean isDetached()

Overrides:

isDetached in class org.apache.flink.client.program.ClusterClient<org.apache.hadoop.yarn.api.records.ApplicationId>

waitForClusterToBeReady

public void waitForClusterToBeReady()

Blocks until all TaskManagers are connected to the JobManager.

Specified by:

waitForClusterToBeReady in class org.apache.flink.client.program.ClusterClient<org.apache.hadoop.yarn.api.records.ApplicationId>

shutDownCluster

public void shutDownCluster()

Overrides:

shutDownCluster in class org.apache.flink.client.program.ClusterClient<org.apache.hadoop.yarn.api.records.ApplicationId>

getApplicationId

public org.apache.hadoop.yarn.api.records.ApplicationId getApplicationId()