org.apache.flink.yarn.highavailability

Class AbstractYarnNonHaServices

java.lang.Object

org.apache.flink.yarn.highavailability.YarnHighAvailabilityServices

org.apache.flink.yarn.highavailability.AbstractYarnNonHaServices

All Implemented Interfaces:

AutoCloseable, org.apache.flink.runtime.highavailability.HighAvailabilityServices

Direct Known Subclasses:

YarnIntraNonHaMasterServices, YarnPreConfiguredMasterNonHaServices

public abstract class AbstractYarnNonHaServices
extends YarnHighAvailabilityServices

Abstract base class for the high availability services for Flink YARN applications that support no master fail over.

Internally, these services put their recovery data into YARN's working directory, except for checkpoints, which are in the configured checkpoint directory. That way, checkpoints can be resumed with a new job/application, even if the complete YARN application is killed and cleaned up.

Field Summary

Fields inherited from class org.apache.flink.yarn.highavailability.YarnHighAvailabilityServices

blobStoreService, FLINK_RECOVERY_DATA_DIR, flinkFileSystem, haDataDirectory, hadoopFileSystem, LOG, workingDirectory

Fields inherited from interface org.apache.flink.runtime.highavailability.HighAvailabilityServices

DEFAULT_JOB_ID, DEFAULT_LEADER_ID

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	AbstractYarnNonHaServices(org.apache.flink.configuration.Configuration config, org.apache.hadoop.conf.Configuration hadoopConf) Creates new YARN high-availability services, configuring the file system and recovery data directory based on the working directory in the given Hadoop configuration.

Method Summary

Modifier and Type	Method and Description
org.apache.flink.runtime.checkpoint.CheckpointRecoveryFactory	getCheckpointRecoveryFactory()
org.apache.flink.runtime.highavailability.RunningJobsRegistry	getRunningJobsRegistry()
org.apache.flink.runtime.jobmanager.SubmittedJobGraphStore	getSubmittedJobGraphStore()

Methods inherited from class org.apache.flink.yarn.highavailability.YarnHighAvailabilityServices

close, closeAndCleanupAllData, createBlobStore, forSingleJobAppMaster, forYarnTaskManager, isClosed

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.runtime.highavailability.HighAvailabilityServices

getDispatcherLeaderElectionService, getDispatcherLeaderRetriever, getJobManagerLeaderElectionService, getJobManagerLeaderRetriever, getJobManagerLeaderRetriever, getResourceManagerLeaderElectionService, getResourceManagerLeaderRetriever, getWebMonitorLeaderElectionService, getWebMonitorLeaderRetriever

Constructor Detail

AbstractYarnNonHaServices

protected AbstractYarnNonHaServices(org.apache.flink.configuration.Configuration config,
                                    org.apache.hadoop.conf.Configuration hadoopConf)
                             throws IOException

Creates new YARN high-availability services, configuring the file system and recovery data directory based on the working directory in the given Hadoop configuration.

This class requires that the default Hadoop file system configured in the given Hadoop configuration is an HDFS.

Parameters:

config - The Flink configuration of this component / process.

hadoopConf - The Hadoop configuration for the YARN cluster.

Throws:

IOException - Thrown, if the initialization of the Hadoop file system used by YARN fails.

Method Detail

getRunningJobsRegistry

public org.apache.flink.runtime.highavailability.RunningJobsRegistry getRunningJobsRegistry()
                                                                                     throws IOException

Throws:

IOException

getCheckpointRecoveryFactory

public org.apache.flink.runtime.checkpoint.CheckpointRecoveryFactory getCheckpointRecoveryFactory()

getSubmittedJobGraphStore

public org.apache.flink.runtime.jobmanager.SubmittedJobGraphStore getSubmittedJobGraphStore()
                                                                                     throws Exception

Throws:

Exception