org.apache.flink.streaming.api.functions.sink.filesystem.rollingpolicies

Class DefaultRollingPolicy<IN,BucketID>

java.lang.Object

org.apache.flink.streaming.api.functions.sink.filesystem.rollingpolicies.DefaultRollingPolicy<IN,BucketID>

All Implemented Interfaces:

Serializable, RollingPolicy<IN,BucketID>

@PublicEvolving
public final class DefaultRollingPolicy<IN,BucketID>
extends Object
implements RollingPolicy<IN,BucketID>

The default implementation of the RollingPolicy.

This policy rolls a part file if:

1. there is no open part file,
2. the current file has reached the maximum bucket size (by default 128MB),
3. the current file is older than the roll over interval (by default 60 sec), or
4. the current file has not been written to for more than the allowed inactivityTime (by default 60 sec).

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	DefaultRollingPolicy.PolicyBuilder A helper class that holds the configuration properties for the DefaultRollingPolicy.

Method Summary

Modifier and Type	Method and Description
static DefaultRollingPolicy.PolicyBuilder	create() Initiates the instantiation of a DefaultRollingPolicy.
boolean	shouldRollOnCheckpoint(PartFileInfo<BucketID> partFileState) Determines if the in-progress part file for a bucket should roll on every checkpoint.
boolean	shouldRollOnEvent(PartFileInfo<BucketID> partFileState, IN element) Determines if the in-progress part file for a bucket should roll based on its current state, e.g.
boolean	shouldRollOnProcessingTime(PartFileInfo<BucketID> partFileState, long currentTime) Determines if the in-progress part file for a bucket should roll based on a time condition.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

shouldRollOnCheckpoint

public boolean shouldRollOnCheckpoint(PartFileInfo<BucketID> partFileState)
                               throws IOException

Description copied from interface: RollingPolicy

Determines if the in-progress part file for a bucket should roll on every checkpoint.

Specified by:

shouldRollOnCheckpoint in interface RollingPolicy<IN,BucketID>

Parameters:

partFileState - the state of the currently open part file of the bucket.

Returns:

True if the part file should roll, false otherwise.

Throws:

IOException

shouldRollOnEvent

public boolean shouldRollOnEvent(PartFileInfo<BucketID> partFileState,
                                 IN element)
                          throws IOException

Description copied from interface: RollingPolicy

Determines if the in-progress part file for a bucket should roll based on its current state, e.g. its size.

Specified by:

shouldRollOnEvent in interface RollingPolicy<IN,BucketID>

Parameters:

partFileState - the state of the currently open part file of the bucket.

element - the element being processed.

Returns:

True if the part file should roll, false otherwise.

Throws:

IOException

shouldRollOnProcessingTime

public boolean shouldRollOnProcessingTime(PartFileInfo<BucketID> partFileState,
                                          long currentTime)

Description copied from interface: RollingPolicy

Determines if the in-progress part file for a bucket should roll based on a time condition.

Specified by:

shouldRollOnProcessingTime in interface RollingPolicy<IN,BucketID>

Parameters:

partFileState - the state of the currently open part file of the bucket.

currentTime - the current processing time.

Returns:

True if the part file should roll, false otherwise.

create

public static DefaultRollingPolicy.PolicyBuilder create()

Initiates the instantiation of a DefaultRollingPolicy. To finalize it and have the actual policy, call .create().