alluxio.master.file.activesync

Class ActiveSyncManager

java.lang.Object

alluxio.master.file.activesync.ActiveSyncManager

All Implemented Interfaces:

Checkpointed, Journaled, JournalEntryIterable

@NotThreadSafe
public class ActiveSyncManager
extends java.lang.Object
implements Journaled

Manager for the Active UFS sync process. There are several threads cooperating to make the active sync process happen. 1. An active polling thread that polls HDFS for change events and aggregates these events. 2. A heartbeat thread that wakes up periodically to consume the aggregated events, and perform syncing if necessary. 3. For initial syncing, we launch a future to perform initial syncing asynchronously. This is stored in mSyncPathStatus.

Constructor Summary

Constructors

Constructor and Description
ActiveSyncManager(MountTable mountTable, FileSystemMaster fileSystemMaster) Constructs a Active Sync Manager.

Method Summary

Modifier and Type	Method and Description
void	applyAndJournal(java.util.function.Supplier<JournalContext> context, File.AddSyncPointEntry entry) Apply AddSyncPoint entry and journal the entry.
void	applyAndJournal(java.util.function.Supplier<JournalContext> context, File.RemoveSyncPointEntry entry) Apply removeSyncPoint entry and journal the entry.
CheckpointName	getCheckpointName()
java.util.concurrent.ExecutorService	getExecutor() Get SyncManager Executor.
java.util.List<AlluxioURI>	getFilterList(long mountId) Get the filter list associated with mount Id.
java.util.Iterator<Journal.JournalEntry>	getJournalEntryIterator()
java.util.concurrent.locks.Lock	getSyncManagerLock() Gets the lock protecting the syncManager.
java.util.List<SyncPointInfo>	getSyncPathList() Get the sync point list.
boolean	isActivelySynced(AlluxioURI path) Check if a URI is actively synced.
void	launchPollingThread(long mountId, long txId) Launches polling thread on a particular mount point with starting txId.
boolean	processJournalEntry(Journal.JournalEntry entry)
void	recoverFromStartSync(AlluxioURI uri, long mountId) Recover from start sync operation.
void	recoverFromStopSync(AlluxioURI uri, long mountId) Recover from a stop sync operation.
void	resetState() It clears all sync points, and stops the polling thread.
MountTable.Resolution	resolveSyncPoint(AlluxioURI syncPoint) Perform various checks of stopping a sync point.
void	setTxId(long mountId, long txId) set the transaction id for a particular mountId.
void	start() start the polling threads.
void	startSyncPostJournal(AlluxioURI uri) Continue to start sync after we have journaled the operation.
void	stop() Stops the sync manager and any outstanding threads, does not change the sync points.
void	stopSyncForMount(long mountId) stop active sync on a mount id.
void	stopSyncInternal(AlluxioURI syncPoint, long mountId) stop active sync on a URI.
void	stopSyncPostJournal(AlluxioURI syncPoint) Clean up tasks to stop sync point after we have journaled.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface alluxio.master.journal.Journaled

applyAndJournal, restoreFromCheckpoint, writeToCheckpoint

Constructor Detail

ActiveSyncManager

public ActiveSyncManager(MountTable mountTable,
                         FileSystemMaster fileSystemMaster)

Constructs a Active Sync Manager.

Parameters:

mountTable - mount table

fileSystemMaster - file system master

Method Detail

isActivelySynced

public boolean isActivelySynced(AlluxioURI path)

Check if a URI is actively synced.

Parameters:

path - path to check

Returns:

true if a URI is being actively synced

getSyncManagerLock

public java.util.concurrent.locks.Lock getSyncManagerLock()

Gets the lock protecting the syncManager.

Returns:

syncmanager lock

start

public void start()
           throws java.io.IOException

start the polling threads.

Throws:

java.io.IOException

launchPollingThread

public void launchPollingThread(long mountId,
                                long txId)

Launches polling thread on a particular mount point with starting txId.

Parameters:

mountId - launch polling thread on a mount id

txId - specifies the transaction id to initialize the pollling thread

applyAndJournal

public void applyAndJournal(java.util.function.Supplier<JournalContext> context,
                            File.AddSyncPointEntry entry)

Apply AddSyncPoint entry and journal the entry.

Parameters:

context - journal context

entry - addSyncPoint entry

applyAndJournal

public void applyAndJournal(java.util.function.Supplier<JournalContext> context,
                            File.RemoveSyncPointEntry entry)

Apply removeSyncPoint entry and journal the entry.

Parameters:

context - journal context

entry - removeSyncPoint entry

stopSyncForMount

public void stopSyncForMount(long mountId)
                      throws InvalidPathException

stop active sync on a mount id.

Parameters:

mountId - mountId to stop active sync

Throws:

InvalidPathException

resolveSyncPoint

@Nullable
public MountTable.Resolution resolveSyncPoint(AlluxioURI syncPoint)
                                                 throws InvalidPathException

Perform various checks of stopping a sync point.

Parameters:

syncPoint - sync point to stop

Returns:

the path resolution result if successfully passed all checks

Throws:

InvalidPathException

stopSyncInternal

public void stopSyncInternal(AlluxioURI syncPoint,
                             long mountId)

stop active sync on a URI.

Parameters:

syncPoint - sync point to be stopped

mountId - mountId

getFilterList

public java.util.List<AlluxioURI> getFilterList(long mountId)

Get the filter list associated with mount Id.

Parameters:

mountId - mountId

Returns:

a list of URIs (sync points) associated with that mount id

getSyncPathList

public java.util.List<SyncPointInfo> getSyncPathList()

Get the sync point list.

Returns:

a list of URIs (sync points)

stopSyncPostJournal

public void stopSyncPostJournal(AlluxioURI syncPoint)
                         throws InvalidPathException

Clean up tasks to stop sync point after we have journaled.

Parameters:

syncPoint - the sync point to stop

Throws:

InvalidPathException

setTxId

public void setTxId(long mountId,
                    long txId)

set the transaction id for a particular mountId.

Parameters:

mountId - mount id

txId - transaction id

getExecutor

public java.util.concurrent.ExecutorService getExecutor()

Get SyncManager Executor.

Returns:

an executor for active syncing

stop

public void stop()

Stops the sync manager and any outstanding threads, does not change the sync points. This stops four things in the following order. 1. Stop any outstanding initial sync futures for the sync points. (syncFuture.cancel) 2. Stop the heartbeat thread that periodically wakes up to process events that have been recorded for the past heartbeat interval. 3. Tell the polling thread to stop monitoring the path for events 4. Stop the thread that is polling HDFS for events

startSyncPostJournal

public void startSyncPostJournal(AlluxioURI uri)
                          throws InvalidPathException

Continue to start sync after we have journaled the operation.

Parameters:

uri - the sync point that we are trying to start

Throws:

InvalidPathException

recoverFromStopSync

public void recoverFromStopSync(AlluxioURI uri,
                                long mountId)

Recover from a stop sync operation.

Parameters:

uri - uri to stop sync

mountId - mount id of the uri

recoverFromStartSync

public void recoverFromStartSync(AlluxioURI uri,
                                 long mountId)

Recover from start sync operation.

Parameters:

uri - uri to start sync

mountId - mount id of the uri

processJournalEntry

public boolean processJournalEntry(Journal.JournalEntry entry)

Specified by:

processJournalEntry in interface Journaled

resetState

public void resetState()

It clears all sync points, and stops the polling thread.

Specified by:

resetState in interface Journaled

getCheckpointName

public CheckpointName getCheckpointName()

Specified by:

getCheckpointName in interface Checkpointed

getJournalEntryIterator

public java.util.Iterator<Journal.JournalEntry> getJournalEntryIterator()

Specified by:

getJournalEntryIterator in interface JournalEntryIterable