alluxio.client.block.policy

Class DeterministicHashPolicy

java.lang.Object

alluxio.client.block.policy.DeterministicHashPolicy

All Implemented Interfaces:

BlockLocationPolicy

@NotThreadSafe
public final class DeterministicHashPolicy
extends java.lang.Object
implements BlockLocationPolicy

This policy maps the blockId to several deterministic Alluxio workers. The number of workers a block can be mapped to can be passed through the constructor. The default is 1. It skips the workers that do not have enough capacity to hold the block. This policy is useful for limiting the amount of replication that occurs when reading blocks from the UFS with high concurrency. With 30 workers and 100 remote clients reading the same block concurrently, the replication level for the block would get close to 30 as each worker reads and caches the block for one or more clients. If the clients use DeterministicHashPolicy with 3 shards, the 100 clients will split their reads between just 3 workers, so that the replication level for the block will be only 3 when the data is first loaded. Note that the hash function relies on the number of workers in the cluster, so if the number of workers changes, the workers chosen by the policy for a given block will likely change.

Nested Class Summary

Nested classes/interfaces inherited from interface alluxio.client.block.policy.BlockLocationPolicy

BlockLocationPolicy.Factory

Constructor Summary

Constructors

Constructor and Description
DeterministicHashPolicy(AlluxioConfiguration conf) Constructs a new DeterministicHashPolicy needed for instantiation in BlockLocationPolicy.Factory.

Method Summary

Modifier and Type	Method and Description
boolean	equals(java.lang.Object o)
java.util.Optional<WorkerNetAddress>	getWorker(GetWorkerOptions options) Gets the worker's network address for serving operations requested for the block.
int	hashCode()
java.lang.String	toString()

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Detail

DeterministicHashPolicy

public DeterministicHashPolicy(AlluxioConfiguration conf)

Constructs a new DeterministicHashPolicy needed for instantiation in BlockLocationPolicy.Factory.

Parameters:

conf - Alluxio configuration

Method Detail

getWorker

public java.util.Optional<WorkerNetAddress> getWorker(GetWorkerOptions options)

Description copied from interface: BlockLocationPolicy

Gets the worker's network address for serving operations requested for the block.

Specified by:

getWorker in interface BlockLocationPolicy

Parameters:

options - the options to get a block worker network address for a block

Returns:

the address of the worker to write to, or empty if no worker can be selected

equals

public boolean equals(java.lang.Object o)

Overrides:

equals in class java.lang.Object

hashCode

public int hashCode()

Overrides:

hashCode in class java.lang.Object

toString

public java.lang.String toString()

Overrides:

toString in class java.lang.Object