org.gridgain.grid.cache.store

Class GridCacheLoadOnlyStoreAdapter<K,V,I>

java.lang.Object

org.gridgain.grid.cache.store.GridCacheLoadOnlyStoreAdapter<K,V,I>

Type Parameters:

K - Key type.

V - Value type.

I - Input type.

All Implemented Interfaces:

GridCacheStore<K,V>

public abstract class GridCacheLoadOnlyStoreAdapter<K,V,I>
extends Object
implements GridCacheStore<K,V>

This adepter designed to support stores with bulk loading from stream-like source.

This class processes input data in the following way:

• Iterator of input record obtained from user-defined inputIterator(Object...).
• Iterator continuously queried for input records and they are grouped into batches of batchSize.
• Batch is placed into processing queue and puled by one of threadsCnt working threads.
• Each record in batch is passed to user-defined parse(Object, Object...) method and result is stored into cache.

Two methods should be implemented by inheritants:

• inputIterator(Object...). It should open underlying data source and iterate all record available in it. Individual records could be in very raw form, like text lines for CSV files.
• parse(Object, Object...). This method should process input records and transform them into key-value pairs for cache.

Field Summary

Fields

Modifier and Type	Field and Description
static int	DFLT_BATCH_QUEUE_SIZE Default batch queue size (max batches count to limit memory usage).
static int	DFLT_BATCH_SIZE Default batch size (number of records read with inputIterator(Object...) and then submitted to internal pool at a time).
static int	DFLT_THREADS_COUNT Default number of working threads (equal to the number of available processors).

Constructor Summary

Constructors

Constructor and Description
GridCacheLoadOnlyStoreAdapter()

Method Summary

Methods

Modifier and Type	Method and Description
int	getBatchQueueSize() Returns batch queue size.
int	getBatchSize() Returns batch size.
int	getThreadsCount() Returns number of worker threads.
protected abstract Iterator<I>	inputIterator(Object... args) Returns iterator of input records.
V	load(GridCacheTx tx, K key) Loads value for the key from underlying persistent storage.
void	loadAll(GridCacheTx tx, Collection<? extends K> keys, GridBiInClosure<K,V> c) Loads all values for given keys and passes every value to the provided closure.
void	loadCache(GridBiInClosure<K,V> c, Object... args) Loads all values from underlying persistent storage.
protected abstract GridBiTuple<K,V>	parse(I rec, Object... args) This method should transform raw data records into valid key-value pairs to be stored into cache.
void	put(GridCacheTx tx, K key, V val) Stores a given value in persistent storage.
void	putAll(GridCacheTx tx, Map<? extends K,? extends V> map) Stores given key value pairs in persistent storage.
void	remove(GridCacheTx tx, K key) Removes the value identified by given key from persistent storage.
void	removeAll(GridCacheTx tx, Collection<? extends K> keys) Removes all vales identified by given keys from persistent storage.
void	setBatchQueueSize(int batchQueueSize) Sets batch queue size.
void	setBatchSize(int batchSize) Sets batch size.
void	setThreadsCount(int threadsCnt) Sets number of worker threads.
void	txEnd(GridCacheTx tx, boolean commit) Tells store to commit or rollback a transaction depending on the value of the 'commit' parameter.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DFLT_BATCH_SIZE

public static final int DFLT_BATCH_SIZE

Default batch size (number of records read with inputIterator(Object...) and then submitted to internal pool at a time).

See Also:

Constant Field Values

DFLT_BATCH_QUEUE_SIZE

public static final int DFLT_BATCH_QUEUE_SIZE

Default batch queue size (max batches count to limit memory usage).

See Also:

Constant Field Values

DFLT_THREADS_COUNT

public static final int DFLT_THREADS_COUNT

Default number of working threads (equal to the number of available processors).

Constructor Detail

GridCacheLoadOnlyStoreAdapter

public GridCacheLoadOnlyStoreAdapter()

Method Detail

inputIterator

protected abstract Iterator<I> inputIterator(@Nullable
                        Object... args)
                                      throws GridException

Returns iterator of input records.

Note that returned iterator doesn't have to be thread-safe. Thus it could operate on raw streams, DB connections, etc. without additional synchronization.

Parameters:

args - Arguments passes into GridCache.loadCache(GridBiPredicate, long, Object...) method.

Returns:

Iterator over input records.

Throws:

GridException - If iterator can't be created with the given arguments.

parse

@Nullable
protected abstract GridBiTuple<K,V> parse(I rec,
                              @Nullable
                              Object... args)

This method should transform raw data records into valid key-value pairs to be stored into cache.

If null is returned then this record will be just skipped.

Parameters:

rec - A raw data record.

args - Arguments passed into GridCache.loadCache(GridBiPredicate, long, Object...) method.

Returns:

Cache entry to be saved in cache or null if no entry could be produced from this record.

loadCache

public void loadCache(GridBiInClosure<K,V> c,
             @Nullable
             Object... args)
               throws GridException

Loads all values from underlying persistent storage. Note that keys are not passed, so it is up to implementation to figure out what to load. This method is called whenever GridCache.loadCache(GridBiPredicate, long, Object...) method is invoked which is usually to preload the cache from persistent storage.

This method is optional, and cache implementation does not depend on this method to do anything. Default implementation of this method in GridCacheStoreAdapter does nothing.

For every loaded value method GridBiInClosure.apply(Object, Object) should be called on the passed in closure. The closure will then make sure that the loaded value is stored in cache.

Specified by:

loadCache in interface GridCacheStore<K,V>

Parameters:

c - Closure for loaded values.

args - Arguments passes into GridCache.loadCache(GridBiPredicate, long, Object...) method.

Throws:

GridException - If loading failed.

getBatchSize

public int getBatchSize()

Returns batch size.

Returns:

Batch size.

setBatchSize

public void setBatchSize(int batchSize)

Sets batch size.

Parameters:

batchSize - Batch size.

getBatchQueueSize

public int getBatchQueueSize()

Returns batch queue size.

Returns:

Batch queue size.

setBatchQueueSize

public void setBatchQueueSize(int batchQueueSize)

Sets batch queue size.

Parameters:

batchQueueSize - Batch queue size.

getThreadsCount

public int getThreadsCount()

Returns number of worker threads.

Returns:

Number of worker threads.

setThreadsCount

public void setThreadsCount(int threadsCnt)

Sets number of worker threads.

Parameters:

threadsCnt - Number of worker threads.

load

public V load(@Nullable
     GridCacheTx tx,
     K key)
       throws GridException

Loads value for the key from underlying persistent storage.

Specified by:

load in interface GridCacheStore<K,V>

Parameters:

tx - Cache transaction.

key - Key to load.

Returns:

Loaded value or null if value was not found.

Throws:

GridException - If load failed.

loadAll

public void loadAll(@Nullable
           GridCacheTx tx,
           @Nullable
           Collection<? extends K> keys,
           GridBiInClosure<K,V> c)
             throws GridException

Loads all values for given keys and passes every value to the provided closure.

For every loaded value method GridInClosure.apply(Object) should be called on the passed in closure. The closure will then make sure that the loaded value is stored in cache.

Specified by:

loadAll in interface GridCacheStore<K,V>

Parameters:

tx - Cache transaction.

keys - Collection of keys to load.

c - Closure to call for every loaded element.

Throws:

GridException - If load failed.

put

public void put(@Nullable
       GridCacheTx tx,
       K key,
       @Nullable
       V val)
         throws GridException

Stores a given value in persistent storage. Note that cache transaction is implicitly created even for a single put. However, if write-behind is configured for a particular cache, transaction object passed in the cache store will be always null.

Specified by:

put in interface GridCacheStore<K,V>

Parameters:

tx - Cache transaction, if write-behind is not enabled, null otherwise.

key - Key to put.

val - Value to put.

Throws:

GridException - If put failed.

putAll

public void putAll(@Nullable
          GridCacheTx tx,
          @Nullable
          Map<? extends K,? extends V> map)
            throws GridException

Stores given key value pairs in persistent storage. Note that cache transaction is implicitly created even for a single put. However, if write-behind is configured for a particular cache, transaction object passed in the cache store will be always null.

Specified by:

putAll in interface GridCacheStore<K,V>

Parameters:

tx - Cache transaction, if write-behind is not enabled, null otherwise.

map - Values to store.

Throws:

GridException - If store failed.

remove

public void remove(@Nullable
          GridCacheTx tx,
          K key)
            throws GridException

Removes the value identified by given key from persistent storage. Note that cache transaction is implicitly created even for a single put. However, if write-behind is configured for a particular cache, transaction object passed in the cache store will be always null.

Specified by:

remove in interface GridCacheStore<K,V>

Parameters:

tx - Cache transaction, if write-behind is not enabled, null otherwise.

key - Key to remove.

Throws:

GridException - If remove failed.

removeAll

public void removeAll(@Nullable
             GridCacheTx tx,
             @Nullable
             Collection<? extends K> keys)
               throws GridException

Removes all vales identified by given keys from persistent storage. Note that cache transaction is implicitly created even for a single put. However, if write-behind is configured for a particular cache, transaction object passed in the cache store will be always null.

Specified by:

removeAll in interface GridCacheStore<K,V>

Parameters:

tx - Cache transaction, if write-behind is not enabled, null otherwise.

keys - Keys to remove.

Throws:

GridException - If remove failed.

txEnd

public void txEnd(GridCacheTx tx,
         boolean commit)
           throws GridException

Tells store to commit or rollback a transaction depending on the value of the 'commit' parameter.

Note that if explicit transactions are not used in code, then it is possible to commit or rollback transactions directly in 'put(..)', or 'remove(..)' methods. In that case, this method should be left empty (GridCacheStoreAdapter provides empty implementation of this method).

Specified by:

txEnd in interface GridCacheStore<K,V>

Parameters:

tx - Cache transaction being ended.

commit - True if transaction should commit, false for rollback.

Throws:

GridException - If commit or rollback failed. Note that commit failure in some cases may bring cache transaction into GridCacheTxState.UNKNOWN which will consequently cause all transacted entries to be invalidated.