Class Zdict
- java.lang.Object
-
- org.lwjgl.util.zstd.Zdict
-
public class Zdict extends java.lang.ObjectNative bindings to the dictionary builder API of Zstandard (zstd).
-
-
Field Summary
Fields Modifier and Type Field Description static intZDICT_CONTENTSIZE_MIN
ZDICT_DICTSIZE_MIN
-
Method Summary
All Methods Static Methods Concrete Methods Modifier and Type Method Description static longnZDICT_finalizeDictionary(long dictBuffer, long dictBufferCapacity, long dictContent, long dictContentSize, long samplesBuffer, long samplesSizes, int nbSamples, long parameters)Unsafe version of:finalizeDictionarystatic intnZDICT_getDictID(long dictBuffer, long dictSize)Unsafe version of:getDictIDstatic longnZDICT_getErrorName(long errorCode)static intnZDICT_isError(long errorCode)static longnZDICT_optimizeTrainFromBuffer_cover(long dictBuffer, long dictBufferCapacity, long samplesBuffer, long samplesSizes, int nbSamples, long parameters)Unsafe version of:optimizeTrainFromBuffer_coverstatic longnZDICT_trainFromBuffer(long dictBuffer, long dictBufferCapacity, long samplesBuffer, long samplesSizes, int nbSamples)Unsafe version of:trainFromBufferstatic longnZDICT_trainFromBuffer_cover(long dictBuffer, long dictBufferCapacity, long samplesBuffer, long samplesSizes, int nbSamples, long parameters)Unsafe version of:trainFromBuffer_coverstatic longZDICT_finalizeDictionary(java.nio.ByteBuffer dictBuffer, java.nio.ByteBuffer dictContent, java.nio.ByteBuffer samplesBuffer, org.lwjgl.PointerBuffer samplesSizes, ZDICTParams parameters)Given a custom content as a basis for dictionary, and a set of samples, finalize dictionary by adding headers and statistics.static intZDICT_getDictID(java.nio.ByteBuffer dictBuffer)ExtractsdictID.static java.lang.StringZDICT_getErrorName(long errorCode)static booleanZDICT_isError(long errorCode)static longZDICT_optimizeTrainFromBuffer_cover(java.nio.ByteBuffer dictBuffer, java.nio.ByteBuffer samplesBuffer, org.lwjgl.PointerBuffer samplesSizes, ZDICTCoverParams parameters)The same requirements astrainFromBuffer_coverhold for all the parameters exceptparameters.static longZDICT_trainFromBuffer(java.nio.ByteBuffer dictBuffer, java.nio.ByteBuffer samplesBuffer, org.lwjgl.PointerBuffer samplesSizes)Train a dictionary from an array of samples.static longZDICT_trainFromBuffer_cover(java.nio.ByteBuffer dictBuffer, java.nio.ByteBuffer samplesBuffer, org.lwjgl.PointerBuffer samplesSizes, ZDICTCoverParams parameters)Train a dictionary from an array of samples using the COVER algorithm.
-
-
-
Field Detail
-
ZDICT_CONTENTSIZE_MIN
public static final int ZDICT_CONTENTSIZE_MIN
- See Also:
- Constant Field Values
-
ZDICT_DICTSIZE_MIN
public static final int ZDICT_DICTSIZE_MIN
- See Also:
- Constant Field Values
-
-
Method Detail
-
nZDICT_trainFromBuffer
public static long nZDICT_trainFromBuffer(long dictBuffer, long dictBufferCapacity, long samplesBuffer, long samplesSizes, int nbSamples)Unsafe version of:trainFromBuffer
-
ZDICT_trainFromBuffer
public static long ZDICT_trainFromBuffer(java.nio.ByteBuffer dictBuffer, java.nio.ByteBuffer samplesBuffer, org.lwjgl.PointerBuffer samplesSizes)Train a dictionary from an array of samples.Redirect towards
optimizeTrainFromBuffer_coversingle-threaded, withd=8andsteps=4.Samples must be stored concatenated in a single flat buffer
samplesBuffer, supplied with an array of sizessamplesSizes, providing the size of each sample, in order.The resulting dictionary will be saved into
dictBuffer.Note:
ZDICT_trainFromBuffer()requires about 9 bytes of memory for each input byte.Tips:
- In general, a reasonable dictionary has a size of ~ 100 KB.
- It's possible to select smaller or larger size, just by specifying
dictBufferCapacity. - In general, it's recommended to provide a few thousands samples, though this can vary a lot.
- It's recommended that total size of all samples be about ~x100 times the target size of dictionary.
- Returns:
- size of dictionary stored into
dictBuffer(≤dictBufferCapacity) or an error code, which can be tested withisError.
-
nZDICT_getDictID
public static int nZDICT_getDictID(long dictBuffer, long dictSize)Unsafe version of:getDictID
-
ZDICT_getDictID
public static int ZDICT_getDictID(java.nio.ByteBuffer dictBuffer)
ExtractsdictID.- Returns:
- zero if error (not a valid dictionary)
-
nZDICT_isError
public static int nZDICT_isError(long errorCode)
-
ZDICT_isError
public static boolean ZDICT_isError(long errorCode)
-
nZDICT_getErrorName
public static long nZDICT_getErrorName(long errorCode)
-
ZDICT_getErrorName
@Nullable public static java.lang.String ZDICT_getErrorName(long errorCode)
-
nZDICT_trainFromBuffer_cover
public static long nZDICT_trainFromBuffer_cover(long dictBuffer, long dictBufferCapacity, long samplesBuffer, long samplesSizes, int nbSamples, long parameters)Unsafe version of:trainFromBuffer_cover
-
ZDICT_trainFromBuffer_cover
public static long ZDICT_trainFromBuffer_cover(java.nio.ByteBuffer dictBuffer, java.nio.ByteBuffer samplesBuffer, org.lwjgl.PointerBuffer samplesSizes, ZDICTCoverParams parameters)Train a dictionary from an array of samples using the COVER algorithm.Samples must be stored concatenated in a single flat buffer
samplesBuffer, supplied with an array of sizessamplesSizes, providing the size of each sample, in order.The resulting dictionary will be saved into
dictBuffer.Note:
ZDICT_trainFromBuffer_cover()requires about 9 bytes of memory for each input byte.Tips:
- In general, a reasonable dictionary has a size of ~ 100 KB.
- It's possible to select smaller or larger szie, just by specifying
dictBufferCapacity. - In general, it's recommended to provide a few thousands samples, though this can vary a lot.
- It's recommended that total size of all samples be about ~x100 times the target size of dictionary.
- Returns:
- size of dictionary stored into
dictBuffer(≤dictBufferCapacity) or an error code, which can be tested withisError.
-
nZDICT_optimizeTrainFromBuffer_cover
public static long nZDICT_optimizeTrainFromBuffer_cover(long dictBuffer, long dictBufferCapacity, long samplesBuffer, long samplesSizes, int nbSamples, long parameters)Unsafe version of:optimizeTrainFromBuffer_cover
-
ZDICT_optimizeTrainFromBuffer_cover
public static long ZDICT_optimizeTrainFromBuffer_cover(java.nio.ByteBuffer dictBuffer, java.nio.ByteBuffer samplesBuffer, org.lwjgl.PointerBuffer samplesSizes, ZDICTCoverParams parameters)The same requirements astrainFromBuffer_coverhold for all the parameters exceptparameters.This function tries many parameter combinations and picks the best parameters.
*parametersis filled with the best parameters found, dictionary constructed with those parameters is stored indictBuffer.- All of the parameters
d,k,stepsare optional. - If
dis non-zero then we don't check multiple values of d, otherwise we checkd = {6, 8, 10, 12, 14, 16}. - If
stepsis zero it defaults to its default value. - If
kis non-zero then we don't check multiple values ofk, otherwise we check steps values in[16, 2048].
Note:
ZDICT_optimizeTrainFromBuffer_cover()requires about 8 bytes of memory for each input byte and additionally another 5 bytes of memory for each byte of memory for each thread.- Returns:
- size of dictionary stored into
dictBuffer(≤dictBufferCapacity) or an error code, which can be tested withisError. On success*parameterscontains the parameters selected.
- All of the parameters
-
nZDICT_finalizeDictionary
public static long nZDICT_finalizeDictionary(long dictBuffer, long dictBufferCapacity, long dictContent, long dictContentSize, long samplesBuffer, long samplesSizes, int nbSamples, long parameters)Unsafe version of:finalizeDictionary
-
ZDICT_finalizeDictionary
public static long ZDICT_finalizeDictionary(java.nio.ByteBuffer dictBuffer, java.nio.ByteBuffer dictContent, java.nio.ByteBuffer samplesBuffer, org.lwjgl.PointerBuffer samplesSizes, ZDICTParams parameters)Given a custom content as a basis for dictionary, and a set of samples, finalize dictionary by adding headers and statistics.Samples must be stored concatenated in a flat buffer
samplesBuffer, supplied with an array of sizessamplesSizes, providing the size of each sample in order.Notes:
dictContentSizemust be ≥CONTENTSIZE_MINbytes.maxDictSizemust be ≥dictContentSize, and must be ≥DICTSIZE_MINbytes.ZDICT_finalizeDictionary()will push notifications intostderrif instructed to, usingnotificationLevel>0.dictBufferanddictContentcan overlap.
- Returns:
- size of dictionary stored into
dictBuffer(≤dictBufferCapacity) or an error code, which can be tested withisError.
-
-