Package org.lwjgl.util.zstd

Class ZstdX

  • public class ZstdX
extends java.lang.Object
    Native bindings to the experimental API of Zstandard (zstd).

    In this proposal, parameters are pushed one by one into an existing context, and then applied on all subsequent compression jobs. When no parameter is ever provided, CCtx is created with compression level ZSTD_CLEVEL_DEFAULT.

    This API is intended to replace all others experimental API. It can basically do all other use cases, and even new ones. In constrast with _advanced() variants, it stands a reasonable chance to become "stable", after a good testing period.

    Block level API

    Block functions produce and decode raw zstd blocks, without frame metadata.

    Frame metadata cost is typically ~18 bytes, which can be non-negligible for very small blocks (< 100 bytes). User will have to take in charge required information to regenerate data, such as compressed and content sizes.

    A few rules to respect:

    • Compressing and decompressing require a context structure. Use createCCtx and createDCtx.
    • It is necessary to init context before starting
      • compression : any ZSTD_compressBegin*() variant, including with dictionary
      • decompression : any ZSTD_decompressBegin*() variant, including with dictionary
      • copyCCtx and copyDCtx can be used too
    • Block size is limited, it must be ≤ getBlockSize ≤ BLOCKSIZE_MAX == 128 KB
      • If input is larger than a block size, it's necessary to split input data into multiple blocks.
      • For inputs larger than a single block size, consider using the regular compress instead. Frame metadata is not that costly, and quickly becomes negligible as source size grows larger.
    • When a block is considered not compressible enough, compressBlock result will be zero. In which case, nothing is produced into dst.
      • User must test for such outcome and deal directly with uncompressed data.
      • decompressBlock doesn't accept uncompressed data as input !!!
      • In case of multiple successive blocks, should some of them be uncompressed, decoder must be informed of their existence in order to follow proper history. Use insertBlock for such a case.

    • Field Detail

      • ZSTD_f_zstd1, ZSTD_f_zstd1_magicless

        ZSTD_format_e
        Enum values:
        • f_zstd1 - zstd frame format, specified in zstd_compression_format.md (default)
        • f_zstd1_magicless - Variant of zstd frame format, without initial 4-bytes magic number. Useful to save 4 bytes per generated frame. Decoder cannot recognise automatically this format, requiring instructions.

      • ZSTD_p_format, ZSTD_p_compressionLevel, ZSTD_p_windowLog, ZSTD_p_hashLog, ZSTD_p_chainLog, ZSTD_p_searchLog, ZSTD_p_minMatch, ZSTD_p_targetLength, ZSTD_p_compressionStrategy, ZSTD_p_enableLongDistanceMatching, ZSTD_p_ldmHashLog, ZSTD_p_ldmMinMatch, ZSTD_p_ldmBucketSizeLog, ZSTD_p_ldmHashEveryLog, ZSTD_p_contentSizeFlag, ZSTD_p_checksumFlag, ZSTD_p_dictIDFlag, ZSTD_p_nbWorkers, ZSTD_p_nonBlockingMode, ZSTD_p_jobSize, ZSTD_p_overlapSizeLog, ZSTD_p_forceMaxWindow, ZSTD_p_forceAttachDict

        ZSTD_cParameter
        Enum values:
        • p_format - Cast selected format as unsigned for CCtx_setParameter compatibility.
        • p_compressionLevel - Update all compression parameters according to pre-defined cLevel table.

          Default level is ZSTD_CLEVEL_DEFAULT==3. Special: value 0 means default, which is controlled by CLEVEL_DEFAULT.

          Notes:

          1. it's possible to pass a negative compression level by casting it to unsigned type.
          2. setting a level sets all default values of other compression parameters.
        • p_windowLog - Maximum allowed back-reference distance, expressed as power of 2.

          Must be clamped between WINDOWLOG_MIN and WINDOWLOG_MAX.

          Special: value 0 means "use default windowLog".

          Note: Using a window size greater than ZSTD_MAXWINDOWSIZE_DEFAULT (default: 2^27) requires explicitly allowing such window size during decompression stage.

        • p_hashLog - Size of the initial probe table, as a power of 2.

          Resulting table size is (1 << (hashLog+2)). Must be clamped between HASHLOG_MIN and HASHLOG_MAX. Larger tables improve compression ratio of strategies ≤ dFast, and improve speed of strategies > dFast.

          Special: value 0 means "use default hashLog".

        • p_chainLog - Size of the multi-probe search table, as a power of 2.

          Resulting table size is (1 << (chainLog+2)). Must be clamped between CHAINLOG_MIN and CHAINLOG_MAX. Larger tables result in better and slower compression. This parameter is useless when using "fast" strategy. Note it's still useful when using "dfast" strategy, in which case it defines a secondary probe table.

          Special: value 0 means "use default chainLog".

        • p_searchLog - Number of search attempts, as a power of 2.

          More attempts result in better and slower compression. This parameter is useless when using "fast" and "dFast" strategies.

          Special: value 0 means "use default searchLog".

        • p_minMatch - Minimum size of searched matches (note: repCode matches can be smaller).

          Larger values make faster compression and decompression, but decrease ratio. Must be clamped between SEARCHLENGTH_MIN and SEARCHLENGTH_MAX.

          Note that currently, for all strategies < btopt, effective minimum is 4.

          Note that currently, for all strategies > fast, effective maximum is 6.

          Special: value 0 means "use default minMatchLength".

        • p_targetLength - Impact of this field depends on strategy.

          For strategies btopt & btultra:

          • Length of Match considered "good enough" to stop search.
          • Larger values make compression stronger, and slower.

          For strategy fast:

          • Distance between match sampling.
          • Larger values make compression faster, and weaker.

          Special: value 0 means "use default targetLength".

        • p_compressionStrategy - See ZSTD_strategy enum definition.

          Cast selected strategy as unsigned for CCtx_setParameter compatibility. The higher the value of selected strategy, the more complex it is, resulting in stronger and slower compression.

          Special: value 0 means "use default strategy".

        • p_enableLongDistanceMatching - Enable long distance matching.

          This parameter is designed to improve compression ratio for large inputs, by finding large matches at long distance. It increases memory usage and window size.

          Note: enabling this parameter increases p_windowLog to 128 MB except when expressly set to a different value.

        • p_ldmHashLog - Size of the table for long distance matching, as a power of 2.

          Larger values increase memory usage and compression ratio, but decrease compression speed. Must be clamped between HASHLOG_MIN and HASHLOG_MAX

          Default: windowlog - 7.

          Special: value 0 means "automatically determine hashlog".

        • p_ldmMinMatch - Minimum match size for long distance matcher.

          Larger/too small values usually decrease compression ratio. Must be clamped between LDM_MINMATCH_MIN and LDM_MINMATCH_MAX.

          Special: value 0 means "use default value" (default: 64).

        • p_ldmBucketSizeLog - Log size of each bucket in the LDM hash table for collision resolution.

          Larger values improve collision resolution but decrease compression speed. The maximum value is LDM_BUCKETSIZELOG_MAX .

          Special: value 0 means "use default value" (default: 3).

        • p_ldmHashEveryLog - Frequency of inserting/looking up entries in the LDM hash table.

          Must be clamped between 0 and (WINDOWLOG_MAX - HASHLOG_MIN). Default is MAX(0, (windowLog - ldmHashLog)), optimizing hash table usage. Larger values improve compression speed. Deviating far from default value will likely result in a compression ratio decrease.

          Special: value 0 means "automatically determine hashEveryLog".

        • p_contentSizeFlag - Content size will be written into frame header whenever known (default:1)

          Content size must be known at the beginning of compression, it is provided using CCtx_setPledgedSrcSize.

        • p_checksumFlag - A 32-bits checksum of content is written at end of frame (default:0).
        • p_dictIDFlag - When applicable, dictionary's ID is written into frame header (default:1).
        • p_nbWorkers - Select how many threads will be spawned to compress in parallel.

          When nbWorkers ≥ 1, triggers asynchronous mode: compress_generic consumes some input, flush some output if possible, and immediately gives back control to caller, while compression work is performed in parallel, within worker threads. (note: a strong exception to this rule is when first invocation sets e_end : it becomes a blocking call).

          More workers improve speed, but also increase memory usage.

          Default value is 0, aka "single-threaded mode": no worker is spawned, compression is performed inside Caller's thread, all invocations are blocking.

        • p_nonBlockingMode - Single thread mode is by default "blocking": it finishes its job as much as possible, and only then gives back control to caller.

          In contrast, multi-thread is by default "non-blocking": it takes some input, flush some output if available, and immediately gives back control to caller. Compression work is performed in parallel, within worker threads. (note : a strong exception to this rule is when first job is called with e_end : it becomes blocking)

          Setting this parameter to 1 will enforce non-blocking mode even when only 1 thread is selected. It allows the caller to do other tasks while the worker thread compresses in parallel.

        • p_jobSize - Size of a compression job.

          This value is enforced only in non-blocking mode. Each compression job is completed in parallel, so this value indirectly controls the nb of active threads. 0 means default, which is dynamically determined based on compression parameters. Job size must be a minimum of overlapSize, or 1 MB, whichever is largest. The minimum size is automatically and transparently enforced.

        • p_overlapSizeLog - Size of previous input reloaded at the beginning of each job.
          • 0: no overlap
          • 6(default): use 1/8th of windowSize
          • >=9: use full windowSize
        • p_forceMaxWindow - Force back-reference distances to remain < windowSize, even when referencing into Dictionary content (default:0)
        • p_forceAttachDict - ZSTD supports usage of a CDict in-place (avoiding having to copy the compression tables from the CDict into the working context). Using a CDict in this way saves an initial setup step, but comes at the cost of more work per byte of input. ZSTD has a simple internal heuristic that guesses which strategy will be faster. You can use this flag to override that guess.

          Note that the by-reference, in-place strategy is only used when reusing a compression context with compatible compression parameters. (If incompatible / uninitialized, the working context needs to be cleared anyways, which is about as expensive as overwriting it with the dictionary context, so there's no savings in using the CDict by-ref.)

          Values greater than 0 force attaching the dict. Values less than 0 force copying the dict. 0 selects the default heuristic-guided behavior.

      • ZSTD_e_continue, ZSTD_e_flush, ZSTD_e_end

        ZSTD_EndDirective
        Enum values:
        • e_continue - collect more data, encoder decides when to output compressed result, for optimal conditions
        • e_flush - flush any data provided so far - frame will continue, future data can still reference previous data for better compression
        • e_end - flush any remaining data and close current frame. Any additional data starts a new frame

      • ZSTD_defaultCMem

        Use this constant to defer to stdlib's functions.

    • Method Detail

      • nZSTD_findFrameCompressedSize

        public static long nZSTD_findFrameCompressedSize​(long src,
                                                 long srcSize)
        Unsafe version of: findFrameCompressedSize
        Parameters:
        srcSize - must be ≥ first frame size

      • ZSTD_findFrameCompressedSize

        public static long ZSTD_findFrameCompressedSize​(java.nio.ByteBuffer src)
        Parameters:
        src - should point to the start of a ZSTD encoded frame or skippable frame
        Returns:
        the compressed size of the first frame starting at src, suitable to pass to decompress or similar, or an error code if input is invalid

      • nZSTD_findDecompressedSize

        public static long nZSTD_findDecompressedSize​(long src,
                                              long srcSize)
        Unsafe version of: findDecompressedSize
        Parameters:
        srcSize - must be the exact size of this series (i.e. there should be a frame boundary exactly at srcSize bytes after src)

      • ZSTD_findDecompressedSize

        public static long ZSTD_findDecompressedSize​(java.nio.ByteBuffer src)
        Notes:
        1. decompressed size is an optional field, that may not be present, especially in streaming mode. When return==ZSTD_CONTENTSIZE_UNKNOWN, data to decompress could be any size. In which case, it's necessary to use streaming mode to decompress data.
        2. decompressed size is always present when compression is done with compress.
        3. decompressed size can be very large (64-bits value), potentially larger than what local system can handle as a single memory segment. In which case, it's necessary to use streaming mode to decompress data.
        4. If source is untrusted, decompressed size could be wrong or intentionally modified. Always ensure result fits within application's authorized limits. Each application can set its own limits.
        5. ZSTD_findDecompressedSize handles multiple frames, and so it must traverse the input to read each contained frame header. This is fast as most of the data is skipped, however it does mean that all frame data must be present and valid.
        Parameters:
        src - should point the start of a series of ZSTD encoded and/or skippable frames
        Returns:
        decompressed size of all data in all successive frames. If the decompressed size cannot be determined: CONTENTSIZE_UNKNOWN. If an error occurred: CONTENTSIZE_ERROR

      • nZSTD_frameHeaderSize

        public static long nZSTD_frameHeaderSize​(long src,
                                         long srcSize)
        Unsafe version of: frameHeaderSize
        Parameters:
        srcSize - must be ≥ FRAMEHEADERSIZE_PREFIX

      • ZSTD_frameHeaderSize

        public static long ZSTD_frameHeaderSize​(java.nio.ByteBuffer src)
        Parameters:
        src - should point to the start of a ZSTD frame
        Returns:
        size of the Frame Header or an error code (if srcSize is too small)

      • nZSTD_sizeof_CCtx

        public static long nZSTD_sizeof_CCtx​(long cctx)

      • ZSTD_sizeof_CCtx

        public static long ZSTD_sizeof_CCtx​(long cctx)

      • nZSTD_sizeof_DCtx

        public static long nZSTD_sizeof_DCtx​(long dctx)

      • ZSTD_sizeof_DCtx

        public static long ZSTD_sizeof_DCtx​(long dctx)

      • nZSTD_sizeof_CStream

        public static long nZSTD_sizeof_CStream​(long zcs)

      • ZSTD_sizeof_CStream

        public static long ZSTD_sizeof_CStream​(long zcs)

      • nZSTD_sizeof_DStream

        public static long nZSTD_sizeof_DStream​(long zds)

      • ZSTD_sizeof_DStream

        public static long ZSTD_sizeof_DStream​(long zds)

      • nZSTD_sizeof_CDict

        public static long nZSTD_sizeof_CDict​(long cdict)

      • ZSTD_sizeof_CDict

        public static long ZSTD_sizeof_CDict​(long cdict)

      • nZSTD_sizeof_DDict

        public static long nZSTD_sizeof_DDict​(long ddict)

      • ZSTD_sizeof_DDict

        public static long ZSTD_sizeof_DDict​(long ddict)

      • ZSTD_estimateCCtxSize

        public static long ZSTD_estimateCCtxSize​(int compressionLevel)
        Estimates memory usage of a future CCtx, before its creation.

        Will provide a budget large enough for any compression level up to selected one. It will also consider src size to be arbitrarily "large", which is worst case. If srcSize is known to always be small, estimateCCtxSize_usingCCtxParams can provide a tighter estimation.

        Note: CCtx size estimation is only correct for single-threaded compression.

      • nZSTD_estimateCCtxSize_usingCCtxParams

        public static long nZSTD_estimateCCtxSize_usingCCtxParams​(long params)
        Unsafe version of: estimateCCtxSize_usingCCtxParams

      • ZSTD_estimateCCtxSize_usingCCtxParams

        public static long ZSTD_estimateCCtxSize_usingCCtxParams​(long params)
        See estimateCCtxSize.

        Can be used in tandem with CCtxParam_setParameter. Only single-threaded compression is supported. This function will return an error code if ZSTD_p_nbWorkers is ≥ 1.

      • ZSTD_estimateDCtxSize

        public static long ZSTD_estimateDCtxSize()
        Estimates memory usage of a future DCtx, before its creation.

      • ZSTD_estimateCStreamSize

        public static long ZSTD_estimateCStreamSize​(int compressionLevel)
        Estimates memory usage of a future CStream, before its creation.

        Will provide a budget large enough for any compression level up to selected one. It will also consider src size to be arbitrarily "large", which is worst case. If srcSize is known to always be small, estimateCStreamSize_usingCCtxParams can provide a tighter estimation.

      • nZSTD_estimateCStreamSize_usingCCtxParams

        public static long nZSTD_estimateCStreamSize_usingCCtxParams​(long params)
        Unsafe version of: estimateCStreamSize_usingCCtxParams

      • ZSTD_estimateCStreamSize_usingCCtxParams

        public static long ZSTD_estimateCStreamSize_usingCCtxParams​(long params)
        See estimateCStreamSize.

        Can be used in tandem with CCtxParam_setParameter. Only single-threaded compression is supported. This function will return an error code if ZSTD_p_nbWorkers is ≥ 1.

      • ZSTD_estimateDStreamSize

        public static long ZSTD_estimateDStreamSize​(long windowSize)
        Estimates memory usage of a future DStream, before its creation.

        ZSTD_DStream memory budget depends on window Size. This information can be passed manually, using ZSTD_estimateDStreamSize, or deducted from a valid frame Header, using estimateDStreamSize_fromFrame.

      • nZSTD_estimateDStreamSize_fromFrame

        public static long nZSTD_estimateDStreamSize_fromFrame​(long src,
                                                       long srcSize)
        Unsafe version of: estimateDStreamSize_fromFrame

      • ZSTD_estimateDStreamSize_fromFrame

        public static long ZSTD_estimateDStreamSize_fromFrame​(java.nio.ByteBuffer src)
        See estimateDStreamSize.

      • ZSTD_estimateCDictSize

        public static long ZSTD_estimateCDictSize​(long dictSize,
                                          int compressionLevel)
        Estimates memory usage of a future CDict, before its creation.

        Will bet that src size is relatively "small", and content is copied, like createCDict.

      • ZSTD_estimateDDictSize

        public static long ZSTD_estimateDDictSize​(long dictSize,
                                          int dictLoadMethod)
        Estimates memory usage of a future DDict, before its creation.
        Parameters:
        dictLoadMethod - one of:
        dlm_byCopydlm_byRef

      • nZSTD_createCCtx_advanced

        public static long nZSTD_createCCtx_advanced​(long customMem)
        Unsafe version of: createCCtx_advanced

      • ZSTD_createCCtx_advanced

        public static long ZSTD_createCCtx_advanced​(ZSTDCustomMem customMem)
        Create a ZSTD compression context using external alloc and free functions.

      • nZSTD_initStaticCCtx

        public static long nZSTD_initStaticCCtx​(long workspace,
                                        long workspaceSize)
        Unsafe version of: initStaticCCtx
        Parameters:
        workspaceSize - use estimateCCtxSize or estimateCStreamSize to determine how large workspace must be to support scenario

      • ZSTD_initStaticCCtx

        public static long ZSTD_initStaticCCtx​(java.nio.ByteBuffer workspace)
        Initialize a fixed-size zstd compression context workspace.

        Notes:

        1. zstd will never resize nor malloc() when using a static cctx. If it needs more memory than available, it will simply error out.
        2. there is no corresponding "free" function. Since workspace was allocated externally, it must be freed externally too.

        Limitations:

        1. currently not compatible with internal CDict creation, such as CCtx_loadDictionary or ZSTD_initCStream_usingDict().
        2. currently not compatible with multi-threading
        Parameters:
        workspace - the memory area to emplace the context into. Provided pointer must 8-bytes aligned. It must outlive context usage.
        Returns:
        pointer to ZSTD_CCtx* (same address as workspace, but different type), or NULL if error (typically size too small)

      • nZSTD_isFrame

        public static int nZSTD_isFrame​(long buffer,
                                long size)
        Unsafe version of: isFrame

      • ZSTD_isFrame

        public static boolean ZSTD_isFrame​(java.nio.ByteBuffer buffer)
        Tells if the content of buffer starts with a valid Frame Identifier.

        Notes:

        1. Frame Identifier is 4 bytes. If size < 4, @return will always be 0.
        2. Legacy Frame Identifiers are considered valid only if Legacy Support is enabled.
        3. Skippable Frame Identifiers are considered valid.

      • nZSTD_createDCtx_advanced

        public static long nZSTD_createDCtx_advanced​(long customMem)
        Unsafe version of: createDCtx_advanced

      • ZSTD_createDCtx_advanced

        public static long ZSTD_createDCtx_advanced​(ZSTDCustomMem customMem)
        Create a ZSTD decompression context using external alloc and free functions.

      • nZSTD_initStaticDCtx

        public static long nZSTD_initStaticDCtx​(long workspace,
                                        long workspaceSize)
        Unsafe version of: initStaticDCtx
        Parameters:
        workspaceSize - use estimateDCtxSize or estimateDStreamSize to determine how large workspace must be to support scenario

      • ZSTD_initStaticDCtx

        public static long ZSTD_initStaticDCtx​(java.nio.ByteBuffer workspace)
        Notes:
        1. zstd will never resize nor malloc() when using a static dctx. If it needs more memory than available, it will simply error out.
        2. static dctx is incompatible with legacy support.
        3. there is no corresponding "free" function. Since workspace was allocated externally, it must be freed externally.

        Limitation: currently not compatible with internal DDict creation, such as ZSTD_initDStream_usingDict().

        Parameters:
        workspace - The memory area to emplace the context into. Provided pointer must 8-bytes aligned. It must outlive context usage.
        Returns:
        pointer to ZSTD_DCtx* (same address as workspace, but different type), or NULL if error (typically size too small)

      • nZSTD_createDDict_advanced

        public static long nZSTD_createDDict_advanced​(long dict,
                                              long dictSize,
                                              int dictLoadMethod,
                                              int dictContentType,
                                              long customMem)
        Unsafe version of: createDDict_advanced

      • ZSTD_createDDict_advanced

        public static long ZSTD_createDDict_advanced​(java.nio.ByteBuffer dict,
                                             int dictLoadMethod,
                                             int dictContentType,
                                             ZSTDCustomMem customMem)
        Create a ZSTD_DDict using external alloc and free, optionally by reference.
        Parameters:
        dictLoadMethod - one of:
        dlm_byCopydlm_byRef
        dictContentType - one of:
        dct_autodct_rawContentdct_fullDict

      • nZSTD_initStaticDDict

        public static long nZSTD_initStaticDDict​(long workspace,
                                         long workspaceSize,
                                         long dict,
                                         long dictSize,
                                         int dictLoadMethod,
                                         int dictContentType)
        Unsafe version of: initStaticDDict
        Parameters:
        workspaceSize - use estimateDDictSize to determine how large workspace must be

      • ZSTD_initStaticDDict

        public static long ZSTD_initStaticDDict​(java.nio.ByteBuffer workspace,
                                        java.nio.ByteBuffer dict,
                                        int dictLoadMethod,
                                        int dictContentType)
        Generate a digested dictionary in provided memory area.

        Note: there is no corresponding "free" function. Since workspace was allocated externally, it must be freed externally.

        Parameters:
        workspace - The memory area to emplace the dictionary into. Provided pointer must 8-bytes aligned. It must outlive dictionary usage.
        dictLoadMethod - one of:
        dlm_byCopydlm_byRef
        dictContentType - one of:
        dct_autodct_rawContentdct_fullDict

      • nZSTD_getDictID_fromDict

        public static int nZSTD_getDictID_fromDict​(long dict,
                                           long dictSize)
        Unsafe version of: getDictID_fromDict

      • ZSTD_getDictID_fromDict

        public static int ZSTD_getDictID_fromDict​(java.nio.ByteBuffer dict)
        Provides the dictID stored within dictionary.
        Returns:
        if 0, the dictionary is not conformant with Zstandard specification. It can still be loaded, but as a content-only dictionary.

      • nZSTD_getDictID_fromDDict

        public static int nZSTD_getDictID_fromDDict​(long ddict)
        Unsafe version of: getDictID_fromDDict

      • ZSTD_getDictID_fromDDict

        public static int ZSTD_getDictID_fromDDict​(long ddict)
        Provides the dictID of the dictionary loaded into ddict.
        Returns:
        if @return == 0, the dictionary is not conformant to Zstandard specification, or empty. Non-conformant dictionaries can still be loaded, but as content-only dictionaries.

      • nZSTD_getDictID_fromFrame

        public static int nZSTD_getDictID_fromFrame​(long src,
                                            long srcSize)
        Unsafe version of: getDictID_fromFrame

      • ZSTD_getDictID_fromFrame

        public static int ZSTD_getDictID_fromFrame​(java.nio.ByteBuffer src)
        Provides the dictID required to decompressed the frame stored within src.
        Returns:
        if 0, the dictID could not be decoded. This could for one of the following reasons:
        • The frame does not require a dictionary to be decoded (most common case).
        • The frame was built with dictID intentionally removed. Whatever dictionary is necessary is a hidden information. Note: this use case also happens when using a non-conformant dictionary.
        • srcSize is too small, and as a result, the frame header could not be decoded (only possible if srcSize < ZSTD_FRAMEHEADERSIZE_MAX).
        • This is not a Zstandard frame.

        When identifying the exact failure cause, it's possible to use getFrameHeader, which will provide a more precise error code.

      • nZSTD_createCStream_advanced

        public static long nZSTD_createCStream_advanced​(long customMem)

      • ZSTD_createCStream_advanced

        public static long ZSTD_createCStream_advanced​(ZSTDCustomMem customMem)

      • nZSTD_initStaticCStream

        public static long nZSTD_initStaticCStream​(long workspace,
                                           long workspaceSize)

      • ZSTD_initStaticCStream

        public static long ZSTD_initStaticCStream​(java.nio.ByteBuffer workspace)

      • nZSTD_createDStream_advanced

        public static long nZSTD_createDStream_advanced​(long customMem)

      • ZSTD_createDStream_advanced

        public static long ZSTD_createDStream_advanced​(ZSTDCustomMem customMem)

      • nZSTD_initStaticDStream

        public static long nZSTD_initStaticDStream​(long workspace,
                                           long workspaceSize)

      • ZSTD_initStaticDStream

        public static long ZSTD_initStaticDStream​(java.nio.ByteBuffer workspace)

      • nZSTD_copyCCtx

        public static long nZSTD_copyCCtx​(long cctx,
                                  long preparedCCtx,
                                  long pledgedSrcSize)
        Unsafe version of: copyCCtx

      • ZSTD_copyCCtx

        public static long ZSTD_copyCCtx​(long cctx,
                                 long preparedCCtx,
                                 long pledgedSrcSize)
        Parameters:
        pledgedSrcSize - can be 0, indicating unknown size. If it is non-zero, it must be accurate. For 0 size frames, use compressBegin_advanced

      • nZSTD_getFrameProgression

        public static void nZSTD_getFrameProgression​(long cctx,
                                             long __result)
        Unsafe version of: getFrameProgression

      • ZSTD_getFrameProgression

        public static ZSTDFrameProgression ZSTD_getFrameProgression​(long cctx,
                                                            ZSTDFrameProgression __result)
        Tells how much data has been ingested (read from input) consumed (input actually compressed) and produced (output) for current frame. Therefore, (ingested - consumed) is amount of input data buffered internally, not yet compressed.

        Can report progression inside worker threads (multi-threading and non-blocking mode).

      • nZSTD_getFrameHeader

        public static long nZSTD_getFrameHeader​(long zfhPtr,
                                        long src,
                                        long srcSize)
        Unsafe version of: getFrameHeader

      • ZSTD_getFrameHeader

        public static long ZSTD_getFrameHeader​(ZSTDFrameHeader zfhPtr,
                                       java.nio.ByteBuffer src)
        Decode Frame Header, or requires larger srcSize.

        Doesn't consume input.

        Returns:
        if 0, zfhPtr is correctly filled. If >0, srcSize is too small, value is wanted srcSize amount, or an error code, which can be tested using isError.

      • nZSTD_copyDCtx

        public static void nZSTD_copyDCtx​(long dctx,
                                  long preparedDCtx)

      • ZSTD_copyDCtx

        public static void ZSTD_copyDCtx​(long dctx,
                                 long preparedDCtx)

      • nZSTD_CCtx_setParameter

        public static long nZSTD_CCtx_setParameter​(long cctx,
                                           int param,
                                           int value)
        Unsafe version of: CCtx_setParameter

      • nZSTD_CCtx_getParameter

        public static long nZSTD_CCtx_getParameter​(long cctx,
                                           int param,
                                           long value)
        Unsafe version of: CCtx_getParameter

      • nZSTD_CCtx_setPledgedSrcSize

        public static long nZSTD_CCtx_setPledgedSrcSize​(long cctx,
                                                long pledgedSrcSize)
        Unsafe version of: CCtx_setPledgedSrcSize

      • ZSTD_CCtx_setPledgedSrcSize

        public static long ZSTD_CCtx_setPledgedSrcSize​(long cctx,
                                               long pledgedSrcSize)
        Sets the total input data size to be compressed as a single frame. This value will be controlled at the end, and result in error if not respected.

        Notes:

        1. 0 means zero, empty.

          In order to mean "unknown content size", pass constant CONTENTSIZE_UNKNOWN. CONTENTSIZE_UNKNOWN is default value for any new compression job.

        2. If all data is provided and consumed in a single round, this value is overridden by srcSize instead.
        Returns:
        0, or an error code (which can be tested with isError)

      • nZSTD_CCtx_loadDictionary

        public static long nZSTD_CCtx_loadDictionary​(long cctx,
                                             long dict,
                                             long dictSize)
        Unsafe version of: CCtx_loadDictionary

      • ZSTD_CCtx_loadDictionary

        public static long ZSTD_CCtx_loadDictionary​(long cctx,
                                            @Nullable
                                            java.nio.ByteBuffer dict)
        Create an internal CDict from dict buffer. Decompression will have to use same dictionary.

        Special: Adding a NULL (or 0-size) dictionary invalidates any previous dictionary, meaning "return to no-dictionary mode".

        Notes:

        1. Dictionary will be used for all future compression jobs. To return to "no-dictionary" situation, load a NULL dictionary.
        2. Loading a dictionary involves building tables, which are dependent on compression parameters. For this reason, compression parameters cannot be changed anymore after loading a dictionary. It's also a CPU consuming operation, with non-negligible impact on latency.
        3. dict content will be copied internally. Use CCtx_loadDictionary_byReference to reference dictionary content instead. In such a case, dictionary buffer must outlive its users.
        4. Use CCtx_loadDictionary_advanced to precisely select how dictionary content must be interpreted.
        Returns:
        0, or an error code (which can be tested with isError)

      • nZSTD_CCtx_loadDictionary_byReference

        public static long nZSTD_CCtx_loadDictionary_byReference​(long cctx,
                                                         long dict,
                                                         long dictSize)
        Unsafe version of: CCtx_loadDictionary_byReference

      • ZSTD_CCtx_loadDictionary_byReference

        public static long ZSTD_CCtx_loadDictionary_byReference​(long cctx,
                                                        @Nullable
                                                        java.nio.ByteBuffer dict)
        See CCtx_loadDictionary.

      • nZSTD_CCtx_loadDictionary_advanced

        public static long nZSTD_CCtx_loadDictionary_advanced​(long cctx,
                                                      long dict,
                                                      long dictSize,
                                                      int dictLoadMethod,
                                                      int dictContentType)
        Unsafe version of: CCtx_loadDictionary_advanced

      • nZSTD_CCtx_refCDict

        public static long nZSTD_CCtx_refCDict​(long cctx,
                                       long cdict)
        Unsafe version of: CCtx_refCDict

      • ZSTD_CCtx_refCDict

        public static long ZSTD_CCtx_refCDict​(long cctx,
                                      long cdict)
        Reference a prepared dictionary, to be used for all next compression jobs.

        Note that compression parameters are enforced from within CDict, and supercede any compression parameter previously set within CCtx.

        The dictionary will remain valid for future compression jobs using same CCtx.

        Special: adding a NULL CDict means "return to no-dictionary mode".

        Notes:

        1. Currently, only one dictionary can be managed. Adding a new dictionary effectively "discards" any previous one.
        2. CDict is just referenced, its lifetime must outlive CCtx.
        Returns:
        0, or an error code (which can be tested with isError)

      • nZSTD_CCtx_refPrefix

        public static long nZSTD_CCtx_refPrefix​(long cctx,
                                        long prefix,
                                        long prefixSize)
        Unsafe version of: CCtx_refPrefix

      • ZSTD_CCtx_refPrefix

        public static long ZSTD_CCtx_refPrefix​(long cctx,
                                       @Nullable
                                       java.nio.ByteBuffer prefix)
        Reference a prefix (single-usage dictionary) for next compression job.

        Decompression need same prefix to properly regenerate data. Prefix is only used once. Tables are discarded at end of compression job. (e_end)

        Special: Adding any prefix (including NULL) invalidates any previous prefix or dictionary.

        Notes:

        1. Prefix buffer is referenced. It must outlive compression job. Its content must remain unmodified up to end of compression (e_end).
        2. Referencing a prefix involves building tables, which are dependent on compression parameters. It's a CPU consuming operation, with non-negligible impact on latency. If there is a need to use same prefix multiple times, consider CCtx_loadDictionary instead.
        3. By default, the prefix is treated as raw content (ZSTD_dm_rawContent). Use CCtx_refPrefix_advanced to alter dictMode.
        Returns:
        0, or an error code (which can be tested with isError)

      • nZSTD_CCtx_refPrefix_advanced

        public static long nZSTD_CCtx_refPrefix_advanced​(long cctx,
                                                 long prefix,
                                                 long prefixSize,
                                                 int dictContentType)
        Unsafe version of: CCtx_refPrefix_advanced

      • ZSTD_CCtx_refPrefix_advanced

        public static long ZSTD_CCtx_refPrefix_advanced​(long cctx,
                                                @Nullable
                                                java.nio.ByteBuffer prefix,
                                                int dictContentType)
        See CCtx_refPrefix.
        Parameters:
        dictContentType - one of:
        dct_autodct_rawContentdct_fullDict

      • nZSTD_CCtx_reset

        public static void nZSTD_CCtx_reset​(long cctx)
        Unsafe version of: CCtx_reset

      • ZSTD_CCtx_reset

        public static void ZSTD_CCtx_reset​(long cctx)
        Return a CCtx to clean state.

        Useful after an error, or to interrupt an ongoing compression job and start a new one. Any internal data not yet flushed is cancelled. The parameters and dictionary are kept unchanged, to reset them use CCtx_resetParameters.

      • nZSTD_CCtx_resetParameters

        public static long nZSTD_CCtx_resetParameters​(long cctx)
        Unsafe version of: CCtx_resetParameters

      • ZSTD_CCtx_resetParameters

        public static long ZSTD_CCtx_resetParameters​(long cctx)
        All parameters are back to default values (compression level is CLEVEL_DEFAULT).

        Dictionary (if any) is dropped. Resetting parameters is only possible during frame initialization (before starting compression). To reset the context use CCtx_reset.

        Returns:
        0 or an error code (which can be checked with isError).

      • nZSTD_compress_generic

        public static long nZSTD_compress_generic​(long cctx,
                                          long output,
                                          long input,
                                          int endOp)
        Unsafe version of: compress_generic

      • ZSTD_compress_generic

        public static long ZSTD_compress_generic​(long cctx,
                                         ZSTDOutBuffer output,
                                         ZSTDInBuffer input,
                                         int endOp)
        Behave about the same as compressStream.

        Notes:

        • Compression parameters are pushed into CCtx before starting compression, using CCtx_setParameter.
        • Compression parameters cannot be changed once compression is started.
        • output->pos must be ≤ dstCapacity, input->pos must be ≤ srcSize.
        • outpot->pos and input->pos will be updated. They are guaranteed to remain below their respective limit.
        • In single-thread mode (default), function is blocking: it completed its job before returning to caller.
        • In multi-thread mode, function is non-blocking: it just acquires a copy of input, and distribute job to internal worker threads, and then immediately returns, just indicating that there is some data remaining to be flushed. The function nonetheless guarantees forward progress: it will return only after it reads or write at least 1+ byte.
        • Exception: in multi-threading mode, if the first call requests a e_end directive, it is blocking: it will complete compression before giving back control to caller.
        • after a e_end directive, if internal buffer is not fully flushed (return != 0), only e_end or e_flush operations are allowed. Before starting a new compression job, or changing compression parameters, it is required to fully flush internal buffers.
        Parameters:
        endOp - one of:
        e_continuee_flushe_end
        Returns:
        provides a minimum amount of data remaining to be flushed from internal buffers or an error code, which can be tested using isError. If @return != 0, flush is not fully completed, there is still some data left within internal buffers. This is useful for e_flush, since in this case more flushes are necessary to empty all buffers. For e_end, @return == 0 when internal buffers are fully flushed and frame is completed.

      • nZSTD_compress_generic_simpleArgs

        public static long nZSTD_compress_generic_simpleArgs​(long cctx,
                                                     long dst,
                                                     long dstCapacity,
                                                     long dstPos,
                                                     long src,
                                                     long srcSize,
                                                     long srcPos,
                                                     int endOp)
        Unsafe version of: compress_generic_simpleArgs

      • ZSTD_compress_generic_simpleArgs

        public static long ZSTD_compress_generic_simpleArgs​(long cctx,
                                                    java.nio.ByteBuffer dst,
                                                    org.lwjgl.PointerBuffer dstPos,
                                                    java.nio.ByteBuffer src,
                                                    org.lwjgl.PointerBuffer srcPos,
                                                    int endOp)
        Same as compress_generic, but using only integral types as arguments.
        Parameters:
        endOp - one of:
        e_continuee_flushe_end

      • ZSTD_createCCtxParams

        public static long ZSTD_createCCtxParams()
        Creates a ZSTD_CCtx_params structure.

      • nZSTD_freeCCtxParams

        public static long nZSTD_freeCCtxParams​(long params)
        Unsafe version of: freeCCtxParams

      • ZSTD_freeCCtxParams

        public static long ZSTD_freeCCtxParams​(long params)
        Frees memory allocated by createCCtxParams.

      • nZSTD_CCtxParams_reset

        public static long nZSTD_CCtxParams_reset​(long params)
        Unsafe version of: CCtxParams_reset

      • ZSTD_CCtxParams_reset

        public static long ZSTD_CCtxParams_reset​(long params)
        Reset params to default values.

      • nZSTD_CCtxParams_init

        public static long nZSTD_CCtxParams_init​(long cctxParams,
                                         int compressionLevel)
        Unsafe version of: CCtxParams_init

      • ZSTD_CCtxParams_init

        public static long ZSTD_CCtxParams_init​(long cctxParams,
                                        int compressionLevel)
        Initializes the compression parameters of cctxParams according to compression level. All other parameters are reset to their default values.

      • nZSTD_CCtxParam_setParameter

        public static long nZSTD_CCtxParam_setParameter​(long params,
                                                int param,
                                                int value)
        Unsafe version of: CCtxParam_setParameter

      • nZSTD_CCtxParam_getParameter

        public static long nZSTD_CCtxParam_getParameter​(long params,
                                                int param,
                                                long value)
        Unsafe version of: CCtxParam_getParameter

      • nZSTD_CCtx_setParametersUsingCCtxParams

        public static long nZSTD_CCtx_setParametersUsingCCtxParams​(long cctx,
                                                           long params)
        Unsafe version of: CCtx_setParametersUsingCCtxParams

      • ZSTD_CCtx_setParametersUsingCCtxParams

        public static long ZSTD_CCtx_setParametersUsingCCtxParams​(long cctx,
                                                          long params)
        Apply a set of ZSTD_CCtx_params to the compression context.

        This can be done even after compression is started, if nbWorkers==0, this will have no impact until a new compression is started. If nbWorkers>=1, new parameters will be picked up at next job, with a few restrictions (windowLog, pledgedSrcSize, nbWorkers, jobSize, and overlapLog are not updated).

      • nZSTD_DCtx_loadDictionary

        public static long nZSTD_DCtx_loadDictionary​(long dctx,
                                             long dict,
                                             long dictSize)
        Unsafe version of: DCtx_loadDictionary

      • ZSTD_DCtx_loadDictionary

        public static long ZSTD_DCtx_loadDictionary​(long dctx,
                                            @Nullable
                                            java.nio.ByteBuffer dict)
        Create an internal DDict from dict buffer, to be used to decompress next frames.

        Special: Adding a NULL (or 0-size) dictionary invalidates any previous dictionary, meaning "return to no-dictionary mode".

        Notes:

        1. dict content will be copied internally. Use DCtx_loadDictionary_byReference to reference dictionary content instead. In which case, the dictionary buffer must outlive its users.
        2. Loading a dictionary involves building tables, which has a non-negligible impact on CPU usage and latency.
        3. Use DCtx_loadDictionary_advanced to select how dictionary content will be interpreted and loaded.
        Returns:
        0, or an error code (which can be tested with isError)

      • nZSTD_DCtx_loadDictionary_byReference

        public static long nZSTD_DCtx_loadDictionary_byReference​(long dctx,
                                                         long dict,
                                                         long dictSize)
        Unsafe version of: DCtx_loadDictionary_byReference

      • ZSTD_DCtx_loadDictionary_byReference

        public static long ZSTD_DCtx_loadDictionary_byReference​(long dctx,
                                                        @Nullable
                                                        java.nio.ByteBuffer dict)
        See DCtx_loadDictionary.

      • nZSTD_DCtx_loadDictionary_advanced

        public static long nZSTD_DCtx_loadDictionary_advanced​(long dctx,
                                                      long dict,
                                                      long dictSize,
                                                      int dictLoadMethod,
                                                      int dictContentType)
        Unsafe version of: DCtx_loadDictionary_advanced

      • nZSTD_DCtx_refDDict

        public static long nZSTD_DCtx_refDDict​(long dctx,
                                       long ddict)
        Unsafe version of: DCtx_refDDict

      • ZSTD_DCtx_refDDict

        public static long ZSTD_DCtx_refDDict​(long dctx,
                                      long ddict)
        Reference a prepared dictionary, to be used to decompress next frames. The dictionary remains active for decompression of future frames using same DCtx.

        Special: adding a NULL DDict means "return to no-dictionary mode".

        Notes:

        1. Currently, only one dictionary can be managed. Referencing a new dictionary effectively "discards" any previous one.
        2. DDict is just referenced, its lifetime must outlive its usage from DCtx.
        Returns:
        0, or an error code (which can be tested with isError)

      • nZSTD_DCtx_refPrefix

        public static long nZSTD_DCtx_refPrefix​(long dctx,
                                        long prefix,
                                        long prefixSize)
        Unsafe version of: DCtx_refPrefix

      • ZSTD_DCtx_refPrefix

        public static long ZSTD_DCtx_refPrefix​(long dctx,
                                       @Nullable
                                       java.nio.ByteBuffer prefix)
        Reference a prefix (single-usage dictionary) for next compression job.

        Prefix is only used once. Reference is discarded at end of frame. End of frame is reached when ZSTD_DCtx_decompress_generic() returns 0.

        Notes:

        1. Adding any prefix (including NULL) invalidates any previously set prefix or dictionary
        2. Prefix buffer is referenced. It must outlive decompression job. Prefix buffer must remain unmodified up to the end of frame, reached when ZSTD_DCtx_decompress_generic() returns 0.
        3. By default, the prefix is treated as raw content (ZSTD_dm_rawContent). Use CCtx_refPrefix_advanced to alter dictMode.
        4. Referencing a raw content prefix has almost no cpu nor memory cost. A fulldict prefix is more costly though.
        Returns:
        0, or an error code (which can be tested with isError)

      • nZSTD_DCtx_refPrefix_advanced

        public static long nZSTD_DCtx_refPrefix_advanced​(long dctx,
                                                 long prefix,
                                                 long prefixSize,
                                                 int dictContentType)
        Unsafe version of: DCtx_refPrefix_advanced

      • ZSTD_DCtx_refPrefix_advanced

        public static long ZSTD_DCtx_refPrefix_advanced​(long dctx,
                                                @Nullable
                                                java.nio.ByteBuffer prefix,
                                                int dictContentType)
        See DCtx_refPrefix.
        Parameters:
        dictContentType - one of:
        dct_autodct_rawContentdct_fullDict

      • nZSTD_DCtx_setMaxWindowSize

        public static long nZSTD_DCtx_setMaxWindowSize​(long dctx,
                                               long maxWindowSize)
        Unsafe version of: DCtx_setMaxWindowSize

      • ZSTD_DCtx_setMaxWindowSize

        public static long ZSTD_DCtx_setMaxWindowSize​(long dctx,
                                              long maxWindowSize)
        Refuses allocating internal buffers for frames requiring a window size larger than provided limit.

        This is useful to prevent a decoder context from reserving too much memory for itself (potential attack scenario). This parameter is only useful in streaming mode, since no internal buffer is allocated in direct mode. By default, a decompression context accepts all window sizes ≤ (1 << ZSTD_WINDOWLOG_MAX)

        Returns:
        0, or an error code (which can be tested with isError)

      • nZSTD_DCtx_setFormat

        public static long nZSTD_DCtx_setFormat​(long dctx,
                                        int format)
        Unsafe version of: DCtx_setFormat

      • ZSTD_DCtx_setFormat

        public static long ZSTD_DCtx_setFormat​(long dctx,
                                       int format)
        Parameters:
        format - one of:
        f_zstd1f_zstd1_magicless

      • nZSTD_getFrameHeader_advanced

        public static long nZSTD_getFrameHeader_advanced​(long zfhPtr,
                                                 long src,
                                                 long srcSize,
                                                 int format)
        Unsafe version of: getFrameHeader_advanced

      • nZSTD_decompress_generic

        public static long nZSTD_decompress_generic​(long dctx,
                                            long output,
                                            long input)
        Unsafe version of: decompress_generic

      • ZSTD_decompress_generic

        public static long ZSTD_decompress_generic​(long dctx,
                                           ZSTDOutBuffer output,
                                           ZSTDInBuffer input)
        Behave the same as decompressStream.

        Decompression parameters cannot be changed once decompression is started.

        Returns:
        an error code, which can be tested using isError. If > 0, a hint, nb of expected input bytes for next invocation. 0 means a frame has just been fully decoded and flushed.

      • nZSTD_decompress_generic_simpleArgs

        public static long nZSTD_decompress_generic_simpleArgs​(long dctx,
                                                       long dst,
                                                       long dstCapacity,
                                                       long dstPos,
                                                       long src,
                                                       long srcSize,
                                                       long srcPos)
        Unsafe version of: decompress_generic_simpleArgs

      • ZSTD_decompress_generic_simpleArgs

        public static long ZSTD_decompress_generic_simpleArgs​(long dctx,
                                                      java.nio.ByteBuffer dst,
                                                      org.lwjgl.PointerBuffer dstPos,
                                                      java.nio.ByteBuffer src,
                                                      org.lwjgl.PointerBuffer srcPos)
        Same as decompress_generic, but using only integral types as arguments.

      • nZSTD_DCtx_reset

        public static void nZSTD_DCtx_reset​(long dctx)
        Unsafe version of: DCtx_reset

      • ZSTD_DCtx_reset

        public static void ZSTD_DCtx_reset​(long dctx)
        Return a DCtx to clean state.

        If a decompression was ongoing, any internal data not yet flushed is cancelled. All parameters are back to default values, including sticky ones. Dictionary (if any) is dropped. Parameters can be modified again after a reset.

      • nZSTD_getBlockSize

        public static long nZSTD_getBlockSize​(long cctx)

      • ZSTD_getBlockSize

        public static long ZSTD_getBlockSize​(long cctx)

      • nZSTD_compressBlock

        public static long nZSTD_compressBlock​(long cctx,
                                       long dst,
                                       long dstCapacity,
                                       long src,
                                       long srcSize)

      • ZSTD_compressBlock

        public static long ZSTD_compressBlock​(long cctx,
                                      java.nio.ByteBuffer dst,
                                      java.nio.ByteBuffer src)

      • nZSTD_decompressBlock

        public static long nZSTD_decompressBlock​(long dctx,
                                         long dst,
                                         long dstCapacity,
                                         long src,
                                         long srcSize)

      • ZSTD_decompressBlock

        public static long ZSTD_decompressBlock​(long dctx,
                                        java.nio.ByteBuffer dst,
                                        java.nio.ByteBuffer src)

      • nZSTD_insertBlock

        public static long nZSTD_insertBlock​(long dctx,
                                     long blockStart,
                                     long blockSize)
        Unsafe version of: insertBlock

      • ZSTD_insertBlock

        public static long ZSTD_insertBlock​(long dctx,
                                    java.nio.ByteBuffer blockStart)
        Insert uncompressed block into dctx history. Useful for multi-blocks decompression.