File LibDeflate.lua
LibDeflate 0.9.0-beta2
Pure Lua compressor and decompressor with high compression ratio using
DEFLATE/zlib format.
Info:
- Copyright: LibDeflate <2018> Haoqian He
- License: GNU General Public License Version 3 or later
This library is implemented according to the following specifications.
Report a bug if LibDeflate is not fully compliant with those specs.
Both compressors and decompressors have been implemented in the library.
1. RFC1950: DEFLATE Compressed Data Format Specification version 1.3
https://tools.ietf.org/html/rfc1951
2. RFC1951: ZLIB Compressed Data Format Specification version 3.3
https://tools.ietf.org/html/rfc1950This library requires Lua 5.1/5.2/5.3 interpreter or LuaJIT v2.0+.
This library does not have any external library dependencies.
(Exception: will register in the World of Warcraft library "LibStub", if detected).
This file "LibDeflate.lua" is the only source file of the library.
Submit suggestions or report bugs to https://github.com/safeteeWow/LibDeflate/issues - Author: Haoqian He (Github: SafeteeWoW; World of Warcraft: Safetyy-Illidan(US))
Functions
LibDeflate:Adler32 (str) | Calculate the Adler-32 checksum of the string. |
LibDeflate:CompressDeflate (str, configs) | Compress using the raw deflate format. |
LibDeflate:CompressDeflateWithDict (str, dictionary, configs) | Compress using the raw deflate format with a preset dictionary. |
LibDeflate:CompressZlib (str, configs) | Compress using the zlib format. |
LibDeflate:CompressZlibWithDict (str, dictionary, configs) | Compress using the zlib format with a preset dictionary |
LibDeflate:CreateCodec (reserved_chars, escape_chars, map_chars) | Create a custom codec with encoder and decoder. |
LibDeflate:CreateDictionary (str, strlen, adler32) | Create a preset dictionary. |
LibDeflate:DecodeForPrint (str) | Decode the string produced by LibDeflate:EncodeForPrint |
LibDeflate:DecodeForWoWAddonChannel (str) | Decode the string produced by LibDeflate:EncodeForWoWAddonChannel |
LibDeflate:DecodeForWoWChatChannel (str) | Decode the string produced by LibDeflate:EncodeForWoWChatChannel. |
LibDeflate:DecompressDeflate (str) | Decompress a raw deflate compressed data. |
LibDeflate:DecompressDeflateWithDict (str, dictionary) | Decompress a raw deflate compressed data with a preset dictionary. |
LibDeflate:DecompressZlib (str) | Decompress a zlib compressed data. |
LibDeflate:DecompressZlibWithDict (str, dictionary) | Decompress a zlib compressed data with a preset dictionary. |
LibDeflate:EncodeForPrint (str) | Encode the string to make it printable. |
LibDeflate:EncodeForWoWAddonChannel (str) | Encode the string to make it ready to be transmitted in World of Warcraft addon channel. |
LibDeflate:EncodeForWoWChatChannel (str) | Encode the string to make it ready to be transmitted in World of Warcraft chat channel. |
Tables
CommandlineOptions | Commandline options |
compression_configs | The description to compression configuration table. |
Functions
- LibDeflate:Adler32 (str)
-
Calculate the Adler-32 checksum of the string.
See RFC1950 Page 9 https://tools.ietf.org/html/rfc1950 for the definition of Adler-32 checksum.Parameters:
- str [string] the input string to calcuate its Adler-32 checksum.
Returns:
-
[integer] The Adler-32 checksum, which is greater or equal to 0,
and less than 2^32 (4294967296).
- LibDeflate:CompressDeflate (str, configs)
-
Compress using the raw deflate format.
Parameters:
- str [string] The data to be compressed.
- configs [table/nil] The configuration table to control the compression . If nil, use the default configuration.
Returns:
- [string] The compressed data
-
[integer] The number of bits padded at the end of output.
0 <= bits < 8
You don't need to use this value unless you want to do some postprocessing to the compressed data.
See also:
- LibDeflate:CompressDeflateWithDict (str, dictionary, configs)
-
Compress using the raw deflate format with a preset dictionary.
Parameters:
- str [string] The data to be compressed.
- dictionary [table] The preset dictionary produced by LibDeflate:CreateDictionary
- configs [table/nil] The configuration table to control the compression . If nil, use the default configuration.
Returns:
- [string] The compressed data
-
[integer] The number of bits padded at the end of output.
0 <= bits < 8
You don't need to use this value unless you want to do some postprocessing to the compressed data.
See also:
- LibDeflate:CompressZlib (str, configs)
-
Compress using the zlib format.
Parameters:
- str [string] the data to be compressed.
- configs [table/nil] The configuration table to control the compression . If nil, use the default configuration.
Returns:
- [string] The compressed data
- [integer] The number of bits padded at the end of output. Should always be 0.
See also:
- LibDeflate:CompressZlibWithDict (str, dictionary, configs)
-
Compress using the zlib format with a preset dictionary
Parameters:
- str [string] the data to be compressed.
- dictionary [table] A preset dictionary produced by LibDeflate:CreateDictionary()
- configs [table/nil] The configuration table to control the compression . If nil, use the default configuration.
Returns:
- [string] The compressed data
- [integer] The number of bits padded at the end of output. Should always be 0.
See also:
- LibDeflate:CreateCodec (reserved_chars, escape_chars, map_chars)
-
Create a custom codec with encoder and decoder.
Credits to LibCompress.Parameters:
- reserved_chars [string] The created encoder will ensure no encoded data will contain any characters in reserved_chars.
- escape_chars [string] The escape character(s) used to escape reserved_chars. The length of param is usally 1 character, but you need to provide one more character if there are more than 127 reseverd chars.
- map_chars [string] The created encoder will encode the input string with mapping every reservedchars[i] (1 <= i <= #mapchars) to map_chars[i]
Returns:
-
[table] The encode/decode table. The table contains two functions
t:Encode(str) returns the encoded string.
t:Decode(str) returns the decoded string if succeeds. nil if fails.Raises:
error if encode/decoder cannot be produced.Usage:
-- Create an encoder/decoder that maps all "\000" to "\003", -- and escape "\001" (and "\002" and "\003") properly local codec = LibDeflate:CreateCodec("\000\001", "\002", "\003") local encoded = codec:Encode(SOME_STRING) local decoded = codec:Decode(encoded) -- assert(decoded == SOME_STRING)
- LibDeflate:CreateDictionary (str, strlen, adler32)
-
Create a preset dictionary.
This function is not fast, and the memory consumption of the produced dictionary is about 50 times of the input string. Therefore, it is suggestted to run this function only once in your program.
It is very important to know that if you do use a preset dictionary, compressors and decompressors MUST USE THE EQUIVALENT dictionary. That is, dictionary must be created using the same string. If you update your program with a new dictionary, people with the old version won't be able to transmit data with people with the new version. Therefore, changing the dictionary must be very careful, and the parameter of the function helps you to avoid accidentally modify the the definition of the dictionary during the program development.
Parameters:
- str
[string] The string used as the preset dictionary.
You should put stuffs that frequently appears in the dictionary string and preferablely put more frequently appeared stuffs toward the end of the string.
Empty string and string longer than 32768 bytes are not allowed. - strlen [integer] The length of 'str'. Please pass in this parameter as a hardcoded constant, in order to verify the content of 'str'.
- adler32 [integer] The Adler-32 checksum of 'str'. Please pass in this parameter as a hardcoded constant, in order to verify the content of 'str'.
Returns:
-
[table] The dictionary used for preset dictionary compression and
decompression.
Raises:
error if 'strlen' does not match the length of the 'str', or if 'adler32' does not match the Adler-32 checksum of the 'str'.Usage:
local dict_str = "1234567890" -- print(dict_str:len(), LibDeflate:Adler32(dict_str)) -- Hardcode the print result below to verify it to avoid acciently -- modification of 'str' during the program development. -- string length: 10, Adler-32: 187433486, -- Don't calculate string length and its Adler-32 at run-time. local dict = LibDeflate:CreateDictionary(dict_str, 10, 187433486)
- str
[string] The string used as the preset dictionary.
- LibDeflate:DecodeForPrint (str)
-
Decode the string produced by LibDeflate:EncodeForPrint
Parameters:
- str [string] The string to be decoded
Returns:
-
[string/nil] The decoded string if succeeds. nil if fails.
- LibDeflate:DecodeForWoWAddonChannel (str)
-
Decode the string produced by LibDeflate:EncodeForWoWAddonChannel
Parameters:
- str [string] The string to be decoded.
Returns:
-
[string/nil] The decoded string if succeeds. nil if fails.
See also:
- LibDeflate:DecodeForWoWChatChannel (str)
-
Decode the string produced by LibDeflate:EncodeForWoWChatChannel.
Parameters:
- str [string] The string to be decoded.
Returns:
-
[string/nil] The decoded string if succeeds. nil if fails.
See also:
- LibDeflate:DecompressDeflate (str)
-
Decompress a raw deflate compressed data.
Parameters:
- str [string] The data to be decompressed
Returns:
- [string/nil] The decompressed data if succeeds. nil if fails.
-
[integer] The number of unprocessed bytes if succeeds. Positive
integer if any extra bytes are appended after the compressed data. If you
consider this as an error, check if this is zero.
UNDEFINED value if fails.
See also:
- LibDeflate:DecompressDeflateWithDict (str, dictionary)
-
Decompress a raw deflate compressed data with a preset dictionary.
Parameters:
- str [string] The data to be decompressed
- dictionary [table] The preset dictionary used by LibDeflate:CompressDeflateWithDict when the compressed data is produced. Decompression and compression must use the same dictionary. Otherwise wrong decompressed data could be produced without generating any error.
Returns:
- [string/nil] The decompressed data if succeeds. nil if fails.
-
[integer] The number of unprocessed bytes if succeeds. Positive
integer if any extra bytes are appended after the compressed data. If you
consider this as an error, check if this is zero.
UNDEFINED value if fails.
See also:
- LibDeflate:DecompressZlib (str)
-
Decompress a zlib compressed data.
Parameters:
- str [string] The data to be decompressed
Returns:
- [string/nil] The decompressed data if succeeds. nil if fails.
-
[integer] The number of unprocessed bytes if succeeds. Positive
integer if any extra bytes are appended after the compressed data. If you
consider this as an error, check if this is zero.
UNDEFINED value if fails.
See also:
- LibDeflate:DecompressZlibWithDict (str, dictionary)
-
Decompress a zlib compressed data with a preset dictionary.
Parameters:
- str [string] The data to be decompressed
- dictionary [table] The preset dictionary used by LibDeflate:CompressDeflateWithDict when the compressed data is produced. Decompression and compression must use the same dictionary. Otherwise wrong decompressed data could be produced without generating any error.
Returns:
- [string/nil] The decompressed data if succeeds. nil if fails.
-
[integer] The number of unprocessed bytes if succeeds. Positive
integer if any extra bytes are appended after the compressed data. If you
consider this as an error, check if this is zero.
UNDEFINED value if fails.
See also:
- LibDeflate:EncodeForPrint (str)
-
Encode the string to make it printable.
Credis to WeakAuras2, this function is equivalant to the implementation it is using right now.
The encoded string will be 25% larger than the origin string. However, every single byte of the encoded string will be one of 64 printable ASCII characters, which are can be easier copied, pasted and displayed. (26 lowercase letters, 26 uppercase letters, 10 numbers digits, left parenthese, or right parenthese)Parameters:
- str [string] The string to be encoded.
Returns:
-
[string] The encoded string.
- LibDeflate:EncodeForWoWAddonChannel (str)
-
Encode the string to make it ready to be transmitted in World of
Warcraft addon channel.
The encoded string is guaranteed to contain no NULL ("\000") character.Parameters:
- str [string] The string to be encoded.
Returns:
-
The encoded string.
See also:
- LibDeflate:EncodeForWoWChatChannel (str)
-
Encode the string to make it ready to be transmitted in World of
Warcraft chat channel.
See also https://wow.gamepedia.com/ValidChatMessageCharactersParameters:
- str [string] The string to be encoded.
Returns:
-
[string] The encoded string.
See also:
Tables
- CommandlineOptions
-
Commandline options
Usage:
lua LibDeflate.lua [OPTION] [INPUT] [OUTPUT] -0 store only. no compression. -1 fastest compression. -9 slowest and best compression. -d do decompression instead of compression. --dict <filename> specify the file that contains he entire preset dictionary. -h give this help. --strategy <fixed/huffman_only/dynamic> specify a special compression strategy. -v print the version and copyright info. --wowdict Use the preset dictionary designed for WoW shipped with LibDeflate. --zlib use zlib format instead of raw deflate.
- compression_configs
-
The description to compression configuration table.
Any field can be nil to use its default.
Table with keys other than those below is an invalid table.Fields:
- level The compression level ranged from 0 to 9. 0 is no compression. 9 is the slowest but best compression. Use nil for default level.
- strategy The compression strategy. "fixed" to only use fixed deflate compression block. "dynamic" to only use dynamic block. "huffman_only" to do no LZ77 compression. Only do huffman compression.