diff options
Diffstat (limited to 'lib_generic/lzma/lzma.txt')
| -rw-r--r-- | lib_generic/lzma/lzma.txt | 663 |
1 files changed, 297 insertions, 366 deletions
diff --git a/lib_generic/lzma/lzma.txt b/lib_generic/lzma/lzma.txt index 5f1a0c994..aa20f9dc5 100644 --- a/lib_generic/lzma/lzma.txt +++ b/lib_generic/lzma/lzma.txt @@ -1,8 +1,6 @@ -LZMA SDK 4.57 +LZMA SDK 4.65 ------------- -LZMA SDK Copyright (C) 1999-2007 Igor Pavlov - LZMA SDK provides the documentation, samples, header files, libraries, and tools you need to develop applications that use LZMA compression. @@ -20,70 +18,7 @@ decompressing. LICENSE ------- -LZMA SDK is available under any of the following licenses: - -1) GNU Lesser General Public License (GNU LGPL) -2) Common Public License (CPL) -3) Simplified license for unmodified code (read SPECIAL EXCEPTION) -4) Proprietary license - -It means that you can select one of these four options and follow rules of that license. - - -1,2) GNU LGPL and CPL licenses are pretty similar and both these -licenses are classified as - - "Free software licenses" at http://www.gnu.org/ - - "OSI-approved" at http://www.opensource.org/ - - -3) SPECIAL EXCEPTION - -Igor Pavlov, as the author of this code, expressly permits you -to statically or dynamically link your code (or bind by name) -to the files from LZMA SDK without subjecting your linked -code to the terms of the CPL or GNU LGPL. -Any modifications or additions to files from LZMA SDK, however, -are subject to the GNU LGPL or CPL terms. - -SPECIAL EXCEPTION allows you to use LZMA SDK in applications with closed code, -while you keep LZMA SDK code unmodified. - - -SPECIAL EXCEPTION #2: Igor Pavlov, as the author of this code, expressly permits -you to use this code under the same terms and conditions contained in the License -Agreement you have for any previous version of LZMA SDK developed by Igor Pavlov. - -SPECIAL EXCEPTION #2 allows owners of proprietary licenses to use latest version -of LZMA SDK as update for previous versions. - - -SPECIAL EXCEPTION #3: Igor Pavlov, as the author of this code, expressly permits -you to use code of the following files: -BranchTypes.h, LzmaTypes.h, LzmaTest.c, LzmaStateTest.c, LzmaAlone.cpp, -LzmaAlone.cs, LzmaAlone.java -as public domain code. - - -4) Proprietary license - -LZMA SDK also can be available under a proprietary license which -can include: - -1) Right to modify code without subjecting modified code to the -terms of the CPL or GNU LGPL -2) Technical support for code - -To request such proprietary license or any additional consultations, -send email message from that page: -http://www.7-zip.org/support.html - - -You should have received a copy of the GNU Lesser General Public -License along with this library; if not, write to the Free Software -Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA - -You should have received a copy of the Common Public License -along with this library. +LZMA SDK is written and placed in the public domain by Igor Pavlov. LZMA SDK Contents @@ -91,87 +26,71 @@ LZMA SDK Contents LZMA SDK includes: - - C++ source code of LZMA compressing and decompressing - - ANSI-C compatible source code for LZMA decompressing - - C# source code for LZMA compressing and decompressing - - Java source code for LZMA compressing and decompressing + - ANSI-C/C++/C#/Java source code for LZMA compressing and decompressing - Compiled file->file LZMA compressing/decompressing program for Windows system -ANSI-C LZMA decompression code was ported from original C++ sources to C. -Also it was simplified and optimized for code size. -But it is fully compatible with LZMA from 7-Zip. - UNIX/Linux version ------------------ -To compile C++ version of file->file LZMA, go to directory -C/7zip/Compress/LZMA_Alone -and type "make" or "make clean all" to recompile all. +To compile C++ version of file->file LZMA encoding, go to directory +C++/7zip/Compress/LZMA_Alone +and call make to recompile it: + make -f makefile.gcc clean all In some UNIX/Linux versions you must compile LZMA with static libraries. -To compile with static libraries, change string in makefile -LIB = -lm -to string +To compile with static libraries, you can use LIB = -lm -static Files --------------------- -C - C source code -CPP - CPP source code -CS - C# source code -Java - Java source code -lzma.txt - LZMA SDK description (this file) +lzma.txt - LZMA SDK description (this file) 7zFormat.txt - 7z Format description -7zC.txt - 7z ANSI-C Decoder description (this file) +7zC.txt - 7z ANSI-C Decoder description methods.txt - Compression method IDs for .7z -LGPL.txt - GNU Lesser General Public License -CPL.html - Common Public License -lzma.exe - Compiled file->file LZMA encoder/decoder for Windows -history.txt - history of the LZMA SDK +lzma.exe - Compiled file->file LZMA encoder/decoder for Windows +history.txt - history of the LZMA SDK Source code structure --------------------- -C - C files - Compress - files related to compression/decompression - Lz - files related to LZ (Lempel-Ziv) compression algorithm - Lzma - ANSI-C compatible LZMA decompressor +C/ - C files + 7zCrc*.* - CRC code + Alloc.* - Memory allocation functions + Bra*.* - Filters for x86, IA-64, ARM, ARM-Thumb, PowerPC and SPARC code + LzFind.* - Match finder for LZ (LZMA) encoders + LzFindMt.* - Match finder for LZ (LZMA) encoders for multithreading encoding + LzHash.h - Additional file for LZ match finder + LzmaDec.* - LZMA decoding + LzmaEnc.* - LZMA encoding + LzmaLib.* - LZMA Library for DLL calling + Types.h - Basic types for another .c files + Threads.* - The code for multithreading. - LzmaDecode.h - interface for LZMA decoding on ANSI-C - LzmaDecode.c - LZMA decoding on ANSI-C (new fastest version) - LzmaDecodeSize.c - LZMA decoding on ANSI-C (old size-optimized version) - LzmaTest.c - test application that decodes LZMA encoded file - LzmaTypes.h - basic types for LZMA Decoder - LzmaStateDecode.h - interface for LZMA decoding (State version) - LzmaStateDecode.c - LZMA decoding on ANSI-C (State version) - LzmaStateTest.c - test application (State version) + LzmaLib - LZMA Library (.DLL for Windows) - Branch - Filters for x86, IA-64, ARM, ARM-Thumb, PowerPC and SPARC code + LzmaUtil - LZMA Utility (file->file LZMA encoder/decoder). Archive - files related to archiving - 7z_C - 7z ANSI-C Decoder - + 7z - 7z ANSI-C Decoder -CPP -- CPP files +CPP/ -- CPP files Common - common files for C++ projects Windows - common files for Windows related code - 7zip - files related to 7-Zip Project + + 7zip - files related to 7-Zip Project Common - common files for 7-Zip Compress - files related to compression/decompression - LZ - files related to LZ (Lempel-Ziv) compression algorithm - - Copy - Copy coder + Copy - Copy coder RangeCoder - Range Coder (special code of compression/decompression) - LZMA - LZMA compression/decompression on C++ + LZMA - LZMA compression/decompression on C++ LZMA_Alone - file->file LZMA compression/decompression - - Branch - Filters for x86, IA-64, ARM, ARM-Thumb, PowerPC and SPARC code + Branch - Filters for x86, IA-64, ARM, ARM-Thumb, PowerPC and SPARC code Archive - files related to archiving @@ -180,67 +99,61 @@ CPP -- CPP files Bundles - Modules that are bundles of other modules - Alone7z - 7zr.exe: Standalone version of 7z.exe that supports only 7z/LZMA/BCJ/BCJ2 - Format7zR - 7zr.dll: Reduced version of 7za.dll: extracting/compressing to 7z/LZMA/BCJ/BCJ2 - Format7zExtractR - 7zxr.dll: Reduced version of 7zxa.dll: extracting from 7z/LZMA/BCJ/BCJ2. + Alone7z - 7zr.exe: Standalone version of 7z.exe that supports only 7z/LZMA/BCJ/BCJ2 + Format7zR - 7zr.dll: Reduced version of 7za.dll: extracting/compressing to 7z/LZMA/BCJ/BCJ2 + Format7zExtractR - 7zxr.dll: Reduced version of 7zxa.dll: extracting from 7z/LZMA/BCJ/BCJ2. - UI - User Interface files + UI - User Interface files - Client7z - Test application for 7za.dll, 7zr.dll, 7zxr.dll + Client7z - Test application for 7za.dll, 7zr.dll, 7zxr.dll Common - Common UI files Console - Code for console archiver -CS - C# files +CS/ - C# files 7zip Common - some common files for 7-Zip Compress - files related to compression/decompression LZ - files related to LZ (Lempel-Ziv) compression algorithm - LZMA - LZMA compression/decompression + LZMA - LZMA compression/decompression LzmaAlone - file->file LZMA compression/decompression RangeCoder - Range Coder (special code of compression/decompression) -Java - Java files +Java/ - Java files SevenZip Compression - files related to compression/decompression - LZ - files related to LZ (Lempel-Ziv) compression algorithm - LZMA - LZMA compression/decompression + LZ - files related to LZ (Lempel-Ziv) compression algorithm + LZMA - LZMA compression/decompression RangeCoder - Range Coder (special code of compression/decompression) -C/C++ source code of LZMA SDK is part of 7-Zip project. - -You can find ANSI-C LZMA decompressing code at folder - C/7zip/Compress/Lzma -7-Zip doesn't use that ANSI-C LZMA code and that code was developed -specially for this SDK. And files from C/7zip/Compress/Lzma do not need -files from other directories of SDK for compiling. +C/C++ source code of LZMA SDK is part of 7-Zip project. 7-Zip source code can be downloaded from 7-Zip's SourceForge page: http://sourceforge.net/projects/sevenzip/ + LZMA features ------------- - Variable dictionary size (up to 1 GB) - - Estimated compressing speed: about 1 MB/s on 1 GHz CPU + - Estimated compressing speed: about 2 MB/s on 2 GHz CPU - Estimated decompressing speed: - - 8-12 MB/s on 1 GHz Intel Pentium 3 or AMD Athlon - - 500-1000 KB/s on 100 MHz ARM, MIPS, PowerPC or other simple RISC - - Small memory requirements for decompressing (8-32 KB + DictionarySize) - - Small code size for decompressing: 2-8 KB (depending from - speed optimizations) + - 20-30 MB/s on 2 GHz Core 2 or AMD Athlon 64 + - 1-2 MB/s on 200 MHz ARM, MIPS, PowerPC or other simple RISC + - Small memory requirements for decompressing (16 KB + DictionarySize) + - Small code size for decompressing: 5-8 KB LZMA decoder uses only integer operations and can be implemented in any modern 32-bit CPU (or on 16-bit CPU with some conditions). -Some critical operations that affect to speed of LZMA decompression: +Some critical operations that affect the speed of LZMA decompression: 1) 32*16 bit integer multiply 2) Misspredicted branches (penalty mostly depends from pipeline length) 3) 32-bit shift and arithmetic operations -Speed of LZMA decompressing mostly depends from CPU speed. +The speed of LZMA decompressing mostly depends from CPU speed. Memory speed has no big meaning. But if your CPU has small data cache, overall weight of memory speed will slightly increase. @@ -251,7 +164,7 @@ How To Use Using LZMA encoder/decoder executable -------------------------------------- -Usage: LZMA <e|d> inputFile outputFile [<switches>...] +Usage: LZMA <e|d> inputFile outputFile [<switches>...] e: encode file @@ -260,11 +173,11 @@ Usage: LZMA <e|d> inputFile outputFile [<switches>...] b: Benchmark. There are two tests: compressing and decompressing with LZMA method. Benchmark shows rating in MIPS (million instructions per second). Rating value is calculated from - measured speed and it is normalized with AMD Athlon 64 X2 CPU - results. Also Benchmark checks possible hardware errors (RAM + measured speed and it is normalized with Intel's Core 2 results. + Also Benchmark checks possible hardware errors (RAM errors in most cases). Benchmark uses these settings: - (-a1, -d21, -fb32, -mfbt4). You can change only -d. Also you - can change number of iterations. Example for 30 iterations: + (-a1, -d21, -fb32, -mfbt4). You can change only -d parameter. + Also you can change the number of iterations. Example for 30 iterations: LZMA b 30 Default number of iterations is 10. @@ -272,52 +185,52 @@ Usage: LZMA <e|d> inputFile outputFile [<switches>...] -a{N}: set compression mode 0 = fast, 1 = normal - default: 1 (normal) + default: 1 (normal) d{N}: Sets Dictionary size - [0, 30], default: 23 (8MB) - The maximum value for dictionary size is 1 GB = 2^30 bytes. - Dictionary size is calculated as DictionarySize = 2^N bytes. - For decompressing file compressed by LZMA method with dictionary - size D = 2^N you need about D bytes of memory (RAM). + The maximum value for dictionary size is 1 GB = 2^30 bytes. + Dictionary size is calculated as DictionarySize = 2^N bytes. + For decompressing file compressed by LZMA method with dictionary + size D = 2^N you need about D bytes of memory (RAM). -fb{N}: set number of fast bytes - [5, 273], default: 128 - Usually big number gives a little bit better compression ratio - and slower compression process. + Usually big number gives a little bit better compression ratio + and slower compression process. -lc{N}: set number of literal context bits - [0, 8], default: 3 - Sometimes lc=4 gives gain for big files. + Sometimes lc=4 gives gain for big files. -lp{N}: set number of literal pos bits - [0, 4], default: 0 - lp switch is intended for periodical data when period is - equal 2^N. For example, for 32-bit (4 bytes) - periodical data you can use lp=2. Often it's better to set lc0, - if you change lp switch. + lp switch is intended for periodical data when period is + equal 2^N. For example, for 32-bit (4 bytes) + periodical data you can use lp=2. Often it's better to set lc0, + if you change lp switch. -pb{N}: set number of pos bits - [0, 4], default: 2 - pb switch is intended for periodical data - when period is equal 2^N. + pb switch is intended for periodical data + when period is equal 2^N. -mf{MF_ID}: set Match Finder. Default: bt4. - Algorithms from hc* group doesn't provide good compression - ratio, but they often works pretty fast in combination with - fast mode (-a0). + Algorithms from hc* group doesn't provide good compression + ratio, but they often works pretty fast in combination with + fast mode (-a0). - Memory requirements depend from dictionary size - (parameter "d" in table below). + Memory requirements depend from dictionary size + (parameter "d" in table below). - MF_ID Memory Description + MF_ID Memory Description - bt2 d * 9.5 + 4MB Binary Tree with 2 bytes hashing. - bt3 d * 11.5 + 4MB Binary Tree with 3 bytes hashing. - bt4 d * 11.5 + 4MB Binary Tree with 4 bytes hashing. - hc4 d * 7.5 + 4MB Hash Chain with 4 bytes hashing. + bt2 d * 9.5 + 4MB Binary Tree with 2 bytes hashing. + bt3 d * 11.5 + 4MB Binary Tree with 3 bytes hashing. + bt4 d * 11.5 + 4MB Binary Tree with 4 bytes hashing. + hc4 d * 7.5 + 4MB Hash Chain with 4 bytes hashing. -eos: write End Of Stream marker. By default LZMA doesn't write - eos marker, since LZMA decoder knows uncompressed size - stored in .lzma file header. + eos marker, since LZMA decoder knows uncompressed size + stored in .lzma file header. - -si: Read data from stdin (it will write End Of Stream marker). - -so: Write data to stdout + -si: Read data from stdin (it will write End Of Stream marker). + -so: Write data to stdout Examples: @@ -345,32 +258,29 @@ Compression ratio hints Recommendations --------------- -To increase compression ratio for LZMA compressing it's desirable +To increase the compression ratio for LZMA compressing it's desirable to have aligned data (if it's possible) and also it's desirable to locate data in such order, where code is grouped in one place and data is grouped in other place (it's better than such mixing: code, data, code, data, ...). -Using Filters -------------- -You can increase compression ratio for some data types, using +Filters +------- +You can increase the compression ratio for some data types, using special filters before compressing. For example, it's possible to -increase compression ratio on 5-10% for code for those CPU ISAs: +increase the compression ratio on 5-10% for code for those CPU ISAs: x86, IA-64, ARM, ARM-Thumb, PowerPC, SPARC. -You can find C/C++ source code of such filters in folder "7zip/Compress/Branch" +You can find C source code of such filters in C/Bra*.* files -You can check compression ratio gain of these filters with such +You can check the compression ratio gain of these filters with such 7-Zip commands (example for ARM code): No filter: 7z a a1.7z a.bin -m0=lzma With filter for little-endian ARM code: - 7z a a2.7z a.bin -m0=bc_arm -m1=lzma - -With filter for big-endian ARM code (using additional Swap4 filter): - 7z a a3.7z a.bin -m0=swap4 -m1=bc_arm -m2=lzma + 7z a a2.7z a.bin -m0=arm -m1=lzma It works in such manner: Compressing = Filter_encoding + LZMA_encoding @@ -383,8 +293,7 @@ since compression ratio with filtering is higher. These filters convert CALL (calling procedure) instructions from relative offsets to absolute addresses, so such data becomes more -compressible. Source code of these CALL filters is pretty simple -(about 20 lines of C++), so you can convert it from C++ version yourself. +compressible. For some ISAs (for example, for MIPS) it's impossible to get gain from such filter. @@ -392,272 +301,294 @@ For some ISAs (for example, for MIPS) it's impossible to get gain from such filt LZMA compressed file format --------------------------- Offset Size Description - 0 1 Special LZMA properties for compressed data - 1 4 Dictionary size (little endian) - 5 8 Uncompressed size (little endian). -1 means unknown size - 13 Compressed data + 0 1 Special LZMA properties (lc,lp, pb in encoded form) + 1 4 Dictionary size (little endian) + 5 8 Uncompressed size (little endian). -1 means unknown size + 13 Compressed data ANSI-C LZMA Decoder ~~~~~~~~~~~~~~~~~~~ -To compile ANSI-C LZMA Decoder you can use one of the following files sets: -1) LzmaDecode.h + LzmaDecode.c + LzmaTest.c (fastest version) -2) LzmaDecode.h + LzmaDecodeSize.c + LzmaTest.c (old size-optimized version) -3) LzmaStateDecode.h + LzmaStateDecode.c + LzmaStateTest.c (zlib-like interface) +Please note that interfaces for ANSI-C code were changed in LZMA SDK 4.58. +If you want to use old interfaces you can download previous version of LZMA SDK +from sourceforge.net site. + +To use ANSI-C LZMA Decoder you need the following files: +1) LzmaDec.h + LzmaDec.c + Types.h +LzmaUtil/LzmaUtil.c is example application that uses these files. Memory requirements for LZMA decoding ------------------------------------- -LZMA decoder doesn't allocate memory itself, so you must -allocate memory and send it to LZMA. - Stack usage of LZMA decoding function for local variables is not -larger than 200 bytes. +larger than 200-400 bytes. + +LZMA Decoder uses dictionary buffer and internal state structure. +Internal state structure consumes + state_size = (4 + (1.5 << (lc + lp))) KB +by default (lc=3, lp=0), state_size = 16 KB. + How To decompress data ---------------------- -LZMA Decoder (ANSI-C version) now supports 5 interfaces: +LZMA Decoder (ANSI-C version) now supports 2 interfaces: 1) Single-call Decompressing -2) Single-call Decompressing with input stream callback -3) Multi-call Decompressing with output buffer -4) Multi-call Decompressing with input callback and output buffer -5) Multi-call State Decompressing (zlib-like interface) - -Variant-5 is similar to Variant-4, but Variant-5 doesn't use callback functions. - -Decompressing steps -------------------- - -1) read LZMA properties (5 bytes): - unsigned char properties[LZMA_PROPERTIES_SIZE]; - -2) read uncompressed size (8 bytes, little-endian) +2) Multi-call State Decompressing (zlib-like interface) -3) Decode properties: +You must use external allocator: +Example: +void *SzAlloc(void *p, size_t size) { p = p; return malloc(size); } +void SzFree(void *p, void *address) { p = p; free(address); } +ISzAlloc alloc = { SzAlloc, SzFree }; - CLzmaDecoderState state; /* it's 24-140 bytes structure, if int is 32-bit */ +You can use p = p; operator to disable compiler warnings. - if (LzmaDecodeProperties(&state.Properties, properties, LZMA_PROPERTIES_SIZE) != LZMA_RESULT_OK) - return PrintError(rs, "Incorrect stream properties"); -4) Allocate memory block for internal Structures: - - state.Probs = (CProb *)malloc(LzmaGetNumProbs(&state.Properties) * sizeof(CProb)); - if (state.Probs == 0) - return PrintError(rs, kCantAllocateMessage); - - LZMA decoder uses array of CProb variables as internal structure. - By default, CProb is unsigned_short. But you can define _LZMA_PROB32 to make - it unsigned_int. It can increase speed on some 32-bit CPUs, but memory - usage will be doubled in that case. - - -5) Main Decompressing - -You must use one of the following interfaces: - -5.1 Single-call Decompressing ------------------------------ +Single-call Decompressing +------------------------- When to use: RAM->RAM decompressing -Compile files: LzmaDecode.h, LzmaDecode.c +Compile files: LzmaDec.h + LzmaDec.c + Types.h Compile defines: no defines Memory Requirements: - Input buffer: compressed size - Output buffer: uncompressed size - - LZMA Internal Structures (~16 KB for default settings) + - LZMA Internal Structures: state_size (16 KB for default settings) Interface: - int res = LzmaDecode(&state, - inStream, compressedSize, &inProcessed, - outStream, outSize, &outProcessed); + int LzmaDecode(Byte *dest, SizeT *destLen, const Byte *src, SizeT *srcLen, + const Byte *propData, unsigned propSize, ELzmaFinishMode finishMode, + ELzmaStatus *status, ISzAlloc *alloc); + In: + dest - output data + destLen - output data size + src - input data + srcLen - input data size + propData - LZMA properties (5 bytes) + propSize - size of propData buffer (5 bytes) + finishMode - It has meaning only if the decoding reaches output limit (*destLen). + LZMA_FINISH_ANY - Decode just destLen bytes. + LZMA_FINISH_END - Stream must be finished after (*destLen). + You can use LZMA_FINISH_END, when you know that + current output buffer covers last bytes of stream. + alloc - Memory allocator. + + Out: + destLen - processed output size + srcLen - processed input size + + Output: + SZ_OK + status: + LZMA_STATUS_FINISHED_WITH_MARK + LZMA_STATUS_NOT_FINISHED + LZMA_STATUS_MAYBE_FINISHED_WITHOUT_MARK + SZ_ERROR_DATA - Data error + SZ_ERROR_MEM - Memory allocation error + SZ_ERROR_UNSUPPORTED - Unsupported properties + SZ_ERROR_INPUT_EOF - It needs more bytes in input buffer (src). + + If LZMA decoder sees end_marker before reaching output limit, it returns OK result, + and output value of destLen will be less than output buffer size limit. + + You can use multiple checks to test data integrity after full decompression: + 1) Check Result and "status" variable. + 2) Check that output(destLen) = uncompressedSize, if you know real uncompressedSize. + 3) Check that output(srcLen) = compressedSize, if you know real compressedSize. + You must use correct finish mode in that case. */ + + +Multi-call State Decompressing (zlib-like interface) +---------------------------------------------------- +When to use: file->file decompressing +Compile files: LzmaDec.h + LzmaDec.c + Types.h -5.2 Single-call Decompressing with input stream callback --------------------------------------------------------- -When to use: File->RAM or Flash->RAM decompressing. -Compile files: LzmaDecode.h, LzmaDecode.c -Compile defines: _LZMA_IN_CB Memory Requirements: - - Buffer for input stream: any size (for example, 16 KB) - - Output buffer: uncompressed size - - LZMA Internal Structures (~16 KB for default settings) + - Buffer for input stream: any size (for example, 16 KB) + - Buffer for output stream: any size (for example, 16 KB) + - LZMA Internal Structures: state_size (16 KB for default settings) + - LZMA dictionary (dictionary size is encoded in LZMA properties header) -Interface: - typedef struct _CBuffer - { - ILzmaInCallback InCallback; - FILE *File; - unsigned char Buffer[kInBufferSize]; - } CBuffer; +1) read LZMA properties (5 bytes) and uncompressed size (8 bytes, little-endian) to header: + unsigned char header[LZMA_PROPS_SIZE + 8]; + ReadFile(inFile, header, sizeof(header) - int LzmaReadCompressed(void *object, const unsigned char **buffer, SizeT *size) +2) Allocate CLzmaDec structures (state + dictionary) using LZMA properties + + CLzmaDec state; + LzmaDec_Constr(&state); + res = LzmaDec_Allocate(&state, header, LZMA_PROPS_SIZE, &g_Alloc); + if (res != SZ_OK) + return res; + +3) Init LzmaDec structure before any new LZMA stream. And call LzmaDec_DecodeToBuf in loop + + LzmaDec_Init(&state); + for (;;) { - CBuffer *bo = (CBuffer *)object; - *buffer = bo->Buffer; - *size = MyReadFile(bo->File, bo->Buffer, kInBufferSize); - return LZMA_RESULT_OK; + ... + int res = LzmaDec_DecodeToBuf(CLzmaDec *p, Byte *dest, SizeT *destLen, + const Byte *src, SizeT *srcLen, ELzmaFinishMode finishMode); + ... } - CBuffer g_InBuffer; - g_InBuffer.File = inFile; - g_InBuffer.InCallback.Read = LzmaReadCompressed; - int res = LzmaDecode(&state, - &g_InBuffer.InCallback, - outStream, outSize, &outProcessed); +4) Free all allocated structures + LzmaDec_Free(&state, &g_Alloc); + +For full code example, look at C/LzmaUtil/LzmaUtil.c code. + + +How To compress data +-------------------- +Compile files: LzmaEnc.h + LzmaEnc.c + Types.h + +LzFind.c + LzFind.h + LzFindMt.c + LzFindMt.h + LzHash.h -5.3 Multi-call decompressing with output buffer ------------------------------------------------ -When to use: RAM->File decompressing -Compile files: LzmaDecode.h, LzmaDecode.c -Compile defines: _LZMA_OUT_READ Memory Requirements: - - Input buffer: compressed size - - Buffer for output stream: any size (for example, 16 KB) - - LZMA Internal Structures (~16 KB for default settings) - - LZMA dictionary (dictionary size is encoded in stream properties) + - (dictSize * 11.5 + 6 MB) + state_size -Interface: +Lzma Encoder can use two memory allocators: +1) alloc - for small arrays. +2) allocBig - for big arrays. - state.Dictionary = (unsigned char *)malloc(state.Properties.DictionarySize); +For example, you can use Large RAM Pages (2 MB) in allocBig allocator for +better compression speed. Note that Windows has bad implementation for +Large RAM Pages. +It's OK to use same allocator for alloc and allocBig. - LzmaDecoderInit(&state); - do - { - LzmaDecode(&state, - inBuffer, inAvail, &inProcessed, - g_OutBuffer, outAvail, &outProcessed); - inAvail -= inProcessed; - inBuffer += inProcessed; - } - while you need more bytes - see LzmaTest.c for more details. +Single-call Compression with callbacks +-------------------------------------- +Check C/LzmaUtil/LzmaUtil.c as example, -5.4 Multi-call decompressing with input callback and output buffer ------------------------------------------------------------------- -When to use: File->File decompressing -Compile files: LzmaDecode.h, LzmaDecode.c -Compile defines: _LZMA_IN_CB, _LZMA_OUT_READ -Memory Requirements: - - Buffer for input stream: any size (for example, 16 KB) - - Buffer for output stream: any size (for example, 16 KB) - - LZMA Internal Structures (~16 KB for default settings) - - LZMA dictionary (dictionary size is encoded in stream properties) +When to use: file->file decompressing -Interface: +1) you must implement callback structures for interfaces: +ISeqInStream +ISeqOutStream +ICompressProgress +ISzAlloc - state.Dictionary = (unsigned char *)malloc(state.Properties.DictionarySize); +static void *SzAlloc(void *p, size_t size) { p = p; return MyAlloc(size); } +static void SzFree(void *p, void *address) { p = p; MyFree(address); } +static ISzAlloc g_Alloc = { SzAlloc, SzFree }; - LzmaDecoderInit(&state); - do - { - LzmaDecode(&state, - &bo.InCallback, - g_OutBuffer, outAvail, &outProcessed); - } - while you need more bytes + CFileSeqInStream inStream; + CFileSeqOutStream outStream; - see LzmaTest.c for more details: + inStream.funcTable.Read = MyRead; + inStream.file = inFile; + outStream.funcTable.Write = MyWrite; + outStream.file = outFile; -5.5 Multi-call State Decompressing (zlib-like interface) ------------------------------------------------------------------- -When to use: file->file decompressing -Compile files: LzmaStateDecode.h, LzmaStateDecode.c -Compile defines: -Memory Requirements: - - Buffer for input stream: any size (for example, 16 KB) - - Buffer for output stream: any size (for example, 16 KB) - - LZMA Internal Structures (~16 KB for default settings) - - LZMA dictionary (dictionary size is encoded in stream properties) +2) Create CLzmaEncHandle object; -Interface: + CLzmaEncHandle enc; - state.Dictionary = (unsigned char *)malloc(state.Properties.DictionarySize); + enc = LzmaEnc_Create(&g_Alloc); + if (enc == 0) + return SZ_ERROR_MEM; - LzmaDecoderInit(&state); - do - { - res = LzmaDecode(&state, - inBuffer, inAvail, &inProcessed, - g_OutBuffer, outAvail, &outProcessed, - finishDecoding); - inAvail -= inProcessed; - inBuffer += inProcessed; - } - while you need more bytes +3) initialize CLzmaEncProps properties; - see LzmaStateTest.c for more details: + LzmaEncProps_Init(&props); + Then you can change some properties in that structure. -6) Free all allocated blocks +4) Send LZMA properties to LZMA Encoder + res = LzmaEnc_SetProps(enc, &props); -Note ----- -LzmaDecodeSize.c is size-optimized version of LzmaDecode.c. -But compiled code of LzmaDecodeSize.c can be larger than -compiled code of LzmaDecode.c. So it's better to use -LzmaDecode.c in most cases. +5) Write encoded properties to header + Byte header[LZMA_PROPS_SIZE + 8]; + size_t headerSize = LZMA_PROPS_SIZE; + UInt64 fileSize; + int i; -EXIT codes ------------ + res = LzmaEnc_WriteProperties(enc, header, &headerSize); + fileSize = MyGetFileLength(inFile); + for (i = 0; i < 8; i++) + header[headerSize++] = (Byte)(fileSize >> (8 * i)); + MyWriteFileAndCheck(outFile, header, headerSize) -LZMA decoder can return one of the following codes: +6) Call encoding function: + res = LzmaEnc_Encode(enc, &outStream.funcTable, &inStream.funcTable, + NULL, &g_Alloc, &g_Alloc); -#define LZMA_RESULT_OK 0 -#define LZMA_RESULT_DATA_ERROR 1 +7) Destroy LZMA Encoder Object + LzmaEnc_Destroy(enc, &g_Alloc, &g_Alloc); -If you use callback function for input data and you return some -error code, LZMA Decoder also returns that code. +If callback function return some error code, LzmaEnc_Encode also returns that code. -LZMA Defines ------------- +Single-call RAM->RAM Compression +-------------------------------- -_LZMA_IN_CB - Use callback for input data +Single-call RAM->RAM Compression is similar to Compression with callbacks, +but you provide pointers to buffers instead of pointers to stream callbacks: -_LZMA_OUT_READ - Use read function for output data +HRes LzmaEncode(Byte *dest, SizeT *destLen, const Byte *src, SizeT srcLen, + CLzmaEncProps *props, Byte *propsEncoded, SizeT *propsSize, int writeEndMark, + ICompressProgress *progress, ISzAlloc *alloc, ISzAlloc *allocBig); -_LZMA_LOC_OPT - Enable local speed optimizations inside code. - _LZMA_LOC_OPT is only for LzmaDecodeSize.c (size-optimized version). - _LZMA_LOC_OPT doesn't affect LzmaDecode.c (speed-optimized version) - and LzmaStateDecode.c +Return code: + SZ_OK - OK + SZ_ERROR_MEM - Memory allocation error + SZ_ERROR_PARAM - Incorrect paramater + SZ_ERROR_OUTPUT_EOF - output buffer overflow + SZ_ERROR_THREAD - errors in multithreading functions (only for Mt version) -_LZMA_PROB32 - It can increase speed on some 32-bit CPUs, - but memory usage will be doubled in that case -_LZMA_UINT32_IS_ULONG - Define it if int is 16-bit on your compiler - and long is 32-bit. -_LZMA_SYSTEM_SIZE_T - Define it if you want to use system's size_t. - You can use it to enable 64-bit sizes supporting +LZMA Defines +------------ + +_LZMA_SIZE_OPT - Enable some optimizations in LZMA Decoder to get smaller executable code. + +_LZMA_PROB32 - It can increase the speed on some 32-bit CPUs, but memory usage for + some structures will be doubled in that case. +_LZMA_UINT32_IS_ULONG - Define it if int is 16-bit on your compiler and long is 32-bit. + +_LZMA_NO_SYSTEM_SIZE_T - Define it if you don't want to use size_t type. C++ LZMA Encoder/Decoder ~~~~~~~~~~~~~~~~~~~~~~~~ C++ LZMA code use COM-like interfaces. So if you want to use it, you can study basics of COM/OLE. +C++ LZMA code is just wrapper over ANSI-C code. -By default, LZMA Encoder contains all Match Finders. -But for compressing it's enough to have just one of them. -So for reducing size of compressing code you can define: - #define COMPRESS_MF_BT - #define COMPRESS_MF_BT4 -and it will use only bt4 match finder. +C++ Notes +~~~~~~~~~~~~~~~~~~~~~~~~ +If you use some C++ code folders in 7-Zip (for example, C++ code for .7z handling), +you must check that you correctly work with "new" operator. +7-Zip can be compiled with MSVC 6.0 that doesn't throw "exception" from "new" operator. +So 7-Zip uses "CPP\Common\NewHandler.cpp" that redefines "new" operator: +operator new(size_t size) +{ + void *p = ::malloc(size); + if (p == 0) + throw CNewException(); + return p; +} +If you use MSCV that throws exception for "new" operator, you can compile without +"NewHandler.cpp". So standard exception will be used. Actually some code of +7-Zip catches any exception in internal code and converts it to HRESULT code. +So you don't need to catch CNewException, if you call COM interfaces of 7-Zip. --- http://www.7-zip.org +http://www.7-zip.org/sdk.html http://www.7-zip.org/support.html |