7Zip (LZMA) In-Memory Compression with C#

Shows a simple C# SevenZip.Compression.Lzma.SevenZipHelper class to enable high-compression of in-memory objects with C#. Adds a new 7Zip format LZMA Cookie Compression class to the previously published CookieCompression utility.

Many developers are familiar with 7-Zip and use it as an alternative to Winzip or Winrar. I use the Windows distribution of 7Zip frequently because of the higher compression ratios it provides, with an excellent balance between speed and high compression with the default settings.

However I would venture to guess that most developers are unaware that Igor Pavlov, the primary developer of SevenZip, provides a full SDK that includes his LZMA source code in several languages, including one in C#. 

The C# code in the SDK is designed to be used in an executable with a command - line interface. However, by recompiling this to a class library, removing the command line parser classes, and adding a simple SevenZipHelper class that uses the default property settings, we can get a very handy and powerful class library for in-memory compression tasks.

Some facts on LZMA (7Zip) Compression:

• Average compression ratio of LZMA is about 30% better than that of gzip, and 15% better than that of bzip2.
• Decompression speed is only little slower than that of gzip, being two to five times faster than bzip2.
• In fast mode, 7Zip compresses faster than bzip2 with a comparable compression ratio.
• Achieving the best compression ratios takes four to even twelve times longer than with bzip2. However. this doesn't affect decompressing speed.

To illustrate the usefulness of this approach, I took the code from a previous article on compressing Http Cookies, and added another CompressedCookie class to it in which I use the 7zip library instead of the default System.IO.Compression deflate mode classes. With LZMA compression (the default for 7Zip), we can get a huge increase in the amount of data that can be stored in the 4,000 byte allowable maximum size for a cookie.

The downloadable Visual Studio 2005 Solution below includes the SevenZip C# LZMA classes, my SevenZipHelper "wrapper class", and a reconstructed CompressedCookies Web application illustrating it's use.

Using my "Helper" class is utterly simple:

The static Compress Method:

byte[] SevenZip.Compression.LZMA.SevenZipHelper.Compress( byte[] )

The static Decompress Method:

byte[] SevenZip.Compression.LZMA.SevenZipHelper.Decompress( byte[] )

I don't think it could be made much simpler!

The new LZMA Cookie Compression class:

using System.Text;
using System.Web;
using System.IO.Compression;
using System.Runtime.Serialization.Formatters.Binary;
using System.IO;
using SevenZip.Compression.LZMA;
namespace CompressedCookies
{
    public static class CpCookiesLzma
    {
        public static int Set(string cookieName, object cookieValue, DateTime expirationDate)
        {
            int siz = 0;
            try
            {
                BinaryFormatter bf = new BinaryFormatter();
                MemoryStream ms = new MemoryStream();
                bf.Serialize(ms, cookieValue);
                byte[] inbyt = ms.ToArray();
                byte[] b = SevenZip.Compression.LZMA.SevenZipHelper.Compress(inbyt);
                string sCookieVal = Convert.ToBase64String(b);
                HttpCookie cook = new HttpCookie(cookieName);
                cook.Value  = sCookieVal;
                siz = sCookieVal.Length;
                cook.Expires = expirationDate;
                HttpContext.Current.Response.Cookies.Add(cook);                 
            }
            catch 
            {
                
                throw ;
            }
            return siz;
        }

        public static object Get(string cookieName)
        {
            object retval = null;
            try
            {
                byte[] bytCook = Convert.FromBase64String(HttpContext.Current.Request.Cookies[cookieName].Value);

                byte[] outByt = SevenZip.Compression.LZMA.SevenZipHelper.Decompress(bytCook);
                
             
                MemoryStream outMs = new MemoryStream(outByt);
                outMs.Seek(0, 0);
                BinaryFormatter bf = new BinaryFormatter();                
                retval = (object)bf.Deserialize(outMs, null);
            }
            catch(Exception ex)
            {                
                throw ex;
            }
            return retval;
        }

        public static bool Delete(string cookieName)
        {
            bool retval = true;
            try
            {
                HttpContext.Current.Request.Cookies[cookieName].Expires = DateTime.Now.AddDays(-365);
            }
            catch
            {
                retval = false;
            }
             return retval;
        }

         
    }
}

Thank you for providing this article it was exactly what I was looking for. I tested it all on a development environment (.net v2, XP) and everything worked fine. I then tested it on a server environment, (,net v2, windows server 2003 enterprise) but it immediately failed with the following error:

System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown. at SevenZip.Compression.LZ.BinTree.Create(UInt32 historySize, UInt32 keepAddBufferBefore, UInt32 matchMaxLen, UInt32 keepAddBufferAfter) at SevenZip.Compression.LZMA.Encoder.Create() at SevenZip.Compression.LZMA.Encoder.SetStreams(Stream inStream, Stream outStream, Int64 inSize, Int64 outSize) at SevenZip.Compression.LZMA.Encoder.Code(Stream inStream, Stream outStream, Int64 inSize, Int64 outSize, ICodeProgress progress) at SevenZip.Compression.LZMA.SevenZipHelper.Compress(Byte[] inputBytes) at CompressedCookies.CpCookiesLzma.Set(String cookieName, Object cookieValue, DateTime expirationDate)

The website hosted on this server continued to work correctly and all other aspx pages were still functioning correctly. Are you able to give any pointers as to what might have caused this?

I've seen this on occasion. I'll take a look at it, the seven zip comes from the SDK and is not my code. Also, it could be that you are attempting to compress more data than a cookie can hold.

Also, try reducing the size of the dictionary in the SevenZipHelper class:

static int dictionary = 1 << 21;

Thank you for the response. Just on your first point, its definitely not the size of the data, I am only testing it with about 30 characters.