Don't click here unless you want to be banned.

LSL Wiki : LibrarySerialize

HomePage :: PageIndex :: RecentChanges :: RecentlyCommented :: UserSettings :: You are crawl427.us.archive.org
Library providing two functions (currently...I plan to implement selective unserialization shortly) for providing storage and transport-safe data list serialization/unserialization.

These functions are derived from ReadyJack 's and EzharFairlight 's previous implementations. (See ExampleListConversion). It incorporates suggestions made by BlindWanderer (Strife) regarding the usage of Float2Hex when serializing float values. You will need to get his code, which can be found at LibraryFloat2Hex.

It also deviates from other implementations by using a 2-stage delimiter. Each list item is completely accessible after the first stage unserialization, which means you don't have to seek through the list if you only want and item at a specific index. Some of you may hate this. I like it.

I'm also working on a PHP library that provides classes that represent all the lsl data types. Included with this is a list object which has the same serialization and unserialization code, which becomes pretty handy now that we have HTTP available. I'll be making the library available soon, and I'll throw a link up here.


Strife's Thoughts:
This script has good bones, my main concern is with using the CSV functions, CSV has a tendancy of eating whitespace. I will post modified versions below that should solve that (and will be better optimized). My tests have shown it to be about 14% faster. Also I have writen a slightly faster Float2Hex.

"Note:
After looking at Strife's code, I dropped mine from the page. His is definitely better, and I don't want to cause confusion over which implementation should be used. I'll be updating my PHP stuff to be compatible with the slightly different serialization format. -- ShadowGretzky"

It's slightly different? I designed and tested the functions to be interchangable. Whats changed? -- BW aka Strife
Ohhhh with the CSV, the LSL CSV functions don't need the whitespace so "a,b" and "a, b" parse to the same thing; CSV is good for everything but keys & strings (keys and strings are really the same data-type). -- BW

"Correct. I was referring more to the changes I need to make on the PHP side of things, where I was expecting the whitespace. :) In any event, I'm happy with the changes you've made. Thanks again for your help. -- ShadowGretzky"

// lib_serializer.lsl
//
// This code will run a bit more slowly than RJ's list_2_string() and
// string_to_list(), but a) they'll be float-safe and b) it'll be
// compatible with my upcoming SecondLife PHP library. Lists suck anyway.
//
// I use 2-space tabwidths....sue me. ;)
//
// props to: Ezhar Fairlight, Strife Onizuka, Ready Jack
// apologies to: Ezhar Fairlight, Strife Onizuka, Ready Jack
//
// Requires: Float2Hex() http://secondlife.com/badgeo/wakka.php?wakka=LibraryFloat2Hex
// #include lib_floats.lsl

string SERIALIZER_DELIMITER = "$!#"; // Purposely different from RJ's delimiter to discourage cross-usage


// Serializes list into a storable, transportable form
// I use comma-separated values to store each variable's properties
// (type and value).  I use SERIALIZER_DELIMITER to separate one variable
// from the next.  That way, it's possible to unserialize a specific list
// position without having to process an entire stream (given the variable
// number of list indexes based on type)
string serializeList(list l) {
    integer i = (l != []);//This is a hack, it gets list lenght.
    if(i)
    {
        string serialized_data = "";
        integer type = 0;
        string result;
        {@loop;
            // this custom loop is about as fast as a while loop.
            // we build the string backwords for memory reasons.
            // My kingdom for select statements....

            if (TYPE_FLOAT == (type = llGetListEntryType(l, (i=~-i))))

                // floats get extra love
                result = Float2Hex(llList2Float(l, i));
            else if (TYPE_VECTOR == type) {
                vector v = llList2Vector(l, i);
                result = Float2Hex(v.x) + "," + Float2Hex(v.y) + "," + Float2Hex(v.z);
            } else  if (TYPE_ROTATION == type) {
                rotation r = llList2Rot(l, i);
                result = Float2Hex(r.x) + "," + Float2Hex(r.y) + "," + Float2Hex(r.z) + "," + Float2Hex(r.s);
            } else //if ((TYPE_INTEGER == type) || (TYPE_STRING ==  type) || (TYPE_KEY == type))
                result = llList2String(l, i);// integers, strings and keys required no voodoo

            if(i)
            {
                //This came to me after reverse engeneering LSL bytecode, the realization that LSL memory management sucks.
                serialized_data = SERIALIZER_DELIMITER + (string)type + (serialized_data = result = ",") + result + serialized_data;
                jump loop;
            }
        }
        return (string)type + (serialized_data = result = ",") + result + serialized_data;
    }
    return "";
}

list unserializeList(string serialized_data) {
    // TODO: add some checking in-case we encounter a poorly formed serialization
    //       consider using the same mem-packing list pushing technique used above
    //       (want to run performace tests first)
    list result = [];
    list t;
    list l = llParseStringKeepNulls(serialized_data, [SERIALIZER_DELIMITER], []);

    string item;
    integer i = (l != []);//This is a hack, it gets list lenght.
    integer type = 0;
    do
    {
        if((type = (integer)(item = llList2String(l, (i=~-i)))))
        {//Little error checking (also takes care of null strings).
            integer p = llSubStringIndex(item, ",");
            item = llDeleteSubString(item, 0, p);
            // How about those switch statements, Lindens???
            if (TYPE_INTEGER == type)
                t = [(integer)item];
            else if (TYPE_FLOAT == type)
                t = [(float)item];
            else if (TYPE_STRING == type)
                t = [item];
            else if (TYPE_KEY == type)
                t = [(key)item];
            else
            {
                if (TYPE_ROTATION ^ type)// if (TYPE_VECTOR == type)
                    t = [(vector)("<" + item + ">")];
                else// if (TYPE_ROTATION == type)
                    t = [(rotation)("<" + item + ">")];
            }
            //when dealing with very long lists it might be advantagous to use the commented out line instead.
            //result = [result = t] + result;
            result = t + result;

        }
    }while(i);
    return result;
}
Comments [Hide comments/form]
How fast is it?
-- ZeeraXi (2006-10-20 17:51:01)
Attach a comment to this page: