Monday, June 8, 2009

Attribute Processors

I've been busy working on other projects, so I haven't worked on the serializer for a while, but I recently got back into things. I've recently added a new feature called Attribute Processors that will be included in an upcoming release. System.Attribute in .NET is a great thing, it allows you to specify extra information about a class, property, field, parameter, etc. right in the code. JsonExSerializer has about 5 attributes that is uses, JsonExIgnore and JsonExProperty probably being the most used. Before, certain places in the code would specifically look for attributes and apply some logic to deal with them. Now I've abstracted that out so that instances of the MetaData class (TypeData, PropertyData, etc) get passed through a list of AttributeProcessors. Those attribute processors can then look for attributes decorating class members or the classes themselves and then modify properties of the MetaData instance.

How is this useful? It's useful to me in that it makes it very easy to add new attributes, and the code is cleanly separated so there is less chance of breaking things. It's useful to everyone else in that you can now add your own custom attributes OR make use of existing attributes that may already be decorating your code.

Here's a good example. Just about every serialization framework out there has some way of excluding properties from being serialized. Me too. Mine is called JsonExIgnore. You may be switching from an existing framework, or using 2 frameworks simultaneously. The built-in XmlSerializer uses the XmlIgnore attribute. I've added an AttributeProcessor class for it which is not turned on by default, but is very easy to add in.


Serializer serializer = new Serializer(typeof(MyClass));

// let's just use the XmlIgnore attribute that we already
// decorated our classes with so we don't have to type anymore
serializer.TypeHandlerFactory.AttributeProcessors.Add(new XmlIgnoreAttributeProcessor());

Here is the code for the XmlIgnoreAttributeProcessor:

public class XmlIgnoreAttributeProcessor : AttributeProcessor
{
public override void Process(MetaDataBase metaData, ICustomAttributeProvider attributeProvider, SerializationContext serializationContext)
{
if (metaData is IPropertyData)
{
IPropertyData property = (IPropertyData)metaData;
if (attributeProvider.IsDefined(typeof(XmlIgnoreAttribute), false))
property.Ignored = true;
}
}
}

TypeData, PropertyData, FieldData all extend from MetaDataBase. They represent Types, properties, and fields respectively. If you're only interested in one type of metadata then you need to check the metaData property to see what it is. IPropertyData is an interface that both PropertyData and FieldData implement. ICustomAttributeProvider is a .NET class that Reflection classes such as Type, MemberInfo, etc implement so that you can read their attributes. You query for the attribute(s) that your processor knows how to handle and then react accordingly if you find any. And that's it.

Sunday, January 25, 2009

Release 3.0 of JsonExSerializer

Release 3.0 of the Serializer is now here. This release contains some key customization enhancements. You now have very fine grained control over the serialization/deserialization process. I'll blog more about this in some later posts.

I also fixed some localization issues plaguing those users outside the U.S. Dates and numbers now use the InvariantCulture to ensure consistent serialization/deserialization regardless of locale. Thanks guys for pointing those out!

Enjoy!

Tuesday, September 16, 2008

Random Thoughts

Sometimes defects make your program go faster. Who needs all that extra input anyway, the user won't miss it. I believe some humans practice this productivity technique as well.

Stacks and Queues are interchangeable AS LONG AS you only ever have one item in them at a time. After that the difference becomes a little more pronounced.

Sunday, September 14, 2008

Pre-Optimization

I'm sure you've heard the saying "Pre-optimization is the root of all evil", or something similar. Well I have too, and I knew it before writing the serializer. But sometimes I can't help myself, I just know that doing it one way is going to be faster that some other way, or so I thought. The TokenStream class splits up the input stream into tokens. When I built the class I decided to only read from the physical reader if necessary when a token was requested by the parser. I'd gotten used to seeing that the Parser and Token stream classes took up most of the time spent during Deserialization when running them under the profiler. I figured it was just I/O taking so long and that I couldn't do anything about it. Well I finally decided to try something a little different, so I made a quick change to read all the tokens up front and store them. Wow! What a difference! That one little change made it about 4 times faster. It went from about an average of 20ms down to 5ms. That may be nitpicky since we're down here in the low millisecond range, but those numbers put us a lot closer to the Binary and Xml Serializers which I usually compare against. So it just goes to show that you should always profile and measure your optimizations to see if you're really improving anything.

Saturday, July 26, 2008

Light-weight Code Gen

I've been toying around with the idea for a while of doing some kind of runtime dynamic code generation to speed up performance of the serializer. Well I've finally gotten around to it. I searched around the net trying to find examples and documentation and finally found some info on Light-weight code gen. Light-weight code gen allows you to generate IL code at runtime. It's advantages over CodeDom are that it doesn't require you to generate an assembly or class to hold your methods and it can be garbage collected when no longer needed just like other classes.

Light-weight Code Gen, LCG for short, is accomplished via the DynamicMethod class. I found the perfect example to follow here. However I needed slightly different code to handle calling getters on a struct. It took me a while to figure out how to do it. My advice if you're trying to do this is to write normal c# code in a method and compile it. Then open up your assembly with IL DASM, which is part of your Visual Studio install. Look at the code and then try and duplicate it with Light-weight Code Gen. This is exactly what I did, however I misread some of the code and ended up with slightly different code which caused exceptions.

Which brings me to my next point. LCG can be hard to debug since there's no source file. I misread one of the op-codes and ended up with some memory exceptions. I didn't have much luck setting up debugging, but I did find this very helpful Debugger Visualizer that allowed me to view the generated code and compare it to my example method that the compiler generated and figure out what was wrong. Here is the code to call a property getter on a struct (Note: you can't call property setters on structs since they are value objects).


/// <summary>
/// Creates a dynamic getter for the property
/// </summary>
/// <param name="propertyInfo">
/// <returns></returns>
private static GenericGetter CreateStructGetMethod(PropertyInfo propertyInfo)
{
/*
* If there's no getter return null
*/
MethodInfo getMethod = propertyInfo.GetGetMethod();
if (getMethod == null)
return null;

/*
* Create the dynamic method
*/
Type[] arguments = new Type[1];
arguments[0] = typeof(object);

string mName = propertyInfo.GetGetMethod().Name;
DynamicMethod getter = new DynamicMethod(
String.Concat("_Get", propertyInfo.Name, "_"),
typeof(object), arguments, propertyInfo.DeclaringType);
ILGenerator generator = getter.GetILGenerator();
generator.DeclareLocal(typeof(object)); // return value
generator.DeclareLocal(propertyInfo.DeclaringType); // struct instance

generator.Emit(OpCodes.Ldarg_0);
generator.Emit(OpCodes.Unbox_Any, propertyInfo.DeclaringType);
generator.Emit(OpCodes.Stloc_1); // get the unboxed value from the stack
//generator.Emit(OpCodes.Ldloc_1);
generator.Emit(OpCodes.Ldloca_S, 1);
generator.EmitCall(OpCodes.Call, getMethod, null);

if (!propertyInfo.PropertyType.IsClass)
generator.Emit(OpCodes.Box, propertyInfo.PropertyType);

generator.Emit(OpCodes.Ret);

/*
* Create the delegate and return it
*/
return (GenericGetter)getter.CreateDelegate(typeof(GenericGetter));
}



So what was the performance increase?

The Old:


Serialization Performance Test
Serializer Type: JsonExSerializer
File Size : 31989 bytes
Iterations : 2500
Object Count : 100
Total Time : 27373ms
Avg Time : 10.949ms per iteration


Deserialization Performance Test
Serializer Type: JsonExSerializer
File Size : 31989 bytes
Iterations : 2500
Object Count : 100
Total Time : 51909ms
Avg Time : 20.764ms per iteration


The New:


Serialization Performance Test
Serializer Type: JsonExSerializer
File Size : 31989 bytes
Iterations : 2500
Object Count : 100
Total Time : 17534ms
Avg Time : 7.014ms per iteration


Deserialization Performance Test
Serializer Type: JsonExSerializer
File Size : 31989 bytes
Iterations : 2500
Object Count : 100
Total Time : 34285ms
Avg Time : 13.714ms per iteration


About 30% faster on serialization, and 35% faster on deserialization.

First Post

This is a blog for the JsonExSerializer project hosted on google code. The JsonExSerializer is a c# library that allows you to serialize C# objects to and from JSON format. I'm the sole developer for the project and I'll use this blog to detail my thoughts as I develop the project and also as a sounding board for new features and ideas on the project.