Parsing “Unsafe” Method Signatures

If you’ve ever used JustMock then you should be familiar with code fragments like this one:

public interface IFoo
{
    byte Bar(int i, string s, long l);
}

var foo = Mock.Create<IFoo>();
Mock.Arrange(() => foo.Bar(123, "test", 321)).Returns(5);

The interesting thing in this code is in the Arrange method. Its signature is as follows:

public static FuncExpectation<TResult> Arrange<TResult>(Expression<Func<TResult>> expression)

The method accepts a single parameter of Expression<TDelegate> type. This allows JustMock to parse the actual method name and the parameter values. In this blog post I am going to show you a simple and yet effective approach for parsing methods that contain pointer parameters.

Lets make our example more interesting.

public unsafe interface IFoo
{
    byte** Bar(int* i, string s, ref long* l);
}

Because Expression<TDelegate> expects a delegate type as a type parameter we can define a new delegate type and use it to construct a proper expression object.

public unsafe delegate byte** BarDel();
//..
int* pi = (int*)IntPtr.Zero;
long* pl = (long*)IntPtr.Zero;
Expression<BarDel> expr = () => Bar(pi, "test", ref pl);

This is all good and nice. Now we can change Arrange method signature as follows:

// to keep it simple, we changed the return type to void
public static void Arrange<TDelegate>(Expression<TDelegate> expr) { }
//..
int* pi = (int*)IntPtr.Zero;
long* pl = (long*)IntPtr.Zero;
Mock.Arrange<BarDel>(() => Bar(pi, "test", ref pl));

The only thing that left is to make the return type independent from the return type of the actual Bar method. This is easy. A pointer to a pointer to a byte (byte**) is just… a pointer! The only tricky thing is that JustMock doesn’t know what actual type is pointed. So, it is a sane decision to leave this knowledge to the unit test author. We need a simple generalization over IntPtr type and that’s all.

namespace Telerik.JustMock
{
    public class PtrBase
    {
        private readonly IntPtr addr;

        protected PtrBase(IntPtr addr)
        {
            this.addr = addr;
        }

        public IntPtr Addr { get { return this.addr; } }
    }

    public delegate T PtrDel<T>() where T : PtrBase;
}

Now, we have to change Arrange method as follows:

public static void Arrange<TPtr>(Expression<PtrDel<TPtr>> expr, params TPtr[] arr)
    where TPtr : PtrBase
{
    var methodCallExpr = (expr.Body as UnaryExpression).Operand as MethodCallExpression;

    // process methodCallExpr
}

Lets see how we have to change our test code so we can call the new Arrange method. Now, we have to take care for all pointers and convert them. This is a trivial task:

public class Ptr : PtrBase
{
    private Ptr(IntPtr addr) : base(addr) { }

    public unsafe static implicit operator Ptr(int* ptr)
    {
        return new Ptr(new IntPtr(ptr));
    }

    public unsafe static implicit operator Ptr(long* ptr)
    {
        return new Ptr(new IntPtr(ptr));
    }

    public unsafe static implicit operator Ptr(byte** ptr)
    {
        return new Ptr(new IntPtr(ptr));
    }
}

Finally, we have to change our Arrange method invocation:

int* pi = (int*)IntPtr.Zero;
long* pl = (long*)IntPtr.Zero;
Mock.Arrange<Ptr>(() => Foo(pi, "test", ref pl), pi, pl);

The code can be improved in many ways. One way to improve it is to replace PtrDel<T> with Func<TResult> though I prefer the explicit constraint on PtrBase. Another way to improved the code is by making PtrBase abstract type and providing a template method that will be called inside the Arrange method, so the actual value can be retrieved. Here is a simple implementation:

namespace Telerik.JustMock
{
    public abstract class PtrBase
    {
        private readonly IntPtr addr;
        private readonly int cookie;

        protected PtrBase(IntPtr addr, int cookie)
        {
            this.addr = addr;
            this.cookie = cookie;
        }

        public IntPtr Addr { get { return this.addr; } }
        public int Cookie { get { return this.cookie; } }

        public abstract object ReadValue();
    }
}

When you create a new type which inherits from PtrBase type you should provide a unique cookie for each pointer type (e.g. 1 for int*, 2 for long*, 3 for byte**, etc.). When Arrange method is called, it will call in turn your implementation of ReadValue method and use the return value. In a matter of fact we don’t need this cookie mechanism because we can get the actual pointer type from methodCallExpr variable. The real purpose of ReadValue is to allow execution of user defined code in a lazy fashion. In case you don’t want this feature you can read the actual pointed value inside Ptr implicit conversion operator implementation and pass it to PtrBase constructor.

JustMock Design Choices

JustMock is a small product. It is around 10,000 lines of code. Despite its small size there are a lot of decisions that were made and many others that have to be made. In this post I would like to shed some light on our decision making process and I’ll try to answer the question why JustMock is built the way it is. The topic is quite large for a single blog post so I am going to focus on small part from JustMock – namely design choices for private method resolution.

First, I would like to emphasize that JustMock was built around C#. Every time when we had to decide how particular feature should be implemented we designed it from C# perspective. As a result JustMock works with concepts that are better expressed in C# rather than VB.NET for example. The reason for this decision is that it seems C# developers tend to use more often mocking tools/frameworks than VB.NET ones. So far, it seems we did the right choice.

As a consequence JustMock is more or less tightly coupled to the C# compiler. So are other mocking libraries. Lets consider the following example with Moq:

    public interface IFoo
    {
        int Bar(string s);
        int Bar(StringBuilder s);
    }

Often, it makes sense to define overloading methods in your code. The way IFoo is defined is perfectly legal. So, lets see a corner case with this interface. Suppose we want to mock Bar(string s) method when the argument is null:

    var mock = new Mock<IFoo>();
    mock.Setup(foo => foo.Bar(null)).Returns(1);

You will quickly find that the this code fragment does not compile. Here goes the error:

error CS0121: The call is ambiguous between the following methods or properties:
'IFoo.Bar(string)' and 'IFoo.Bar(System.Text.StringBuilder)'

This error has nothing to do with Moq. It is related to the C# compiler which in this case does not have enough information how to do method resolution. So, it is our fault and we have to fix it.

Here comes the second guide line we follow when design JustMock – there are no “right” or “wrong” choices. It is not white or black; rather it is like different gray shades. We could employ C# syntax in JustMock API to provide single method resolution (in most cases) but we didn’t. The reason is that different C# developers have different strategies to correct this issue. Here is a sample list with the most common fixes:

    // option 1
    mock.Setup(foo => foo.Bar((string)null)).Returns(1);

    // option 2
    mock.Setup(foo => foo.Bar(null as string)).Returns(1);

    // option 3
    mock.Setup(foo => foo.Bar(default(string))).Returns(1);

    // option 4
    string s = null;
    mock.Setup(foo => foo.Bar(s)).Returns(1);

After all, it’s matter of taste. Please notice that while I provided the sample above with Moq the issue is valid for JustMock. Lets make the things more interesting. Suppose we have the following concrete type:

    public sealed class Foo
    {
        public Foo() { ... }
        // ...
        private int Bar(string s) { ... }
        private int Bar(StringBuilder s) { ... }
    }

This time Bar(string s) and Bar(StringBuilder s) are private methods in a sealed type. Now we have to design an API for mocking private methods.

Note: Mocking private methods is a controversial thing. There are two equally large camps and a hot discussion between them. For good or bad, JustMock offers this feature.

We can provide the following method:

    public static void Arrange(object instance, string methodName, params object[] args) 
    { ... }

And then use it as follows:

    var foo = Mock.Create<Foo>();
    Mock.Arrange(foo, "Bar", null);

    string s = null;
    int actual = foo.Bar(s);

The problem with this approach is that the last argument passed to Arrange method is null and thus we cannot resolve the correct Bar(…) method. This time options 1 to 4 provided above don’t solve the issue. Lets see what can we do.

Approach 1: no method resolution

The problem is all about method resolution so if we don’t have to do it then we are good. Lets provide the API:

    public static void Arrange(object instance, MethodInfo method, params object[] args)
    { ... }

We have to change our test code:

    Type[] argTypes = { typeof(string) };
    var bar = typeof(Foo).GetMethod("Bar", argTypes);

    var foo = Mock.Create<Foo>();
    Mock.Arrange(foo, bar, null);

The test code is more messy. Now we have to deal with System.Reflection stuff which is not related to our code. Some developers are fine with this approach though. While this approach is suitable for API-to-API scenarios a lot of people prefer something more “friendly”.

Approach 2: pass method parameter types

We can pass the actual parameter types along with the parameter values. Lets change the Arrange method signature:

    public static void Arrange(object instance, string methodName, Type[] types, params object[] args) 
    { ... }

We added Type[] argument to Arrange method so we have to change our test code:

    var foo = Mock.Create<Foo>();
    Type[] argTypes = { typeof(string) };
    Mock.Arrange(foo, "Bar", argTypes, null);

    // use foo instance

This solution is easy to understand. It can also handle ref and out parameters and other more complex scenarios.

Approach 3: pass method parameter types as generic parameters

In case we decide to support private methods with simple signatures (no ref and out parameters and other fancy stuff) we can modify our previous approach by providing generic version of Arrange method (we can use T4 template for code generation; for the sake of simplicity we add a single generic parameter):

    public static void Arrange<T>(object instance, string methodName, params object[] args) 
    { ... }

We change our test code accordingly to:

    var foo = Mock.Create<Foo>();
    Mock.Arrange<string>(foo, "Bar", null);

    // use foo instance

This all looks good and nice but now we have to provide generic parameter types even when we don’t need them.

    var foo = Mock.Create<Foo>();
    Mock.Arrange<string>(foo, "Bar", "test");

    // use foo instance

It is not that bad but this can be annoying if Bar(…) method accepts, lets say, 5 integer parameters.

Approach 4: use typed references

The CLR and .NET framework already provide support for typed references but for our need we can come up with something much simpler. We have to provide support for our corner case when values are null. So we can provide a simple Null<T> type as follows:

    public sealed class Null<T>
        where T : class
    {
    }

Every time when we have to pass null value we are going to replace it with an instance of Null<T> type:

    var foo = Mock.Create<Foo>();
    Mock.Arrange(foo, "Bar", new Null<string>());

    // use foo instance

As the previous approach this one does not support methods with ref and out parameters. We can try to fix it by providing another constructor for Null<T> type:

    public sealed class Null<T>
        where T : class
    {
        public Null() {}

        public Null(System.Reflection.ParameterAttributes attrs)
        {
            this.Attrs = attrs;
        }

        public ParameterAttributes Attrs { get; private set; }
    }

Suppose we add the following method to Foo:

    private int Bar(ref string s) { ... }

If we want to mock it then our test code should like something like:

    var foo = Mock.Create<Foo>();
    Mock.Arrange(foo, "Bar", new Null<string>(ParameterAttributes.Out));

    // use foo instance

It doesn’t look nice to me. The syntax becomes more verbose and doesn’t flow naturally. We can mitigate it by adding new helper type as follows:

    public static class Arg
    {
        public static Null<T> Null<T>()
            where T : class
        {
            return new Null<T>();
        }
    }

Now we can compare the two options

    var foo = Mock.Create<Foo>();

    //option 1
    Mock.Arrange(foo, "Bar", new Null<string>());

    //option 2
    Mock.Arrange(foo, "Bar", Arg.Null<string>());

    // use foo instance

The readability seems a little bit better but many developers find the syntax verbose.

Approach 5: using dynamic (C# 4 and higher)

This is a good approach for a lot of developers. There is a caveat though. There are a plenty of customers that are still using .NET 3.5 and earlier. Shall we abandon them? A tough question.

Which one to use?

First, it is a wrong question. There is no silver bullet and the best thing we can do is to decide on a case-by-case basis. So far, we saw a few possible options; there are other alternatives as well. The options are not mutually exclusive which makes it even harder. If you are curious how we implemented it you can download a trial JustMock version.

NUI or GUI or … not

Today there are a lot of devices (smartphones, tablets, handheld game consoles, etc.) with support for gestural interface. We often call such interfaces natural user interfaces (NUI). These interfaces are in contrast with the traditional graphical user interfaces (GUI). In this post I am going to share my thoughts and experience with NUI.

Every software engineer knows that one should not make definitive assumptions how IT systems will be used. Often IT systems are used in unexpected and unpredicted ways. However such assumptions are rarely considered when it comes to user interfaces.

Natural user interfaces strive to offer more intuitive and easy human-technology interaction. This sounds all good and nice but lets focus on the word natural. In my understanding, natural means that the user doesn’t have to use artificial input/interaction devices such as keyboard and mouse. However, this does not mean that NUI is easy and intuitive for everyone. Users have still to learn it. While NUI is crucial for fast adoption of a new product it doesn’t mean it is easy to achieve it.

Today’s NUIs are not natural. Every company provides its own NUI standard and its per se artificial gestural language. Trying forcefully to apply NUI standard is not a solution. Cultural aspects should be preserved and considered. For example, almost every smartphone does not have support for left handed people.

My experience with different Android, iOS and Windows (Phone) 8 devices shows me that there is inconsistency between all of them. Often different applications for a particular platform use different gestures for the same command. Sometimes the companies use different NUI vocabularies for the same gesture. This could be confusing for the users.

In closing, I think that natural interfaces should allow us to interact with devices in the manner we interact with objects in everyday life. The devices should be able to learn user’s natural gestures/language and adapt to it. The companies should provide their natural interfaces as a fallback option.

 

Thoughts on C# Compiler

In 2010 I wrote the blog post Fun with pointers in C#. Back then, I thought it was fun. Today, I am not so sure. Lets take a look at the following code fragment:

using System;

namespace ClassLibrary1
{
    public class Class1
    {
        unsafe public void Method1(ref object* obj)
        {
            Console.WriteLine(obj->ToString());
        }
    }
}

If you try to compile this code with C# compiler distributed with .NET 1.1 you will get the following error:

error CS1005: Indirection to managed type is not valid

This is all good and nice because error CS1005 is aligned with the C# language specification which states:

Unlike references (values of reference types), pointers are not tracked by the garbage collector—the garbage collector has no knowledge of pointers and the data to which they point. For this reason a pointer is not permitted to point to a reference or to a struct that contains references, and the referent type of a pointer must be an unmanaged-type.

An unmanaged-type is any type that isn’t a reference-type or constructed type, and doesn’t contain reference-type or constructed type fields at any level of nesting. In other words, an unmanaged-type is one of the following:

sbyte, byte, short, ushort, int, uint, long, ulong, char, float, double, decimal, or bool.

Any enum-type.

Any pointer-type.

Any user-defined struct-type that is not a constructed type and contains fields of unmanaged-types only.

However since .NET 2.0 Microsoft introduced a bug that allows you to compile the code fragment above. Currently, I use C# compiler version 4.0.30319.17929 and I can still compile the code. If you run peverify.exe tool on the produced assembly you will get the following error:

[IL]: Error: [ClassLibrary1.dll : ClassLibrary1.Class1::Method1][offset 0x00000001]
Unmanaged pointers are not a verifiable type.
1 Error(s) Verifying ClassLibrary1.dll

Mono C# compiler (version 3.0.6) does it right. It fails, as expected, with error CS0208:

Class1.cs(7,40): error CS0208: Cannot take the address of, 
get the size of, or declare a pointer to a managed type `object'

Considering the fact that .NET 2.0 was released in 2005 it is hard to believe that Microsoft has not fixed the bug for the last 7 years. The only explanation I have here is that the bug is quite esoteric.

RVA Static Fields

In JustTrace Q1 2013 we added support for analyzing GC roots that are static fields. The implementation of this feature uses ICorProfilerInfo2::GetAppDomainStaticAddress,  ICorProfilerInfo2::GetThreadStaticAddress and so on. Among all these methods, there is a very interesting one, namely ICorProfilerInfo2::GetRVAStaticAddress. In this post I am going to focus on a little known CLR feature that is closely related to this method.

What I find so interesting in ICorProfilerInfo2::GetRVAStaticAddress method is the RVA abbreviation. It stands for relative virtual address. Here is the definition from Microsoft PE and COFF Specification:

In an image file, the address of an item after it is loaded into memory, with the base address of the image file subtracted from it. The RVA of an item almost always differs from its position within the file on disk (file pointer).

Once we know what RVA is, we can make a few reasonable guesses about RVA static fields. Since RVA should be known at compile/link time, it is reasonable to guess that the static field should be a value type and should not contain any reference types. We can use the same argument so that any RVA field should be static since it does not make sense to have multiple instance fields occupying the same RVA.

Lets try find out whether our guesses are correct. Because VB.NET\C# can specify only application domain static and thread static fields we should look at Standard ECMA-335. We are interested in RVA static fields, so it makes sense to look at the field definition specification (II.16 Defining and referencing fields)

Field ::= .field FieldDecl
FieldDecl ::=
[ ‘[’ Int32 ‘]’ ] FieldAttr* Type Id [ ‘=’ FieldInit | at DataLabel ]

The interesting thing here is the at clause. This clause is used together with a DataLabel, so we have to find out what DataLabel is. Reading the document further we can see that II.16.3 Embedding data in a PE file paragraph starts with the following words:

There are several ways to declare a data field that is stored in a PE file. In all cases, the .data directive is used.

The good thing is that the document provides the following example code:

.data theInt = int32(123)
.data theBytes = int8 (3) [10]

After reading a little bit further we find the following text:

[…]In this case the data is laid out in the data area as usual and the static variable is assigned a particular RVA (i.e., offset from the start of the PE file) by using a data label with the field declaration (using the at syntax).

This mechanism, however, does not interact well with the CLI notion of an application domain (see Partition I). An application domain is intended to isolate two applications running in the same OS process from one another by guaranteeing that they have no shared data. Since the PE file is shared across the entire process, any data accessed via this mechanism is visible to all application domains in the process, thus violating the application domain isolation boundary.

Now, we know enough about RVA static fields so lets create a test scenario. I tried to keep it as simple as possible. I decided to create a console app that uses a class library, so I can replace the class library assembly with different implementation. Here is the source code for the console app:

using System;

namespace ConsoleApplication1
{
    class Program
    {
        static void Main(string[] args)
        {
            PrintStaticVar();

            AppDomain app = AppDomain.CreateDomain("MyDomain");
            app.DoCallBack(PrintStaticVar);

            AppDomain.Unload(app);
        }

        unsafe static void PrintStaticVar()
        {
            fixed (int* p = &ClassLibrary1.Class1.MyInt)
            {
                IntPtr ptr = new IntPtr(p);
                Console.WriteLine("app {0}, static {1:X}, addr {2:X}",
                                AppDomain.CurrentDomain.FriendlyName,
                                ClassLibrary1.Class1.MyInt,
                                ptr.ToInt64());
            }
        }
    }
}

As you can see, it prints the value and the address of MyInt static variable and it does so for the two application domains. Here is the source code for the class library:

using System;

namespace ClassLibrary1
{
    public class Class1
    {
        public static int MyInt = 0x11223344;
    }
}

The output from running the console app is as follows:

app ConsoleApplication1.exe, static 11223344, addr AF3C74
app MyDomain, static 11223344, addr E23EAC
Press any key to continue . . .

As you can see, the app prints unique address for each application domain. Now, it is time to provide different implementation for ClassLibrary1. This time we should write it in ILAsm:

.assembly extern mscorlib
{
  .publickeytoken = (B7 7A 5C 56 19 34 E0 89)
  .ver 4:0:0:0
}
.assembly ClassLibrary1
{
  .ver 1:0:0:0
}

.data MyInt_Data = int32(0x11223344)

.class public auto ansi ClassLibrary1.Class1
	extends [mscorlib]System.Object
{
  .field public static int32 MyInt at MyInt_Data
}

The last thing we have to do is to run the console app once again. Here is output:

app ConsoleApplication1.exe, static 11223344, addr 7A4000
app MyDomain, static 11223344, addr 7A4000
Press any key to continue . . .

As expected, this time the app prints the same address for both application domains. If you run the following command

dumpbin.exe /all ClassLibrary1.dll

and examine the output then you should see something similar

SECTION HEADER #2
  .sdata name
       4 virtual size
    4000 virtual address (00404000 to 00404003)
     200 size of raw data
     600 file pointer to raw data (00000600 to 000007FF)
       0 file pointer to relocation table
       0 file pointer to line numbers
       0 number of relocations
       0 number of line numbers
C0000040 flags
         Initialized Data
         Read Write

RAW DATA #2
  00404000: 44 33 22 11                                      D3".

  Summary

        2000 .sdata

We can see that the ILAsm compiler emitted MyInt_Data value in the .sdata section. Cross checking with ILDasm only assures us that FieldRVA table contains the correct RVA.rvastaticfield

Lets check our guess that RVA static fields should be value type only. It is easy to modify our code by adding the following lines appropriately:

.data MyString_Data = char*("test")
//...
.field public static string MyString at MyString_Data

If you try to run the console app again you will get System.TypeLoadException.

In closing, I think RVA static fields are little known CLR feature because they aren’t very useful. It is good know that the CLR has this feature but I guess its practical usage is limited.

 

JustTrace Q1 2013 SP1 is out

The last two weeks I was busy with the latest release of JustTrace. We had some hard times with JustTrace Q1 2013 release but after all we managed to fix a critical bug and we shipped SP1 after one week. There are a lot of new features in this release but I am going to focus on one thing only:

Visual Studio integration is out-of-process

If you use Visual Studio 2012 then you probably noticed a new process XDesProc.exe on your machine.outofproc1
outofproc2

This is the new XAML UI Designer. Lets use Spy++ tool on the XAML designer in Visual Studio 2012 and see what’s interesting.outofproc3
outofproc4

If you convert 0x1554 to decimal number you will get 5460 and that’s the XDesProc.exe process id shown on the screen shot above. So, what we see in Visual Studio is actually a window that belongs to XDesProc.exe process. Moving the XAML UI Designer out of Visual Studio has a lot of benefits – better isolation, improved performance and higher resource utilization. The same approach is used by Internet Explorer, Firefox and Chrome. You can read the full story about XDesProc.exe here.

So, why is it important to use out-of-process integration with Visual Studio? The short answer is more memory space. At present Visual Studio is 32bit process and this means that it can use at most 4GB (in most cases 2GB). This is a serious limitation for most profiling tools including JustTrace. For a long time, we wanted to move JustTrace out of Visual Studio and finally we did it. At the same time, JustTrace was built around the idea to provide seamless integration with Visual Studio. I am happy to say that the new JustTrace gives you the best of both worlds. Try it and give us your feedback.

 

Technical Debt

There are a lot of articles explaining what technical dept is, so why another one? A lot of smart people has written about it (see the references at the end of the post). Despite of this, technical debt seems to be a hot topic over and over again and here I put my two cents in.

I like Eric Allman’s article Managing Technical Debt. My favorite quote from it is:

Technical debt is inevitable.

Some may find this statement a bit controversial. I had my own successes and failures in trying to manage the technical debt. My experience shows that one can avoid technical debt for small and simple projects only. However, nowadays we often work on large and complex projects. As a result we start our projects with insufficient understanding about them which in turn naturally leads to acquiring technical debt.

Lets define what technical debt is. Usually technical debt is defined as bad coding practices and “dirty hacks” that patch the software product instead of building it. Most of time the technical debt is attributed to lazy and/or inexperienced software developers. Often the reasons for acquiring technical debt are project specific but the most common are project cost, short deadlines, lack of experienced software engineers and so on.

A lot of managers and software developers are afraid of taking technical debt. I don’t think technical debt is a scary thing in case it is well-managed. The well-managed technical debt could save time and money. Today customers buy features, they are usually not interested in maintaining the source code. Shipping the right set of features on time can be a huge win for everyone.

It is all about the risks and managing technical debt. Unmanaged technical debt can be devastating. It tends to accumulate until you cannot pay it back. Every effort to maintain and/or extend the source code becomes harder. Eventually it slows the project and in the worst case the project is cancelled.

Sometimes acquiring technical debt cannot be observed or predicted. One can take technical debt intentionally or unintentionally. Unintentional technical debt could be dangerous if remains unnoticed for a long time. Intentional technical debt also can be dangerous if the risks taken are high. Martin Fowler provides a practical decision making approach when to take technical debt (see the references).

Common practices that help for better technical debt management are:

  • experienced software developers on the team
  • reasonable ship date/deadline
  • short release cycles
  • automate as many as possible simple-but-tedious tasks
  • spread knowledge across the team/remove one-person-bottlenecks

In closing, I think well-managed technical debt is a good thing. As every debt it allows you to do important things right now and pay the cost later. Consequences from bad technical debt are yet another reason to improve our skills in technical debt managements.

References:

  1. Ward Cunningham, The WyCash Portfolio Management System
  2. Steve McConnell, Technical Debt
  3. Martin Fowler, Technical Debt Quadrant
  4. Eric Allman, Managing Technical Debt
  5. Wikipedia, Technical Dept

 

Profiler types and their overhead

It is a common opinion that profiling tools are slow. Every time I stumble upon this statement I ask for the definition of slow. Most of the time I get the answer that a profiler is slow when it adds more than 100% overhead.

At present there many commercial profilers that are fast (according to the definition above). So, why don’t people use profiling tools then?

I think the confusion comes from the fact that there are different profiler types and some of them are fast while others are slow. Lets see what these profiler types are. It is common to classify profiling tools into two major categories:

  • memory profilers
  • performance profilers

Memory profilers are used when one wants to solve memory related issues like memory leaks, high memory consumption and so on. Performance profilers are used when one wants to solve performance related issues like high CPU usage or concurrency related problems. These categories are not set in stone though. For example too much memory allocation/consumption can cause performance issues.

Lets see why some performance profilers are fast while others are slow. All profilers, and performance profilers in particular, can be classified in yet another two categories:

  • event-based profilers (also called tracing profilers)
  • statistical profilers (also called sampling profilers)

Event-based profilers collect data on events from a well-defined event set. Such event set may contain events for enter/leave function, object allocation, thrown exception and so on. Statistical profilers usually collect data/samples on regular intervals (e.g. take a sample on every 5 milliseconds).

At first, it is not obvious whether event-based/tracing profilers are faster or slower than statistical/sampling ones. So, lets first have a look the the current OOP platforms. For the sake of simplicity we will have a look at the current .NET platform.

Each .NET application makes use of the .NET Base Class Library (BCL). Because of current OOP design principles most frameworks/libraries have a small set of public interfaces and a fair amount of private encapsulated APIs. Lets look at the picture above. As you see your application can call only a small number of public BCL interfaces while they in turn can call much more richer APIs. So, you see only the “tip of the iceberg”. It is a common scenario when a single call to a public BCL interface results in a few dozen private interface calls.

Lets have an application that runs for 10 seconds and examine the following two scenarios.

Scenario 1

The application makes heavy usage of “chatty” interface calls. It is easy to make 1000 method calls per second. In case of event-based/tracing performance profiler you have to process 20000 events (10000 enter function events + 10000 leave function events). In case of statistical/sampling performance profiler (assuming that the profiler collects data every 5 ms) your profiler have to process 2000 events. So, it is relatively safe to conclude that the tracing profiler will be slower than the sampling one. And this is the behavior that we most often see.

Scenario 2

Suppose your application is computation bound and performs a lot of loops and simple math operations. It is even possible that your “main” method calls a single BCL method (e.g. Console.WriteLine) only. In this case your event-based/tracing performance profiler have to process a few events only while the statistical/sampling performance profiler have to process 2000 events again. So, in this scenario is much safe to say that the tracing profiler will be faster than the sampling one.

In reality, statistical/sampling profilers have constant 2-10% overhead. Event-based/tracing profilers often have 300-1000% overhead.

Tracing or Sampling Profiler

The rule of thumb is that you should start with a sampling profiler. If you cannot solve the performance issue then you should go for a tracing profiler. Tracing profilers usually collect much more data that helps to get better understanding about the performance issue.

[Note: If you are not interested in the theoretical explanation you can skip the following two paragraphs.]

If you’ve read carefully the last sentence then you’ve seen that I’ve made the implication that the more data the profiler has collected the easier you are going to solve the performance problem. Well, that’s not entirely true. You don’t really need data. As Richard Hamming said “The purpose of computing is insight, not numbers”. So, we don’t need data but rather “insight”. How do we define “insight” then? Well, the answer comes from relatively young information management and knowledge management. We define data, information, knowledge and wisdom as follows:

  • data: numbers/symbols
  • information: useful data that helps to answer “who”, “what”, “where” and “when” questions; information is usually processed data
  • knowledge: further processed information; it helps to answer “how” questions
  • wisdom: processed and understood knowledge; it helps to answer “why” questions

So, it seems that we are looking for “information”. Here the algorithmic information theory comes to help. This theory is a mixture of Claude Shannon’s information theory and Alan Turing’s theory of computation. Andrey Kolmogorov and more recently Gregory Chaitin had defined quantitative measures of information. Though they followed different approaches an important consequence they made is that the output from any computation cannot contain more information than was the input in first place.

Conclusion

Drawing parallel back to profiling we now understand why sometimes we have to use event-based/tracing profilers. As always, everything comes at a price. Don’t be biased that profiling tools are slow. Use them and make your software better.

It’s all about the version

Have ever you noticed that people are obsessed by versioning the things? Yep, it’s true. Today, everything has a version. Even you. I am not sure why the versioning is so important. I guess it is related to the notion of improving the things.

Here is a short list of what I encounter most often in my job.

  • Web (1.0), Web 2.0, Web 3.0
  • Project Management (PM 1.0), PM 2.0, PM 3.0
  • ALM (1.0), ALM 2.0, ALM 3.0
  • Agile (1.0), Agile 2.0
  • Scrum (1.0), Scrum 2.0
  • PRINCE, PRINCE2
  • UML 1.x, UML 2.x

I can easily add a few dozen more items to the list. The things I selected can be classified as ideas, concepts, processes and standards. I won’t bother showing you software product versioning. However you may be surprised to see that versioning is applied to other things.

  • Science 2.0
  • Health 2.0
  • Business 2.0
  • Enterprise 2.0

Versioning is applied even to food.

  • Milk 2.0
  • Bread 2.0

It’s not that we version the things around us only. We version ourself as well.

  • Übermensch
  • Human 2.0
  • People 2.0

I am still playing with the idea of what would be the next version of me. Or you 😉