Using the Decorator Pattern to Auto-Instrument .Net Classes With Otel Tracing

.NET has definitely embraced OpenTelemetry in its recent releases. Its support for defining traces and spans using the built-in System.Diagnostics classes greatly simplifies the task of instrumenting code. My only nitpicky complaint about the .NET implementation is the terminology confusion it manages to create by diverging from the standard OTEL naming.

For example, an OpenTelemetry Span is called an ‘Activity’ in .NET, while ‘ActivitySource’ is what’s usually referred to as a ‘Tracer.” OpenTelemetry can be confusing enough without having to translate back and forth between language-specific naming for basic entities.

I recently had a chance to work on some sample projects when something else caught my attention: I noticed how I’m repeatedly copy/pasting the boilerplate setup code for spans/activities. It actually requires discipline to make sure to include these declarations in key areas of the code.

To compound the problem, I always seemed to miss the most critical sections — perhaps because my mind was occupied with actual code decisions when working on them so the copy-paste ceremony was forgotten.

Below you can see what the required boilerplate for Span/Activity looks like:

private async Task<Account> RetrieveAccount(long id)
{
    //boilerplate code that really just repeats that can be 
    //naturally documented with function naming
    using (var activity = Activity.StartActivity("Retrieving account", ActivityKind.Internal))
    {
        return await moneyVault.Accounts.FindAsync(id);

    }

}

Naming spans can also be an annoying pass-time. Maintaining consistent span naming and granularity often feels like an exercise in taxonomy. For the sake of simplification, I often would actually just set the span name to be identical to the enclosing method name, if I managed to make it clear and descriptive enough.

The decorator pattern can help remedy some of these issues and provides an easy and elegant way to wrap operations with OTEL activities. Furthermore, by separating the logic related to creating spans from the application domain classes, we can more easily change, manage and standardize the cross-application aspects of tracing.

Decorators, of course, also have their overheads and limitations, I would definitely not recommend using a decorator with low-level classes or with extremely short and performance-sensitive operations.

In this post, we’ll walk through the steps of creating a Decorator to handle creating Activities or OTEL Spans for different service methods in a sample ASP.NET Core MVC application. If you just want to use this Decorator in your code and don’t much care about the process, it has also been added to the Digma OpenTelemetry helper Nuget Package and maintained in this repo.

Creating a Basic Tracing Decorator With DispatchProxy

The .NET DispatchProxy class, allows us to create a generic wrapper for any interface by creating a proxy object implementing that interface. It is commonly used to separate out specific concerns that are orthogonal to the code being evoked such as logging, caching, etc.

We’ll be creating a subclass of DispatchProxy which we’ll name TracingDecorator. Similar to the base class, we’ll use a static constructor that accepts the object to decorate and instantiates our proxy object:

public class TraceDecorator<TDecorated> : DispatchProxy
{
    
    private ActivitySource _activity;
    private TDecorated _decorated;

    public static TDecorated Create(TDecorated decorated)
    {
        object proxy = Create<TDecorated, TraceDecorator<TDecorated>>()!;
        ((TraceDecorator<TDecorated>)proxy!).SetParameters(decorated);

        return (TDecorated)proxy;
    }
    
    private void SetParameters(TDecorated decorated)
    {
        _decorated = decorated;
        _activity = new(_decorated!.GetType().FullName!);
    }

Notice that as a part of the initial object configuration we set up some basic object parameters including a reference to the decorated object, and the ActivitySource object we’ll use to create new Activity instances.

Finally, we need to override the DispatchProxy Invoke method that gets called whenever one of the interface methods is executed on the proxy object. Our implementation will delegate the call to the decorated object, and wrap the entire operation with an Activity.

protected override object? Invoke(MethodInfo? targetMethod, object?[]? args)
{
    using var activity = _activity.StartActivity($"{_decorated.GetType().FullName}.{targetMethod.Name}");
    try
    {
        var result = targetMethod.Invoke(_decorated, args);
        return result;
    }
    catch (Exception e)
    {
        activity.RecordException(e);
        throw;

    }
}

We can use any naming convention for the activity we created. In this case, we’ll just create a fully qualified name using the decorated object type and the method name (we’ll improve on that later). Finally, we make sure to record any exception as an Activity event before rethrowing it so as not to interfere with the normal handling flow.

Using the Decorator

Now that we have an initial version more or less ready, we can test it out by wrapping an application class. For this example, I used a sample application for OTEL I had at hand. I chose to decorate one of the domain services as that seemed like a safe enough use case for a decorator. As a high level construct it is not as sensitive to minor performance degradations, which are probably also overshadowed by the many different layers and abstractions that the request already has to go through.

The initialization is pretty straightforward using the static constructor we defined above:

IMoneyTransferDomainService decoratedInstance = TraceDecorator<IMoneyTransferDomainService>.Create(domainServiceInstance)

If you’re using the .NET DI framework, however, it can be much easier to simply override the registered object with our new decorated proxy. There are multiple ways to implement that change. The simplest one I found was to use the excellent Scrutor Nuget package. Scrutor provides a handy extension method called Decorate that does just what we need. This is what the code looks like:

builder.Services.Decorate<IMoneyTransferDomainService>((decorated) =>
    TraceDecorator<IMoneyTransferDomainService>.Create(decorated));

Basically, we replace the current mapping in the ServiceCollection so that it will use our decorated instance. We pass a factory lambda that accepts the original undecorated object (after it has been initialized and its dependencies already injected) with the TracingDecorator proxy for the same interface.

If we re-run the application now and remove the boilerplate code for tracing from one of the methods, we’ll be able to see how it is still automatically instrumented by our TracingDecorator class:

Improving Our Initial Implementation: Recording Exceptions, Supporting Custom Span Names, and More

To make our decorator less rigid we’ll add a few attributes that will allow us to customize some aspects of its behavior. For example, the user might wish to use a different Activity name, or define the exception recording behavior explicitly.

To start, we’ll define an attribute that contains the information we need:

[AttributeUsage(AttributeTargets.Method, Inherited = false)]
public class TraceActivityAttribute : Attribute
{
    public string? Name { get; }
    public bool RecordExceptions { get; }
    
    public TraceActivityAttribute(string? name=null, bool recordExceptions=true)
    {
        Name = name;
        RecordExceptions = recordExceptions;
    }
}

Next, we can change the decorator implementation to retrieve the attribute values and use them:

protected override object? Invoke(MethodInfo? targetMethod, object?[]? args)
{
    using var activity = _activity.StartActivity($"{_decorated.GetType().FullName}.{targetMethod.Name}");
    var activityAttribute = targetMethod.GetCustomAttribute<TraceActivityAttribute>(false);
    using var activity = _activity.StartActivity(activityAttribute?.Name ?? defaultSpanName);
    if (activityAttribute?.RecordExceptions==false)
    {
          return InvokeDecoratedExecution(targetMethod, args);
    }

Now we can easily define how we want our spans to behave by setting the right attribute values on the decorated class:

A much cleaner implementation, removing the unneeded repetition and better handling of the trace concern in a standardized manner.

Further Improvements

Working on this example I found myself tempted to add additional useful features to the TracingDecorator, including the following:

• The ability to define whether to automatically instrument all interface methods or just those marked with the TraceActivity attribute.
• The ability to set span attributes on the interface globally, and have them be injected into every span automatically for that interface.

• Define an ISpanNamingSchema interface that allows setting up the decorator with a standard naming schema that may be different from the default one we’ve implemented above.

What Else Would You Add?

You can find the full source code in the Digma OpenTelemetry repository. Let me if this is useful in your projects! Also, if there are any features you feel would be useful, feel free to reach out or open an issue or a PR on Github.

If you’re interested in OTEL and observability, I’ve written some other posts on the topic, specifically about how to leverage OTEL in dev that you can find here and here.

Want to Connect? You can reach me, Roni Dover, on Twitter at @doppleware.Follow my open source project for continuous feedback at https://github.com/digma-ai/digma.