Span<T>: Working efficiently with contiguous memory in .NET

.NET developers are often faced with the challenge of working with large arrays and data structures in a memory-efficient way. Especially in writing performance-critical server applications and scalable cloud-based services that are sensitive to memory consumption. To address these scenarios, .NET introduces flagship type into the ecosystem, namely Span<T>, which is used to provide scalable APIs that don’t allocate buffers and avoid unnecessary data copies. This article tries to introduce this .NET feature.

The problem

When it comes to utilizing various types of memory, C# offers us different kinds versatility. However, the vast majority of developers exclusively utilize the managed one. Let’s take a quick look at what C# has to offered us for working with memory:

• Stack memory: Allocated using the stackalloc keyword on the Stack. Allocation and deallocation happen quite quickly. The Stack’s size is small (often less than 1 MB) and well suited for CPU cache. However, attempting to allocate more results in a StackOverflowException that cannot be handled and immediately terminates the entire process. For small and short tasks where managed memory cannot be allocated, stackalloc is frequently utilized.
• Unmanaged memory: Invoking the Marshal.AllocHGlobal or Marshal.AllocCoTaskMem procedures will allocate memory on the unmanaged heap (which is hidden from GC). The developer must explicitly release this memory by calling Marshal.FreeHGlobal or Marshal.FreeCoTaskMem. Utilizing it doesn’t put the GC under any more pressure. The most frequent usage of Unmanaged memory is to avoid GC in situations where you would typically allocate huge arrays of value types without pointers.
• Managed memory: The new operator can be used to allocate managed memory. It is managed since the Garbage Collector (GC) is in charge of managing it. Developers don’t have to worry about it because the GC decides when to free the memory.

The problem is related to when to work with arrays. When it comes to working with arrays and other data collections in C#, developers frequently confront performance and memory management challenges. In particular, copying or allocating extra memory can be a costly operation, especially for large arrays. As an example, consider the following code. An array data structure with 1024 elements and is filled with initial values.

Now suppose that we want to calculate the sum of values between specific continuous elements (e.g., The third quarter section of the desired array). The general solution for this is to create a new array with 250 elements and put the desired values in it and then perform the sum calculations on the new array (code below).

This way has a problem. The problem is the overhead of reading the original array and copying the desired values to the new array or initialze and new object in memory. This overhead problem will have a direct impact on performance. Especially by increasing the number of elements of the main array or increasing the number of references to the main array. The solution that Microsoft has included in C# is to use Span<T>.

What is Span<T>?

Span<T> is a value type in .NET within the System namespace that provides a safe and editable view into any arbitrary contiguous block of memory with no-copy semantics. One of the primary advantages of using Span is that it helps you to avoid making duplicate and unnecessary copies of data. When you pass an array to a function, for example, a new copy of the array is created on the heap. This is inefficient, particularly when working with huge and large arrays. You can pass a reference to the original data rather than a copy when using Span, which can enhance efficiency. You can use Span<T> as an abstraction to uniformly represent arrays, strings, memory allocated on the stack, and unmanaged memory. In some ways, it’s analogous to C# arrays, but with the added ability to create a view of a portion of the array without allocating a new object on the heap or copying the data. This feature is called slicing and types with this feature are known as sliceable types. Span promises type and memory safety with checks to avoid out-of-bounds access, but that type of safety comes with certain usage restrictions enforced by the C# compiler and runtime. The Figure-1 illustrates a schematic view of how Span<T> select the desired blocks from a contiguous segments in memory.

Figure-1: Span<T> provides type-safe access to a adjacent area of memory

NOTE: Span<T> is introduced in C# 7.2. Furthermore, Span<T> is considered as a build-in library in.NET core 2.1 and higher. Meanwhile, in order to use the span capability in the.NET Framework, you must add the System.Memory package from nuget to your code.

Benefits of using Span<T>

• Reduced GC pressure: By minimizing the amount of memory allocation and copying, Span<T> can help lessen the burden on the garbage collector (GC), resulting in better performance and less memory fragmentation.
• Memory safety: Span<T> provides a type-safe and bounds-checked method of accessing memory, assisting in the prevention of frequent memory-related issues such as buffer overflows and null reference exceptions.
• Improved performance: Span<T> can help enhance the efficiency and performance of array and memory manipulation operations by reducing the need to copy or allocate additional memory.
• Lower memory utilization: By avoiding the need to copy or allocate additional memory, Span<T> can help minimize the memory usage of application that operate with big arrays or collections of data.
• Interoperability: Span<T> can be used to operate with unmanaged memory.
• Parallelism: Span<T> can be used to efficiently work with data in parallel.
• Code simplification: Span<T> provides a uniform and straightforward approach to work with arrays and memory, minimizing the amount of boilerplate code required for routine tasks.

Limitations of using Span<T>

• Boxing: Span<T> can’t be boxed or put on the heap.
• Reference-type: Span<T> can’t be fields in a reference type.
• Asynchronous operations: Span<T> can’t be used within asynchronous methods and across await and yield boundaries.
• Generic-type argument: Span<T> can’t be used as a generic type argument.
• Type Objects: Span<T> can’t be assigned to variables of type Object, dynamic or to any interface type.

Important methods and properties of Span<T>

Here is a list of some of the most essential methods and properties accessible in the Span<T> class, along with a brief description:

• Span<T>.IsEmpty: Gets an empty Span<T>.
• Span<T>.Length: Gets the length of the Span<T>.
• Span<T>.Slice(int start): Returns a Span<T> that starts at the specified index and includes all elements to the end of the Span<T>.
• Span<T>.Slice(int start, int length): Returns a Span<T> that starts at the specified index and includes the specified number of elements.
• Span<T>.ToArray(): Copies the elements of the Span<T> to a new array.
• Span<T>.CopyTo(Span<T> destination): Copies the elements of the Span<T> to the specified destination Span<T>.
• Span<T>.SequenceEqual(ReadOnlySpan<T> other): Returns a value indicating whether the elements of the Span<T> are equal to the elements of the specified ReadOnlySpan<T>.
• Span<T>.Fill(T value): Sets all elements in the Span<T> to the specified value.
• Span<T>.IndexOf(T value): Searches for the specified value in the Span<T> and returns the index of the first occurrence, or -1 if the value is not found.
• Span<T>.LastIndexOf(T value): Searches for the specified value in the Span<T> and returns the index of the last occurrence, or -1 if the value is not found.
• Span<T>.GetEnumerator(): Returns an enumerator that iterates through the elements of the Span<T>.
• Span<T>.TryCopyTo(Span<T> destination): Attempts to copy the elements of the Span<T> to the specified destination Span<T>. Returns true if the operation succeeded, or false if the Span<T> is too large to fit in the destination Span<T>.
• Span<T>.TryGet(ref T value): Attempts to get the first element of the Span<T>. Returns true if the operation succeeded, or false if the Span<T> is empty.

Performance benchmark

In this section, the performance of Span<T> against array manipulation will be investigated. BenchmarkDotNet library is used for this purpose. Indeed, BenchmarkDotNet is a lightweight, open-source, powerful .NET library that can transform your methods into benchmarks, track their performance, and share reproducible measurement experiments.

In the code below, we have defined three functions. All three functions perform the same action. All of them add data in the target data structure, as much as the value of the Count variable, and finally return the filled object. The benchmark has been done for two values of 10, 1000, and 10000.

The following code block also shows the entry point of the program, which is the point of calling and starting the benchmark. Write the blow codew in Program.cs file:

Run the benchmark by entering the following command in the .NET CLI or any other way you prefere. Just keep in mind that you need to run this benchmark as Release configuration, not Debug.

dotnet run -c release

After running the benchmark, the following results will be obtained.

Figure-2: The performance of Span<T> against LINQ, and array manipulation

NOTE: It is important to mention that the benchmark results obtained is different from one machine to another (according to the specifications).

As you can see, in the Figure-2, the Span<T> has a better performance and faster than the CopyArray and LINQ expression. For 10, 1000, and 10000 capacity, it works better in the terms of getting memory segments. Also, notice Span<T> has 0 allocation while the other ways have some overhead and this amount will increase as the array capacity increases.

Project Link

Here, you can find the related project on Github.

Conclusion

In C#, the Span<T> type offers a powerful and efficient approach to work with arrays and memory. Span<T> can provide considerable speed gains by avoiding the need to copy or allocate additional memory, especially for large arrays or collections of data. Span<T> can also be used to interoperate with unmanaged memory that expect contiguous blocks of memory. Developers can increase the performance and memory efficiency of their C# programs by employing Span<T>. At the end, a benchmark was conducted between three different methods to calculate the sum of an array for different capacity, and it was found that the Span<T> approach has a better performance in terms of speed and memory allocation.