Achieving a 12% performance lift migrating Raygun's API to .NET Core 3.1

Here at Raygun, improving performance is baked into our culture. We don’t just think about our application performance, but more broadly, we look at our own infrastructure and ask if there’s anything we can do to make it more performant for our business and for our customers.

Two years ago, we switched our API from Node.js to .NET Core and achieved a 2000% increase in throughput. To continue that story, we recently upgraded .NET Core 2.1 to 3.1 and saw a 12% increase in performance.

We enjoy presenting our performance findings, so in this post, we’d like to give some context into why we upgraded and the conditions that helped us achieve the 12% performance lift.

Why upgrade from .NET Core 2.1 to 3.1?

There were three reasons behind Raygun making the switch from .NET Core 2.1 to 3.1, all of which may also be relevant to your team.

1. Performance is a feature in the .NET Core team, and they’ve been doing a lot of work to improve the performance of the .NET Core framework. By upgrading to the latest version, you get the benefit of these performance improvements without having to change any of your code. These performance improvements have been documented in great detail on their blog.

2. Long term support (LTS) for 2.1 ends in August 2021 and 3.1 is the new LTS version of .NET Core. To ensure we’re using a supported version of .NET Core and getting the most up to date patches and security fixes, it’s important to keep moving forward with LTS versions.

3. The best way to prepare for .NET 5 is to move all of our .NET Core applications to 3.1. (Microsoft has made it clear that it’s still safe to leave your apps on the .NET Framework, but you should consider using .NET Core 3.1 for all your new applications.)

Raygun’s scenario

Our primary ELB deals with around 25,000 requests per second during peak times. For this, we’re running 12 API nodes with plenty of overhead for spikes in traffic. With this change, we could run one or two fewer servers during this peak period and still be able to handle the same traffic load.

The code changes to support the version change were very minimal and could have been as simple as changing a setting to support our use of reading the request stream synchronously. However, we decided to change our usage of reading the request stream to asynchronous (for good reason as it can cause thread starvation and application hangs. More on this in Microsoft’s blog post.)

Business outcomes

For our API nodes, upgrading to the latest version of .NET Core means we can now handle 12% more volume at no additional costs.

We also saw a significant improvement in the average response time over the load test. This means requests to the API from our customers are faster and allow them to send more data in less time.

How we tested .NET Core performance

Tests were run using a c5.large AWS instance running Ubuntu 18.04 with Nginx acting as a proxy to the Kestrel web server on the same machine.

We used Apache JMeter to post sample Raygun Crash Reporting payloads to the API. JMeter can simulate a very high load with many concurrent requests. We tweaked this to the point where the version running 2.1 started to max out the CPU, but before it got overloaded and was not able to handle all requests (so 100% success rate for the requests).

We then ran the same test against the .NET Core 3.1 version, where the only difference was the installation of the .NET Core 3.1 runtime and the server running the new build of the API application.

JMeter ran multiple 10-minute tests, where it measures each of the requests and generates a summary report at the end of each run. We then averaged the results of the multiple test runs to come up with the final results below.

The performance results of .NET Core 2.1 vs. 3.1

.NET Core 2.1 had an average response time of 1.8ms vs. an average response time of 1.6ms for .NET Core 3.1, representing a 12.5% improvement in response time (a win for our customer.)

.NET Core 2.1 processed 3,609 transactions per second vs. .NET Core 3.1, which processed 4,035 transactions per second, representing a 12% improvement (a win for us.)

Why .NET Core?

The Raygun API has been through a few iterations in its lifetime. It started its life with Mono, then we moved it to Node.js before we rewrote it in .NET Core when V1 was in pre-release.

We’re always excited about the direction Microsoft is taking with .NET Core. With every version, there have been massive performance improvements that you get for free just by upgrading.

We know the better performance we deliver, the happier we make our customers, and the more efficient we can run our infrastructure. (40% of users will stop using a site that takes longer than three seconds to be ready.)

So, making the platform more efficient with better systems means we can deliver more for the same price. Squeezing more out of our infrastructure investment means we can direct that spending into building features our customers need and love.

Closing thoughts

No doubt that now a preview of .NET 5 has been released, you’ll be looking to upgrade to 3.1 soon. In our experience, the performance promises of a faster and leaner experience were fulfilled. Our performance improvements reflected both a win for us in how much data we can process and a faster experience for our customers.

We did a detailed dive into some more complex .NET Core performance problems with .NET performance expert, Matt Warren. You can watch the full webinar here.