Node.js performance vs Hapi, Express, Restify, Koa & More

By Alex | Posted Apr 18, 2017 | 5 min. (1061 words)

In 2015 and 2016, Raygun has tested the Node.js framework against other popular frameworks including Hapi, Express.js, Restify and Koa. This year (2017), we’ve added some more frameworks due to popular request; Sails.js and Adonis.js.

The aim of these performance tests is to help you benchmark popular frameworks so you can see which one best suits your project.

Raygun lets you detect and diagnose errors and performance issues in your codebase with ease

It takes minutes to add Raygun into your software. Be alerted to issues affecting end users and replicate problems 1,000x faster than using logs and incomplete information from users. Learn more and try Raygun free for 14 days.

As always, we’ve broken the results down and compared them to last year. We’ve also included instructions on how to reproduce the test.

Node.js performance tests were performed on the Ubuntu subsystem on Windows 10, along with a VM provisioned from Digital Ocean.

The tests only utilize the most basic capabilities of the frameworks in question, therefore the main goal was to show the relative overhead these frameworks add to the handling of a request. This is not a test of the absolute performance as this will vary greatly depending on the environment and network conditions.

This test also doesn’t cover the utility each framework provides and how this enables complex applications to be built with them.

Here’s a quick recap of what we found in our previous Node.js performance tests in 2015 and 2016:

Node.js performance 2015

In 2015, we tested tested Hapi, ExpressJS, Restify and naked Node with no middleware. The results were clear:

Node.js performance June 2016

This year, we added one addition framework, the results of which you can see below:

Node.js performance December 2016

It turned out that total.js was the fastest of the frameworks tested, being only 15% slower than using the raw node.js http library.

The other frameworks, koa, restify and express all performed similarly and hapi performed the worst. For this update, we introduced a secondary test environment:

Node.js performance 2017: the test

The original reasoning behind these posts was an exploration of Node.js frameworks for use in the development of a lightweight public facing ingestion API for possible use in developing Raygun’s public API.

For this round of testing, as requested by our readers, we added two new frameworks (Sails.js and Adonis.js) to the test. We also used the updated versions of the previously tested frameworks which included a major version bump for Koa to Koa 2.0 and minor updates for some of the other frameworks.

As with the previous tests, I used Apache Bench to fire GET requests at the endpoint for each sample app. We configured the applications to respond with a simple;Hello World!; string. Note that this does;t necessarily correlate with any desired real world behavior but is rather testing the theoretical maximum number of requests which can be handled.

Apache Bench was configured to make 100 requests concurrently until 50,000 requests had completed or 20 seconds had elapsed. The request responses per second were then recorded.

Both Apache Bench and the server were hosted within the same environment removes any confounding factors network stability could introduce.

All tests were repeated five times in each environment. The displayed figures refer to the averaged values produced by the test.

The rough code used in the test is available on Github here. Depending on the environment additional packages or permissions may be required to run the tests.

Software versions tested

Runtime:

Node.js v7.9.0

Frameworks:

Hapi 16.1.1 Express 4.15.2 Restify 4.3.0 Koa 2.0.0 Total.js 2.4.0 Sails 0.12.13 Adonis 3.0.12

Test environments

The Ubuntu subsystem environment running on Windows 10 PC; 32 GB RAM, i7-4790 CPU
A $20 Digital Ocean VM running Ubuntu 16.04 , 2GB Memory, 2 Cores

The Results

Ubuntu Subsystem on Windows 10:

Ubuntu VM (Digital Ocean):

What these results mean for you

The results of this test shouldn’t be taken as an indication of the framework’s merits or flaws as a whole, as this test is focused on a simplistic operation.

If you are going to write a high performance API, we recommend having some idea about the base performance overhead. This will better inform whether or not it is suitable.

What the performance on the different systems mean

As the test was performed on environments which aren’t similar to a professionally hosted service’s environment, the raw numbers here mean little.

The windows subsystem is mainly useful for emulating nix-like functionality within Windows with ease and a $20 VM is very much at the lower end of the scale for any professional webservice. Because of this the main takeaway should be the relative performance of each framework.

Comparing with prior tests

The results of the tests were similar to previous runs. There was very little variation in performance between the VM performance of the last test and the most recent one. This make sense as Node.js has not undergone any major updates since the last test was run. However, we wanted to monitor the performance against other frameworks as they have had minor updates.

New frameworks:

Both of the new frameworks were relatively opinionated and dependency heavy frameworks came with many out-of-the-box features.

That they performed somewhat unfavorably and this micro-benchmark was not surprising. Both were leaning towards the creation of larger applications rather than simple API endpoints. Their performance was surprising good on the Subsystem environment. Having access to a larger amount of memory may have been a factor in both these frameworks performing better than on the relatively resource light Virtual Machine.

How to reproduce the test

These tests can be replicated by cloning this repository and running:

./install.sh
./run.sh

The first script should install all the required tooling to run the test as well as npm and the latest version of Node.js. The second script then runs all the servers and benchmarks in sequence outputting the results to a file. We highly recommended reviewing the scripts to ensure it will not interfere with any other processes on your machine before running it.

Overall

Overall these tests showed that Total.js remains the fastest framework with performance within 25% of a raw Node.js service. The large amount of built-in features might make them unsuitable for use of an API project without significant modifications to their default behavior.

C# performance tips and tricks

Node.js v10.2.1 Documentation