Improving ASP Performance With Asynchronous Coding

Studies have shown that the average internet user will abandon a page that takes more than four seconds to load.

If you are using ASP.NET, and you aren't using asynchronous programming, then you may have a problem.

Success may literally kill your site.

How I Bumbled Into The Problem

I discovered this problem a couple of years ago with a web app I was load testing.

Quick summary of what I found:

Two copies of same app were running on the same server. I was load testing one.

As I ramped up beyond about 300-350 concurrent users bad things happened.

The time it took to return a page skyrocketed up to 30 seconds and beyond!

CPU and Memory on the server were barely being used.

I logged in to the second app and guess what? It was responding as quickly as ever.

Hmm.

Dirty Secret

As I researched this issue I came across an MSDN article from 2007 by Jeff Prosise called Scalable Apps with Asynchronous Programming in ASP.NET.

Jeff is one of those “Card-Carrying Genius” types. He has written probably more books than I've read and was one of the founders of Wintellect. I've known several folks who worked for Wintellect, and they have all been damned sharp.

Here's the opening of his article:

Do you want to know a secret? A deep, dark, dirty secret? One that, if revealed, would cause great angst in the ASP.NET community and prompt shouts of “Aha!” from the anti-Microsoft crowd?

Most Web sites I've seen built with ASP.NET aren't very scalable, not because of a flaw in ASP.NET, but because of how the technology is used. They suffer a self-imposed glass ceiling that limits the number of requests they can process per second. These sites scale just fine until traffic rises to the level of this invisible ceiling. Then throughput begins to degrade. Soon after, requests start to fail, usually returning “Server unavailable” errors.

– Jeff Prosise

Why?

In a word: Threads. In two words: Thread Pool. Three words? Thread Pool Saturation.

Each person who connects to your app gets a thread from the .Net CLR Thread Pool.

As long as you don't have more people than threads, everything's great!

Unfortunately once you get more people than threads, new users go into a queue to wait for the next available thread.

Browsers waiting for connections lead to frustrated users leaving the page.

The real fun happens when they leave, and then a thread is assigned to them and waits for them to respond.

The thread doesn't know they left, so it waits until it reaches its timeout period. That means even though they left, they still ate up a thread.

Going back to my story of the two identical apps, the reason the one not being load tested kept working was because it had its own thread pool. The thread pool is at the CLR instance level. In ASP terms that means Application or App Pool level.

How Bad Is It Really?

Bad. Really bad.

I've created a super simple MVC app to illustrate.

To keep it simple I just did ye olde File: New Project and accepted the default MVC app.

The Home controller has three actions:

1. Index
2. About
3. Contact

These do nothing but return their view.

In the real world, they would probably look something up in the database, so I put in a random delay to simulate that.

So if you hit one of the pages, the app waits between .5 and 1.5 second before responding.

Here's a bit of the code:

Random randy = new Random();

public ActionResult Contact()
{
    ViewBag.Message = "Your contact page.";
    Thread.Sleep(GetSleepTime());
    return View();
}

protected int GetSleepTime()
{
    return randy.Next(500, 1500);
}

Next I used the Load Testing built into Visual Studio Ultimate to see how it performed.

I started with 200 users and added 200 every 10 seconds until I reached 2,000.

I held at 2,000 and ran for a total of five minutes.

Here is a chart of the results:

Random randy = new Random();

public async Task<ActionResult> Contact()
{
    ViewBag.Message = "Your contact page.";
    await Task.Delay(GetSleepTime());
    return View();
}

protected int GetSleepTime()
{
    return randy.Next(500, 1500);
}

I think you'll agree that's quite an improvement.

Yes we had a few that took too long, but the average time to load dropped by an order of magnitude.

Our proverbial average user wouldn't abandon our site just because of the time to load a page.

As you can see there are a couple of gaps in the chart.

Unfortunately my notebook tends to have I/O log jams and the test was a bit too much for it.

I could move the test up to Visual Studio Online, but it would exceed my monthly allotment of testing “virtual user minutes.” Since these results demonstrate the point I'm making, I don't think I'll buy extra minutes to perfect the test.

Quick Tip:

My first test was with 10,000 users and my notebook shut itself down because the CPU was overheating!

Fortunately 2,000 users sufficiently demonstrates the point.

Caveats

You knew there had to be a catch, right? It couldn't be this easy.

No, it's not quite this easy.

In the real world you will be getting stuff from databases, file systems, and web APIs.

If any of those do something synchronously, you will lose the ability to relinquish the thread back to the pool.

Entity Framework has been an offender in this area, but from EF6 onwards there are asynchronous methods you can use instead of synchronous ones.

I haven't worked with asynchronous EF yet, so I can't say much about it.

Conclusion

If you are creating anything other than a departmental app, you should be coding asynchronously.

Sure you can throw more hardware at the problem, but what a waste!

You could spin up the biggest, baddest, fire-breathing monster of a server and you would still be limited by the number of threads in the thread pool.

You can increase the number of connections in the thread pool (and you very well might anyway), or spread the load across multiple web servers (again, you will probably do so), but the biggest bang for the buck comes with asynchronous code.

Asynchronous Resources

Hopefully I've piqued your interest in asynchronous ASP. Hopefully you'll starting doing it ASAP.

To help you learn more check out these links:

• Scalable Apps with Asynchronous Programming in ASP.NET. Prosise's article. Defines the problem, but his solutions are circa 2007.

• Asynchronous Programming with Async and Await Good intro to asynchronous coding with .Net.

• Nodejs.org It wouldn't hurt to spend some time with Node to see how an asynchronous or die philosophy works. If nothing else it will make you appreciate how much less complicated .Net now makes asynchronous code.

• Intro To Node With Microsoft Tools Presentation A presentation I did about using Node.js with Microsoft tools. I talked about this issue as well.

• Async Query & Save (EF6 onwards)

• Scaling a standard Azure website to 380k queries per minute of 163M records with loader.io Excellent post by Troy Hunt.

• Tasks, IDisposables and async/await Great post from Mark Rendle about not botching your code by trying to pass along an already disposed task. He makes a great point about the incremental time added by async/await and when it is worth the wait. Or await.