Clearly, even just a second's worth of caching can have a big impact on performance for a high-volume application. It's easy to see why this is the case if we look at the database's performance during this test. I ran the test with two performance counters:
SQLServer:SQL Statistics\Batch Requests/sec
ASP.NET Apps v1.1.4322\Requests/Sec\myapplication
In the non-caching test, the SQL Server requests/sec counter averaged 56.2 requests per second. In the micro-caching test, it averaged 1.0 request per second. Obviously, the database was not having to do nearly as much work. However, not all of the savings were on the database side. Loading data into a DataSet is relatively expensive as well, as is data binding. Although there aren't any performance counters for these activities specifically, we know they were occurring once per request in the first case, and once per second in the latter case, so we dropped a lot of work from the ASP.NET engine's plate as well.
Typically, the database and the web server will reside on separate boxes. The reduction in database load would clearly benefit the database server, which is a very good thing since, historically, database servers are much more difficult to scale out than web servers. In addition, it will reduce network traffic between the web and database servers, which will further enhance performance, and the reduction in web server load is also helpful, of course, since it reduces how many web servers would be needed (or how powerful each one would need to be).
Summary
Caching is a very useful tool to take advantage of when designing applications. It is not a panacea, and like any tool, it can be used improperly. But even in situations where data needs to be up to the second, adding one second's worth of caching can make a big impact on application performance. Remember to ask your users how much they can tolerate out-of-date data and try to get them to at least sign off on one-second-old data if you can, since that will allow you to take advantage of techniques like the one described here.