Caching frequently changing search results

I am maintaining a intranet web application that allows users to Search/Add/Edit items and run reports.

When the user searches for items or runs a report, the items/report-records were stored into Session so that I didn't have to recreate the data source for functions like paging, sorting, editing etc.

Whenever a user would open a new tab, I'd check to see if there were search results or a report in Session and if so I would warn the user about the possibility of messing things up in the other tab.

A new requirement has been added to the application that states that users should be allowed to work in multiple tabs.

I've modified the application so that it no longer stores the the search results and reports in Session. Instead it caches the search criteria used to generate the search results/report in ViewState and uses that to recreate the datasource each postback.

I found that when the user's search resulted in 4000+ items it took 13 seconds to run the search. That means that after the user has done the search...and if they caused a postback they'd have to wait another 13 seconds (this was particularity bad when the user does something small like sorting/paging etc).

I wasn't happy with this in the slightest but eventually I was told to leave it as it is because the user is expected to refine their search results to just a few items...in which case it took less than second to return.

I'm working on the reports right now.
Reports can be Very large (well over 4000 records) and I can see this lag-time being a problem.

I could continue to use Session to "cache" the search results and the reports...and create unique IDs for each tab.

The thing is that I'm worried about Session growing too large as multiple users may be connecting with multiple tabs, running search and reports in each tab. Session could potentially grow too large and cause the application to be recycled.

I'm considering using ASP.NET's "Cache" but from what I've been reading it's not really meant to be used with data that is frequently changing like this...it's really meant to be used in situations where the data remains constant for long periods of time and doesn't really change much.

I am looking for any input on this how I should cache these results (I'm even thinking of using an XML file server side...since I don't have access to a database)

Thanks a lot

-Frinny

how much RAM is on the server?  Is the server used for other resource intensive sites or processes?

This application is distributed to customers who purchase it.

I can't remember the exact system requirements that we've stated are required for the web server to have in order to run the web application but I know it's a minimum of 2GB of RAM (because that's what my DEV  machine has...)

Every server that the software is installed on is going to have a different amount of RAM though.

Sorry I didn't answer your question completely.

Other than the 2 GB RAM, the number of concurrent users is probably going not going to be very many (this is an intranet application).

This is probably going to be limited more than usually because we have a licensing package that, to begin with, allows 5 users to be "connected" to the web application at the same time. They can purchase licenses to allow more users to connect (packages include 1 license, 10 licenses). However, the likelihood that  the number of concurrent users connected at the same time is higher than 5 is very low and it would be up to the customer's IT staff to configure IIS to allow for more connections.

The servers may be used for other websites...depending on what other websites/applications are currently being served on the customer's web server

Using the ASP.NET cache is fine.  Session is faster serializing and deserializing the data though.

I think using the cache with a 2-3 minute sliding expiration, is probably the best option off the top of my head.  This would make sure that you are not leaving lots of data in memory for extended periods of time while still addressing your needs for postbacks sorting data etc...

Even at 4000+ records for larger reports, with 5-10 users you still won't come close having issues with memory.  I'd double check my queries and make 100% sure you need each and every column though.

Thank you very much for your time!

I just have one more quick question...does Cache, like Session, also cause the application to recycle if it grows to big. I haven't really been able to find a lot of information on this type of thing...I've mainly been finding articles on how to use Cache.

Thanks again!

-Frinny

For your application pool, you'd want to set your memory limit up above normal.

Thanks again for your help Robbe