posted on Wednesday, June 11, 2003 4:59 AM
by
DougSeven
Server Out of Control
This evening I spent most of my time (and some of Russ's time) trying to isolate the reason why the DNJ server was performing so poorly. It seems that something was causing the aspnet_wp.exe processes (2 of them, one for each processor) to restart every 3-5 mintues. Each one kept growing in memory until they reached 75% of available memory, then BLAM! process restart.
After a bit of poking around, chating and perf monitoring I decided it was directly related to the ASP.NET Cache growing out of control. Further investigation showed that Output Cache was under control, but the Cache API entries would grow by 100 entries every 5-8 seconds until they reached around 5,000-6,000 and then the processes would restart.
In the hijacked forums code we use (the ASP.NET Forums) we had to alter some code to integrate with our existing security (authentication/authorization) model. In doing this we were putting the forum name for the users in cache. I decided to do some code reworking to see if this was the problem. After altering 7 or 8 classes in the forums codebase and recompiling we saw a perf increase. We still have an issue, but now the processes are restarting every 30 minutes instead of every 3 minutes.
Donny and I are going to run some tools on our dev machines over the next week or so to try and isolate the problems. I'll keep ya posted.