[Helma-user] file read and processing pretty slow. ideas?

[Joshua Paine]

I upgraded to jdk 6u2. With the same java options (only the memory stuff 
set) it works faster. With the -server option (which now works) it's 
faster still, and my most common runtime (for the whole code) is down to 
about 1.8 sec. But each page load still makes memory usage jump 10-15MB 
until it levels off at something above 90MB, where it doesn't change at 
all. But I think there's something wrong with your reading on this one, 
or else your GC is just much more aggressive, b/c PHP also takes about 
15MB memory to run the equivalent, and it doesn't seem out of place that 
the VM would wait until somewhat later to GC the memory (apparently when 
it reaches some magic memory consumption number of its own).

> I can't seem to reproduce your results. The very first time I ran your
> code it took about 2.4sec to do the file read, but after that it went
> down to 0.01 sec and then 0.000 every time after that, going back to
> 0.01 after a complete restart of helma (not just the app).
> 
> That's pretty much what I would expect - the first time round the OS
> (winXP in this case) would be caching the file (400k easily fits in its
> in-mmeory filecache) and after that its pretty much instant access.

That makes sense to me, too, but that feature doesn't seem to be working 
on my system. My initial file read under the new java settings was 0.735 
sec. After that it varied a bit, dropping as low as 0.203 with 0.219 
being the most common.

> So I'm not sure why you would keep getting such slow reads of the file -
> could you let us know what class of machine you are running on? (mine is
> just a bog standard, fairly old P4 3.2Ghz with a very slow HDD)

Pentium D, first-gen lowest-end dual core. 2 identical 160GB SATA HDDs 
configured in RAID 1. 1.5GB of RAM which is never full, presumably 
leaving plenty of space for caching the file.

> If you can, try downloading the latest JRE and seeing if it makes a
> difference as you memory usage whould not be spiking like that per each
> request.

The latest JRE/JDK with the -server option has tremendously improved the 
runtime. Initial run with all the processing was 2.84s, then 2.27, then 
1.84, then various around there with one spike up to 2.05 and one drop 
to just under 1.8. It seems like this jre could actually be doing some 
intelligent compiling stuff that the earlier consumer jre did not. In 
any case it cut a solid 35% off the run time.

Still no clue why my file reading is so slow, though.