[Helma-user] Parsing JSON

[Kris Leite]

Hi Hannes,

It appears to me that the Rhino leakage is always occurring.  It has 
been a long on going struggle trying to pin down where the leakage was 
coming from.

I found that whenever I parse a simple JSON string, many char[] and 
Strings are left hanging around in memory that are never recovered in 
GC.  As a result, the server will run out of memory after about 20K 
parses of JSON strings.  (My memory limit was set to 256MB for catching 
memory leaks.)  Once I replaced the Rhino eval with a javascript routine 
that parses the JSON string, then my memory usage was a stable.

As for code, try a simple loop  on the attached JSON string and monitor 
the memory.  For example, start up Helma with JVM max memory usage of 
256MB and then run the following:

for(var i=1; i<15000; i++) { var o = test.parseJSON(); }

The variable 'test' contains the attached file as a string.  (Note:  I 
tested using Helma Inspector and put the code into the text area for 
running scripts.)  I believe you will notice that memory is used and 
never recovered and the server will run out of memory.

Thanks,
Kris





Hannes Wallnoefer wrote:
> Hi Kris,
>
> On 11/26/07, Kris Leite <kleite at imcsoftware.com> wrote:
>   
>> In case anybody is parsing a large number of JSON strings and found
>> memory leakage issues you might consider replacing the Rhino Eval
>> command with another JSON parser.
>>     
>
> Is this something you can reproduce, or does it happen intermittently?
> Because if it is a problem with Rhino, I'd rather have this fixed.
>
>   
>> I found that using the JSON parser in Helma, that uses the Rhino Eval
>> command, to cause unexplained memory lost when using it to parse some
>> 75K+ small JSON strings.  This occurred with a single long running task
>> that using JSON strings for passing data between a Helma server and
>>     
>
> Can you send me some test data or code?
>
>   
>> another computer.  When I replaced the Rhino Eval with a javascript JSON
>> parser, my memory lost disappeared.
>>
>> I also found a couple changes to Property.java to be useful also:
>>
>> 1) Changed the line in 'setStringValue' from:
>>
>> value = str:
>>
>> to:
>>
>> value = (str == null) ? null : new String(str);
>>
>> 2) Changed the line 'Property(String propname, Node node)' from:
>>
>> this.propname = propname;
>>
>> to:
>>
>> this.propname = (propname == null) ? null : new String(propname);
>>     
>
> That would be strange, but it would be possible if these strings are
> somehow used in weak references. I'll look into this.
>
> hannes
>
>   
>> Your Mileage May Vary,
>> Kris
>>
>> _______________________________________________
>> Helma-user mailing list
>> Helma-user at helma.org
>> http://helma.org/mailman/listinfo/helma-user
>>
>>     
> _______________________________________________
> Helma-user mailing list
> Helma-user at helma.org
> http://helma.org/mailman/listinfo/helma-user
>
>   
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: json.txt
Url: http://helma.org/pipermail/helma-user/attachments/20071127/2d763515/attachment.txt