[Helma-user] Parsing JSON

[Kris Leite]

Hi Hannes,

>Maybe it has to do with the rhino version or settings.

Wow, I didn't think of that.  I am using the following settings for Rhino:

rhino.strictvars = true
rhino.debug = true

Since I didn't think of changing these settings I took them out and got 
the same
results as you did, no leakage at all.  So I activated the strictvars 
option and
still no leakage.  When I activated the rhino.debug option, that is when 
I got the
leakage. 

It appears that setting rhino.debug to true is causing the leakage in 
the eval function.

Thank you for looking into this.  As a workaround, I am checking to see 
if the
Rhino.debug option is set.  If true then use my javascript parser, if 
not, then
use the standard Helma parseJSON routine.

Thanks,
Kris


Hannes Wallnoefer wrote:
> Thanks for the code, Kris. Strangely, I ran the code with 15000
> iterations ~10 times without seeing any increase in memory usage. Do I
> have to run it more often?
>
> Maybe it has to do with the rhino version or settings. I've run the
> test with a current Helma svn snapshot, which has a rhino slightly
> newer than the one shipped with 1.6.0, and with a rather fresh rhino
> CVS snapshot.
>
> Do you have any special rhino specific settings, such as for rhino.optlevel?
>
> hannes
>
> 2007/11/27, Kris Leite <kleite at imcsoftware.com>:
>   
>> Hi Hannes,
>>
>> It appears to me that the Rhino leakage is always occurring.  It has
>> been a long on going struggle trying to pin down where the leakage was
>> coming from.
>>
>> I found that whenever I parse a simple JSON string, many char[] and
>> Strings are left hanging around in memory that are never recovered in
>> GC.  As a result, the server will run out of memory after about 20K
>> parses of JSON strings.  (My memory limit was set to 256MB for catching
>> memory leaks.)  Once I replaced the Rhino eval with a javascript routine
>> that parses the JSON string, then my memory usage was a stable.
>>
>> As for code, try a simple loop  on the attached JSON string and monitor
>> the memory.  For example, start up Helma with JVM max memory usage of
>> 256MB and then run the following:
>>
>> for(var i=1; i<15000; i++) { var o = test.parseJSON(); }
>>
>> The variable 'test' contains the attached file as a string.  (Note:  I
>> tested using Helma Inspector and put the code into the text area for
>> running scripts.)  I believe you will notice that memory is used and
>> never recovered and the server will run out of memory.
>>
>> Thanks,
>> Kris
>>
>>
>>
>>
>>
>> Hannes Wallnoefer wrote:
>>     
>>> Hi Kris,
>>>
>>> On 11/26/07, Kris Leite <kleite at imcsoftware.com> wrote:
>>>
>>>       
>>>> In case anybody is parsing a large number of JSON strings and found
>>>> memory leakage issues you might consider replacing the Rhino Eval
>>>> command with another JSON parser.
>>>>
>>>>         
>>> Is this something you can reproduce, or does it happen intermittently?
>>> Because if it is a problem with Rhino, I'd rather have this fixed.
>>>
>>>
>>>       
>>>> I found that using the JSON parser in Helma, that uses the Rhino Eval
>>>> command, to cause unexplained memory lost when using it to parse some
>>>> 75K+ small JSON strings.  This occurred with a single long running task
>>>> that using JSON strings for passing data between a Helma server and
>>>>
>>>>         
>>> Can you send me some test data or code?
>>>
>>>
>>>       
>>>> another computer.  When I replaced the Rhino Eval with a javascript JSON
>>>> parser, my memory lost disappeared.
>>>>
>>>> I also found a couple changes to Property.java to be useful also:
>>>>
>>>> 1) Changed the line in 'setStringValue' from:
>>>>
>>>> value = str:
>>>>
>>>> to:
>>>>
>>>> value = (str == null) ? null : new String(str);
>>>>
>>>> 2) Changed the line 'Property(String propname, Node node)' from:
>>>>
>>>> this.propname = propname;
>>>>
>>>> to:
>>>>
>>>> this.propname = (propname == null) ? null : new String(propname);
>>>>
>>>>         
>>> That would be strange, but it would be possible if these strings are
>>> somehow used in weak references. I'll look into this.
>>>
>>> hannes
>>>
>>>
>>>       
>>>> Your Mileage May Vary,
>>>> Kris
>>>>
>>>> _______________________________________________
>>>> Helma-user mailing list
>>>> Helma-user at helma.org
>>>> http://helma.org/mailman/listinfo/helma-user
>>>>
>>>>
>>>>         
>>> _______________________________________________
>>> Helma-user mailing list
>>> Helma-user at helma.org
>>> http://helma.org/mailman/listinfo/helma-user
>>>
>>>
>>>       
>> {"merchcommission":"","merchcolor":"","merchsearch":"BINOCULARS 8-20 POWER 50MM","merchlistdisc":"","merchprice":"111000","merchwarrantable":"Y","merchchange":"","lastmodified":{},"parent":"8fab6be0-e4be-4183-8912-792c57d6c196","merchtype":"","merchvendordesc":"","name":"12100027","merchstyle":"T1-.5-G.SI2","applyon":"","merchgroupprice":"","merchmsgcode":"SCS","merchsellinglimit":"","applyid":"P56IC590","merchringsize":"","merchcentercwt":"","merchdesc3":"rubber coated zoom by Tasco","merchdesc2":"8-20 Power x 50mm Binoculars","merchserials":"","merchpurity":"","merchdesc1":"Tasco 8-20x50mm Binoculars","merchgroupcode":"","merchalt":"","merchlistprice":"","merchaltacq":"001","merchvendor":"37775","merchsellingprice":"","merchcertificate":"N","merchcerts":"","status":"A","merchsellingtype":"R","merchsubtype":"","merchgroupdisc":"","applyto":"merchandise","merchsellingcode":"S","merchinstock":"Y","merchserial":"N","merchcode10":"","merchsku":"121-0002-7   - -","merchclass":"
>>     
>  119","merchplu":"12100027","applydelete":"","merchclarity":"","json":"","merchpricedate":"2003/07/24","merchcode9":"","merchlastpurdate":"","merchcode8":"","merchdept":"","merchforsale":"Y","merchcode7":"","merchcode6":"","merchcode5":"SCS","merchcode4":"","merchcode3":"","merchcode2":"","merchmetal":"14Y","merchcode1":"","merchprompt":"","merchsellingdisc":"","merchtotalcwt":"","merchlastactdate":"2009/07/24","merchreturnable":"Y"}
>   
>> _______________________________________________
>> Helma-user mailing list
>> Helma-user at helma.org
>> http://helma.org/mailman/listinfo/helma-user
>>
>>
>>     
> _______________________________________________
> Helma-user mailing list
> Helma-user at helma.org
> http://helma.org/mailman/listinfo/helma-user
>
>