[Helma-user] Parsing JSON

[Hannes Wallnoefer]

Thanks for the code, Kris. Strangely, I ran the code with 15000
iterations ~10 times without seeing any increase in memory usage. Do I
have to run it more often?

Maybe it has to do with the rhino version or settings. I've run the
test with a current Helma svn snapshot, which has a rhino slightly
newer than the one shipped with 1.6.0, and with a rather fresh rhino
CVS snapshot.

Do you have any special rhino specific settings, such as for rhino.optlevel?

hannes

2007/11/27, Kris Leite <kleite at imcsoftware.com>:
> Hi Hannes,
>
> It appears to me that the Rhino leakage is always occurring.  It has
> been a long on going struggle trying to pin down where the leakage was
> coming from.
>
> I found that whenever I parse a simple JSON string, many char[] and
> Strings are left hanging around in memory that are never recovered in
> GC.  As a result, the server will run out of memory after about 20K
> parses of JSON strings.  (My memory limit was set to 256MB for catching
> memory leaks.)  Once I replaced the Rhino eval with a javascript routine
> that parses the JSON string, then my memory usage was a stable.
>
> As for code, try a simple loop  on the attached JSON string and monitor
> the memory.  For example, start up Helma with JVM max memory usage of
> 256MB and then run the following:
>
> for(var i=1; i<15000; i++) { var o = test.parseJSON(); }
>
> The variable 'test' contains the attached file as a string.  (Note:  I
> tested using Helma Inspector and put the code into the text area for
> running scripts.)  I believe you will notice that memory is used and
> never recovered and the server will run out of memory.
>
> Thanks,
> Kris
>
>
>
>
>
> Hannes Wallnoefer wrote:
> > Hi Kris,
> >
> > On 11/26/07, Kris Leite <kleite at imcsoftware.com> wrote:
> >
> >> In case anybody is parsing a large number of JSON strings and found
> >> memory leakage issues you might consider replacing the Rhino Eval
> >> command with another JSON parser.
> >>
> >
> > Is this something you can reproduce, or does it happen intermittently?
> > Because if it is a problem with Rhino, I'd rather have this fixed.
> >
> >
> >> I found that using the JSON parser in Helma, that uses the Rhino Eval
> >> command, to cause unexplained memory lost when using it to parse some
> >> 75K+ small JSON strings.  This occurred with a single long running task
> >> that using JSON strings for passing data between a Helma server and
> >>
> >
> > Can you send me some test data or code?
> >
> >
> >> another computer.  When I replaced the Rhino Eval with a javascript JSON
> >> parser, my memory lost disappeared.
> >>
> >> I also found a couple changes to Property.java to be useful also:
> >>
> >> 1) Changed the line in 'setStringValue' from:
> >>
> >> value = str:
> >>
> >> to:
> >>
> >> value = (str == null) ? null : new String(str);
> >>
> >> 2) Changed the line 'Property(String propname, Node node)' from:
> >>
> >> this.propname = propname;
> >>
> >> to:
> >>
> >> this.propname = (propname == null) ? null : new String(propname);
> >>
> >
> > That would be strange, but it would be possible if these strings are
> > somehow used in weak references. I'll look into this.
> >
> > hannes
> >
> >
> >> Your Mileage May Vary,
> >> Kris
> >>
> >> _______________________________________________
> >> Helma-user mailing list
> >> Helma-user at helma.org
> >> http://helma.org/mailman/listinfo/helma-user
> >>
> >>
> > _______________________________________________
> > Helma-user mailing list
> > Helma-user at helma.org
> > http://helma.org/mailman/listinfo/helma-user
> >
> >
>
> {"merchcommission":"","merchcolor":"","merchsearch":"BINOCULARS 8-20 POWER 50MM","merchlistdisc":"","merchprice":"111000","merchwarrantable":"Y","merchchange":"","lastmodified":{},"parent":"8fab6be0-e4be-4183-8912-792c57d6c196","merchtype":"","merchvendordesc":"","name":"12100027","merchstyle":"T1-.5-G.SI2","applyon":"","merchgroupprice":"","merchmsgcode":"SCS","merchsellinglimit":"","applyid":"P56IC590","merchringsize":"","merchcentercwt":"","merchdesc3":"rubber coated zoom by Tasco","merchdesc2":"8-20 Power x 50mm Binoculars","merchserials":"","merchpurity":"","merchdesc1":"Tasco 8-20x50mm Binoculars","merchgroupcode":"","merchalt":"","merchlistprice":"","merchaltacq":"001","merchvendor":"37775","merchsellingprice":"","merchcertificate":"N","merchcerts":"","status":"A","merchsellingtype":"R","merchsubtype":"","merchgroupdisc":"","applyto":"merchandise","merchsellingcode":"S","merchinstock":"Y","merchserial":"N","merchcode10":"","merchsku":"121-0002-7   - -","merchclass":"119","merchplu":"12100027","applydelete":"","merchclarity":"","json":"","merchpricedate":"2003/07/24","merchcode9":"","merchlastpurdate":"","merchcode8":"","merchdept":"","merchforsale":"Y","merchcode7":"","merchcode6":"","merchcode5":"SCS","merchcode4":"","merchcode3":"","merchcode2":"","merchmetal":"14Y","merchcode1":"","merchprompt":"","merchsellingdisc":"","merchtotalcwt":"","merchlastactdate":"2009/07/24","merchreturnable":"Y"}
> _______________________________________________
> Helma-user mailing list
> Helma-user at helma.org
> http://helma.org/mailman/listinfo/helma-user
>
>