Saturday, July 10, 2010

More JavaScript performance data

After yesterday's exercise comparing the execution speed of Rhino-standalone versus Rhino-embedded-in-JRE, I decided to carry out a couple more performance investigations.

I was curious to find out, first of all, how much slower Rhino (a pure-Java implementation of JavaScript) would be than, say, Spidermonkey (the C++ implementation in Firefox). I repeated the "numerical efficiency" test from yesterday (Euler's expansion series for pi) and got the following results:


Execution time (not speed, but time; higher values are worse) is plotted vertically. The surprising result? Firefox's C++ interpreter is not all that much faster than standalone Rhino. The Y-axis here goes from zero to 1200 milliseconds. Trials, on the X-axis, run from zero to values of 200000, 400000, 600000, 800000, and 1000000 for the terms argument to yesterday's pi() function. So in other words, it took Rhino 1212 milliseconds to calculate pi from a million terms, whereas it took Firefox/Spidermonkey 984 ms to do the same thing, making Firefox's interpreter about 20% faster. Rhino comes out looking quite good, considering it is, after all, "only Java."

A more shocking comparison emerged when I tested Rhino against Acrobat. As you may know, Adobe Acrobat comes with a JavaScript engine, presumably written in C++. I expected it to blow the doors off Rhino. Not so! Here are the results:

Astonishingly, Acrobat's interpreter was a full order of magnitude slower than Rhino, requiring 11765 milliseconds to calculate pi from a million terms (versus 1212 ms for Rhino). Why so slow? I have no idea. Perhaps someone from Adobe can comment.

Bottom line: Don't assume that because something is written in Java, it's (automatically) slow. It depends on the runtime environment and what it is, exactly, you're trying to do.

After this, my respect for Rhino only rises. (And it was high to begin with.)

3 comments:

  1. You're making weird assumptions, or possibly your terminology is off. First of all, it's not the "Rhino-embedded-in-the-JRE," it's simply an older version of Rhino shipped with the JRE. It does not include many improvements that have been added by Google since then. Not sure why.

    Next you seem to be confused as to the nature of Rhino. It is quite optimized, and if I'm not mistaken (but I could be wrong about this) it can generate JVM bytecode. The JVM has had 15 years of intense bytecode execution optimization; it's not surprising that it should demolish a rarely used JavaScript runtime. Who cares about Acrobat JS performance? Adobe does not care about Acrobat performance.

    ReplyDelete
  2. Your test really isn't measuring much of the JavaScript engine.

    The pi() function you have is so simple and being executed so many times, the bulk of the time will be spent handling the arithmetic and Math.sqrt() call.

    Thus your only really measuring those two things, which I guess if that's all your gonna use Rhino for, you'll only be about 20% slower as you mentioned.

    I'm guessing if you had a more robust set of tests, one that calls more JavaScript functions, recursive calls, etc. that you would see engines like TraceMonkey and v8 pull further away from Rhino.

    ReplyDelete

Add a comment.

Note: Only a member of this blog may post a comment.