Welcome!

Machine Learning Authors: Elizabeth White, Yeshim Deniz, Liz McMillan, Pat Romanski, Shelly Palmer

Related Topics: Machine Learning , Java IoT

Machine Learning : Article

Lesson Learnd from Using Rhino to Process JavaScript

This entry documents a few tips related to using Rhino JavaScript Engine to process JavaScript code

This entry documents a few tips related to using Rhino JavaScript Engine to process JavaScript code. If you are using Rhino, you probably won’t run into the issues covered in this post during development or even testing. However, you are fairly likely to run into these issues after your system goes live. It would easily result in days or even weeks of soul searching (speaking from my personal experience:-)). Part of the problem seems to be the lack of documentation from the web. The other part of the problem is that the problem is rather a Java language problem(very convoluted). - Java limits the maximum method size to be 64KB.

1. JVM Byte Code Size Limit Problem

On rare occasions, you will see exceptions like the followings from Rhino when processing JavaScript files:

Exception in thread “main” java.lang.IllegalArgumentException: out of range index

at org.mozilla.classfile.ClassFileWriter?.add(ClassFileWriter?.java:541) at org.mozilla.classfile.ClassFileWriter?.addLoadConstant(ClassFileWriter?.java:601) at org.mozilla.classfile.ClassFileWriter?.addPush(ClassFileWriter?.java:837) at org.mozilla.javascript.optimizer.BodyCodegen?.visitSpecialCall(Codegen.java:2571) at org.mozilla.javascript.optimizer.BodyCodegen?.generateExpression(Codegen.java:1763) at org.mozilla.javascript.optimizer.BodyCodegen?.visitSetProp(Codegen.java:3743) at org.mozilla.javascript.optimizer.BodyCodegen?.generateExpression(Codegen.java:2118) at org.mozilla.javascript.optimizer.BodyCodegen?.generateStatement(Codegen.java:1660) at org.mozilla.javascript.optimizer.BodyCodegen?.generateStatement(Codegen.java:1510) at org.mozilla.javascript.optimizer.BodyCodegen?.generateBodyCode(Codegen.java:1181) at org.mozilla.javascript.optimizer.Codegen.generateCode(Codegen.java:285) at org.mozilla.javascript.optimizer.Codegen.compileToClassFile(Codegen.java:157) at org.mozilla.javascript.optimizer.Codegen.compile(Codegen.java:67) at org.mozilla.javascript.Context.compileImpl(Context.java:2327) at org.mozilla.javascript.Context.compileString(Context.java:1323) at org.mozilla.javascript.Context.compileString(Context.java:1312) at org.mozilla.javascript.tools.shell.Main.loadScriptFromSource(Main.java:500) at org.mozilla.javascript.tools.shell.Main.processFileSecure(Main.java:439) at org.mozilla.javascript.tools.shell.Main.processFile(Main.java:406) at org.mozilla.javascript.tools.shell.Main.processSource(Main.java:397) at org.mozilla.javascript.tools.shell.Main.processFiles(Main.java:181) at org.mozilla.javascript.tools.shell.Main$IProxy.run(Main.java:102) at org.mozilla.javascript.Context.call(Context.java:540) at org.mozilla.javascript.ContextFactory?.call(ContextFactory?.java:447) at org.mozilla.javascript.tools.shell.Main.exec(Main.java:164) at org.mozilla.javascript.tools.shell.Main.main(Main.java:142)

Here are a few posts you can find when you do a web search that are related to this issue (though don’t seem to offer a solution):

This can be extremely frustrating. For example, if you are a Dojo ShrinkSafe user, you know ShrinkSafe works because you have used it millions of times on many projects successfully. But on a certain situations, you will run into the above problem. When you search specifically for this ShrinkSafe problem, here is the best that I can find:

“While it’s not recommended that you develop 2MB JS files, it seems that it *does* happen, and it’s about the time that your files get this large that a project (naturally!) turns to the Dojo compressor. Unfortunantly, the default set of arguments doesn’t hold up to a file this big, and strange exceptions result. Here is a modified recipe for use with the kinds of files that really, *really* need the Dojo compressor:
java -Xmn100M -Xms500M -Xmx500M -jar custom_rhino.jar -opt -1 -c HUGE.js > smaller.js

This is not very helpful in my own experience. First of all, there are tons of reasons that Rhino needs to process big JavaScript files (for example, you want to use Rhino to analyze a bunch of JavaScript files concatenated together). Secondly, very little explanation is given about this command line “magic”. Thirdly, this command line “magic” seems to be misleading because the problem is not related to JVM memory limit (no need to do -Xms500M -Xmx500M in my experience).

The source of the problem is not your code, not Rhino, but rather a Java language limit. See http://java.sun.com/docs/books/jvms/second_edition/html/ClassFile.doc.html, section 4.10:

The amount of code per non-native, non-abstract method is limited to 65536 bytes by the sizes…

Rhino implemented correctly by sticking to the Java language specification. In fact, you can see the following comments from <code>org.mozilla.classfile.ClassFileWriter<code>:

if (attrLength > 65536) {
// See http://java.sun.com/docs/books/jvms/second_edition/html/ClassFile.doc.html,
// section 4.10, “The amount of code per non-native, non-abstract
// method is limited to 65536 bytes…
throw new ClassFileFormatException(
“generated bytecode for method exceeds 64K limit.”);
}

The Dojo ShrinkSafe issue is fundamentally caused by this problem. Any other software that relies on Rhino will have this problem too.

2. So What Is the Solution?

The short answer is, well, there is no solution, if you rely on Rhino to compile JavaScript into Java code. Well, Java code has to subject to the 64KB Java language limit.

If you are in desperate need for a solution, you might try to “pre-process” your JavaScript code by breaking apart large JavaScript methods into a bunch of smaller methods, which may yield smaller than 64KB Java byte code for each compiled method. -Thought this approach looks really painful.

However, if you are not using Rhino to compile JavaScript to Java, there are easier ways to get around the problem. Rhino does not have to “compile JavaScript to Java code”. You can configure Rhino to run at an “Interpretive mode”. Given that there is Java byte code involved at all, this limit naturally does not apply. This is simple to do. It takes one line of code:

context.setOptimizationLevel(-1);

The above set the Rhino Context object to “interpreative” mode. See Rhino Optimization Level for detailed information.

3. Is it time for us to treat the 64KB JVM limit as a bug, rather that a Java language spec item?

In the world of JavaScript, well, it is not usual to have methods that far exceeds the 64KB limit. For example, it is very common these days for JavaScript libraries to wrap a lot of functions inside one parent function for the sake of encapsulation and scoping. For example, jQuery wraps all code inside one function block (function(){…})(). Other JavaScript libraries like Dojo use similar techniques too. In such situations, it is fairly easy to exceed the 64KB limit when compiled into Java byte code.

For dynamic languages, especially languages that developers tend to nest functions inside functions (”inner function”), the 64KB limit seems to be awfully low.

Of course, this problem is not unique to JavaScript developers. For example, JSP developers can easily run into this problem too. See http://forums.sun.com/thread.jspa?threadID=472929&messageID=2296445 for an example.

Interesting enough, bug 4262078 at bugs.sun.com, “Maximum method size is too small (64Kb)“, is exactly on this issue. See http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4262078. The bug was submitted in 1999. Nine years later, in 2008, we are still not sure whether this will be fixed.

More Stories By Coach Wei

Coach Wei is founder and CEO of Yottaa, a web performance optimization company. He is also founder and Chairman of Nexaweb, an enterprise application modernization software company. Coding, running, magic, robot, big data, speed...are among his favorite list of things (not necessarily in that order. His coding capability is really at PowerPoint level right now). Caffeine, doing something entrepreneurial and getting out of sleeping are three reasons that he gets up in the morning and gets really excited.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


@CloudExpo Stories
SYS-CON Events announced today that Twistlock, the leading provider of cloud container security solutions, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Twistlock is the industry's first enterprise security suite for container security. Twistlock's technology addresses risks on the host and within the application of the container, enabling enterprises to consistently enforce security policies, monitor...
SYS-CON Events announced today that Outscale, a global pure play Infrastructure as a Service provider and strategic partner of Dassault Systèmes, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Founded in 2010, Outscale simplifies infrastructure complexities and boosts the business agility of its customers. Outscale delivers a secure, reliable and industrial strength solution for its customers, which i...
SYS-CON Events announced today that Peak 10, Inc., a national IT infrastructure and cloud services provider, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Peak 10 provides reliable, tailored data center and network services, cloud and managed services. Its solutions are designed to scale and adapt to customers’ changing business needs, enabling them to lower costs, improve performance and focus intern...
A strange thing is happening along the way to the Internet of Things, namely far too many devices to work with and manage. It has become clear that we'll need much higher efficiency user experiences that can allow us to more easily and scalably work with the thousands of devices that will soon be in each of our lives. Enter the conversational interface revolution, combining bots we can literally talk with, gesture to, and even direct with our thoughts, with embedded artificial intelligence, whic...
Everywhere we turn in our industry we can find strong opinions about the direction, type and nature of cloud’s impact on computing and business. Another word that is used in every context in our industry is “hybrid.” In his session at 20th Cloud Expo, Alvaro Gonzalez, Director of Technical, Partner and Field Marketing at Peak 10, will use a combination of a few conceptual props and some research recently commissioned by Peak 10 to offer a real-world consideration of how the various categories of...
While some vendors scramble to create and sell you a fancy solution for monitoring your spanking new Amazon Lambdas, hear how you can do it on the cheap using just built-in Java APIs yourself. By exploiting a little-known fact that Lambdas aren’t exactly single threaded, you can effectively identify hot spots in your serverless code. In his session at 20th Cloud Expo, David Martin, Principal Product Owner at CA Technologies, will give a live demonstration and code walkthrough, showing how to ov...
DevOps is often described as a combination of technology and culture. Without both, DevOps isn't complete. However, applying the culture to outdated technology is a recipe for disaster; as response times grow and connections between teams are delayed by technology, the culture will die. A Nutanix Enterprise Cloud has many benefits that provide the needed base for a true DevOps paradigm. In his Day 3 Keynote at 20th Cloud Expo, Chris Brown, a Solutions Marketing Manager at Nutanix, will explore t...
In order to meet the rapidly changing demands of today’s customers, companies are continually forced to redefine their business strategies in order to meet these needs, stay relevant and continue to see profitable growth. IoT deployment and development is integral in this transformation, and today businesses are increasingly seeing the value of investing their resources into IoT deployments. These technologies are able increase ROI through projects such as connecting supply chains or enabling sm...
This talk centers around how to automate best practices in a multi-/hybrid-cloud world based on our work with customers like GE, Discovery Communications and Fannie Mae. Today’s enterprises are reaping the benefits of cloud computing, but also discovering many risks and challenges. In the age of DevOps and the decentralization of IT, it’s easy to over-provision resources, forget that instances are running, or unintentionally expose vulnerabilities.
In his session at 20th Cloud Expo, Brad Winett, Senior Technologist for DDN Storage, will present several current, end-user environments that are using object storage at scale for cloud deployments including private cloud and cloud providers. Details on the top considerations of features and functions for selecting object storage will be included. Brad will also touch on recent developments in tiering technologies that deliver single solution and an end-user view of data across files and objects...
Multiple data types are pouring into IoT deployments. Data is coming in small packages as well as enormous files and data streams of many sizes. Widespread use of mobile devices adds to the total. In this power panel at @ThingsExpo, moderated by Conference Chair Roger Strukhoff, panelists will look at the tools and environments that are being put to use in IoT deployments, as well as the team skills a modern enterprise IT shop needs to keep things running, get a handle on all this data, and deli...
SYS-CON Events announced today that SoftLayer, an IBM Company, has been named “Gold Sponsor” of SYS-CON's 18th Cloud Expo, which will take place on June 7-9, 2016, at the Javits Center in New York, New York. SoftLayer, an IBM Company, provides cloud infrastructure as a service from a growing number of data centers and network points of presence around the world. SoftLayer’s customers range from Web startups to global enterprises.
Regardless of what business you’re in, it’s increasingly a software-driven business. Consumers’ rising expectations for connected digital and physical experiences are driving what some are calling the "Customer Experience Challenge.” In his session at @DevOpsSummit at 20th Cloud Expo, Marco Morales, Director of Global Solutions at CollabNet, will discuss how organizations are increasingly adopting a discipline of Value Stream Mapping to ensure that the software they are producing is poised to ...
You know you need the cloud, but you’re hesitant to simply dump everything at Amazon since you know that not all workloads are suitable for cloud. You know that you want the kind of ease of use and scalability that you get with public cloud, but your applications are architected in a way that makes the public cloud a non-starter. You’re looking at private cloud solutions based on hyperconverged infrastructure, but you’re concerned with the limits inherent in those technologies.
IBM helps FinTechs and financial services companies build and monetize cognitive-enabled financial services apps quickly and at scale. Hosted on IBM Bluemix, IBM’s platform builds in customer insights, regulatory compliance analytics and security to help reduce development time and testing. In his session at 20th Cloud Expo, Tom Eck, Industry Platforms CTO at IBM Cloud, will discuss how these tools simplify the time-consuming tasks of selection, mapping and data integration, allowing developer...
For financial firms, the cloud is going to increasingly become a crucial part of dealing with customers over the next five years and beyond, particularly with the growing use and acceptance of virtual currencies. There are new data storage paradigms on the horizon that will deliver secure solutions for storing and moving sensitive financial data around the world without touching terrestrial networks. In his session at 20th Cloud Expo, Cliff Beek, President of Cloud Constellation Corporation, ...
Interested in leveling up on your Cloud Foundry skills? Join IBM for Cloud Foundry Days on June 7 at Cloud Expo New York at the Javits Center in New York City. Cloud Foundry Days is a free half day educational conference and networking event. Come find out why Cloud Foundry is the industry's fastest-growing and most adopted cloud application platform.
With major technology companies and startups seriously embracing Cloud strategies, now is the perfect time to attend @CloudExpo | @ThingsExpo, June 6-8, 2017, at the Javits Center in New York City, NY and October 31 - November 2, 2017, Santa Clara Convention Center, CA. Learn what is going on, contribute to the discussions, and ensure that your enterprise is on the right path to Digital Transformation.
Five years ago development was seen as a dead-end career, now it’s anything but – with an explosion in mobile and IoT initiatives increasing the demand for skilled engineers. But apart from having a ready supply of great coders, what constitutes true ‘DevOps Royalty’? It’ll be the ability to craft resilient architectures, supportability, security everywhere across the software lifecycle. In his keynote at @DevOpsSummit at 20th Cloud Expo, Jeffrey Scheaffer, GM and SVP, Continuous Delivery Busine...
SYS-CON Events announced today that delaPlex will exhibit at SYS-CON's @CloudExpo, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. delaPlex pioneered Software Development as a Service (SDaaS), which provides scalable resources to build, test, and deploy software. It’s a fast and more reliable way to develop a new product or expand your in-house team.