GraalVM — Episode 2: “The Holy Grail”

Published in

FAUN — Developer Community 🐾

7 min readSep 11, 2020

In Episode 1, we looked at how JVM evolved into an optimum VM, with JIT C1, C2, and HotSpot. In this blog, we will see how Graal replaces the C2 compiler and brings in a larger ecosystem of frameworks to support multiple non-JVM languages…

“CloudFirst”

With large enterprises embracing the CloudFirst approach, more and more we are building/moving critical enterprise applications onto Cloud. To truly realize the value of the cloud, and derive the most optimum TCO, it's important that these applications are built on optimized runtimes.

I had listed a few critical non-functional requirements in Episode 1 (smaller footprint, faster startup, etc), The following are a few more, that needs to be noted before we ask “why graal”

Polyglot & Interoperability: Polyglot is the reality, each language has its own strengths and will continue to have, so we need to embrace this fact. If you look at the core logic of the interpreter/compiler, they are all the same. They all try to achieve similar levels of optimization, to generate the fastest running machine code, with the smallest footprint. How about, if we can take the maturity that Java VM has achieved over a period of last 10+ years and use that to our advantage?? and have all these new-age programming languages leverage that (Graal), and also build on top of that (Truffle, JVMCI)…even better how about getting these different languages, run on the same VM & talk to each other?? that would be a “Perfect Cloud Native world”.
Embeddable runtimes (VMs): As we moved to the container technologies, we are looking for embeddable VMs for “immutable” approaches towards building the container, for better management and versioning.

Graal & Truffle Stack

Let's look at the stack that is published by Graal, and go through in detail, how it works…

JVM (Hotspot): This is the traditional JVM HotSpot.

I published a detailed blog on the evolution of Java JIT & Hotspot, in Episode 1 Pease go through that here in detail.

JVM Compiler Interface: In Java 9, JVMCI (JVM Compiler Interface) was introduced. This allowed writing compilers, as plugins, that JVM can call for dynamic compilation. It provides an API and a protocol to build compilers with custom implementations and optimisations.

The word compiler, I am referring to here, is not the javac type of compiler, this is the compiler within the JVM, (like C1, C2), that converts the bytecode to optimized machine code (instead of interpreter, as explained in the previous section)

Substrate VM: Native Images that are compiled “Ahead of Time(AOT)” into a standalone executable, run on the Substrate VM. Substrate VM builds in all the JVM functionality into the native image (such as memory management garbage collection, thread scheduling etc).

Graal Compiler & tooling: Graal compiler is built on JVMCI, and provides a better JIT compiler (C2 — that we covered in Episode 1) implementation, with further optimizations. We will go through in detail later in the blog. It also provides an AOT option, where we can create a standalone, machine code directly.

To understand how GraalVM is better than JVM and how it achieves this “Holy Grail” VM for all language…we need to dig deeper on 2 key components — Truffle and Graal VM.

Before we dig deeper, we need to have complete clarity on how JVM works, and optimizes the code, using the interpreters and compilers (C1, C2), please make sure you have already gone through Episode 1

Graal & Truffle

Truffle is a language interpreter framework. It's a framework, that the language programmers implement and let their languages run on GraalVM, leveraging the optimizations that GraalVM provides.

To understand this better, if I am a language programmer, what I need to do is to write an interpreter using “Tuffle Language Implementation Framework”, and package them as components, that I can run on GraalVM, not worrying about writing complicated optimizations. GraalVM will take care of running the code optimally. I can choose to compile “Ahead-Of-Time” to Graal Native Image or “Just-In-Time”.

Truffle also provides a framework called “Truffle Instrument API” for building tools. Instruments provide fine-grained VM level runtime events, which can be used to built profiling, tracing analyzing, debugging tools. The best part is, if you have your language interpreters written with Truffle, you can use the ecosystem of Truffle instruments already built. (example VisualVM, Chrome Debugger, GraalVM Visual Studio Code Extension etc)

Graal JIT

Graal can compile the code Just-In-Time (JIT, like C2) or Ahead of Time (AOT) directly to native image.

There is a general myth that AOT is faster, which is very true in the first few runs, but there is a possibility that the JIT might outperform the AOTs, as JIT is constantly optimizing (Graal VM/C2) based on the feedback it gets from profiling. JIT normally has a larger footprint, than AOT.

For Serverless — it makes more sense to go towards AOT, while for long-running container-based/VM based deployments, JIT might make more sense.

The diagram below shows how the Graal JIT compiler works (you might find it very close to C2 compilers in what we covered in Episode 1

Abstract Syntax Tree is the intermediate representation. AST provides a very optimum way to represent the syntactic structure of the language, where typically the parent node is the operator, and the children node represents the operands or operators (based on cardinality).

In this statement, a, b, c can be any variables (for loosely typed languages). The interpreter starts assuming “Generics”, based on the profiling, of various executions, it starts assuming the specifics & then will optimize the code, using partial evaluation.

Partial Evaluation & De-Optimization: Truffle (language interpreters written with Truffle) runs as an interpreter, and Graal JIT kicks in to start identifying optimizations in the code. Here are some of the optimizations

Inlining: The compiler identifies small methods, and inline them into the code. This makes this method calls inexpensive.
Escape Analysis: The compiler detects local variables, and allocates them in CPU registers for faster access.
Dead Code: JIT only compiles the code that is used.

The above optimizations are based on speculation, and eventually, if the speculation is proven wrong at runtime, the JIT will re-optimize and recompile (as shown in the diagram above). Re-optimization & recompiling is an expensive task.

Partial Evaluation creates an intermediate representation of the language, from the code and the data, and as it learns, and identifies new data types, it will deoptimize to AST Interpreter, applies optimizations, and does the node-rewriting and recompiles. After a certain point, it would have the most optimum representation.

Graal AOT

Graal Ahead of Time compilation is a very powerful way to create native images, for a particular targeted OS/Architecture. For cloud-native workloads and serverless, this is a very powerful option, for lower footprint, faster startups, and more importantly embeddable runtimes (providing immutability). Imagine you Tekton/Jenkins pipelines build your code with this option, you already get a super optimum native image (assuming we are running them on a specific container runtime platform — which in most cases is built on Linux)

Graal Polyglot Interoperability

GraalVM supporting and allowing applications written in multiple languages, it also provides ways to pass data between these applications, that are written using different languages. Truffle provides “Polyglot Interoperability Protocol”. The Interoperability protocol defines the message, that each language needs to implement, to support the passing of data between the polyglot applications.

Sulong

Sulong is a high-performance LLVM bitcode runtime that is built on Graal VM. Sulong uses Truffle and Graal as dynamic compiler.

LLVM is an open-source project which is a collection of modular, reusable compilers and toolchains. There are a lot of languages (C, C++, Fortran, Rust, Swift etc) compilers that are built on LLVM, where LLVM provides the intermediate representation (LLVM-IR).

Sulong pipeline looks different from what we looked at other language compilers, that are running on Truffle., please refer to the diagram above, on how C/C++ code gets compiled.

Frameworks built on GraalVM

There are various frameworks that are developed on GraalVM to provide a further optimizations, for building MicroServices, and go cloud-native. Quarkus, Micronaut, Helidon, and Spring are some of the popular frameworks, built on Graal.

We will be covering this in the next episode, in our journey towards the most optimum MicroServices architecture…

ttyl ;-) stay safe!!!