V8 Torque user manual · V8V8 Torque user manual V8 Torque is a language that allows developers contributing to the V8 project to express changes in the VM by focusing on the intent of their changes to the VM, rather than preoccupying themselves with unrelated implementation details. The language was designed to be simple enough to make it easy to directly translate the ECMAScript specification into an implementation in V8, but powerful enough to express the low-level V8 optimization tricks in a robust way, like creating fast-paths based on tests for specific object-shapes.Torque will be familiar to V8 engineers and JavaScript developers, combining a TypeScript-like syntax that eases both writing and understanding V8 code with syntax and types that reflects concepts that are already common in the CodeStubAssembler. With a strong type system and structured control flow, Torque ensures correctness by construction. Torque’s expressiveness is sufficient to express almost all of the functionality that is currently found in V8’s builtins. It also is very interoperable with CodeStubAssembler builtins and macros written in C++, allowing Torque code to use hand-written CSA functionality and vice-versa.Torque provides language constructs to represent high-level, semantically-rich tidbits of V8 implementation, and the Torque compiler converts these morsels into efficient assembly code using the CodeStubAssembler. Both Torque’s language structure and the Torque compiler’s error checking ensure correctness in ways that were previously laborious and error-prone with direct usage of the CodeStubAssembler. Traditionally, writing optimal code with the CodeStubAssembler required V8 engineers to carry a lot of specialized knowledge in their heads — much of which was never formally captured in any written documentation — to avoid subtle pitfalls in their implementation. Without that knowledge, the learning curve for writing efficient builtins was steep. Even armed with the necessary knowledge, non-obvious and non-policed gotchas often led to correctness or security bugs. With Torque, many of these pitfalls can be avoided and recognized automatically by the Torque compiler.Getting started # Most source written in Torque is checked into the V8 repository under the src/builtins directory, with the file extension .tq. Torque definitions of V8's heap-allocated classses are found alongside their C++ definitions, in .tq files with the same name as corresponding C++ files in src/objects. The actual Torque compiler can be found under src/torque. Tests for Torque functionality are checked in under test/torque, test/cctest/torque, and test/unittests/torque.To give you a taste of the language, let’s write a V8 builtin that prints “Hello World!”. To do this, we’ll add a Torque macro in a test case and call it from the cctest test framework.Begin by opening up the test/torque/test-torque.tq file and add the following code at the end (but before the last closing }):@export macro PrintHelloWorld(): void { Print('Hello world!'); } Next, open up test/cctest/torque/test-torque.cc and add the following test case that uses the new Torque code to build a code stub:TEST(HelloWorld) { Isolate* isolate(CcTest::InitIsolateOnce()); CodeAssemblerTester asm_tester(isolate, JSParameterCount(0)); TestTorqueAssembler m(asm_tester.state()); { m.PrintHelloWorld(); m.Return(m.UndefinedConstant()); } FunctionTester ft(asm_tester.GenerateCode(), 0); ft.Call(); } Then build the cctest executable, and finally execute the cctest test to print ‘Hello world’:$ out/x64.debug/cctest test-torque/HelloWorld Hello world! How Torque generates code # The Torque compiler doesn’t create machine code directly, but rather generates C++ code that calls V8’s existing CodeStubAssembler interface. The CodeStubAssembler uses the TurboFan compiler’s backend to generate efficient code. Torque compilation therefore requires multiple steps:The gn build first runs the Torque compiler. It processes all *.tq files. Each Torque file path/to/file.tq causes the generation of the following files:path/to/file-tq-csa.cc and path/to/file-tq-csa.h containing generated CSA macros.path/to/file-tq.inc to be included in in a corresponding header path/to/file.h containing class definitions.path/to/file-tq-inl.inc to be included in the corresponding inline header path/to/file-inl.h, containing C++ accessors of class definitions.path/to/file-tq.cc containing generated heap verifiers, printers, etc.The Torque compiler also generates various other known .h files, meant to be consumed by the V8 build.The gn build then compiles the generated -csa.cc files from step 1 into the mksnapshot executable.When mksnapshot runs, all of V8’s builtins are generated and packaged in to the snapshot file, including those that are defined in Torque and any other builtins that use Torque-defined functionality.The rest of V8 is built. All of Torque-authored builtins are made accessible via the snapshot file which is linked into V8. They can be called like any other builtin. In addition, the d8 or chrome executable also includes the generated compilation units related to class definitions directly.Graphically, the build process looks like this:Torque tooling # Basic tooling and development environment support is available for Torque.There is a Visual Studio Code plugin for Torque, which uses a custom language server to provide features like go-to-definition.There is also a formatting tool that should be used after changing .tq files: tools/torque/format-torque.py -i <filename>Troubleshooting builds involving Torque # Why do you need to know this? Understanding how Torque files get converted into machine code is important because different problems (and bugs) can potentially arise in the different stages of translating Torque into the binary bits embedded in the snapshot:If you have a syntax or semantic error in Torque code (i.e. a .tq file), the Torque compiler fails. The V8 build aborts during this stage, and you will not see other errors that may be uncovered by later parts of the build.Once your Torque code is syntactically correct and passes the Torque compiler’s (more or less) rigorous semantic checks, the build of mksnapshot can still fail. This most frequently happens with inconsistencies in external definitions provided in .tq files. Definitions marked with the extern keyword in Torque code signal to the Torque compiler that the definition of required functionality is found in C++. Currently, the coupling between extern definitions from .tq files and the C++ code to which those extern definitions refer is loose, and there is no verification at Torque-compile time of that coupling. When extern definitions don’t match (or in the most subtle cases mask) the functionality that they access in the code-stub-assembler.h header file or other V8 headers, the C++ build of mksnapshot fails.Even once mksnapshot successfully builds, it can fail during execution. This might happen because Turbofan fails to compile the generated CSA code, for example because a Torque static_assert cannot be verified by Turbofan. Also, Torque-provided builtin that are run during snapshot creation might have a bug. For example, Array.prototype.splice, a Torque-authored builtin, is called as part of the JavaScript snapshot initialization process to setup the default JavaScript environment. If there is a bug in the implementation, mksnapshot crashes during execution. When mksnapshot crashes, it’s sometimes useful to call mksnapshot passing the --gdb-jit-full flag, which generates extra debug information that provides useful context, e.g. names for Torque-generated builtins in gdb stack crawls.Of course, even if Torque-authored code makes it through mksnapshot, it still may be buggy or crash. Adding test cases to torque-test.tq and torque-test.cc is a good way to ensure that your Torque code does what you actually expect. If your Torque code does end up crashing in d8 or chrome, the --gdb-jit-full flag is again very useful.constexpr: compile-time vs. run-time # Understanding the Torque build process is also important to understanding a core feature in the Torque language: constexpr.Torque allows evaluation of expressions in Torque code at runtime (i.e. when V8 builtins are executed as part of executing JavaScript). However, it also allows expressions to be executed at compile time (i.e. as part of the Torque build process and before the V8 library and d8 executable have even been created).Torque uses the constexpr keyword to indicate that an expression must be evaluated at build-time. Its usage is somewhat analogous to C++’s constexpr: in addition to borrowing the constexpr keyword and some of its syntax from C++, Torque similarly uses constexpr to indicate the distinction between evaluation at compile-time and runtime.However, there are some subtle differences in Torque’s constexpr semantics. In C++, constexpr expressions can be evaluated completely by the C++ compiler. In Torque constexpr expressions cannot fully be evaluated by the Torque compiler, but instead map to C++ types, variables and expressions that can be (and must be) fully evaluated when running mksnapshot. From the Torque-writer’s perspective, constexpr expressions do not generate code executed at runtime, so in that sense they are compile-time, even though they are technically evaluated by C++ code external to Torque that mksnapshot runs. So, in Torque, constexpr essentially means “mksnapshot-time”, not “compile time”.In combination with generics, constexpr is a powerful Torque tool that can be used to automate the generation of multiple very efficient specialized builtins that differ from each other in a small number of specific details that can be anticipated by V8 developers in advance.Files # Torque code is packaged in individual source files. Each source file consists of a series of declarations, which themselves can optionally wrapped in a namespace declaration to separate the namespaces of declarations. The following description of the grammar is likely out-of-date. The source-of-truth is the grammar definition in the Torque compiler, which is written using contex-free grammar rules.A Torque file is a sequence of declarations. The possible declarations are listed in torque-parser.cc.Namespaces # Torque namespaces allow declarations to be in independent namespaces. They are similar to C++ namespaces. They allow you to create declarations that are not automatically visible in other namespaces. They can be nested, and declarations inside a nested namespace can access the declarations in the namespace that contains them without qualification. Declarations that are not explicitly in a namespace declaration are put in a shared global default namespace that is visible to all namespaces. Namespaces can be reopened, allowing them to be defined over multiple files.For example:macro IsJSObject(o: Object): bool { … } // In default namespace namespace array { macro IsJSArray(o: Object): bool { … } // In array namespace }; namespace string { // … macro TestVisibility() { IsJsObject(o); // OK, global namespace visible here IsJSArray(o); // ERROR, not visible in this namespace array::IsJSArray(o); // OK, explicit namespace qualification } // … }; namespace array { // OK, namespace has been re-opened. macro EnsureWriteableFastElements(array: JSArray){ … } }; Declarations # Types # Torque is strongly typed. Its type system is the basis for many of the security and correctness guarantees it provides.For many basic types, Torque doesn’t actually inherently know very much about them. Instead, many types are just loosely coupled with CodeStubAssembler and C++ types through explicit type mappings and rely on the C++ compiler to enforce the rigor of that mapping. Such types are realized as abstract types.Abstract types # Torque’s abstract types map directly to C++ compile-time and CodeStubAssembler runti…