Five Challenges When Using WebAssembly, Golang, and TypeScript

Programming

In a recent project, I needed to develop an SSH wrapper and an AskPass program using Golang to enhance Git commands with custom authentication. These programs also needed to share some business logic with another program based on NodeJS and potentially a future browser app.

After careful consideration, I decided to use WebAssembly as a bridge between the Golang and NodeJS runtimes. WebAssembly is an effective solution for API communication between different language runtimes in web development. However, my experience implementing it turned out less straightforward than anticipated.

Issue #1: WASM File Compilation Error

The first challenge I encountered when compiling Golang into WASM is a compilation error, which presents itself as a message stating: 

CompileError: WebAssembly.instantiate(): expected magic word 00 61 73 6d, found 21 3c 61 72 @+0
CompileError: WebAssembly.instantiate(): expected magic word 00 61 73 6d, found 21 3c 61 72 @+0

This error message indicates that the WASM module was not correctly compiled. The expected magic word 00 61 73 6d expected magic word 00 61 73 6d refers to the header that every valid WASM file should start with, which is the hexadecimal representation of the ASCII string asm asm , preceded by a null byte.

The error message suggests that instead of the expected header, the file starts with a different sequence of bytes. The hexadecimal sequence 21 3c 61 72 21 3c 61 72 translates to the ASCII string !<ar !<ar , which implies that the file might be an archive file, not a WebAssembly binary.

Solution

To solve this problem, ensure to set the environment variables and use the proper command to compile Golang: 

GOOS=js GOARCH=wasm go build -o main.wasm main.go
GOOS=js GOARCH=wasm go build -o main.wasm main.go

The two environment variables GOOS GOOS and GOARCH GOARCH are essential to specify the target operating system and architecture for which the Golang code should be compiled.

  • GOOS=js GOOS=js sets the target operating system to JavaScript. It might sound a bit of a misnomer since JavaScript is not an operating system, but in this context, it’s used to indicate the target environment is a JavaScript engine that can run WebAssembly
  • GOARCH=wasm GOARCH=wasm sets the target architecture to WebAssembly. WebAssembly is a binary instruction format that’s designed to be portable target for the compilation of high-level languages like C, C++, Golang and Rust, enabling them to run on the web platforms

Issue #2: Missing WASM Import Arguments

The second issue related to missing import arguments. When trying to instantiate a WASM module, I encountered an error message stating: 

TypeError: WebAssembly.instantiate(): Imports argument must be present and must be an object
TypeError: WebAssembly.instantiate(): Imports argument must be present and must be an object

This error indicates that I didn’t pass the required import object when calling the WebAssembly.instantiate WebAssembly.instantiate function - the first argument is the WebAssembly binary code and the second is the import object.

The import object is a JavaScript object that provides functions, JavaScript objects, or WebAssembly Memory instances that can be called or used by the WebAssembly code. These are essentially dependencies for a WebAssembly module to operate correctly.

In the context of Golang, the import object typically includes the following:

  • go.importObject.env go.importObject.env provides the environment for the WebAssembly instance. It includes functions for memory management, handling system calls and other operations that the Golang runtime needs to function
  • go.iimportObject.go go.iimportObject.go is an object that the Golang runtime uses internally to manage the execution of Golang code. It includes functions for scheduling and running Golang routines

Solution

To solve this problem, ensure to import the wasm_exec.js wasm_exec.js file, provided by Golang, before loading the WASM module:

  1. First, we need to copy the support file wasm_exec.js wasm_exec.js from $(go env GOROOT)/misc/wasm/wasm_exec.js $(go env GOROOT)/misc/wasm/wasm_exec.js
  2. Secondly, load the support file and construct a Golang object
  3. Finally, we can successfully instantiate the WASM module
/**
 * @see https://github.com/golang/go/wiki/WebAssembly#getting-started
 */
import './wasm_exec.js';

const go = new globalThis.Go();
WebAssembly.instantiate('my.wasm', go.importObject).then((wasmModule) => {
  go.run(wasmModule.instance);
  // ... do something with WASM API
});
/**
 * @see https://github.com/golang/go/wiki/WebAssembly#getting-started
 */
import './wasm_exec.js';

const go = new globalThis.Go();
WebAssembly.instantiate('my.wasm', go.importObject).then((wasmModule) => {
  go.run(wasmModule.instance);
  // ... do something with WASM API
});

Issue #3: Unavailable crypto.getRandomValues

The third issue is the unavailability of crypto.getRandomValues crypto.getRandomValues , the function used to generate cryptographically strong random values.

This issue occurs when running in a certain NodeJS (or browser) runtime where crypto.getRandomValues crypto.getRandomValues is not available.

Solution

The solution is to provide a polyfill and you can also copy the polyfill code provided by Golang: 

/**
 * @see `$(go env GOROOT)/misc/wasm/wasm_exec_node.js`
 */
const fs = require('node:fs');
const crypto = require('node:crypto');
const util = require('node:util');

globalThis.require = globalThis.require ?? require;
globalThis.fs = globalThis.fs ?? fs;
globalThis.TextEncoder = globalThis.TextEncoder ?? util.TextEncoder;
globalThis.TextDecoder = globalThis.TextDecoder ?? util.TextDecoder;

globalThis.performance = globalThis.performance ?? {};
globalThis.performance.now = globalThis.performance.now ?? () => {
  const [sec, nsec] = process.hrtime();
  return sec * 1000 + nsec / 1000000;
};

globalThis.crypto = globalThis.crypto ?? {};
globalThis.crypto.getRandomValues = globalThis.crypto.getRandomValues ?? (b) => crypto.randomFillSync(b);
/**
 * @see `$(go env GOROOT)/misc/wasm/wasm_exec_node.js`
 */
const fs = require('node:fs');
const crypto = require('node:crypto');
const util = require('node:util');

globalThis.require = globalThis.require ?? require;
globalThis.fs = globalThis.fs ?? fs;
globalThis.TextEncoder = globalThis.TextEncoder ?? util.TextEncoder;
globalThis.TextDecoder = globalThis.TextDecoder ?? util.TextDecoder;

globalThis.performance = globalThis.performance ?? {};
globalThis.performance.now = globalThis.performance.now ?? () => {
  const [sec, nsec] = process.hrtime();
  return sec * 1000 + nsec / 1000000;
};

globalThis.crypto = globalThis.crypto ?? {};
globalThis.crypto.getRandomValues = globalThis.crypto.getRandomValues ?? (b) => crypto.randomFillSync(b);

Issue #4: Declaring Variables in globalThis in TypeScript

While developing the NodeJS program using TypeScript, I encountered the fourth challenge, which was related to TypeScript and the usage of the globalThis globalThis object.

globalThis globalThis is a standard JavaScript global object, introduced in ECMAScript 2020. It refers to the global object, no matter what the environment is.

  • In a browser, globalThis globalThis is the same as window window
  • In NodeJS, globalThis globalThis is the same as global global

It is often used to store global variables. But when using TypeScript, due to its static typing system, any addition to the globalThis globalThis object requires a corresponding type declaration. Without the type declaration, if we try to add a variable to globalThis globalThis , TypeScript will throw an error, as it’s not aware of the new property.

Solution

To solve this issue, we can augment the global type definitions. Here’s an example: 

import './wasm_exec.js';

declare namespace globalThis {
  const Go: any;
  const otherGlobalVar: string;
}

globalThis.otherGlobalVar = 'Hello, world!';
import './wasm_exec.js';

declare namespace globalThis {
  const Go: any;
  const otherGlobalVar: string;
}

globalThis.otherGlobalVar = 'Hello, world!';

In the above example, we use the declare namespace declare namespace to extend the TypeScript definition for globalThis globalThis , and inside the namespace declaration, we declare additional constants: Go Go and myGlobalVar myGlobalVar . Since the declare namespace declare namespace is used to add type information, the actual global variables of these names are not being created in the globalThis globalThis object, and so we need to assign them first before access, otherwise we would get undefined undefined .

Issue #5: Golang + WASM is an Application, Not a Library

When it comes to WebAssembly, we usually treat it as a JavaScript library, converted from C/C++, Rust or Golang, and it serves as a set of functions that we can call out to.

However, Golang takes a different approach. When compiling Golang code to WASM, we are not creating a library, but rather a standalone application that can run within the WASM virtual machine in the browser. The Golang runtime is also included in the WASM binary, and it takes over the main thread of execution, initializes itself and starts up our Golang program. In other words, the combination of Golang + WASM creates a powerful tool for developing efficient, high-performance web applications.

This leads us to the fifth challenge: once we launch the Golang program, it runs, then exits, leaving us unable to interact with it further, as it has completed and cleaned up - it’s as if the bridge we built collapses right after its construction 💥

Solution

To solve the problem, we have to prevent the Golang runtime from shutting down, and this is where Golang’s Channel Channel comes in handy.

A Channel Channel is something that awaits data to be sent into it and keeps the main goroutine in a waiting stage until it receives data and completes execution. This is often done using an empty struct as a signaling mechanism. 

func main() {
  done := make(chan struct{})

  js.Global().Set("sayHi", js.FuncOf(sayHi))

  fmt.Println("Waiting...")

  <-done

  fmt.Println("Shut down!!!")
}
func main() {
  done := make(chan struct{})

  js.Global().Set("sayHi", js.FuncOf(sayHi))

  fmt.Println("Waiting...")

  <-done

  fmt.Println("Shut down!!!")
}

In the above example, since we never send any data to the done done channel, the Golang runtime continues to run for as long as needed, thereby allowing us to call anything from it - the bridge remains intact 🎉

Category Programming
Published
Tags