TypeScript modules With Emscripten and CMake, part 2
When I set out to create an NPM package for SoundSwallower, I was unable to find much relevant information in the Emscripten documentation or elsewhere on the Web, so I have written this guide, which walks through the process of compiling a library to WebAssembly and packaging it as a CommonJS or ES6 module.
This is part of a series of posts. Start here to read from the beginning.
Exporting functions
In the previous episode, we successfully built
a CommonJS module and accompanying WebAssembly file, which we loaded
in Node with require, but which contains no useful code. There are
two reasons for this:
- Emscripten requires functions defined in C to be exported in order to be accessed from JavaScript, and…
- Emscripten also does “dead code elimination”, and since no functions were exported, they were all considered to be dead, deceased, expired, run down the curtain and joined the choir invisible, and were… eee-liminated.
The first order of business, then, is to get some functions exported.
There are exactly four of these that we care about, and while we
could put them on the emcc command-line or declare them as exported
in the code, it is better in the long run to list them in a text file.
So, create the file exported_functions.txt containing:
_kiss_fftr_alloc
_kiss_fftr
_malloc
_free
You’ll notice we had to add a leading underscore to the names, which is an Emscripten convention, for some reason. Now, if you run:
emcc -o kissfft.js jsbuild/libkissfft-float.a -sMODULARIZE=1 \
-sEXPORTED_FUNCTIONS=@exported_functions.txt
You should see a kissfft.wasm of a more impressive size. More to
the point, you can, actually, import the module and run these
functions. Create the script test_kissfft.js containing:
const assert = require("assert");
require("./kissfft.js")().then((kissfft) => {
let fftr = kissfft._kiss_fftr_alloc(16, 0, 0, 0);
assert.ok(fftr);
console.log(`fftr is ${fftr}`);
});When you run it, it will produce some quite useless output resembling:
$ node test_kissfft.js
fftr is 70216
Um, hooray? This takes a bit of explaining, which will also explain
why kissfft.js exists and why it’s so incredibly huge (don’t worry,
we will make it smaller eventually).
In the beginning, Emscripten would compile all your code into
JavaScript, and exported functions would be attributes on a global
module object called, obviously, Module. This is not so great if
you want to make a CommonJS or ES6 module instead of just mashing
everything into the global namespace like it’s 1995. Also, if your
code is quite large, you might want to do other things (like display a
web page) while it loads and compiles.
So, when you pass -sMODULARIZE=1 to emcc, what you get when you
call require on the generated JavaScript is not the module itself,
but rather a function returning a Promise to return that module.
This is necessary because currently you can’t directly import
WebAssembly from JavaScript code but must use the asynchronous-only
WebAssembly
API.
The various code in kissfft.js handles the loading of the
WebAssembly and exporting its functions into a module object.
Although we use this Promise directly above, it is usually more
convenient to await on it, like this:
const createModule = require("./kissfft.js");
const kissfft = await createModule();Calling C functions directly
With all that explanation out of the way, let’s get down to the business of actually using the functions we exported above.
As noted previously these become properties of the object that we get
when Promise returned by calling the function that require returns
resolves (what a mouthful!). They otherwise work just like they do in
C, with the obvious exception that all of their arguments are simple
JavaScript numbers with no type-checking whatsoever.
What does this mean if you have strings, pointers, etc? Well, you can
use the Emscripten utilities ccall and
cwrap
for simple cases, especially involving C strings. The rest of that
page is not worth reading (WebIDL, inlining JavaScript in C… who
does that?) with the exception of the section on directly accessing
memory,
which is super important, because if you have any kind of interesting
data, that is what you will have to do, but also super wrong (as of
writing this), because it misses one extremely important detail. Read
on to find out which one!
As a reminder, we are doing all this because we have some time-domain
data, which we would like to transform into frequency-domain data.
Specifically, we are using the real-valued FFT function kiss_fftr,
which reads from an array of nfft real values (in this case, float
which is 32 bits by definition) and writes to an array of nfft / 2 + 1
complex numbers, which are represented with a struct that looks
like this:
typedef struct {
kiss_fft_scalar r; // (NOTE: these are floats)
kiss_fft_scalar i;
} kiss_fft_cpx;No, it’s not really documented how Emscripten organizes and aligns
structs in memory, but we can be reasonably sure that they are packed,
and we can assume an array of 9 kiss_fft_cpx is equivalent to an
array of 18 float.
So let’s create these arrays! We’ll do this by creating a
Float32Array on the JavaScript side then copying it into the
module’s memory space (sadly there is no more efficient way to do
this). To keep things manageable, we’ll do a 16-point FFT of a
waveform with a single component at half the Nyquist frequency (yes,
really). You can see the equivalent C code
here.
const timedata = new Float32Array([0, 0.5, 0, -0.5,
0, 0.5, 0, -0.5,
0, 0.5, 0, -0.5,
0, 0.5, 0, -0.5]);Now we’ll figure the length of the input and the output and allocate
them in the module’s address space (you may be asking, but why
aren’t we checking the return value of _malloc? By default,
Emscripten is configured to panic if malloc
fails
instead of returning NULL):
const nfft = timedata.length;
const nfreq = nfft / 2 + 1;
const ctimedata = kissfft._malloc(nfft * 4); // float
const cfreqdata = kissfft._malloc(nfreq * 4 * 2); // complex
And we will copy the data to the address we allocated using
set
on the module’s HEAP8 array, first taking a view of it as a
Uint8Array. THIS IS SUPER IMPORTANT! as otherwise
JavaScript will helpfully convert each of your floating-point values
to an 8-bit integer, which is definitely not what you want:
kissfft.HEAP8.set(new Uint8Array(timedata.buffer), ctimedata);Great! Now we can just call the function with the “pointers” (which
are really just indices into HEAP8) we created above:
kissfft._kiss_fftr(fftr, ctimedata, cfreqdata);To get the output array, we use slice to make a copy
(confusingly, this is the oppposite of what a “slice” does in every
other programming language in existence) of the memory and then take
a view of it as a Float32Array:
const freqdata = new Float32Array(
kissfft.HEAP8.slice(cfreqdata, cfreqdata + nfreq * 4 * 2).buffer);And finally, we should deallocate everything so that we don’t run out of memory:
kissfft._free(ctimedata);
kissfft._free(cfreqdata);
kissfft._free(fftr);You can download the full test script here.
Obviously, it’s not a great idea to have an API full of functions with
arbitrary number parameters and no type-checking, though this is
JavaScript, after all, so perhaps some people consider that to be
totally acceptable. In the next installment, we will find out how to make a safer
and more programmer-friendly API!