When I set out to create an NPM package for SoundSwallower, I was unable to find much relevant information in the Emscripten documentation or elsewhere on the Web, so I have written this guide, which walks through the process of compiling a library to WebAssembly and packaging it as a CommonJS or ES6 module.

This is part of a series of posts. Start here to read from the beginning.

Optimizing for Size

Really, “for Size” is redundant here. It’s the Web. People are loading your page/app/whatever on mobile phones over metered connections. There is no other optimization you should reasonably care about. So, let’s see how we’re doing:

$ ls -lh jsbuild
-rw-rw-r--  1 dhd dhd  64K fév 27 12:09 kissfft.cjs.js
-rwxrwxr-x  1 dhd dhd 197K fév 27 12:09 kissfft.cjs.wasm

Not good! Well, we did configure this with CMAKE_BUILD_TYPE=Debug, and we didn’t bother adding any optimization flags at compilation or link time. When optimizing for size, however, we should not immediately switch to Release build, as the minimization that it does makes it impossible to understand what parts of the output are wasting space.

Let’s set up CMake to build everything with maximum optimization with the -Oz option, which should be passed at both compile and link time. (Note: I will not discuss -flto here, because it is only useful when dealing with the eldritch horrors of C++). While we’re at it we’ll also disable support for the longjmp function which we know our library doesn’t use:

target_compile_options(kissfft PRIVATE -Oz -sSUPPORT_LONGJMP=0 -sSTRICT=1)
target_link_options(kissfft.cjs PRIVATE
  -sMODULARIZE=1 -Oz -sSUPPORT_LONGJMP=0
  -sEXPORTED_FUNCTIONS=@${CMAKE_CURRENT_SOURCE_DIR}/exported_functions.txt)
target_link_options(kissfft.esm PRIVATE
  -sMODULARIZE=1 -sEXPORT_ES6=1 -Oz -sSUPPORT_LONGJMP=0
  -sEXPORTED_FUNCTIONS=@${CMAKE_CURRENT_SOURCE_DIR}/exported_functions.txt)

Now let’s see where we’re at:

$ cmake --build jsbuild
$ ls -lh jsbuild
-rw-rw-r--  1 dhd dhd  40K fév 27 12:11 kissfft.cjs.js
-rwxrwxr-x  1 dhd dhd 139K fév 27 12:11 kissfft.cjs.wasm

Already much better! Now, if you have WABT installed, you can use wasm-objdump to quickly see which functions are taking the most space:

$ wasm-objdump -x jsbuild/kissfft.cjs.wasm  | grep size
 - func[13] size=5194 <dlmalloc>
 - func[14] size=1498 <dlfree>

Now, Emscripten has an option to use a smaller, less full-featured memory allocator, which, since we know that our library is quite simple and doesn’t do a lot of allocation, is a good idea. Let’s change the link flags again:

target_link_options(kissfft.cjs PRIVATE
  -sMODULARIZE=1 -Oz -sSUPPORT_LONGJMP=0 -sMALLOC=emmalloc
  -sEXPORTED_FUNCTIONS=@${CMAKE_CURRENT_SOURCE_DIR}/exported_functions.txt)
target_link_options(kissfft.esm PRIVATE
  -sMODULARIZE=1 -sEXPORT_ES6=1 -Oz -sSUPPORT_LONGJMP=0 -sMALLOC=emmalloc
  -sEXPORTED_FUNCTIONS=@${CMAKE_CURRENT_SOURCE_DIR}/exported_functions.txt)

This saves another 20K (unminimized and uncompressed):

$ cmake --build jsbuild
$ ls -lh jsbuild
-rw-rw-r--  1 dhd dhd  40K fév 27 12:14 kissfft.cjs.js
-rwxrwxr-x  1 dhd dhd 119K fév 27 12:14 kissfft.cjs.wasm

If we look again at wasm-objdump we can see that there isn’t much else we can do, as what’s left consists of runtime support, including some stubs to allow debug printf() to work.

What about the .js file? Here’s where it gets a bit more complicated. First, let’s rebuild in Release mode and see where we’re at after minimization. It’s also important to look at the compressed size of the .js and .wasm files, as a good webserver should be configured to serve them with gzip compression:

$ emcmake cmake -S. -B jsbuild -DCMAKE_BUILD_TYPE=Release
$ cmake --build jsbuild
$ ls -lh jsbuild
-rw-rw-r--  1 dhd dhd  12K fév 27 12:32 kissfft.cjs.js
-rwxrwxr-x  1 dhd dhd  12K fév 27 12:32 kissfft.cjs.wasm
$ gzip -c jsbuild/kissfft.cjs.js | wc -c
3599
$ gzip -c jsbuild/kissfft.cjs.wasm | wc -c
7307

So, our total payload size is about 11K. This is quite acceptable in most circumstances, so you may wish to skip to the next section at this point.

Now, Emscripten also has an option -sMINIMAL_RUNTIME=1 (or 2) which can shrink this a bit more, but the problem is that it doesn’t actually produce a working CommonJS or ES6 module with -sMODULARIZE=1 and -sEXPORT_ES6=1, and worse yet, it cannot produce working code for the Web or ES6 modules, because it loads the WebAssembly like this:

if (ENVIRONMENT_IS_NODE) {
  var fs = require('fs');
  Module['wasm'] = fs.readFileSync(__dirname + '/kissfft.cjs.wasm');
}

Basically your only option if you use -sMINIMAL_RUNTIME is to postprocess the generated JavaScript to work properly in the target environment, because even if you enable streaming compilation, it will still include the offending snippet above, among other things. Doing this is quite complex and beyond the scope of this guide, but you can look at the build.js script used by wasm-audio-encoders, for example.

The other option, if your module is not too big and you don’t mind that it all gets loaded at once by the browser, is to do a single-file build:

Single-File Builds (WASM or JS-only)

In many cases it is a super huge pain to get the separate .wasm file packaged and loaded correctly when you are using a “framework” or even just a run-of-the-mill JavaScript bundler like Webpack or ESBuild. This is, for instance, the case if you use Angular, which requires a custom Webpack configuration in order for it to work at all with modules that use WebAssembly. (note that by default, Webpack works just fine, as long as you have a pretty recent version)

We will go into the details of this in the next installment, but suffice it to say that, if you have a small enough library, you can save yourself a lot of trouble by simply making a single-file build, which you can do by adding -sSINGLE_FILE=1 to the linker options. This gives a quite acceptable size, which is only slightly large due to the fact that the WebAssembly gets encoded as base64:

$ ls -lh jsbuild
-rw-rw-r--  1 dhd dhd  29K fév 27 16:07 kissfft.cjs.js
$ gzip -c jsbuild/kissfft.cjs.js | wc -c
13321

Note, however, that in this case if you load the resulting JavaScript before your page contents, your users will have to wait until it downloads to see anything, whereas with a separate .wasm file, the downloading can be done asynchronously.

Alternately, if you want to support, say, Safari 13, iOS 12, or anything else that predates the final WebAssembly spec, you can simply disable WebAssembly entirely and compile to JavaScript with -sWASM=0. Sadly, at the moment, this is also incompatible with -sEXPORT=ES6=1.

In the next episode, stay tuned for how to actually use this module in a simple test application!

When I set out to create an NPM package for SoundSwallower, I was unable to find much relevant information in the Emscripten documentation or elsewhere on the Web, so I have written this guide, which walks through the process of compiling a library to WebAssembly and packaging it as a CommonJS or ES6 module.

This is part of a series of posts. Start here to read from the beginning.

Building with CMake

Just by screwing around on the command line, we were previously able to produce a more or less useful CommonJS module wrapping the real-valued FFT function from the Kiss FFT library (though not as useful as the existing one on npmjs.org). Now let’s look at how we can build a module with CMake as part of the library’s build system.

As a reminder, we configured CMake to build the library with:

emcmake cmake -S . -B jsbuild -DCMAKE_BUILD_TYPE=Debug \
    -DKISSFFT_TOOLS=OFF -DKISSFFT_STATIC=ON -DKISSFFT_TEST=OFF

When configuring using emcmake, the EMSCRIPTEN variable is defined, so if we want to make all of those flags the defaults, we can add this to CMakeLists.txt after the option definitions (line 54 in the current source):

if(EMSCRIPTEN)
  set(KISSFFT_TOOLS OFF)
  set(KISSFFT_STATIC ON)
  set(KISSFFT_TEST OFF)
endif()

Now let’s add a target to build our module. This is a bit “special” for two reasons:

• The CMake functions for Emscripten treat any output (even a module) as an “executable”, so we have to make believe we’re linking a program.
• Even though all of the C code is already in the libkissfft-float.a library, which CMake references with the kissfft target, it still expects to have at least one source file to link into our “executable”.

To satisfy CMake, we will first simply create an empty C file:

touch api.c

We may at some point want to add helper functions for our API, so this isn’t entirely useless - see the corresponding file in SoundSwallower for an example.

Now we will add the necessary CMake configuration to the end of CMakeLists.txt:

if(EMSCRIPTEN)
  add_executable(kissfft.cjs api.c)
  target_link_libraries(kissfft.cjs kissfft)
  target_link_options(kissfft.cjs PRIVATE
    -sMODULARIZE=1
    -sEXPORTED_FUNCTIONS=@${CMAKE_CURRENT_SOURCE_DIR}/exported_functions.txt)
  em_link_post_js(kissfft.cjs api.js)
endif()

A few things to note here:

• em_link_post_js is not documented, but should be.
• We have to add ${CMAKE_CURRENT_SOURCE_DIR} to the path to exported_functions.txt so that CMake can find it, since we are building in a separate directory.
• We can’t use kissfft as the target name since that is already taken by the C library.

Emscripten will automatically append .js and .wasm to the target name, so, after adding this, if you run:

emcmake cmake -S . -B jsbuild -DCMAKE_BUILD_TYPE=Debug
cmake --build jsbuild

You should find the files kissfft.cjs.js and kissfft.cjs.wasm in the jsbuild directory.

Building an ES6 module

Up to this point we have built a CommonJS module, since they are simpler to use in Node.js, but in reality, all the cool kids are now using ES6 modules, and they are particularly preferred when using a bundler for the Web like Webpack or Esbuild. The latest versions of Emscripten do have built-in, if occasionally buggy, support for producing ES6 modules. So, we can add an extra target inside the if(EMSCRIPTEN) block at the end of CMakeLists.txt:

add_executable(kissfft.esm api.c)
target_link_libraries(kissfft.esm kissfft)
target_link_options(kissfft.esm PRIVATE
  -sMODULARIZE=1 -sEXPORT_ES6=1
  -sEXPORTED_FUNCTIONS=@${CMAKE_CURRENT_SOURCE_DIR}/exported_functions.txt)
em_link_post_js(kissfft.esm api.js)

Sadly, there is no way in the Emscripten CMake support to choose a different file extension for a specific target, so we can’t call this kissfft.esm.mjs. In addition, the boilerplate loader code that Emscripten gives us won’t allow us to share the WebAssembly (which is identical) between targets. For the moment we will end up with kissfft.esm.js and kissfft.esm.wasm in the jsbuild directory, and this is a Problem, as we will see soon.

Packaging with NPM

Now that everything is built, it is actually quite simple to package this as an NPM package. No other action is required on your part… well, not quite. First, let’s create a package.json file, which will have one big problem, that we’ll get to later:

{
  "name": "kissfft-example",
  "version": "0.0.1",
  "description": "A very simple example of packaging WebAssembly",
  "types": "./index.d.ts",
  "main": "./jsbuild/kissfft.cjs.js",
  "exports": {
    ".": {
      "types": "./index.d.ts",
      "require": "./jsbuild/kissfft.cjs.js",
      "import": "./jsbuild/kissfft.esm.js",
      "default": "./jsbuild/kissfft.esm.js"
    }
  },
  "author": "David Huggins-Daines <dhd@ecolingui.ca>",
  "homepage": "https://ecolingui.ca/en/blog/emguide",
  "license": "MIT",
  "scripts": {
    "test": "npx tsc test_realfft.ts && node test_realfft.js"
  },
  "files": [
    "index.d.ts",
    "jsbuild/kissfft.*.js",
    "jsbuild/kissfft.*.wasm"
  ],
  "devDependencies": {
    "@types/node": "^18.14.1",
    "typescript": "^4.9.5"
  },
  "dependencies": {
    "@types/emscripten": "^1.39.6"
  }
}

Of note above:

• We use the exports field to supply different entry points for import and require (but note that this won’t actually work… more below).
• We just package the stuff we built in place, by including only the files we need with the files field.
• We point to the type definition file with the types field in two places, for good luck.
• Although the node we get with emsdk includes @types/emscripten by default, others will not, so it is a package (and not dev) dependency.

Now, assuming you have you have previously created test_realfft.ts (if not, download it here), you should be able to run:

npm install
npm test

And you should see the same output we saw previously. But, did we say there was a problem? Yes. The nifty ES6 model built above won’t actually work in Node, because the Node developers somehow can’t agree to not depend on file extensions to select module systems. Since our package contains both CommonJS (loaded with require) and ES6 (loaded with import) modules, we have to change the file extension on at least one of them to satisfy Node’s simplistic view of the world.

The path of least resistance to fix this and still stay CMakically correct is to add a custom command that copies the built .js file for the ES6 module to a .mjs file:

add_custom_command(
  OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/kissfft.esm.mjs
  DEPENDS kissfft.esm
  COMMAND ${CMAKE_COMMAND} -E copy
  ${CMAKE_CURRENT_BINARY_DIR}/kissfft.esm.js
  ${CMAKE_CURRENT_BINARY_DIR}/kissfft.esm.mjs)
add_custom_target(copy-mjs-bork-bork-bork ALL
  DEPENDS ${CMAKE_CURRENT_BINARY_DIR}/kissfft.esm.mjs)

Now we will modify package.json by changing kissfft.esm.js to kissfft.esm.mjs everywhere, and modifying files to specifically only include the files we need:

  "files": [
    "index.d.ts",
    "jsbuild/kissfft.esm.mjs",
    "jsbuild/kissfft.cjs.js",
    "jsbuild/kissfft.*.wasm"
  ],

You can download the updated version here. And now we can test that both import types work by creating a directory called kissfft-test alongside kissfft, creating the files index.mjs (download here) and index.cjs (download here) in it, then running:

npm link ../kissfft
node index.mjs
node index.cjs

Congratulations! You now have a WebAssembly module that will work as both ES6 and CommonJS, and can also be uploaded to NPM (but please don’t do that). To see what would be packaged, you can run:

npm publish --dry-run

In the next installment, we will see what we can do to make the module as small as possible.

When I set out to create an NPM package for SoundSwallower, I was unable to find much relevant information in the Emscripten documentation or elsewhere on the Web, so I have written this guide, which walks through the process of compiling a library to WebAssembly and packaging it as a CommonJS or ES6 module.

This is part of a series of posts. Start here to read from the beginning.

Creating an API wrapper

Where we left off, we were able to call into the Kiss FFT library from JavaScript code, but the interface left a lot to be desired, as we were messing around with arbitrary number values pointing into the module’s memory space. What we would like to do is to wrap these dangerous functions in easier to use functions/methods, and also provide some type definitions so TypeScript can complain when you try to do some Undefined Behaviour.

Despite what you may have been led to believe, it’s almost never a good idea to generate these sorts of wrappers automatically as it leads to un-idiomatic APIs and handling the special cases is usually more time-consuming than just writing (and testing) the necessary functions by hand.

Like most respectable C libraries, Kiss FFT encapsulates state using an opaque pointer with functions to allocate and free this, um, object. This easily lends itself to being wrapped in an object-oriented TypeScript API. Let’s start by creating the type definitions for that API, in index.d.ts (you must give it this name so that TypeScript’s inscrutably Byzantine lookup rules can find it):

/// <reference types="emscripten" />
export class RealFFT {
  fft(timedata: Float32Array): Float32Array;
  delete(): void;
};
export interface KissFFTModule extends EmscriptenModule {
  RealFFT: {
    new(nfft: number): RealFFT;
  };
};
declare const createModule: EmscriptenModuleFactory<KissFFTModule>;
export default createModule;

Note the somewhat curious way in which the RealFFT constructor is declared. Because of the way we load the module object, we can’t actually export any functions or classes (which are functions, remember, this is JavaScript) from it directly, but must instead define them as properties on its interface. Luckily, TypeScript gives us at least one way to do this, which is shown above.

Also note that we have an explicit delete method. THIS WILL NOT BE CALLED AUTOMATICALLY because JavaScript is still a defective language in 2023 and JavaScript programmers do not care about memory leaks. The consequence of not calling delete when your RealFFT object goes out of scope is that, eventually, the module’s memory space (which is some finite amount, 4MB by default I think) will be used up and its malloc will panic. This is less bad than crashing your browser, but still kind of bad, so please call delete, as shown in the example below. Alternately, you can make the API stateless and allocate and deallocate the FFT state on each call to the FFT code, which is what kissfft-wasm does, and in this case is probably quite acceptable.

You can now create a TypeScript file which uses the interface in test_realfft.ts:

const assert = require("assert");
require("./kissfft.js")().then((kissfft) => {
  const fftr = new kissfft.RealFFT(16);
  const timedata = new Float32Array([0, 0.5, 0, -0.5,
                                     0, 0.5, 0, -0.5,
                                     0, 0.5, 0, -0.5,
                                     0, 0.5, 0, -0.5]);
  const freqdata = fftr.fft(timedata);
  for (let i = 0; i < freqdata.length / 2; i__) {
    console.log(`${i}: ${freqdata[i * 2]} + ${freqdata[i * 2 + 1]}j`);
  }
  fftr.delete(); // please do this
});

And you can already test-compile it to make sure the types are good:

npm install --save-dev typescript @types/node
npx tsc test_realfft.ts

Cool! Now we just have to implement it ☺. We will use the --post-js flag to emcc to attach some JavaScript code to kissfft.js at “link” time. In this case we will make a file called api.js containing this code which you can see closely resembles our previous script that directly called the module functions, with the exception that we don’t need to refer to them as attributes on the module, since this code will be inserted inside the module loading code:

class RealFFT {
  constructor(nfft) {
    this.fftr = _kiss_fftr_alloc(nfft, 0, 0, 0);
  }
  fft(timedata) {
    const nfft = timedata.length;
    const nfreq = nfft / 2 + 1;
    const ctimedata = _malloc(nfft * 4);
    const cfreqdata = _malloc(nfreq * 4 * 2);
    HEAP8.set(new Uint8Array(timedata.buffer), ctimedata);
    _kiss_fftr(this.fftr, ctimedata, cfreqdata);
    const freqdata = new Float32Array(
        HEAP8.slice(cfreqdata, cfreqdata + nfreq * 4 * 2).buffer);
    _free(ctimedata);
    _free(cfreqdata);
    return freqdata;
  }
  delete() {
    _free(this.fftr);
  }
}
Module.RealFFT = RealFFT;

(You may once again be asking, but why aren’t we checking the return value of _malloc? Because Emscripten is configured to panic if malloc fails instead of returning NULL).

Nou you can recompile the module to include this code:

emcc -o kissfft.js jsbuild/libkissfft-float.a --post-js api.js \
    -sMODULARIZE=1 -sEXPORTED_FUNCTIONS=@exported_functions.txt

And we can run our (transpiled) test code and verify that it produces the expected output:

$ node test_realfft.js 
0: 0 + 0j
1: 0 + 0j
2: 0 + 0j
3: 0 + 0j
4: -4.898587410340671e-16 + -4j
5: 0 + 0j
6: 0 + 0j
7: 0 + 0j
8: 0 + 0j

Cool! We have successfully wrapped a C library in a more or less friendly TypeScript module. If you aren’t planning to publish this module anywhere and don’t care about testing, repeatable builds, or code size, you can stop here.

In the next installment, we will handle setting up CMake to build our module as part of the library build when run under Emscripten, and package the resulting module for NPM (but please don’t actually upload it, since, as mentioned before, an exsting, full-featured module already exists there).