MMO: Lightning fast webpacking packing for Kotlin UMD modules

Jul 27 2018

For my Korlibs and my MMO, I was using UMD JavaScript module system. That module system is compatible with AMD, CommonJS or plain JS loading in order.

I tried to use webpack, but it was pretty slow even in dev builds and added another step to the configuration. Also wanted to serve the gzip file.

So I did a quick and dirty hack to do this.

First I checked the JS entrypoint, then searched for all the require(‘module’) in the file to locate all the dependencies of the entrypoint and dependencies. With that graph, I did a topology ordering (a slow buy simple implementation for just a bunch of dependencies). Then I concatenated all the files in memory in a string, generated a bytearray and compressed with gzip level 9 in a bytearray. Then I’m serving one of those bytearrays depending on the Accept header from the client.

The topology sort is used for sorting dependencies. It work on a digraph without cycles. The unoptimized but simple version looks like this:

The idea is to grab one node with no outgoing edges (without dependencies), place in the output, and then remove all the incoming edges to that node and repeat.

To optimize this we can use a PriorityQueue that keeps the number of nodes as ordering algorithm, and a list of edges per node to remove them quickly.

This algorithm is always possible for graphs without cycles.