Suggestion: rewrite file format handlers in Rust

[perlun]

Hi @JHalme, first I want to say thank you for an incredibly nice little thing here. 😄 And thanks for making it FOSS!

I guess I'm feeling a bit nostalgic right now, recorded an .avi of an old MSDOS intro I made 20 years ago and put up on YouTube the other day. And now I'm looking into making a web port of this intro, just for the sake of it...

I integrated the player in my web app quite easily, using this code (documenting this since it might help others who also want to integrate the player into their intros/whatever). Only the files below are needed if you don't want the UI; no jQuery etc is necessary.

<script src="webaudio-mod-player/utils.js" type="text/javascript"></script>
<script src="webaudio-mod-player/player.js" type="text/javascript"></script>
<script src="webaudio-mod-player/pt.js" type="text/javascript"></script>

And then in a button event handler I kick it live like this:

window.module = new Modplayer();
window.module.setrepeat(true);

// Because the file gets loaded asynchronously, we need to wait until it's ready before we
// start playing the mod file.
window.module.onReady = () => {
  window.module.play();

  // ...start my animation etc, which is handled by some Rust code.
}
window.module.load("music.mod");

This all works quite well, and the performance is satisfying (only tested on Chrome on desktop yet.) With the https://mod.haxor.fi/ player, I noted some files (probably more complex ones) were stuttering when I played them on my iPhone - like you write on the web page, the Javascript runtimes can very greatly in speed.

---

I think we should consider rewriting the inner loops here in Rust, targetting WebAssembly. As you probably already know, webassembly is supported in all the bigger browsers these days and it's providing a lot better performance than JavaScript will ever be able to deliver, because of its compiled and statically typed nature.

So, I think we should think about this. Maybe this is something you have already thought about? Or what do you think? This demo from hellorust.com is what inspired me to start playing around a bit with WebAssembly and Rust, and it's actually really fun and works well. You have to use a nightly Rust compiler at the moment (webassembly is not supported by the stable releases yet) but apart from that, it works pretty much flawlessly and provides a really nice & capable platform for "low-level web stuff" - which .mod playing happens to be.

If you have a particular file handler (S3M? XM?) that is more performance-demanding than the others, this could perhaps be a suitable candidate for trying this idea out.

All in all, thanks for a great tool now already! 👍

[electronoora]

Hello, and thank you for the kind words - I really appreciate it! :)

I am aware of the performance issue, and it actually seems be more frequent on desktop Chrome in comparison to macOS Safari or Firefox Quantum. Some XM modules with a large number of channels are especially prone to it, although the stuttering seems to always occur at particular positions in the song. As you also mentioned, I think that it is due to the heavy inner loop in the software mixing and interpolation code. It becomes even more apparent when switching the audio output sample rate from 44/48kHz to 96kHz.

Currently, Firefox seems to be the browser least affected by the issue. It seems that different browsers might have different priorities for the thread running Web Audio ScriptProcessorNode.onaudioprocess -callbacks.

As it turns out, I too had been thinking about converting the mixing code into using WebAssembly, although my plan was to use a language more familiar to me - plain old C. :) It's more of a gut feeling at the moment, but I think that converting the channel loop inside .mix on each player into WebAssembly would provide a marked improvement in performance.

As soon as I have some more time to work on this project, I'll start looking a bit closer into experimenting with this - perhaps even before continuing with .IT player code.

[RyanBram]

Great news to hear.
Thanks @JHalme .

Out if curiosity, is it possible to just use C code from this https://github.com/cmatsuoka/libxmp/tree/master/src/loaders with little modification and convert it to asm.js or web assembly.

[perlun]

As it turns out, I too had been thinking about converting the mixing code into using WebAssembly, although my plan was to use a language more familiar to me - plain old C. :) It's more of a gut feeling at the moment, but I think that converting the channel loop inside .mix on each player into WebAssembly would provide a marked improvement in performance.

I have no problem with this; it's your project so you can decide. What I like with Rust is that it's more modern, potentially more secure (unless you use a lot of unsafe blocks...) and the tooling is also simpler. No need for emscripten, which is nice - I have a little toy repo you can look at here if you like: https://github.com/perlun/gameland-wasm

Having that said, I won't mind if you decide to go with the "plain old C" approach. Like @RyanBram implies, it may actually mean that we could port code from libxmp with little effort, which could have seriously very nice implications... (caveat: haven't actually looked at this at any detail at all)

[perlun]

For reference, there is actually another player which already has an asm.js version: https://github.com/deskjet/chiptune2.js/

(it's based on a C/C++ library, compiled to asm.js using emscripten.)

I don't know how feature complete chiptune2.js is compared to webaudio-mod-player.

[perlun]

FWIW, I compared the performance of these two a bit. webaudio-mod-player is a bit more CPU consuming, but still doing pretty good for a pure-js implementation.

I played Daddy Freddy's "mod.are_you_excited" with chiptune2.js (using their online player at https://deskjet.github.io/chiptune2.js/ which is pretty nice, you can drag-and-drop a local .mod file to the web page to play it) and the mod.under_the_bridge file at https://mod.haxor.fi/Daddy_Freddy/mod.under_the_bridge, with visualizations disabled.

chiptune2.js seemed to be using (for the FirefoxCP Web Content process) between 0.5-8.0% CPU. It would actually often lay below 1% or below 3%, which is rather impressive - this is with asm.js, so a pure webassembly compilation should potentially be even faster.

webaudio-mod-player seems to use around 6-9% for me, with the tab not visible in the browser (which means the remaining visualizations shouldn't have to render to the actual screen, potentially reducing the load a bit.)

Here is the call tree including some usage metrics. @JHalme, this supports your suggestion that just porting the mixing code (pt.js in this case) to WebAssembly would be a good starting point for any perf. improvements or investigations.

[electronoora]

That's an excellent analysis and comparison - thank you!

The lack of interpolation and volume ramping makes the Protracker mixing code perform reasonably well, but ST3 and FT2 mixing hits the CPU quite hard. Ping-pong looping in FT2 code also has lots of branches and nested contexts, so it could definitely use some improvement.

Some XM modules seem to be especially prone to stuttering. These two, for example:

https://mod.haxor.fi/Hunz/clone_it.xm
https://mod.haxor.fi/Mystical/system_override.xm

Intrestingly, the stuttering seems to always occur at the same position within the module - for clone_it.xm, it is just before position 0x0a. I noticed that at least on Chrome, the heap usage goes absolutely bananas right at that position:

So using Webassembly for the mixing code may also help in providing finer control over memory allocation and when GC is allowed to run.