Playing with the Aurora LED Panels

I recently got my hands on a bunch of Nanoleaf Light Panels for a decent price. To be honest, I have been thinking about buying some for some time now, but IMHO they are quite expensive at about 20 Euros per piece.

There is a nice Teardown on EEVblog, but it stops just where things get interesting from a hacker's point of view: the communication protocol you need if you want to control the tiles yourself. I decided to reverse engineer this protocol and to document it in this post.

Physical Layer

Let's first look at the physical layer. Each tile is connected to it's neighbors via a 4 pin connector. The 4 Pins are:

1. Power Supply (24V)
2. Ground
3. Select (3.3V active high)
4. Comm (24V active low, open collector)

Pins 1, 2 and 4 are physically connected between the three connectors on the tile's edges. Pin 3 is of each edge is connected to the tile's microcontroller and can be controlled individually for each edge of the tile.

The Comm Pin

The Comm Pin is used for data communication between the tiles. It is a open collector bus which can be driven to low by any tile. The protocol must take care of collision avoidance. RS232 at 115200 Baud, 8N1 is used for the communication on this pin. Inactive level is 24V, Active level is 0V. The signal is pulled to 24V by the controller. The tiles drive the signal to GND via a open collector circuit.

The Select Pin

The Select Pin is used to enumerate the tiles and to discover topology. Usually the Select Pin is a Ground level, but it can be pulled to 3.3V for each edge of a tile separately if requested by the controller. 

Protocol on Comm Pin

The protocol on the comm pin is not too complicated. The first byte in each command is always the command type. The second byte is always the short id of the addressed tile (0xFF addresses all tiles). The rest of the bytes are the optional command payload. Some commands are followed by two bytes checksum (CRC16 CCIT). Some commands trigger a response by the addressed tile which follows immediately after the command. Some responses also are followed by a CRC16.

Short and long tile IDs

Each tile has a unique 8 Byte long ID that uniquely identifies this specific tile. This long ID is used in the enumeration phase and for topology discovery. In the enumeration phase each tile is assigned a 1 Byte short ID that from then on used to address the tile.

Commands known so far

0x01 - Assign short ID by long ID

This command is used to assign a tile a short ID if it's long ID is known. If the long ID of the tile matches the long ID in the message, the tile sets i't short ID to the short ID in the message (Byte 11) and responds. The message looks like this:

Byte	Explanation
1	Command Byte (0x01)
2	Short ID byte (always 0xFF since addressing is done via Long ID)
3-10	Long ID of addressed tile
11	Short ID to be assigned to this tile
12-13	CRC16 of bytes 1-11

The tile with the Long ID matching the Long ID in the message responds with 80 00 00 if present.

0x02 - Set Edge Pin

This command is used to set the Edge Pins of the addressed tile. The only payload byte is a bitmask of the levels to be set on the three Edge Pins (0x01 / 0x02 / 0x04). There is no CRC for this command.

Byte	Explanation
1	Command Byte (0x02)
2	Short ID byte
3	Edge Pin configuration

The tile with the Short ID matching the Short ID in the message sets it's Edge Pins according to the message and responds with 80 00 00 if present.

0x03 - Read Long ID if selected

This command is used to read the long ID from a tile that is selected by pulling one of it's Edge Pins high from a neighbor tile. A tile only responds to this message if one of it's Edge Pins is pulled high from the outside.

Byte	Explanation
1	Command Byte (0x03)
2	Short ID byte (always 0xFF since addressing is done via Edge Pin)

The tile responds with the following sequence:

Byte	Explanation
1	Response Byte (0x83)
2-3	Unknown
4-11	Long ID of responding tile
12-13	CRC16 of bytes 1-11

0x08 - Fade Tile to Color

This command is used to set the Color of a specific tile or all tiles. A fade time can be given. There is no CRC and no response for this command.

Byte	Explanation
1	Command Byte (0x08)
2	Short ID byte (or 0xFF for all tiles)
3	Red color value from 0x00 to 0xFF
4	Green color value from 0x00 to 0xFF
5	Blue color value from 0x00 to 0xFF
6	White color value from 0x00 to 0xFF
7-8	Fade time MSB first 0x0000 to 0xFFFF

0x09 - Call for unconfigured tiles

This command is used to check if new tiles have been added to the network. Only tiles which have not been assigned a short ID will respond to this command with 80 00 00. There is no CRC for command or response.

Byte	Explanation
1	Command Byte (0x09)
2	Short ID byte (always 0xFF)

0x0A - Check tile presence

This command is used to check if a specific tiles has been removed from the network. The addressed tile will respond to the command with 80 00 00. There is no CRC for command or response.

Byte	Explanation
1	Command Byte (0x09)
2	Short ID byte

0x0B - Set Brightness

This command is used to set the brightness of a specific tile or all tiles.

Byte	Explanation
1	Command Byte (0x0B)
2	Short ID byte
3	Brightness Value from 0x00 to 0xFF

0x0C - Read tile info

This command is used to read some version information from the tile. The version numbers are encoded in ASCII, unused bytes are set to 0x00. The addressed tile responds with its version information and the CRC.

Byte	Explanation
1	Command Byte (0x0C)
2	Short ID byte

The tile responds with the following sequence:

Byte	Explanation
1	Response Byte (0x8C)
2-20	Version Information
21-22	CRC16 of bytes 1-20

Kronos hacking for a broader audience - Step 1: User Interface

In the last few days, the interest in this kronos hacking has increased significantly. Unfortunately this blog isn't worth much to someone without deep knowledge about Linux and other stuff. And on top of that, there is no structure since I just wrote down stuff as I researched.

Marcan (whom I deeply respect for his work on the Wii) just filled this gap with a very nice blog post that summarizes my research and adds a proper introduction for the beginners as well as some detailed background.

As he wrote in this thread, the possibility to provide feedback and to interact with the user is the foundation for a safe and clean way to bring whatever Kronos modifications we might come up with to a broader and less technical audience.

Fortunately Korg's original drivers provide some basic functionality.

The framebuffer /dev/fb1

The kernel module OmapVideoModule.ko provides a framebuffer device that interfaces the Kronos' LCD display. The resolution is 800x600 pixels and it has to be used in 8bpp palette mode. This means that you can choose up to 256 colors that can be shown on the display at the same time.

Since the display is connected to the NKS4 board, the real framebuffer is on the OMAP. This means that all changes in the fake framebuffer on the PC must be sent to the OMAP via USB. There is a special ioctl (OMAPFB_IOCTL_PIXELREGION) to trigger this update for a specific rectangle on the display. Without this ioctl, you will see nothing on the display. Unfortunately, this means that standard applications that work on /dev/fbX will not work.

There is also a ioctl to setup the palette (OMAPFB_IOCTL_PIXELREGION). Up to 256 palette entries can be populated with RGB colors.

Then, there are some less important ioctls:

• OMAPFB_IOCTL_FILLDATA is an accelerated fill command that fills a rectangle with a specified color
• OMAPFB_IOCTL_GETPROGRESS, OMAPFB_IOCTL_SETPROGRESS, OMAPFB_IOCTL_INCPROGRES, OMAPFB_IOCTL_ADDTOPROGRESS are helpers to draw the progress bar that you see when the Kronos is booting. This progress bar is actually handled by the OMAP so that early boot stages can increment it without knowing about graphics.
• OMAPFB_IOCTL_INITLCDREGS, OMAPFB_IOCTL_XAXISBYTESIZE can be used to configure the LCD configuration registers in the OMAP. As the OMAP configures those registers independently of the PC when showing the title screen, we do not need to do that.
• OMAPFB_IOCTL_GETTITLESCREENVERSION returns some version number that is probably used by the updater to determine where to draw it's messages so that they look nice with different versions of the OMAP software.
  

The keypad /proc/OmapNKS4

The kernel module OmapNKS4.ko creates a proc file that can be used to read part of the control surface (Keys 0-9, Enter, Exit and the scrollwheel). The interface is ugly - you can only read one keypress each time you open the file, but at least it works...

A demo application

I have hacked together a simple demo that shows working framebuffer and control surface access here. If you read through the source, you'll find definitions and wrapper for the ioctls and the keypad interface.

Kronos encrypted file systems

Korg uses Cryptoloop to hide the important parts of it's Kronos firmware from prying eyes. The mount mechanism is somewhat obfuscated and hidden in loadmod.ko and loadoa. Here is a brief description:

1. loadoa loads the kernel module loadmod.ko
2. loadmod.ko replaces the kernel's mount syscall with it's own implementation (but only if filesystem check, kernel check and communication with the security chip are ok).
3. loadmod.ko reads and decrypts /.pairFact3 to retrieve the Cryptoloop keys. The Cryptoloop keys are the same for all devices, but the keys to decrypt /.pairFact3 are individual for each device. When you "authorize" your Kronos, you actually get the Cryptoloop keys encrypted with your individual device keys.
4. loadoa mounts issues a mount command that looks like this:
   /bin/mount -n -t ignoreType ignoreDev /korg/rw/PCM/WaveMotion
5. loadmod.ko gets the syscall and recognizes the "magic" mount point name (/korg/rw/PCM/WaveMotion). Now it initializes the Cryptoloop device and calls the kernel's normal mount syscall handler to mount the loop device.

Now, it would be nice to have the Cryptoloop keys for the encrypted file systems inside the Kronos firmware. You can easily extract the encrypted files from the Kronos firmware updates published by Korg. If we just had the keys, we could mount and examine them on any PC around.

The possibility to boot our own kernel helps a lot, here. One could just add some printk() to a function that handles the cryptoloop keys (for example loop_set_status).

This is the kernel patch. Let's see what happens...

loop_set_status(0): lo_file_name=/korg/ro/Eva.img lo_encrypt_type=18 lo_crypt_name=aes lo_encrypt_key=342ee59d549c7d329d835537be0540d
loop_set_status(1): lo_file_name=/korg/ro/WaveMotion.img lo_encrypt_type=18 lo_crypt_name=aes lo_encrypt_key=3e72c0e59fc017a9eb7d7e1168a4cdb
loop_set_status(2): lo_file_name=/korg/ro/Mod.img lo_encrypt_type=18 lo_crypt_name=aes lo_encrypt_key=a336a15cd841ec8926b99e7c3884eaa

Crazy keys

Looking at struct loop_info64 it is pretty clear that the simple printk("... lo_encrypt_key=%s ...", status->lo_encrypt_key) should not have worked. lo_encrypt_key should be binary, not a null terminated string!

Someone had no real clue about how to use loop_set_status() and used 31 (plus a \0 byte for string termination) hex digits (containing only 124 bits of information) as AES256 key.

I am still wondering if this is just a stupid mistake or just one more strange obfuscation. 

A short look into the past: OASYS

The Kronos' history actually began more than ten years ago with the OASYS. Back then, when the OASYS was released, I really wanted to have one, but I also really could not afford it. So I decided to hack other stuff and kind of forgot about the OASYS.

Now, ten years later, I got the chance to look into the architecture and the software of the OASYS and the similarities to the Kronos are astonishing. The architecture is almost the same and I would be tempted to say that the Kronos is in fact a "OASYS Lite V2".

Why "Lite"? Because the Kronos misses many expensive parts (LEDs around knobs/sliders, trigger buttons, large screen, CD drive, ...). It looks like the goal set for the Kronos product engineering was to build a more affordable version of the OASYS (which makes perfect sence, if you look at the price tag of the OASYS).

Why "V2"? Because the Kronos has some features (more engines, sample streaming) that the OASYS lacks. I am wondering why Korg did not upgrade the OASYS software to include those features. Maybe hardware limitations were the reason. The OASYS has no SSD, so disk access latency might be the reason there is no sample streaming. Also, the OASYS has a single core CPU (Pentium 4) while the Kronos has a dual core (Atom). It would be interesting to benchmark the single core P4 against the dual core Atom... (Update: Comparison between OASYS and Kronos CPUs. Looks like the P4 is faster in single core workloads, but the Atom is faster in multi core workloads. A synthesizer is almost a perfect multi core workload, so the Kronos is in fact more powerful than the OASYS).

However, the software and hardware architecture between OASYS and Kronos are so similar that most articles in this blog are - to a certain degree - also valid for the OASYS.

Here are - in no particular order - some things I noticed while comparing OASYS and Kronos:

• Both use Linux with RTAI realtime extensions. However, OASYS uses Linux 2.4, Kronos uses Linux 2.6
• Security stuff (filesystem integrity check, security ic, authorization file) is almost identical
• The OASYS uses the VGA output of the mainboard for it's display while the Kronos uses a display connected to the OMAP. This means that the Kronos has to transfer all display data over USB. I am unsure why they did this - the Kronos mainboard also has a VGA connector, but it is unused.
• In the OASYS the whole control surface (including the keybed) is connected via USB. On the Kronos, the keybed is connected to the mainboard via RS232 - the rest of the controls are still connected via USB. Maybe latency reduction?
• The OASYS uses a custom PCI sound card while the Kronos uses a custom USB sound card for Audio/Midi I/O

Looking at all this, I am wondering why Korg sticks to meaningless upgrades like different colors or different end-plates for it's new/special editions. How about a bigger screen, LEDs for knobs and sliders and some pressure sensitive drum pads? Add a decent mainboard for increased polyphony and call it "platinum edition". KORG, are you listening???

Streamlining the Kronos boot process / what does loadmod.ko do?

I have already sched some light on the Kronos boot process in an older article (What does /sbin/loadoa do?). There you can see that a kernel module called "loadmod.ko" is loaded just before the encrypted loop files are mounted.

loadmod.ko plays a vital role in booting the Kronos and is the center of Korg's attempts to protect the Kronos from hacking.

So, what does loadmod.ko do?

1. First, loadmod.ko performs a file system integrity check. It opens and reads a lot of files and hashes them using MD5. The hash is then compared to a constant value encoded in loadmod.ko. If the values do not match, loadmod.ko will bail out with error code 1.
   
2. Next, the Linux kernel is checked. loadmod.ko calls the functions register_cdrom() and init_cdrom_command() with some magic values. A normal Linux kernel will just return error codes because the parameters make no sense. The Linux kernel delivered by Korg has modified versions of this functions that return other magic values to signal loadmod.ko that the kernel is "genuine".
   
   Korg has not published the source code for those modifications to the Linux kernel even though they are distributing a binary version of Linux with those modifications. This is a clear violation of the GPL.
   
   You can find a more detailed description of the kernel authentication mechanism in About the Linux kernel used in KORG Kronos.
   
   I provide a patch that reimplements the modified register_cdrom()/init_cdrom_command() functions here. With this patch you can compile your own kernel that will be accepted by loadmod.ko.
   
   If the kernel check fails, loadmod.ko will bail out with error code 3.
   
3. After filesystem and kernel have been checked, loadmod.ko tries to read the authorization file called "pairFact". This file contains the encryption keys for the encrypted file systems in the Kronos (detailed explanation). If loadmod.ko cannot read that authorization data, it will bail
   out with error code 4.
   
4. The contents of "pairFact" are encrypted with a key stored in the security IC. So, the next step for loadmod.ko is to talk to the security IC (the primitives for the communication are provided by OmapNKS4Module.ko. The Public ID of the Kronos is read and the key needed to decrypt the authorization data is read. If communication with the security IC fails, loadmod.ko will bail out with error code 5.
   
5. If all checks were successful, a magic value (0x22FB39CC) is stored at a certain address in kernel memory. This magic value will be checked by the synthesizer module (OA.ko) later. If it is not there, OA.ko will skip some important initializations which will cause the sound output to be bad. So even if you do not need loadmod.ko anymore because you have copied the synthesizer files out of the encrypted file systems, you still need to run loadmod.ko (or at least write this magic value) to have useful sound output.
   
6. The error code (or zero if there were no errors) from the checks above is written to "/tmp/stgStatus". This is used by /sbin/loadoa to display a more or less useful error message (or the reauthentication screen) if one of the integrity checks failed.
   
7. Now, things are getting crazy: loadmod.ko replaces some of the kernel's syscalls by overwriting sys_call_table. The following syscalls are overwritten: sys_mount, sys_ioctl, sys_oldumount, sys_umount, sys_create_module, sys_init_module.
   
   The syscalls sys_mount and sys_oldumount are replaced with versions that "magically" mount a cryptoloop filesystem if one of the magic paths (/korg/Eva, /korg/Mod, /korg/WaveMotion) is mounted.
   
   The other syscall replacements don't do anything but call the original handlers. It looks like Korg wanted to prevent users from loading additional modules (maybe to prevent access to the kernel memory), but that mechanism was not finished.
   
8. Finally, a kernel thread is started. This kernel thread will handle system updates by listening on /proc/.update and running /sbin/UpdateOS or /tmp/UpdateOS as a usermode helper if necessary.

So, why is that interesting?

From a users point of view, loadmod.ko doesn't do many useful things. The filesystem integrity checks take precious boot time and prevent you from changing your file system. The encrypted loop file systems also degrade system performance and you can easily copy their contents onto the unencrypted file system, 

However, if you decide to save boot time and trouble by getting rid of loadmod.ko, you still have to write the magic value or else the Kronos' sound will be crippled.

I have updated my kernel patches and my precompiled kernel to do that, Using this kernel, you do not need to load loadmod.ko. You can get rid of the filesystem/kernel integrity check and still OA.ko will work perfectly.

Kronos Polyphony and Performance

As explained in many discussions, the Kronos has no fixed polyphony. The number of sounding voices differs depending on the engine used for each voice and even depending on which effects are currently active.

But how does the sound engine know how many voices can be active? 

The answer is a file called CostProfile in /korg/rw/Startup on your Kronos (this file is not included in the filesystem integrity check, so you can modify this file via FTP/SCP after rooting your Kronos. In this file, the resource cost is stored for each engine and for each effect. The engine uses this file to constantly keep track of the current resource usage. As soon as there are not enough resources for all voices and effects that should be running, some algorithm carefully selects the least audible voices and drops them to make room for new (probably more audible voices).

So, if you upgrade the CPU power of your synthesizer, you still need to patch CostProfile to get increased polyphony. I have prepared a CostProfile for my hardware upgrade here. The proposed setup has enough power to archive 200 note polyphony across all engines and regardless of the active effects,

Patching CostProfile to reduce the resource cost on the original main board (i.e. without additional CPU power) leads to some interesting behavior when too many voices are triggered: instead of stuttering and/or clicking, single voices drop out at random. This is because of a second mechanism called EmergencyVoiceStealer that seems to just drop one or more voices as a last measure to prevent a complete audio dropout. It looks like Korg spent a lot of effort to make sure there are no hard audio dropouts. Wow!

Finally, there is a hard limit of 200 voices. This limit is probably a #define in the synthesizer source code and was set arbitrarily by Korg.

Precompiled kernel for the Kronos with XHCI support

If you want to roll your own hardware upgrade, and don't want to dive into the details of compiling the Linux kernel, use this precompiled Kronos-compatible kernel with XHCI support.

The kernel was compiled from the ingredients explained here. All patches applied to the Linux kernel 2.6.32.11 to compile the binary linked above are available here.

Update (April 3rd 2016):

• Enabled IPv6 in kernel config (required for Intel NIC drivers)
• Fixed problem with selecting OA.ko's heap memory size (orig_mem_size)
• Driver modules for both Intel NICs on the Supermicro X11SSV-Q (Compiled from e1000e-3.3.3.tar and igb-5.3.4.4.tar.gz).

Linux Kernel

Applied Patches

Driver Modules for Intel NICs on Supermicro X11SSV-Q (extract to /lib/modules/2.6.32.11-korg)

Update (May 5th 2016):

• Added magic value to register_cdrom() so that OA.ko still works properly if loadmod.ko was not loaded.

Linux Kernel

Applied Patches

Have fun!