NetHarmony on M5Stack Core2

* Thanks for the source code and project information provided by @リス子 @lalalaroot196

Story

Development material

System configuration

System

M5Stack Core2
USB MIDI Instrument

NetHarmony is an ensemble support system using the Internet. This time, we used M5Stack Core2 to verify the concept and partially implement it.

Each terminal plays the received MIDI signals and sends them to other performers. It also plays a sound when it receives MIDI signals from other performers. By doing this with low latency, remote ensemble performance becomes possible.

Development Background

There is an online game that has a music performance function, but I was disappointed that there was a lot of delay due to the communication being done through the game server, and I couldn't play together with people who were far away.

Development began with the idea that if musicians could communicate directly with each other (p2p), they could perform together even if they were physically separated in the network environment.

That's the start of NetHarmony development.

What's special about NetHarmony?

In order to achieve a comfortable ensemble performance, we implemented it with low latency. I2S sound generation also takes advantage of the ESP32's dual-core feature to minimize the buffer and minimize the delay from signal reception to vocalization.

Future plans - What's next?

Although we were able to prove the concept this time, there is still room for improvement in sound quality and polyphony. Currently, there is a limitation due to the slow PSRAM, but M5Stack S3 (ESP32-S3) has dramatically increased speed, so a significant improvement can be expected.