Virtual Music Jamming with Low Latency Networked Systems

85 views
Download
  • Share
Create Account or Sign In to post comments
#audio systems #audio recording #low-latency networks #multipath TCP #forward error connection #SoC #consumer electronics #virtual meetings #music jamming

This video discusses the technical issues required for virtual music jamming, demonstrates how to implement it using inexpensive hardware and open-source software, discusses limiting factors, and proposes future technical advances.

With covid-19 restrictions on indoor gatherings, it has been difficult for musicians to get together for in-person musical performances or practice sessions. However, there are a number of networked system applications that potentially allow on-line "virtual" jamming. The key to good performance is low-latency network connections and audio systems with low-latency ADC/DAC components and computationally efficient algorithms for network communications. Another stumbling block for many musicians is that the technology is often hard to understand or use. Working with John Lombardo, an IEEE-Madison member and leader of a local Jazz group, SwingTime Music, Tom Kaminski built and tested a number of "musical appliances" based on the Raspberry Pi with sound cards or USB connected DAC/ADC boxes.

In addition, Anton Kapela and 5Nines, a Madison-based data technology company, put up virtual servers and helped establish low-latency routing from local musicians to the server. The result is an acceptable level of delay that supports virtual jamming. A second goal was to develop a simple appliance that students of music could use with little or no fuss and that could be controlled by a simple browser. That goal was met by using a pre-built operating system (Jambox) with real-time preemption and a VNC interface that is accessed with a browser.

The system supports both client/server jamming and peer-to-peer jamming without having to configure complex firewall routers on the typical home ISP box. It does, however, have to be connected via a wired Ethernet cable for lowest, consistent latency. In technical part of the presentation, Anton Kapela speaks to the general question of "what causes latency?" Major contributors include oversampling ADC/DAC techniques, USB, SPI, popular CODECs, and other input/output components of modern computer systems, as well as various software abstractions to use them. Almost unavoidably, latency creeps in. A

nton also discusses the current and previous work in various CODEC and forward error correction spaces with regards to forms of reliable and multi-path networks. Finally, he touches upon the latest work in Network Coding going on at MIT and Multipath TCP within the Internet Engineering Task Force.

This video discusses the technical issues required for virtual music jamming, demonstrates how to implement it using inexpensive hardware and open-source software, discusses limiting factors, and proposes future technical advances.

Speakers in this video