World Music Visualisation Aggregator Platform - Musical Immersion

I suspect musical immersion means different things to different people. For the online guitar learner, audio, notation and synched video or an instrument finger- or keyboard display may be sufficient. A performer would presumably be content with the in-ear audio feed from other band members, whereas a session player would put the emphasis on direct eye-to-eye contact, the immediate instrumental constellation and the musical craic.

For all others: the world musicians interested in exotic instruments, the advance learner thirsting for immediately relevant theoretical knowledge, the virtuoso ethnic teacher keen to reach out to the world, it's a very different story.

With the best of current instrument learning platforms tied to specific instruments and fingerings, only the most basic of notation interactions available, little means of interrogating graphical screen objects, music theory still pretty much locked away in books or standalone apps, and ethnic virtuosos struggling with video chat's lack of tool support, we are still far removed from any notion of online musical immersion. The time is ripe for a data-driven aggregator platform bringing all the learning strands together.

Musically economic and focussed platforms such as Soundslice are a step in the right direction, but are not equipped to handle the wider diversity and interchangeability of world music instruments. Indeed, they will have huge trouble scaling from guitar (their primary target) to even a small range of other popular instruments.

A data-driven, single page application (SPA) carefully constructed, has the potential to bring immediate and immersive musical teaching and learning to both mobile and fixed environments, and online and remote teaching and learning. And that for nearly all world music instruments.

Big, brave, open-source, non-profit, community-provisioned, cross-cultural and pine tar crazy. → Like, share, back-link, pin, tweet and mail. Hashtags? For the crowdfunding: #VisualFutureOfMusic. For the future live platform: #WorldMusicInstrumentsAndTheory. Or just register as a potential crowdfunder..

Why A Music Visualisation Aggregator Platform?

Mobile Fragmentation vs Desktop Aggregation

The majority of (tongue in cheek: 'legacy') graphical mobile music apps necessarily focus on a very limited range of features. There are several reasons for this:

• limitations on mobile bandwidth and processing capabilities leading to a 'domain fragmentation' or 'app' mindset
• the lack of mobile device frameworks capable of fluidly handling large numbers of graphics objects
• the ongoing lack of a lightweight, fully accessible and addressable, in-browser scalar vector graphics (SVG) based notation framework
• the inability (lack of APIs) to feed musical information to other applications during playback.

Musical Immersion

Originally, mobile device manufacturers had to find a way of making lacklustre processing power and bandwidth acceptable to end users. Enter the app.

This essentially brought about a domain fragmentation, a dumbing-down both in application scope and, hence, the user. The accustomed drip feed, but in another guise.

Mobile vs fixed broadband speeds remain a bummer, but -at least under high-end mobile data plans- the worst of onboard processing restrictions are slowly coming to an end.
Current music app offerings, nevertheless, still tend to focus on just a few of many potential musical uses. This is an area of opportunity, but -thanks to the 'app store' distribution model, only a tiny proportion of developers benefit. 

Current offerings are pretty much limited to:

• possibly multipart notation + chords
• single-voice notation plus time-synchronised video
• audio + theory tool

Musical Immersion

Potential Crowdfunder? First Name Email Address

With the steady advance of web standards, the raw beauty and power of data visualisation libraries and improvements in processing power, the conditions are in place to fuel a wave of aggregation and growth into fully immersive, device-independent web applications.

It is entirely possible to build the entire front-end of a Single Page Application (SPA) using javascript and a little CSS in conjunction with a visualisation library such as D3.js, leaving heavy-duty, resource-hungry frameworks untouched.

The criteria for this are:

1. underlying data structures reflect a natural, indexed order or classification, allowing referred to by URLs (physical addresses) rather than URIs (abstract properties serving as, for example, search terms). This eliminates any need for routing and hence much of the overhead associated with frameworks.
2. rather than rely on prepackaged and possibly redundant GUI libraries, place full trust in D3.js's ability to populate and manipulate complex graphical objects (including menu structures) in the browser's DOM (domain object model).
3. rigorous attention to reuse in the instrument model and theory tool construction hierarchy
4. keep it simple: complexity makes idiots of us all.

With base functionality established, the question of integration into the mobile apps landscape can be addressed. With clean and consistent interworking between score and the various models and tools established within the browser, this should not be too difficult.
Nevertheless, given the pace of advance, innovators may have to plan for technology mature only in a few years time. Current data visualisation libraries, for example, are still poorly supported on mobile devices, even if such barriers tend to fall quickly.

It is in this sense that this project was started. A decision was made to exploit the full power of the workstation-grade browser DOM through the latest in data visualisation libraries, and cross other bridges when as and when they arrive.

Immersion Goes Remote

Though built to test feasibility rather than seduce viewers the above video gives a taste of the potential for more immersive music applications.

It combines not just notation and an instrument model, but a music theory tool (circle of fifths) and a representation of the underlying physical waveforms (using Teoria.js).

All are intimately synchronised with the score during playback. This is an entirely in-browser, single page application.

The final platform will be entirely customisable to preferred instruments and theory tools, will incorporate sound, and, ultimately, allow P2P sessions with other users across the internet parallel to a video link.

This opens up the possibility of direct, person-to-person teaching and learning across geographical, musical, cultural and timezone boundaries.

What ARE we waiting for..? :-)
The online HTML tidy tool lets you compose your articles easily in your web browser without downloading and installing other programs and HTML tools.

---

Keywords

---

online music learning, online music lessons	distance music learning, distance music lessons	remote music lessons, remote music learning
p2p music lessons, p2p music learning	music visualisation music visualization	musical instrument models interactive music instrument models
music theory tools musical theory	p2p music interworking p2p musical interworking	comparative musicology ethnomusicology
world music international music	folk music traditional music	P2P musical interworking, Peer-to-peer musical interworking
WebGL, Web3D, WebVR, WebAR	Virtual Reality, Augmented or Mixed Reality	Artificial Intelligence, Machine Learning
Scalar Vector Graphics, SVG	3D Cascading Style Sheets, CSS3D	X3Dom, XML3D