Music Visualisation: Technical Details

From a creative freedom point of view, there are two important foundational elements in a world music visualisation platform such as described here.

Potential Crowdfunder? First Name Email Address

These are that the core musical notation is 1) entirely data-driven and 2) scalar vector graphics (SVG) based.

With these in place, we have a technical foundation for -on the one hand- data manipulation, labelling and presentation freedoms, and on the other a host of transformations such as scaling, zooming, panning, rotation, flipping, reordering, colouring, mapping and visual linking.

With the web stack as context and visual creativity central to the whole concept, an end-to-end javascript-based approach is also implied. With few suitable music parsing libraries available and their scope often limited to western 12-tone equal temperament, much must be implemented more or less from scratch.

Here we give a brief glimpse of some of the guiding principles underlying the platform.

Despite a general dislike on my part mixing musical concept and technology, this post gets a little technical in places.

Big, brave, open-source, non-profit, community-provisioned, cross-cultural and gobsmacked crazy. → Like, share, back-link, pin, tweet and mail. Hashtags? For the crowdfunding: #VisualFutureOfMusic. For the future live platform: #WorldMusicInstrumentsAndTheory. Or simply register as a potential crowdfunder..

---

P2P Interworking

---

The platform in focus here will provide support for video chat teaching. This implies some form of interworking/control messaging, of which WebRTC (open source) or Firebase (proprietary and at cost) are candidates.

WebRTC is open-source, entirely free, is designed for use by web (as opposed to -say- VoIP) developers (familiar syntax), is supported natively in most browsers, is already being used by hundreds of vendors (including Facebook and Whatapp), and (though of questionable relevance to our visualization aggregator platform) even has some mobile support.

There are already a number of potential open-source toolkits available in this area, but a 'standout' feature among some is the concept of distributed WebRTC servers - hence with no single point of control or failure. An example of such an approach can be found at Matrix.org. More on basic P2P music communications as relates to our visualization aggregator platform >here<. Nevertheless, watch this space..

---

Geekspeak (Technical Delights)

---

Flexible Reuse Hierarchies Implement Once. And Only Once.

On-Demand Loading, Dynamic Change For The On-Demand Economy

Consequently Data-Driven Unparalleled Presentation Freedoms

---

The impact of a data-driven approach on musical notation, instrument models & theory tools is profound. It opens a vast, subtle compendium of musical differentiations and transformations - which we can only hint at here. The best is undoubtably to come.

---

Features Some Achieved, Some Still Adrift

Leveraging Instrument & Tool Data Trees Coding Examples? Post-Crowdfunding :-)

Slice & Dice The Data With D3.js - Any Way Ya Want

---

Simple Core Architecture

"The solution to complexity is abstraction, also known as information hiding. Abstraction is simply the removal of unnecessary detail. The idea is that to design a part of a complex system, you must identify what about that part others must know in order to design their parts, and what details you can hide. The part others must know is the abstraction". - cs.cornell.edu

The DOM-based structures as shown in the banner image at the head of this blog are at heart built around a few simple, interlocking mechanisms. These allow us, from raw data, to build static, on-screen models which can then be overlaid with fully dynamic, score-driven and synchronised animated graphics. Despite their visual complexity, with only few parts moving at any one time, processing overheads remain manageable.

There is nothing mysterious about the core architecture, indeed it is much like a number of other music platforms featuring simple interactive instruments such as piano keyboard or guitar fingerboard.

What really distinguishes the system, however, are on the one hand a consequent approach to musical property abstraction and reuse, and on the other, the mechanisms for personalisation: unhindered instrument model and theory tool selection and exchange. Though a substantial open source modeling phase will be needed to populate the platform with instruments and theory tools, there are clear underlying structures and processes. Here their (modeling) core:

World Music Aggregator Platform: The Foundations Of Animation
(Further Animation Types -Esoterica, Visual Storytelling, Psychophysics etc- Can Be Added At Will)

The mechanisms for the interactive classification, discovery, selection, population, synchronization, sharing and person-to-person interworking will have to wait until after the crowdfunding. Patience..

Musical property reuse, on the other hand is so simple, it can be summed up in one diagram:

Musical Property Abstraction And Reuse
In Instrument And Theory Tool Modeling

Key Achievements So Far

To round up this post, here some of the key technical problems overcome so far, using a 'red, amber, green' traffic light metaphor to represent progress. Left unresolved, any one of these areas would severely impede the platform's usefulness. Keep in mind these represent 'proof-of-concept' and nothing near production-ready states.

Basically, the further towards the top right hand corner of the diagram, the closer one is to an well-defined animation of real, practical use in supporting P2P interworking. Clearly the interworking controls are a topic of key future interest.

In as far as useful to open source integration work, further details will be revealed following successful crowdfunding and initial release.

Itching For Detail

For those with a slightly more persistent itch for detail, try this description of some of the technologies underpinning online gaming. In the long run, these are likely to feature prominently in our implementation.

---

Keywords

---

online music learning, online music lessons	distance music learning, distance music lessons	remote music lessons, remote music learning
p2p music lessons, p2p music learning	music visualisation music visualization	musical instrument models interactive music instrument models
music theory tools musical theory	p2p music interworking p2p musical interworking	comparative musicology ethnomusicology
world music international music	folk music traditional music	P2P musical interworking, Peer-to-peer musical interworking
WebGL, Web3D, WebVR, WebAR	Virtual Reality, Augmented or Mixed Reality	Artificial Intelligence, Machine Learning
Scalar Vector Graphics, SVG	3D Cascading Style Sheets, CSS3D	X3Dom, XML3D