World Music Visualization Aggregator Platform: Key Features

We have the foundation for a music visualization (or visualisation) aggregator platform. What are it's key characteristics, and how do they benefit learners?

A number of platforms have begun to offer online (aka P2P, peer to peer, remote or distance) music lessons, but none so far have been able to offer a supporting toolset - providing a wide range of interactive, score-driven world music instrument models and music theory tools.

Remote Or Distance Music Learning

Those that succeed in bringing such tools to online world music teaching open the doors to highly interactive technical music visualization, an music-cultural avalanche, a revolution in comparative musicology, and will make a huge contribution to cultural understanding. Just as importantly, they will bring hitherto unknown freedom of choice.

Choice Of Notation	Choice Of Instrument	Choice Of Theory Tool
Choice Of Mentor	Choice Of Supporting Tools	Color Prefs etc

Peer To Peer Music Teaching And Learning Via Video Chat

In remote interworking between a teacher and learners, clearly simultaneous play will long remain a pipe dream, but a two-way musical conversation lies well within capabilities, offering the opportunity to listen, absorb, repeat - and, of course, question.

This dialog mimics the style of conventional (offline) musical workshops, where students are accustomed for example to exploring melody, fingering and chord progression alternatives, getting a tighter grip on the specifics of style and ornamentation, or absorbing a wide range of anchoring cultural information. These important and personal aspects of musical immersion are often ignored in many conventional online video courses.

The tools enrich this by on the one hand taking much of the effort out of preparation, and on the other generating an immersive learning environment and then driving it using the notation. The system can be considered "peer-to-peer" in that user environments are directly linked by shared controls. Notation, instrument model and theory tool behaviours on one end of the connection are shared with the others.

GOAL: A Toolset Supporting P2P (i.e. Remote) Teaching Via Video Chat
With The Virtuoso At The Musical Epicentre

Ease of configuration and tight synchronization between the various forms media are central, but beyond that, our focus must be teacher, student, music (exercise), and instrument. Simplicity and transparency is all.

Judging by the lack of response to what has been documented so far, there is some scepticism to be overcome. I take heart, though, from the lessons from the past. :-)

A Galaxy Of Supporting Music Visualization Possibilities

If the core visualizations or animations are notation, instrument models and theory tools, secondary to these (we can imagine them as clustered around them) are supporting applications, handling, for example, teacher, student, band and other affiliations, and related planning or scheduling tools.

A third (and outer) zone of relevance would be that of related integrations - in which we can include standalone applications derived from core platform components. Here we might encounter integrations focussed on the impact of music on the psyche, esoterica such as music-astrological mappings, animated dance instructions, storytelling, or simply personal 'Internet Of Things' preferences associated with specific learning environments or pieces of music.

World Music Online Learning Platform Integration Zoning
Core, Supporting And Embedded Applications

With open source (commons-based peer-produced) models, the only limitation is the collective imagination. What one mind can visualise and articulate, others will be able to implement.

P2P Music Learning: Simple, Flexible, Personal

It's not rocket science. The development tools and libraries central to a comprehensive world music teaching platform such as this have for the most part long been proven. Scalar vector graphics (SVG) has, for example, been around for approaching two decades, and has long been supported in all major browsers.

Supporting Peer To Peer Video Chat Teaching
Core Platform Features

The truly transformative link in our aggregator platform's technology chain is the visualization library D3.js, which provides a consistent and DOM-affine foundation for SVG manipulation.

With a surprising economy and elegance of expression, D3.js provides the means to dramatically augment the range of tools available to teachers and learners. D3.js tends to hide tedious detail (the 'how to do'), allowing developers to concentrate on tasks (the 'what to do'). It has a reputation for a steep learning curve, but this is dramatically eased with access to the code for established, open source instrument and theory tool models.

World Music Online Learning Platform: Simple Core Architecture

The DOM-based structures as shown in the banner image at the head of this blog are built around a few simple, interlocking mechanisms. These allow us, from raw data, to build static, on-screen models which can then be overlaid with fully dynamic, score-driven and synchronized animated graphics.

Taking no account of the mechanisms for instrument and theory tool exchange, we can view the system much as shown below. This view is trivial in the sense that it resembles that of a number of existing notation systems offering basic instrument (guitar, piano) display. Any similarities end in the detail, however, as in place of an Java / XSLT / XSL / XDom / XQuery coding constellation, we are working directly with javascript, D3.js and a couple of small helper libraries.

World Music Aggregator Platform: The Foundations Of Animation
(Further Animation Types -Dance Moves, Esoterica, Visual Storytelling, Psychophysics etc- Can Be Added At Will)

What really distinguishes the visualization aggregator platform as shown in the banner image are the mechanisms for personalization and comprehensive instrument model and theory tool exchange. Again there are clear structures and processes to these which dramatically extend the above model. In as far as useful to open source integration work, these will be revealed following successful crowd funding and initial release to the open source community.

Big, brave, open-source, non-profit, community-provisioned, cross-cultural and batshit crazy. → Like, share, back-link, pin, tweet and mail. Hashtags? For the crowdfunding: #VisualFutureOfMusic. For the future live platform: #WorldMusicInstrumentsAndTheory. Or simply register as a potential crowdfunder..

World Music Instrument Models - Diversity In Surfeit

Instrumental Diversity
Of A Vast Global Heritage, A Tiny Selection

With time, the platform can be expected to feature score-driven, interchangeable models of almost every world music instrument -of which there are certainly many thousands- and theory tools -of which there are hundreds- and allow straightforward behavioural comparison between them. How is this possible?

World Music Instrument Models: As Few Steps To Any Instrument Configuration

Using the 'lute' instrument family as an example, here we see how we might configure a Turkish bağlama or saz:

Closing Online Modelling Gaps
Step-By-Step To Hundreds Of Configurations (Analagous Process For Other Instruments)

Modelling steps can be expected to mimic historical instrument construction and evolution. For simpler instrument such as those of the lute family in equal temperament (the default), configuration can be completed in as few as five steps. More complex instrument families may require a few more, but will always be manageable.

Complex wind instruments such as the modern flute, clarinet and saxophone can be expected to present among the greatest challenges, but manufacturers may be willing to assist. Our goal here is pragmatic visual modelling - and certainly not the fine tolerances associated with manufacturing.

World Music Instrument And Theory Tool Visualization: Social Value Wellspring

Each configuration can be saved for immediate, general use by the community. In this way, any generic instrument base can be used to generate hundreds of configuration variants subsequently available to the community via the classification trees (visual hierarchies) accessible only through the top-level menus row.

Non-Profit: Social Value Wellspring
Rote Reinvestment, No Wealth Hoarding

These entirely dynamic, interactive models represent a wellspring of social value in three distinct ways:

1. used directly on the aggregator site, they provide a secure base toolset for remote P2P teaching by anyone
2. embedded in an own, external site, they will allow users to build custom offerings, nevertheless around proven models
3. because the models are to be open source and liberally licensed, they can be reused and adapted at will

The former will further understanding and community among musical cultures ('comparative musicology'), the latter two will allow anyone to develop and showcase own teaching content around specific instrument models. This might include entire instrumental courses, where even the simplest of exercises drives networked and synchronised understanding across animated notation, instruments and theory.

MusicXML: Serving The 2D And 3D Web

Potential Crowdfunder? First Name Email Address

The exchange file format 'MusicXML' represents a piece of music as data in a serialised form suitable for streaming. In this sense each exchange file can be thought of as a rather wilfully structured database.

In contrast to a conventional database, though, to get 'useful' data (a melody line, the notes suggesting a chord), you are obliged to reconstruct the score and apply filters to it. Depending on the intended application, these may target (say) a score part associated with a specific instrument, a note selection at a particular position during playback, the scope of a loop, the tonal delta between two notes, or indeed the range of dynamics glyphs applied.

In this sense, the filters act as a form of database query, the score something of a query playground. Literally anything can be selected.

Note: there is another approach using XSLT stylesheets to query streaming MusicXML with XPath expressions. As this would be more difficult to steer and we are at no point dealing with actual streaming MusicXML, this approach is discarded.

The decisive factor is likely to be user convenience and freedom of choice: it is easier to handle filtering visually, on a reconstructed score. D3.js's selection mechanisms do just this - flexibly, precisely and concisely. Filtered data is then passed to dependent animations.

This data could also, however, be passed -via API- to other, external, applications.

Music Exchange File As Data Source For 2D and 3D Web - Overview
DOM or WebGL-Based

The platform has the potential to exploit three transformational technologies: music visualization, artificial intelligence, and augmented or virtual reality. The above diagram attempts to illustrate use of MusicXML in two of these scenarios.

On the top left-hand side we have predominantly (but not exclusively) 2D music visualization, which are score-driven animations of music instrument models and theory tools built using SVG (scalar vector graphics) on the browser's DOM (domain object model). On the top right, we see how the same base musical data might be used to drive dramatic or gamified 3D narratives in a WebGL-based virtual world.

There are many possible configurations (directly via the API, using an embedded and interactive score display, or indeed app to app), but the central point is simply that any selections made on the notation could be shared with the 3D WebGL environment and hence fulfil considerably wider needs.

World Music Online Learning Speed And Focus

The brain is by all accounts a remarkably elastic organ. Given the right field of specialization and learning environment, a strong and free will and steady support, I have the impression we may all be capable of something approaching the extraordinary.

Our goal is as musically open and immersive an environment as possible, but with the live, virtuoso / teacher firmly at the centre. To that end, here an overview of some of the key musical qualities (key performance indicators or KPIs, if you will) aspiring musicians can be expected to hunger after, and which any learning environment should consciously support.

By structure, incidentally, the mental maps underlying instrumental navigation. Clearly, these can be literal or figurative.

Learning Sweet Spot (LHS) + Musicality Focus (RHS)

With technical advance goes cultural erosion. We are left -on a purely technical level- dissatisfied with the media used in the direct teachings of earlier, culturally iconic players. In discarding the media, we discard what are often the last remnants of a culture.
a
Moreover, no single technology can replicate in all it's subtlety the directness of face-to-face teaching of a specific example of fine motor movement. This is why -despite technical barriers- it is so very essential that where we work online, it is with live, virtuoso musicians.

At the end of the day, we have feeds needing synchronised and merged to give our best shot at a semblance of immersion. Looking in a little more detail, we see how those feeds might flower into 2- and 3D animations across a whole range of technology stacks.

A Natural Step In The Move Towards Immersive Musical Learning

The traditional, face-to-face approach to teaching and learning mentioned at the bottom LHS of the following diagram embodied more or less everything needed to master an instrument over time. Indeed, in as much as there was also companionship and wide personal engagement, it also served social ends.

Until now online music learning has more or less just mimicked, step for step, that old mode of learning - if at distance. Technology will only really come into it's own when it breaks with conventional learning, and develops new, unimagined forms. How this happens -and whether, with diluted forms of contact, this proves a good thing- is anyone's guess.
ƒ

Immediacy And Immersion Over Time
An Ongoing Game Of Leap-Frog Between Technology And It's Application

For the meantime, immersion in online music learning is technology-led, and any gain in immediacy simply a reflection of how it is applied. There is, however, always this sense of increasingly 'distancing' oneself from the original, living source. This makes me uneasy. Nevertheless, like it or not, technology is with us to stay, and so it is in our interest to make the best possible use of it that we can.

World Music Visualization Aggregator Platform: An End-To-End Value Chain

The musical data available at various points in the platform workflows are a resource to which both algorithm and machine intelligence can be applied, and which could be made available on-demand.

User Value Chain
Orange = Web Stack, Cream = Value Chain, Red = Artificial Intelligence (AI)

Online Music Learning: Liberate The Emotional, Delegate The Banal

Whether through soaring genre fluency, subtle contrasts in tempo, pitch, timbre or rhythm, or simply triggering each other's creativity through clear role punctuation, musical virtuosity is an act of profound empathy and intuition, of listening and reacting to other's play is ways that develop the emotional tension of the whole.

'Artificial intuition' is something of a hot topic in advanced computing, but is likely to take decades to find general application. Until then we have models such as the following for relationship between the emotional and the (for want of a better word) banal.

Threshold Between Musically Banal (Algorithm-Delegable) and Emotional/Creative (Human)

There is a threshold above which algorithm (and here I include artificial intelligence's pattern matching orgies) cede ground to simple human intuition and emotional intelligence. This is an area where creativity has free rein, and, naturally, where musicians love to hang out..

Driving Social Value: The 'Open Everything' Community Provisioning Stack

In the context of the web browser and DOM, there is a well-established (if perhaps hitherto not explicitly documented) open source resource pyramid providing a solid base for commons-based peer production. It lies in our hands to ensure that, for world music, the upper, as yet only marginally explored tiers are also brought into the realm of commons based peer production.

The 'Open Everything' Community Provisioning Stack.

Our aggregator platform has the potential to provide the skeleton or framework upon which to integrate such services. How technologies such as artificial intelligence or virtual reality are integrated will at least be open to exploration and experiment.

World Music Visualization Aggregator Platform: Fully Configurable, From User Menus Down To Note Colours

The entire system has been conceived for personalization, both in terms of own (side) menu content, but also generally applicable conventions such as note colourings and fine settings such as tunings, custom fingering layouts and the use of aids such as capo.

Perhaps contrary to expectations, this is no rocket science, but simply reflects the power of modern data visualization libraries.

Instead of an MVC approach, the focus is on the data-driven creation of all possible configurations from generic family bases, by progressively cycling through all parameters and saving each variant. This has consequences for the entire platform:

• eliminates any need for MVC-style decoupling silos.
• the user has a choice from thousands of configurations, and so profound control over own learning environment.
• each graphical model configuration is stored as a simple (and small) JSON text file comprising simple key-value pairs, such as 'courses: 6', making the system light and fast.
• a given configuration, taken in the context of current user preferences, allows consistency (colors, timers, -normally implemented using templating- naming) across all dependent visualization models.

End-To-End Customizable

For all our technological advance, we are barely able to replicate millennia-old face-to-face (F2F) learning practices at distance - via video chat. Soundslice has set the bar high for synchonised video. Our aim is to go a little further, add as much value, immersion and immediacy to that dialog as current means allow.

World Music Online Learning: Cross-Cultural, Immersive, Global

Working with our hands seems to answer a fundamental human need - satisfied both in the high craftsmanship of instrument building, and in musicianship itself. With the inevitable arrival of a post-work society, I feel fairly sure both will enjoy a thoroughgoing renaissance.

World Music Aggregator Platform Aim
Global P2P Teaching And Learning

Anyone familiar with the smell of freshly freshly shellacked wood, cured leather, machined synthetic ivory or the twang of a set of brass-wound steel strings will know just how satisfying a visit to an instrument builder's workshop or an instrumental workshop can be. It is our intention to catch and ride this wave, bringing what we can to the home, but more importantly, motivating people to get together.

Online Learning With Support For Any Music System (Notation), Instrument Or Theory Tool

The prize is sensational: the ability to configure the platform to your own needs, and then have the animations dance together -in tight synchronization- as the score plays.

Cultural Passport, Social Currency
Music And Dance

The freedom to load one's own or a teachers immediate choice of exercise or score, have it drive an own, immediate selection of instrument models and theory tools, and the ability to visually follow and understand mappings between each.

Just as with the original, instrument models will be fine-configurable to fit own, immediate needs. Choose between different African Kora tunings, just bağlama or equal temperament oud, Venezuelan diatonic or classical chromatic harp, Bb, C or other clarinet (and particular historical form), A B- or C- system chromatic accordion, or simply between instruments of different scale lengths.

Visual Classification Of World Music Theory Tools

Going by Google Blogger's (anonymised) statistics, few visitors take the time to explore the Instrument and Theory Tool menus above, yet together with the notation, these form the core, if exchangeable, visual elements of the framework.

2 and 3D music-theoretical models normally represent some form of abstraction (for example reducing octave equivalents to some 'octave-agnostic' value) with the aim of exposing lean, underlying structure. Carefully aligned for manipulation by advanced, data-driven visualization libraries such as D3.js, though, data can be transformed and mapped in ways that we can barely imagine.

Mirroring Hornbostel-Sachs' visual classification for musical instruments, out aim is a simple, pragmatic classification system for theory tools - all of which, in the context of our aggregator platform, are all of course graphical in nature.

Pragmatic (And Ongoing) Visual Classification of Theory Tools

Online World Music Learning: Shared P2P Interworking Controls

Shared, P2P Interworking controls are foreseen on three levels>

1. Configuration
2. Environmental
3. Playback

Configuration, Environmental And Playback Controls

Configuration controls focus on the setting or sharing of instrument and theory tool definitions governing form and function.

Environmental controls govern the setting or sharing of preferences such as colour schemes, sound libraries and layouts.

Playback controls govern the speed, scope, and other behaviours of shared score playback.

Each control type could be applied across several peers. It would be up to the peers themselves to decide if they wish to retain these settings for future sessions.

World Music Visualization Platform: Value Added Core Strengths

The platform's core strengths are ripe for augmenation by two transformative technologies: artificial intelligens, and, by preference the Web3D forms of virtual and augmented reality.

Core Platform Strengths (LHS) - Music Data Enhancement (RHS)

The focus of the initial crowdfunding, however, is the browser DOM, as that is where this journey starts. Any expertise won there can be reapplied in other the other areas.

Music Visualization: A Foundation For Musical Storytelling

To date, timeline applications using D3.js or it's derivatives have been focussed on conventional data sets, but (analogously to our instrument and theory tool animations and using using either SVG or WebGL) we could equally well envisage stories being animated during score playback.

Animate Musical Storytelling
Data, Goal, Story, Metaphor

This suggests the merging of data supporting storyline development with that of data generated during score playback.

How 'creatives' develop this opportunity - and especially if it leads to new tools blending score and storyline - is something I'll be following with great interest.

---

Keywords

---

online music learning, online music lessons	distance music learning, distance music lessons	remote music lessons, remote music learning
p2p music lessons, p2p music learning	music visualisation music visualization	musical instrument models interactive music instrument models
music theory tools musical theory	p2p music interworking p2p musical interworking	comparative musicology ethnomusicology
world music international music	folk music traditional music	P2P musical interworking, Peer-to-peer musical interworking
WebGL, Web3D, WebVR, WebAR	Virtual Reality, Augmented or Mixed Reality	Artificial Intelligence, Machine Learning
Scalar Vector Graphics, SVG	3D Cascading Style Sheets, CSS3D	X3Dom, XML3D