Alternative Music Visualization Or Animation Scenarios

Despite it's potential, visualization of music is hamstrung by a widespread misconception that it comprises nothing more than swirling patterns of color driven algorithmically by a music source.

Big, brave, open-source, non-profit, community-provisioned, cross-cultural and aquabats crazy. → Like, share, back-link, pin, tweet and mail. Hashtags? For the crowd funding: #VisualFutureOfMusic. For the future live platform: #WorldMusicInstrumentsAndTheory. Or just register as a potential crowd funder..

Note: the images in this post are drawn from external sites which you are encouraged to visit for further information.

Animated music visualization's wider acceptance lies perhaps in relating musical structure's most abstract, pure and visualized forms to the supposedly intuitive, emotional and artistic aspects.

It may take time, but this will be where I gather examples that attempt to merge the two - dynamically.

Care to see some retro music visualization from the era of Adobe's (in principle legacy, but still widely installed) Flash? If your browser still supports it (read "your security is lax"), the image caption below serves as a link to the original..

Interference Theory's Visualisation Dashboard

Though apparently neither interactive nor in any way customizable, this is, for my tastes, a remarkable piece of work, combining many of the visual features of the aggregator platform at the centre of this blog's focus, but with slightly more artistic flair.

On the other hand, there are plenty of what you might describe as purely 'cosmetic' visualizations. Just one example? Envato. Our aggregator platform, on the other hand, really focusses on giving coherent, visual and above all educational expression to music's underlying mathematical patterns.

Alternative Music Visualization/Animation Scenarios

Storm Waves on Lewis

Potential Crowdfunder? First Name Email Address

Liberating the Imagination

Almost 200 years ago, Mendelssohn's tone poem, the overture 'Die Hebriden', Op. 26 helped followers of classical music to a mental picture of the raw majesty of Scotland's atlantic coastline.

One of today's visualization challenges is similar, but moving from the physical to the abstract: from appreciation of nature to our ability to appreciate and understand music itself -in all it's forms.

We are at a point where we can finally expose the mathematical landscapes underlying our vast and developing musical heritage - and bring these (so to say) to life as the music plays.

In doing so, we will overcome not just geographical barriers, but those of culture, symbolism, language, time zone - and technology.

Like the Scottish coastal landscape, music is in a sense a shifting landscape. New forms and conventions are adopted, old discarded. Cultures clash, mix and settle into new. Taste for adventure? Tourist or individualist? Music visualization -as a branch of music education or cultural expression- is a young discipline, the field large, pretty level, and full of opportunity for generation of >social value.
Let's explore some of the implementation alternatives to the aggregator platform featured here..

Tools And Approaches

The range of potential music visualization scenarios and tools and libraries available is large:

• Notation engraving libraries such as Verovio, (engraves MEI music notation into SVG)

Verovio

• Vexflow is an open-source online music notation rendering API. It is written completely in JavaScript, and runs right in the browser, supports HTML5 Canvas and SVG. A notation engraver system takes it's instructions from an own music formatting language, with the goal of generating high-quality (pseudo-engraved) music scores. It also features a music-mathematical query engine able to answer simple questions about the tonal (modal and harmonic) relations between notes.
• Game engines and tools (perhaps of particular interest: those that simply work on the web, but can be published to Android and iOS), plus ports of standalone game engines (such as Unreal Engine) to the web. Of Unreal Engine?: "By simply entering a URL, web browser users can access a beautifully rendered, full screen console game experience running at 60 frames per second, with very little download time and no installation required." Smack.ma.gob..
• Three.js is a cross-browser JavaScript library/API used to create and display animated 3D computer graphics in a web browser. You can find some examples here - with the option of involving other helper libraries such as Tween.js (some examples here).
• P5.js is a JS client-side library for creating graphic and interactive experiences, based on the core principles of Processing. It supports a WebGL mode, but can also be used in conjunction with D3.js.
• WebGL and (care!) asm.js
• Virtual (danger?) and Augmented Reality

Moreover, W3C has begun work towards a new music notation container standard for the web, known -rather cryptically- as MNX.
Various W3C members have begun to showcase experimental scenarios.

You may also like to explore some of the issues (past, present and future) surrounding music and audio in Virtual Reality applications.

Our Scenario

The visualization aggregator platform proof-of-concept on which this blog is focussed has been built using javascript, d3.js and (for example for certain animations) the light but promising music theory library teoria.js.
Proof-of-concept focus has been firmly on:
o ease and consistency of integration o ease of personalization (aggregation) o maximum expressive power in visualisations.
From an implementation point of view, this path allows for a high degree of reuse, and integration of fast and potentially highly sophisticated animations.
Should crowd funding prove successful, the free but expressively limited Gonville SVG font is likely to be replaced by an SMuFL-compliant font such as Bravura.

Challenges

The key challenges I see in any instrument modeling or theory tool context are those of:

• style (the often finely nuanced differences in local or regional playing dynamics which ultimately distinguish individual players within an idiom or genre)
• method (the underlying exercises and techniques by whose means individual style is achieved)
• tension - primarily a quality of play. There is no secret to producing something tedious from the best of resources.

Whether made at score, instrument model or theory level, there is no support for these inherent to any technical approach. As exemplified by SoundSlice, the best approach is an immersive, multi-media one.

---

Keywords

---

online music learning, online music lessons	distance music learning, distance music lessons	remote music lessons, remote music learning
p2p music lessons, p2p music learning	music visualisation music visualization	musical instrument models interactive music instrument models
music theory tools musical theory	p2p music interworking p2p musical interworking	comparative musicology ethnomusicology
world music international music	folk music traditional music	P2P musical interworking, Peer-to-peer musical interworking
WebGL, Web3D, WebVR, WebAR	Virtual Reality, Augmented or Mixed Reality	Artificial Intelligence, Machine Learning
Scalar Vector Graphics, SVG	3D Cascading Style Sheets, CSS3D	X3Dom, XML3D