[Keeping Tempo With Music Biz] Is 2021 “The Year of Search?” — A Conversation with Hazel Savage and Team Musiio
The introduction of artificial intelligence (AI) and machine learning into a well-established, storied industry like ours has presented both exciting possibilities and potential concerns. Specific applications are a perfect fit for AI tech, particularly those that require menial data-entry tasks. Enter Musiio, who have programmed an AI that can “listen” to up to 1 million tracks a day, categorize it based on metadata points on a compositional level, and suggest similar songs based on roughly 1,500 data points. We recently discussed this tech and the advantages it presents with members of the Musiio team, led by CEO and Co-Founder Hazel Savage. We also cover the common misconceptions about AI adoption, what “The Year of Search” means for our global business community, and more!
Music Biz: Would you share what led to the formation of Musiio — i.e., what needs did you seek to fulfill by integrating music AI into music search and curation?
Hazel Savage, CEO and Co-Founder: One of the many reasons I launched Musiio in late 2018 was to address a problem I had while working at a previous music tech business. The company had a huge community of creators, but being a completely free app, the range of quality was extremely wide. While there were some creators that had a lot of talent, there was a high percentage of users that were new to the music creation process, so finding great music was pretty difficult. I had a full-time staff member who would search every hour of the day on the platform, listening to anywhere between 3,000 and 5,000 tracks on a weekly basis. Some weeks he would have a playlist, some weeks he would have three. There were quite a few weeks, however, where we would have no playlists to send as part of our weekly marketing initiatives. With a fast-growing community now in the tens of millions, it was also getting increasingly impossible to listen to everything. My question at that point in time was: is it possible to get a machine to do the first listen and separate the great quality music from the poor quality music? Years later, it turns out that is not only possible, but we can also train a machine to do so much more.
Music Biz: Outside of traditional metadata information, what unique identifiers does your Tag platform check for that a music professional might not normally think of?
Aron Pettersson, CTO and Co-Founder: Musiio’s proprietary AI can identify anywhere up to 1,500 data points per track. If you compare this to a traditional set of metadata, which tops out at about 20 individual pieces of text, this starts to give you some idea of the power of the technology and an understanding of what it is capable of. As we always say, if the human ear can hear it, we can teach our AI to tag it.
We’re able to tag up to 1 million tracks in a day with genre, key, bpm, mood, emotional valence, instruments, quality, vocal presence and more. In fact, we recently added use-case or context to our tagging capabilities.
For example, when analyzing the track from The Strokes below, the AI recognized that it would be well-suited to Extreme Sports related videos.
Music Biz: Musiio can analyze and assign tag tracks to new music as they are uploaded to a new digital platform. Did analyzing existing catalog music to create the foundation of your tags come first, or was the tagging process developed with consideration for newer music?
Olivia Coleon, Head of Music: Training AI is an extremely intensive process, and it’s taken us years to get to where we are today. Any time we want to teach the AI something new, the first thing that we need to do is build large datasets of music which are accurate representations of that given tag in all its forms. If we look at House music for example, we must include all types of House music, from Lo-Fi House with slower tempos and spoken word samples to Classic House with a faster tempo and soulful vocals.
Our AI can currently tag 84 genres with 90-99% accuracy. When we first launched, Musiio was able to tag 21 genres. When choosing which genres to add to the pool, it was customer feedback that guided our efforts. We currently have 40 paying clients and most are pretty outspoken when it comes to tags that we are missing or where they would like to see improvements. Having access to so much music and data, it’s pretty obvious where the gaps are, and we use this to our benefit. The music team at Musiio runs quality assurance on every POC we put out, so we have a very good handle on how our tags perform and the areas that we’d like to improve next. This continuous feedback loop naturally gives us a solid AI training roadmap, and all I will say is there are some very exciting updates to come! When you join as a new Musiio client, you start at our baseline, but you can certainly expect your tagging to continue to improve as our AI gets increasingly smarter.
Music Biz: In talking to the Musiio team, we’ve heard you bill 2021 as “The Year of Search” — what are the implications of that term for the music industry as a whole, and for specific niche areas of the business most suited to benefit from Musiio’s toolset?
HS: At launch, most of our communication was focussed on educating people about the benefits of AI while also dispelling the belief that AI was coming to take people’s jobs. Uptake for AI at the time was tentative and really reserved only for early adopters and folks with job titles such as Innovation Director. We saw tagging pick up relatively quickly, and that trend continued to gain traction in 2019 and 2020. While automated tagging has been our greatest commercial success to date, it’s our search technology that will revolutionize the industry and have the greatest, widest impact on how the world interacts with and consumes music.
With Musiio’s Search product, catalogs are more searchable than ever before. With a YouTube link or an MP3, users can now paste the link or upload the audio into the search bar. Our AI will then scan the audio file, turn it into a series of fingerprints, and search the catalog for matches based on 1,500 data points per track. You can search a catalog with millions of tracks in seconds with no need to understand how their taxonomy is put together. This technology is vastly more accurate than anything on the market today.
We are calling this the “Year of Search” because we have seen a huge uptick in interest in this technology since January, and you will be seeing more and more catalogs adopting this feature in the months and years ahead. 2021 marks a new age of searching music, with music!
Music Biz: Much of the debate on applying AI to the music business lies in the idea that it will eliminate the human element in our industry, particularly where songwriting and music curation are concerned. Can you speak on how Musiio has sought to blend technology with a human touch in how it collects, presents and applies its data?
HS: I’ve always been pretty outspoken about AI being used for music creation. It’s not that I don’t find it interesting or cool. But from a business perspective, with Spotify announcing that they now publish 60,000 new tracks every single day, I just don’t see the business need. The world isn’t short of music and with humans creating so much of it, I don’t see AI-generated music ever really competing with the real thing.
When it comes to curation though, there is still a lot of work to be done to meet the standards of today’s users. Most large streaming services have the same 40 million tracks, so it’s no longer a race for content. It’s a race to curate the best and most suitable music for every user. Curating personalized playlists for millions of streamers at once, understanding their nuances, and understanding the last 100 tracks from a detailed audio perspective requires the very latest in AI technology.
As a company, we are advocates of using AI to do the things that humans can’t or don’t want to do. Tagging is a laborious and pretty boring process to do manually, so whether you’re running a small label with a thousand tracks and don’t want to have to tag every detail, or you’re a streaming service publishing 60,000 tracks per day and don’t have the manpower, AI can do the job for you.
As for music search, AI assists you to find the music you are looking for much faster than ever before, but it’s still up to use as the supervisor, curator or DJ as to which music you will actually choose.
Music Biz: To that point, DJing as a culture and art form has evolved through the years to incorporate advanced technology while maintaining the power of the artist’s ear to create mixes. How do Musiio’s tools fit into that balanced equation of automation and human curation?
Mack Hampson, Commercial Director and DJ: DJing has been an artform which has got progressively more accessible throughout the years. I originally learned in the late 90s on belt drive decks, and ordered six vinyl records which I used for the first few months to learn how to beatmatch. It took me more than six months to learn just how to do that. With today’s technology, I could teach someone how to do the same with a DJ controller in less than a day. Where there used to be a select few people at a party that could DJ, now it seems to be a skill that most people with a little bit of time and $200 can pick up. While technology has changed exponentially and continues to do so, being a successful professional that is able to cut through still takes a combination of taste, passion, skill and determination. You have to put the time into search deep inside catalogues if you want to separate yourself from the pack.
The difficulty today is that you don’t have a record shop with thousands of records. You have digital music stores and streaming services with millions. It’s getting harder and harder to navigate through large catalogues because everyone has the same access to the same storefront. So for now, DJs looking to separate themselves simply have to spend more time searching.
With AI, this will all change. DJs will have the power to build sets from the spark of an idea. Imagine starting with an idea that all of your set is going to be based purely on music created by an 808. Today it would be practically impossible or would take you a very long time and a lot of research — the music data just isn’t there.
Imagine being able to design your two-hour set as you are searching, slotting in tracks with increasing energy, or differences in musical shape, as you search for them. AI will allow DJs to express themselves as individuals and as artists more than ever before. Now that’s an exciting opportunity!
Music Biz: You recently published a piece to the Musiio blog about how AI was able to determine nuances between four different recordings of the same piece of music. For an example like that, what distinctions between recordings does your platform look for?
Xiao’an – Music Strategist: Even though these tracks were all of a solo violin, there is a great deal of detail that distinguishes one from the other. Little pauses taken, how long one holds a note, what sort of vibrato is being used, how dynamic the performance is, etc. These and more all contribute to the overall “feel” of a piece — its energy, its mood, whether a fixed tempo can be detected. Sometimes, even a choice to place an emphasis on different notes in the same piece can change how our AI perceives the key!
Music Biz: More and more, modern music consumers are abandoning the concept of genre. Any given curated playlist on a streaming service may, for example, have artists and songs that folks may never expect to see next to one another. Does this present difficulties in tagging & categorizing music, and how has Musiio responded to these challenges?
HS: It’s an interesting topic and one that aligns itself well with our search technology. Interestingly, our search technology doesn’t need any tag information at all to curate highly accurate search results or curate playlists. All it needs is the fingerprints of the audio files. The nuances it can pick up on are often pretty eye-opening to our clients — it can tell the difference and make quite different playlists.
For example, when given the seed track “Bodak Yellow” by Cardi B, vs “Beez in the Trap” by Nicki Minaj, it is true that both tracks could be described as Hip-Hop- or Trap-based. But “Bodak Yellow” has high Pop sensibilities, clean edits, and a swooping melody; whereas Nicki’s Beez is a sparser instrumentation, a more distorted and grimey back-beat, and more monotone-style repetition in the vocal. Don’t get me wrong I LOVE both tracks, but both would lead to a very different sounding playlist if used as a Seed Track for the Musiio AI, because we are looking at all the nuances of the audio files, not just the tag “Hip-Hop” to group tracks together. If you get this wrong as a streaming service, you end up with bad recommendations and you risk losing the listeners attention.
While I don’t ever feel as if we will see a complete swing away from using tags to describe music — after all, it is a part of the very bedrock of music culture — there is a lot of special sauce in-between tags if we process data on a granular level, and technology does a great job at the necessary heavy lifting.
Music Biz: As we wrap up, can you speculate what applications you foresee your technology being used for in the future?
HS: There are so many different exciting applications of this technology. As we continue to build outwards when it comes to our tags, we continue to stretch the boundaries of what is possible.
As was recently covered in Music Business Worldwide, our AI has just been trained to understand whether a track has the ability to be a hit record. Record companies call it gut instinct, TV shows call it the “X factor,” but now our AI can hear the hit potential of a record. With more music being published to platforms now than ever before, and with artists becoming overnight stars on platforms like TikTok, the ability to find and surface these stars before they blow up (or don’t), is an extremely exciting prospect, one which we will be looking forward to in the years to come.
Before I go, I’d just like to say that we have a lot of exciting products we’re working on behind the scenes. If people would like to to hear more, they can contact us via our website at musiio.com or by emailing me at hazel@musiio.com .
You can read past “Keeping Tempo” articles via the portal linked here. And, stay tuned for more insightful discussions from our members and industry partners!
