AI Music Lab @ hfm-nuernberg

Congrats! to our team! »First Encounter« has won the 2nd price of jazzki award 2023 @elbjazz with a trio of two saxophones and an AI-driven grand piano #elbjazz23 hfm-nuernberg.de/en/news/detail…

Exciting news! 📢 The MusAI public seminar series in collaboration with @turinginst is now on Youtube. Join in the in-depth discussions on various subjects from copyright in AI music to recommendation systems. youtube.com/playlist?list=… #AIMusic #AI

ChatMusician Understanding and Generating Music Intrinsically with LLM While Large Language Models (LLMs) demonstrate impressive capabilities in text generation, we find that their ability has yet to be generalized to music, humanity's creative language. We introduce ChatMusician, an open-source LLM that integrates intrinsic musical abilities. It is based on continual pre-training and finetuning LLaMA2 on a text-compatible music representation, ABC notation, and the music is treated as a second language. ChatMusician can understand and generate music with a pure text tokenizer without any external multi-modal neural structures or tokenizers. Interestingly, endowing musical abilities does not harm language abilities, even achieving a slightly higher MMLU score. Our model is capable of composing well-structured, full-length music, conditioned on texts, chords, melodies, motifs, musical forms, etc, surpassing GPT-4 baseline. On our meticulously curated college-level music understanding benchmark, MusicTheoryBench, ChatMusician surpasses LLaMA2 and GPT-3.5 on zero-shot setting by a noticeable margin. Our work reveals that LLMs can be an excellent compressor for music, but there remains significant territory to be conquered. We release our 4B token music-language corpora MusicPile, the collected MusicTheoryBench, code, model and demo in GitHub.

Recent papers I'm excited about on controlling diffusion models w/ musicality: - ditto-music.github.io/web/ - musiccontrolnet.github.io/web/ - machinelearning.apple.com/research/contr… Ideas from image diffusion that seem promising for music: - readout-guidance.github.io

Do you want to study interdisciplinary music research with a focus on AI and music? Apply for our online/hybrid Master’s Program at Nuremberg University of Music until April 2nd #creativeai #studyprograms #musicandai #musicresearch hfm-nuernberg.de/en/study/cours…

A little moment in the sun of the @mitpress homepage. The book is FREE as an ePub or PDF.

For all those interested in AI's impact on music and creative culture, do check out this awesome series of seminars hosted by the Alan Turing Institute (@turinginst) and organised by the MusAI research group (@MusAIproject), an ERC-funded project led by Prof Georgina Born.

If you're at @BeyondCnf in London over the next couple of days please say hi. I'm talking early tomorrow, 9.30am, on a panel discussing generative AI and the creative industries. beyondconference.org.

🎙️ Emilia Parada-Cabaleiro talks about "Machine Learning in Vocal Training" at the 16th International Symposium on Computer Music Multidisciplinary Research (CMMR). 🎶 Watch her talk: youtube.com/watch?v=Gq4d6a… (from 1:22:10). zenodo.org/records/101253…. 🚀 #musicpedagogy #AI

#RoboPercussion on stage again — Heute beim 13. Symposium zur Kultur- und Kreativwirtschaft in der @MetropolregionN #creativeai

Very inspiring talks with colleagues from all over the world on music and AI at the CreAItivity conference in beautiful Speinshart Monastery turquoise-mouse-mwl4.squarespace.com/creativityandai #hightechagenda #creativeai

@bastustrump discussing and performing with our robotic percussion @LeonardoZentrum event Artistic Intelligence – AI in Culture and Arts

On July 4th @bastustrump will perform the latest version of RoboPercussion at PegnitzWaves as part of the @nuedigital festival. nuernberg.digital/programm/detai…

ICCC’23 Best short paper: “Diversity is Not a One-Way Street: Pilot Study on Ethical Interventions for Racial Bias in Text-to-Image Systems” By Kathleen Fraser, Svetlana Kiritchenko and Isar Nejadgholi Read the paper: computationalcreativity.net/iccc23/confere…

We’re releasing the Anticipatory Music Transformer: a controllable generative model for symbolic music (like MIDI). Read about the model on the CRFM blog: crfm.stanford.edu/2023/06/16/ant… 🧵👇

We are hiring – join our lab as AI lab engineer at Nuremberg University of Music stellen.hfm-nuernberg.de/jobposting/59f… #musicAI #HightechAgenda

The Artificial Intelligence Music Creativity conference theme this year is Intelligent Performance Systems. We are opening up for contributions of late-breaking research both for our Alt-AIMC strand and the Doctoral Consortium. Info here: aimc2023.pubpub.org/call-alt-aimc-…