An international team led by researchers at the University of Michigan School of Music, Theatre & Dance has been awarded more than $340,000 from Schmidt Sciences through its Humanities and AI Virtual Institute (HAVI). The project, “Strokes of Sonic Tradition,” will develop novel artificial intelligence methodologies to unlock one of the largest yet least studied musical canons – more than 30,000 surviving compositions for the guqin, one of the world’s oldest musical traditions.
The project aims to bridge the gap between surviving historical jianzipu notations and the living, performative practices of the guqin. Pictured playing the guqin is Ming Wilson, PhD student in music theory at the University of Michigan and member of the project team.
The guqin, a seven-string zither inscribed on UNESCO’s Intangible Cultural Heritage list, was the primary musical instrument for the Chinese literati from the Song through the Qing dynasties. While an estimated 30,000 to 100,000 historical pieces survive in a shorthand tablature called jianzipu, decoding this massive corpus remains a daunting challenge. The jianzipu notation is a component-based, character-like system that encodes hand placement and plucking techniques, but it omits temporal duration, rhythm, and phrasing, relying heavily on oral tradition for interpreting the performative parameters.
To unlock this vast, low-resource musical corpus, the research team will develop data-efficient AI methodologies, including optical music recognition (OMR) models designed specifically for the unique graphological structure of jianzipu characters. The team will also develop music information retrieval (MIR) and computer vision tools to align historical scores with modern audio and video recordings.
The jianzipu annotation studio developed by the team, featuring an AI-powered stroke detection algorithm that reduces the time and effort required to annotate jianzipu characters. Lead developer is Yuxuan (Glen) Xin, an undergraduate student in computational physics at the University of Edinburgh and member of the project team.
“By building tools that can read centuries-old tablature and align it with the kinesthetic realities of modern performance, we are teaching AI to understand music not just as data, but as a living, breathing cultural practice,” said Hao-Wen (Herman) Dong, assistant professor of performing arts technology in the University of Michigan School of Music, Theatre & Dance and principal investigator of the grant.
“The guqin corpus is one of the largest yet least studied notated repertories in the world. This human-AI collaboration will allow us to uncover historical patterns, aesthetic values, and stylistic shifts across dynasties at a scale that was previously impossible for human scholars alone,” Dong stated.
Hao-Wen Dong
By combining historically informed music theory with modern AI analysis tools, the initiative will map the semantic and performative underpinnings of guqin music. This multimodal approach will allow researchers to connect surviving musical scores with the poetry, literature, and historical treatises that informed the literati who played them. The team aims to illuminate the profound relationships between the guqin and ancient imperial rituals, Neo-Confucian politics, and Daoist and Buddhist metaphysics. By rediscovering the actual “sound” of these embedded historical narratives and myths, the researchers hope to establish a vibrant new portal into the rich artistic and cultural landscape of early East Asia.
The two-year project brings together U-M principal investigators Hao-Wen Dong, David Sears, Nathan Martin, and Joseph Lam, alongside researchers Eduardo Veas from the Graz University of Technology and Ming Tak Ted Hui from the University of Oxford. The current project team also includes Ming Wilson (University of Michigan), Tristan Repolusk (University of Graz), Yuxuan Xin (University of Edinburgh), Zuchen Li (University of Michigan), and Yikang Hong (University of Michigan).
Group photo of U-M principal investigators (from left to right) Hao-Wen Dong, Nathan Martin, David Sears, and Joseph Lam.
The U-M researcher-led project is one of 19 interdisciplinary teams receiving a combined $10 million from Schmidt Sciences’ Humanities and AI Virtual Institute (HAVI) program to expand the study of human history, culture, and expression while helping researchers develop and test new AI capabilities. Teams will openly share the tools and resources they develop.