I was commissioned to work on a new version of the Movement in Time, Part 2 project for the exhibition, By the People: Creative Chinese Characters. In the exhibition, I created two versions of the artwork.
The two artworks are the re-interpretation and re-development of the original computational cinema artwork. The original inspiration came from the legend that when the great Chinese calligrapher, Zhang Xu watched the sword dance of the Lady Gongsun, he invented the wild cursive style calligraphy. With the use of computer vision and basic machine learning technologies, I speculated on the possibility to automatically transform a martial art fighting sequence into cursive style Chinese calligraphy.
Movement in Time, Part 2, 2016 new version
It analyses the fighting sequences from two classical martial art films, Burning of the Red Lotus Monastery, 1963 and Dragon Inn, 1967 and transforms those sequences into cursive style Chinese calligraphy with characters generated from the famous text, Thousand Character Classic.
I developed a custom software in Python to extract the motion data with optical flow analysis. Ten frames of the motion data are summarised into 10 3×3 grid, similar to the one we used to learn Chinese calligraphy in our childhood. The motion data from the fighting sequences is matched against a database of 1,000 Chinese characters written in cursive script style using the same 10x3x3 model. An animation of the matched character is displayed on the screen according to the proper calligraphic stroke order and thickness.
Movement in Time, Part 2, 2022 version
It is an interactive edition of the former one, interpreting the live movement of the audience member and transforms it into cursive style Chinese calligraphy in real time to be shown in the exhibition venue in the Hong Kong Museum of Art.
The underlying mechanism of the interactive version is similar to the cinematic one. Instead of using optical flow to analyse the motion data, the interactive version uses the Google MediaPipe machine learning library for gesture tracking. The captured data is also matched against the same database of 1,000 characters with the 10 frames of 3×3 grid model. In addition to showing the cursive style writing of the Chinese character, the software also simulates the dancing movement of two ribbons attaching to the two hands of the audience member. It creates the intersection of virtual dance movement and real physical body movement, with reference to another performance piece from the Taiwanese Cloud Gate Dance Theatre. Below is a test of concept using the dance sequence from a similar work by the Cloud Gate Dance Theatre related to Chinese calligraphy.
The cursive style Chinese calligraphy of the Thousand Character Classic was written by Ms Lam Lai-sha, Lisa based on the version of Yu Youren.