peep the horrors

Some thoughts on music synchronization in games

I've been thinking about the idea of synchronizing music to gameplay in games and decided to write a little about it. This post doesn't really talk about story, but it has minor gameplay spoilers for several Touhou games and Final Fantasy XIV, and some major gameplay spoilers for Perfect Cherry Blossom and Void Stranger. If you plan to play the latter two games I would really recommend to do that before reading this.

Read More

Void Stranger

Void Stranger is a sokoban-style puzzle game with an OST that's genuinely one of my favorites of all time. Generally, puzzle games don't really ask the player to act under time pressure; part of the intended gameplay loop is to just sometimes stare blankly at the screen while you think the puzzle through, sometimes not directly interacting with the game for several minutes. And, for most of Void Stranger, the soundtrack is just background music to that kind of contemplative puzzle-solving. But there are two moments where I think the soundtrack particularly shines.

To explain the first moment it's probably good to give some context. Void Stranger has a life system that appears very punishing at first. You have to spend a life to retry any puzzle, and the chests that give lives are generally locked around optional mini-puzzles. You eventually figure out that there's a lot of ways to get easy lives, but you'll probably run out of lives before you discover even one of them.

However, when you run out of lives you don't necessarily have to return to the start: you get the option to eat a fruit that makes it so you don't spend lives when you die anymore. This will happen to most players very early in the game, so it's easy to assume dying is an intended story beat and eat the fruit. This VOIDs the player, locking them into a bad ending that will trigger when you beat the game, likely several hours later. That bad ending is the moment I'm talking about.

In retrospect, this moment is basically a cutscene: it doesn't really matter what the player does here, beyond maybe making sure they take some screenshots to catch hints for later puzzles. But when you're playing it for the first time, it feels like the music is breaking the puzzle game promise to let you solve things at your own pace. The first rooms it teleports you to are impossible, but you see them for so little time that you might just assume they had some hidden trick, and you could have figured them out if the game hadn't teleported you away. However, when the music crescendos and it starts teleporting you to rooms without an exit, or which just start you over a hole, or which are a garbled mess of blocks, you begin to realize that this is pointless, and you might as well just wait for it to end. It's a moment that does a great job slowly instilling helplessness into the player, and it wouldn't work the same (or at all!) if it wasn't for the music synchronization. Really great use of the idea.

The other moment is more prosaic. At the very end of the game, it shifts genres into a shmup:

I don't think the music synchronization in this section is particularly impressive to people that regularly play shmups (I'm sure the studio's full shmup game, ZeroRanger, has more interesting moments), but I was pretty impressed at how well it orchestrated the enemy attacks. Speaking of shmups

Touhou

Levels in Touhou will generally have two themes: the stage theme (that plays during the initial portion, where you fight little mooks and the midboss) and the boss theme (that plays during the boss at the end of the stage). The start of most stages will synchronize changes in enemy type, background and sometimes even the actual attacks to the stage music.

The midboss sometimes throws a wrench into this. In Touhou, most bosses are divided into healthbars that you have to deplete to make the boss advance. This means that both the full midboss fight and the individual spells/nonspells can take a variable amount of time, depending on the player's damage output. However, even given this, I do feel like most midbosses end up having a well-defined spot in the stage music once you're doing them competently and bombless.

The boss themes are generally a wash when it comes to at least author-intended music synchronization, though. A lot of bosses will start playing their theme during the pre-battle conversation, which means the player could ruin the synchronization if they read too slowly, or they screenshot it and take a moment to post it to their liveblog thread, or if they had to get up and pace around their room for ten minutes after Keiki said they would turn them into her "finest idol". The previously-mentioned healthbar problem is also worse on the stage bosses, since they often have several different spells and nonspells that the player can take different amounts of time to beat.

That being said, since boss themes tend to have a lot of distinct parts, some synchronization does emerge between cards and the music. If a card falls on the transition to a cool part of the song, then that feels special even if it wasn't intended by the developer. That feeling is particularly strong if you're doing something that requires a lot of trial-and-error (like challenge runs or Lunatic); I definitely still associate certain portions of Kanako's theme or Utsuho's theme with spellcards that I had to retry a ton of times to get through.

One interesting final word is about survival cards. These are spellcards where the boss has no healthbar, and you have to just survive for some amount of time to get past it. Since they always take a fixed amount of time, they've been used twice (that I can remember) to create fun music moments:

First, the whole concept behind Raiko's timeout is that she's playing a little tune with her danmaku. This tune fits in at any point in the BGM, and the fact it's a survival card makes you listen to all of it:

Second, Yuyuko's timeout comes after you "defeat" her. The music cuts out, but then a new song starts and you have to get through a final survival card. It's 🎥:

Did I miss anything interesting? Let me know but don't be mean about it. I don't watch the touhou video essays

Final Fantasy XIV

FFXIV is a game where the majority of combat content is heavily scripted, to the point where players can know down to the second when the boss will do certain mechanics. It also heavily emphasizes repetition, both in the Touhou trial-and-error sense (for high-end raid progression) and in the Genshin Impact "you have to do this fight once a week if you want 10 Incomparable Gemstones" sense (for casual content). This means it is, excluding actual rhythm games, maybe one of the most well-suited games in history to explore the idea of gameplay and music synchronization.

And yet it rarely does. The main problem is a systems one: the music for most fights doesn't start when the boss is engaged in combat, but when people enter the arena. It also doesn't restart the music after a wipe (total party kill). This means that the actual combat can and will start at any point in the song, which doesn't allow even for unintended synchronization to emerge like it did for Touhou bosses.

There are some situations where the start of the song is fixed to the combat timeline, though. Many bosses in FFXIV will transition to a different song at a certain point in the fight, and they do try to make that transition between songs memorable.

However, more interestingly, transitions (without a checkpoint) mean that any mechanics that happen after the transition will always happen at the same part of the song. The most famous example of this (and the only one I think was intentional) are the tankbusters in the last phase of T13, a fight from the base game:

Ultimate fights in particular have several transitions, no checkpoints (excepting DSR) and very mechanic-dense phases. This means that a lot of ultimate phases end up, intentionally or not, synchronizing really well with the music. Some non-exhaustive examples (the mechanics will probably be a bit illegible if you're not familiar with the game, though):

This kind of synergy between combat and music is what makes Ultimate fights bombastic and fun to play for high-end raiders. And, though easier high-end content doesn't tend to allow for that kind of experience, we have seen some of it on recent fights. In the second phase of M4S, a fight from the latest content drop whose impressive transition I showed earlier, the music begins to crescendo at the start of the last big mechanic:

Zeromus EX, one of the last fights from last expansion, also has a final phase where each section of the music lines up with a mechanic:

In general, though, I just think it would be nice for the game to at least experiment with making boss music start when combat starts. I don't think the majority of these examples are intentionally crafted interactions, which would mean just locking the music to the fight timeline would already help create way more of these moments and provide players with a more cohesive experience.