Back in 2021 I made a mod for Deus Ex 1 that fixes the lipsyncing and blinking, which, I betcha didn't know, was broken since ship. Everything I wrote about it is on Twitter, and it oughta be somewhere else, so here's a post about it.
I guess I was playing DX1 and thinking, geez, was this lipsync always this bad? In a weird way? It's insta-snapping mouth shapes, but they're not always the same mouth shapes. Is this broken? I couldn't find anything online about it, but I did find this article: an interview with Chris Norden, a coder on DX, where he goes into the lipsyncing and how it was, at one point, super elaborate and amazing, and they had to pare it back for performance reasons. I thought I'd check how much of this was done in Unrealscript (since the C++ source for DX is nowhere) and whether I could un-pare it. It turns out it was an extremely simple fix to get it as good as I got it, and I think that's as good as you can get it until someone leaks the source code.
I'd messed around with lipsyncing stuff before and was familiar with the broad strokes of how it tends to work via my intense familiarity with Half-Life 2: you figure out, hopefully automatically, the sounds (phonemes) present in a sound file ("oo", "ah", whatever) and map those to mouth shapes (visemes), then when the audio plays, move the mouth into the right shape for the phoneme we're in at this moment. The figuring-out process is called "phoneme extraction", at least by Valve, and Valve do this offline, because it takes a sec. In Valve's case they append this phoneme information to the end of the .wav file, and it looks like this:
