Hi John,
No not yet,
But sooner or later I've got to go there.
I had a few thinks about this before. I think it would be too complext to 'deform geometry' (for me at least;)
So my approach would be to create different textures for a characters mouth- and replace/animate (multiple frames texture) them according to the speech. If I can't replace his mouth texture - I'll stick and match a sprite in front of his mouth to get the same effect.
Then basically making (for example) 8 different 'mouth poses' for the A, E, I, O, U, etc. (also, a smile, grin, and other basic expressions.
To 'sync' then with the exactual voice (=wav file) I'd create a long string like this for example:
"E...L.L..O.... E..A.A..G...ER..." wich could translate in something like 'hello stranger' ;)
Where each letter corresponds to one of the x textures (or A, E, I, O, U, 'phonyms'). And each character in that string would ie. be 1/4 of a second. That way to 'only' have to translate each wav file to one of those text strings. This should be super-easy to code (and doesn't cost you anything else except time ;)
Actuall speach-to-text or speach-to-text would be too complex I guess and take too much processor time. Also remember you don't have to translate each letter to get proper lip-sync - quite the contrary.
The 'string' technique works really well, I've used something like this for many years doing commercials and such in 3D. Ofcourse I didn't do it using textures for mouth positions but with actual 3D blend shapes. Textures will work for most basic human characters I figure, although for a 'dragon's mouth' an animated texture would look rather silly I guess ;)
But, as I said before, I've never actually tried this in Blitz :)
Most importantly I still think that it's better NOT to have any lip-sync, than BAD lip-sync...
Hope this helps,
Danny.
No not yet,
But sooner or later I've got to go there.
I had a few thinks about this before. I think it would be too complext to 'deform geometry' (for me at least;)
So my approach would be to create different textures for a characters mouth- and replace/animate (multiple frames texture) them according to the speech. If I can't replace his mouth texture - I'll stick and match a sprite in front of his mouth to get the same effect.
Then basically making (for example) 8 different 'mouth poses' for the A, E, I, O, U, etc. (also, a smile, grin, and other basic expressions.
To 'sync' then with the exactual voice (=wav file) I'd create a long string like this for example:
"E...L.L..O.... E..A.A..G...ER..." wich could translate in something like 'hello stranger' ;)
Where each letter corresponds to one of the x textures (or A, E, I, O, U, 'phonyms'). And each character in that string would ie. be 1/4 of a second. That way to 'only' have to translate each wav file to one of those text strings. This should be super-easy to code (and doesn't cost you anything else except time ;)
Actuall speach-to-text or speach-to-text would be too complex I guess and take too much processor time. Also remember you don't have to translate each letter to get proper lip-sync - quite the contrary.
The 'string' technique works really well, I've used something like this for many years doing commercials and such in 3D. Ofcourse I didn't do it using textures for mouth positions but with actual 3D blend shapes. Textures will work for most basic human characters I figure, although for a 'dragon's mouth' an animated texture would look rather silly I guess ;)
But, as I said before, I've never actually tried this in Blitz :)
Most importantly I still think that it's better NOT to have any lip-sync, than BAD lip-sync...
Hope this helps,
Danny.