You know what, I think AI/deepfakes could have a legitimate use in generating fully realistic dubs, provided the appropriate parties are compensated fairly. It must be annoying as hell seeing English-language movies clumsily dubbed over with mismatched voices and completely out of sync lip movements.
Deepfake lip syncs video to edit out profanity, match translation
Submitted 11 months ago by cyu@sh.itjust.works to technology@lemmy.world
Comments
RagingNerdoholic@lemmy.ca 11 months ago
wreckage@lemmy.world 11 months ago
It must be annoying as hell seeing English-language movies clumsily dubbed over with mismatched voices and completely out of sync lip movements.
It is. On my country we only use subtitles (except for movies aimed to kids) and I think it’s much better
Aceticon@lemmy.world 11 months ago
Same in mine, Portugal.
It has the added benefit that people naturaly improve their learning of foreign languages when foreign language programming on TV is subtitled: just compare the percentage of people who speak “good” english in Portugal with that of next door Spain were everything is dubbed.
That and I always found it weird when I went to Spain in vacations and saw things like Westerns with John Wayne were his voice was dubbed by some (comparativelly) whinny spanish guy.
CobblerScholar@lemmy.world 11 months ago
Well you think not every language uses the exact same number of words or syllables to convert the same amount of information. So on translation not only do people have to translate the words they have to basically rewrite the entire movie for every language because the speech time has to match the screen time
RagingNerdoholic@lemmy.ca 11 months ago
Of course not, but a well-developed deepfake is so robust that you can shorten or lengthen the dialog and rate of speech and it has no problem compensating.
chickenwing@lemmy.world 11 months ago
More tech to change things in post. I feel like Hollywood is relying too much of this kind of technology to fix poor planning and writing. Look at all reports about marvel films being radically changed in post. You can tell when a film is a patch job.
Creddit@lemmy.world 11 months ago
Pretty sure Netflix has been doing this for overdubs for the last 2+ years. The actors mouths approximately match the overdubbed language, at least for English.
brsrklf@compuverse.uk 11 months ago
Are you sure they’re not just decent dubs? Non English-speaking countries have been used to synced dubs forever.
This is why adapting (well) for dubs is way more complicated than just translating dialogues. You have to find an equivalent line that matches visible mouth movements.
Creddit@lemmy.world 11 months ago
That’s true I suppose. No, I can’t be sure. It could just be really good dub writers, I guess?
krayj@lemmy.world 11 months ago
The actors mouths approximately match the overdubbed language, at least for English.
I wish this were true. Netflix is a worst-in-class offender imo. They will shoot a film in three different native languages so that about 1/3 the dialog sounds and loks correct for your native language. And then for the other 2/3 of the dialog, it’s a horribly butchered dub job from 2 different languages back to english. I can’t watch them because my brain rejects the horrible dub and it pre-occupies my thoughts rather than me just being able to the movie.
DreamerOfImprobableDreams@kbin.social 11 months ago
Also, for some reason the dub and subtitles use different translations, so the character will be saying one thing and the subs will read something completely different. It's confusing AF. Often end up getting so frustrated I just turn off the dub and focus exclusively on the subs.
PipedLinkBot@feddit.rocks [bot] 11 months ago
Here is an alternative Piped link(s): piped.video/watch?v=iQ1OPpj8gPA
Piped is a privacy-respecting open-source alternative frontend to YouTube.
I’m open-source, check me out at GitHub.
Ocelot@lemmies.world 11 months ago
This is really cool and would take dubbed movies to a whole new level
ahbi_santini@kbin.social 11 months ago
I have been waiting for this for dubbed movies
The thing that will really take them to the next level is using audio AI to do the dubbing, instead of voice actors.
That way you hear the real actors voices but in the native language.
Ocelot@lemmies.world 11 months ago
I think we’ll get there soon. Right now AI voice is close but it still sounds a little unnatural/off. Getting better by leaps and bounds really quick though.
At some point we are going to have movies created entirely by AI, even the film itself is auto generated.
TheKarion@lemmy.world 11 months ago
I need this
LexiconDexicon@lemmy.world 11 months ago
I like the profanity, only some Christian Conservative from the 1980’s would argue against it
GONADS125@lemmy.world 11 months ago
Can’t say fuck, but totally okay to see someone’s brains bashed out! It’s so idiotic and illogical…
krayj@lemmy.world 11 months ago
Even worse than that is acceptance of “freaking” in 0lace of “fucking”. The words are internalized inside the brain to have the same exact meaning in this context, it’s just they take offense to certain arrangements of consonants andd vowel combinations to deliver the same message. It’s like they think their god is stupid enough to fall for a semantical trick.
LexiconDexicon@lemmy.world 11 months ago
Yeah I know, they love gore and violence but the second they see a nipple it’s like we’re back in Puritan New England or something
sundrei@lemmy.sdf.org 11 months ago
“The horror… the horror…”
FaceDeer@kbin.social 11 months ago
You could use the same technology to add extra profanities into the dialogue as well.
Charlton Heston in the 10 Commandments, except everyone's swearing like a sailor the whole time.
DreamerOfImprobableDreams@kbin.social 11 months ago
Yeah, to be honest I'm a bit worried studios might use this technology to start changing lines in old movies, so subtly only film buffs who know the film line-for-line would ever notice.
ChrisLicht@lemm.ee 11 months ago
Same thing for books, and once they’re all digital, it will be far worse…
The only good use for blockchain I’ve heard of is publishing important works to it, so there is an immutable record.