You know what, I think AI/deepfakes could have a legitimate use in generating fully realistic dubs, provided the appropriate parties are compensated fairly. It must be annoying as hell seeing English-language movies clumsily dubbed over with mismatched voices and completely out of sync lip movements.
Deepfake lip syncs video to edit out profanity, match translation
Submitted 1 year ago by cyu@sh.itjust.works to technology@lemmy.world
Comments
RagingNerdoholic@lemmy.ca 1 year ago
wreckage@lemmy.world 1 year ago
It must be annoying as hell seeing English-language movies clumsily dubbed over with mismatched voices and completely out of sync lip movements.
It is. On my country we only use subtitles (except for movies aimed to kids) and I think it’s much better
Aceticon@lemmy.world 1 year ago
Same in mine, Portugal.
It has the added benefit that people naturaly improve their learning of foreign languages when foreign language programming on TV is subtitled: just compare the percentage of people who speak “good” english in Portugal with that of next door Spain were everything is dubbed.
That and I always found it weird when I went to Spain in vacations and saw things like Westerns with John Wayne were his voice was dubbed by some (comparativelly) whinny spanish guy.
CobblerScholar@lemmy.world 1 year ago
Well you think not every language uses the exact same number of words or syllables to convert the same amount of information. So on translation not only do people have to translate the words they have to basically rewrite the entire movie for every language because the speech time has to match the screen time
RagingNerdoholic@lemmy.ca 1 year ago
Of course not, but a well-developed deepfake is so robust that you can shorten or lengthen the dialog and rate of speech and it has no problem compensating.
chickenwing@lemmy.world 1 year ago
More tech to change things in post. I feel like Hollywood is relying too much of this kind of technology to fix poor planning and writing. Look at all reports about marvel films being radically changed in post. You can tell when a film is a patch job.
Creddit@lemmy.world 1 year ago
Pretty sure Netflix has been doing this for overdubs for the last 2+ years. The actors mouths approximately match the overdubbed language, at least for English.
brsrklf@compuverse.uk 1 year ago
Are you sure they’re not just decent dubs? Non English-speaking countries have been used to synced dubs forever.
This is why adapting (well) for dubs is way more complicated than just translating dialogues. You have to find an equivalent line that matches visible mouth movements.
Creddit@lemmy.world 1 year ago
That’s true I suppose. No, I can’t be sure. It could just be really good dub writers, I guess?
krayj@lemmy.world 1 year ago
The actors mouths approximately match the overdubbed language, at least for English.
I wish this were true. Netflix is a worst-in-class offender imo. They will shoot a film in three different native languages so that about 1/3 the dialog sounds and loks correct for your native language. And then for the other 2/3 of the dialog, it’s a horribly butchered dub job from 2 different languages back to english. I can’t watch them because my brain rejects the horrible dub and it pre-occupies my thoughts rather than me just being able to the movie.
DreamerOfImprobableDreams@kbin.social 1 year ago
Also, for some reason the dub and subtitles use different translations, so the character will be saying one thing and the subs will read something completely different. It's confusing AF. Often end up getting so frustrated I just turn off the dub and focus exclusively on the subs.
PipedLinkBot@feddit.rocks [bot] 1 year ago
Here is an alternative Piped link(s): piped.video/watch?v=iQ1OPpj8gPA
Piped is a privacy-respecting open-source alternative frontend to YouTube.
I’m open-source, check me out at GitHub.
Ocelot@lemmies.world 1 year ago
This is really cool and would take dubbed movies to a whole new level
ahbi_santini@kbin.social 1 year ago
I have been waiting for this for dubbed movies
The thing that will really take them to the next level is using audio AI to do the dubbing, instead of voice actors.
That way you hear the real actors voices but in the native language.
Ocelot@lemmies.world 1 year ago
I think we’ll get there soon. Right now AI voice is close but it still sounds a little unnatural/off. Getting better by leaps and bounds really quick though.
At some point we are going to have movies created entirely by AI, even the film itself is auto generated.
TheKarion@lemmy.world 1 year ago
I need this
LexiconDexicon@lemmy.world 1 year ago
I like the profanity, only some Christian Conservative from the 1980’s would argue against it
GONADS125@lemmy.world 1 year ago
Can’t say fuck, but totally okay to see someone’s brains bashed out! It’s so idiotic and illogical…
krayj@lemmy.world 1 year ago
Even worse than that is acceptance of “freaking” in 0lace of “fucking”. The words are internalized inside the brain to have the same exact meaning in this context, it’s just they take offense to certain arrangements of consonants andd vowel combinations to deliver the same message. It’s like they think their god is stupid enough to fall for a semantical trick.
LexiconDexicon@lemmy.world 1 year ago
Yeah I know, they love gore and violence but the second they see a nipple it’s like we’re back in Puritan New England or something
sundrei@lemmy.sdf.org 1 year ago
“The horror… the horror…”
FaceDeer@kbin.social 1 year ago
You could use the same technology to add extra profanities into the dialogue as well.
Charlton Heston in the 10 Commandments, except everyone's swearing like a sailor the whole time.
DreamerOfImprobableDreams@kbin.social 1 year ago
Yeah, to be honest I'm a bit worried studios might use this technology to start changing lines in old movies, so subtly only film buffs who know the film line-for-line would ever notice.
ChrisLicht@lemm.ee 1 year ago
Same thing for books, and once they’re all digital, it will be far worse…
The only good use for blockchain I’ve heard of is publishing important works to it, so there is an immutable record.