Facebook bans deceptive deepfakes and some misleadingly modified media

Facebook wants to be the arbiter of truth after all. At least when it comes to intentionally misleading deepfakes and heavily manipulated and/or synthesized media content, such as AI-generated photorealistic human faces that look like real people but aren’t.

In a policy update announced late yesterday, the social network’s VP of global policy management, Monika Bickert, writes that it will take a stricter line on manipulated media content from here on in — removing content that’s been edited or synthesized “in ways that aren’t apparent to an average person and would likely mislead someone into thinking that a subject of the video said words that they did not actually say”.

However edits for quality or cuts and splices to videos that simply curtail or change the order of words are not covered by the ban.

Which means that disingenuous doctoring — such as this example from the recent UK General Election (where campaign staff for one political party edited a video of a politician from a rival party who was being asked a question about brexit to make it look like he was lost for words when in fact he wasn’t) — will go entirely untouched by the new ‘tougher’ policy. Ergo there’s little to trouble Internet-savvy political ‘truth’ spinners here. The disingenuousness digital campaigning can go on.

Instead of grappling with that sort of subtle political fakery, Facebook is focusing on quick PR wins — around the most obviously inauthentic stuff where it won’t risk accusations of partisan bias if it pulls bogus content.

Hence the new policy bans deepfake content that involves the use of AI technologies to “merge, replace or superimpose content onto a video, making it appear to be authentic” — which looks as if it will capture the crudest stuff, such as revenge deepfake porn which superimposes a real person’s face onto an adult performer’s body (albeit nudity is already banned on Facebook’s platform).

It’s not a blanket ban on deepfakes either, though — with some big carve outs for “parody or satire”.

So it’s a bit of an open question whether this deepfake video of Mark Zuckerberg, which went viral last summer — seemingly showing the Facebook founder speaking like a megalomaniac — would stay up or not under the new policy. The video’s creators, a pair of artists, described the work as satire so such stuff should survive the ban. (Facebook did also leave it up at the time.)

But, in future, deepfake creators are likely to further push the line to see what they can get away with under the new policy.

The social network’s controversial policy of letting politicians lie in ads also means it could, technically, still give pure political deepfakes a pass — i.e. if a political advertiser was paying it to run purely bogus content as an ad. Though it would be a pretty bold politician to try that.

More likely there’s more mileage for political campaigns and opinion influencers to keep on with more subtle manipulations. Such as the doctored video of House speaker Nancy Pelosi that went viral on Facebook last year, which had slowed down audio that made her sound drunk or ill. The Washington Post suggests that video — while clearly potentially misleading — still wouldn’t qualify to be taken down under Facebook’s new ‘tougher’ manipulated media policy.

Bickert’s blog post stipulates that manipulated content which doesn’t meet Facebook’s new standard for removal may still be reviewed by the independent third party fact-checkers Facebook relies upon for the lion’s share of ‘truth sifting’ on its platform — and who may still rate such content as ‘false’ or ‘partly false’. But she emphasizes it will continue to allow this type of bogus content to circulate (while potentially reducing its distribution), claiming such labelled fakes provide helpful context.

So Facebook’s updated position on manipulated media sums to ‘no to malicious deepfakes but spindoctors please carry on’.

“If a photo or video is rated false or partly false by a fact-checker, we significantly reduce its distribution in News Feed and reject it if it’s being run as an ad. And critically, people who see it, try to share it, or have already shared it, will see warnings alerting them that it’s false,” Bickert writes, claiming: “This approach is critical to our strategy and one we heard specifically from our conversations with experts.

“If we simply removed all manipulated videos flagged by fact-checkers as false, the videos would still be available elsewhere on the internet or social media ecosystem. By leaving them up and labelling them as false, we’re providing people with important information and context.”

Last month Facebook announced it had unearthed a network of more than 900 fake accounts that had been spreading pro-Trump messaging — some of which had used false profile photos generated by AI.

The dystopian development provides another motivation for the tech giant to ban ‘pure’ AI fakes, given the technology risks supercharging its fake accounts problem. (And, well, that could be bad for business.)

“Our teams continue to proactively hunt for fake accounts and other coordinated inauthentic behavior,” suggests Bickert, arguing that: “Our enforcement strategy against misleading manipulated media also benefits from our efforts to root out the people behind these efforts.”

While still relatively nascent as a technology, deepfakes have shown themselves to be catnip to the media which loves the spectacle they create. As a result, the tech has landed unusually quickly on legislators’ radars as a disinformation risk — California implemented a ban on political deepfakes around elections this fall, for example — so Facebook is likely hoping to score some quick and easy political points by moving in step with legislators even as it applies its own version of a ban.

Bickert’s blog post also fishes for further points, noting Facebook’s involvement in a Deep Fake Detection Challenge which was announced last fall — “to produce more research and open source tools to detect deepfakes”.

While says Facebook has been working with news agency Reuters to offer free online training courses for journalists to help reporters identify manipulated visuals.

“As these partnerships and our own insights evolve, so too will our policies toward manipulated media. In the meantime, we’re committed to investing within Facebook and working with other stakeholders in this area to find solutions with real impact,” she adds.

source https://techcrunch.com/2020/01/07/facebook-bans-deceptive-deepfakes-and-some-misleadingly-modified-media/

Douyin, TikTok app in China, hits 400 million daily active users

China’s TikTok, which has taken the world by storm, is working its magic in its home nation, too.

The Chinese version of TikTok, called Douyin, has amassed 400 million daily active users, parent company ByteDance revealed in its annual report this week (in Chinese). This is an impressive growth for the addictive video app, which had 250 million daily active users in January last year. (A ByteDance spokesperson confirmed the figures to TechCrunch.)

The report, which describes the user behavior and trends, illustrates the cultural difference between China and the U.S., said Katherine Wu, an investor at New York-based firm Notation Capital.

“Things that trend in these two countries are insanely different. For example: knowledge-based content is extremely popular in China, and less so in the U.S. Also, this was wild to me: those creators that did the most dance videos in China are users born in the 60s (!!), whereas in the US, it seems that it’s mostly teenagers who are creating the dances,” she wrote.

ByteDance claimed that Douyin has established itself as the largest knowledge, culture, art, and non-promo platform in China. (Douyin is only available in mainland China.) 14.89 million “knowledge-based content videos” were shared on the app last year, it said.

Citing an example, one of the world’s most valued startup said that one user alone who posts videos on chemistry reached 130 million people last year. On the art and culture front, videos that revolved around those topics had 543.1 billion plays on Douyin last year.

Education has become a crucial use case for ByteDance. TikTok, which is estimated to have clocked more than $50 million in revenue last year, already counts educational content as one of the most consumed categories on its app across the world. The app launched an educational campaign in India, where it has amassed more than 200 million users, last year.

“The behavioral differences also underscore one major thing to me: there really isn’t a ‘one-size-fit-all’ social app that exists. Different regions like different types of content, even though the medium itself (in this case, short-form video) is the same. Cultural factors (geography, age, local pop culture) play a huge role in how users end up using a product,” Wu added.

Marketing research firm eMarketer said on Monday that Douyin’s user growth outpaces those of WeChat and Weibo. But it’s facing challenge from newer entrants such as Tencent-backed video app Kuaishou.

According to eMarketer, Douyin already makes up 67.9% of China’s mobile social network users and 59% of smartphone users. “These figures will grow to 68.3% and 59.6%, respectively, in 2020,” it estimated.

Such user growth and penetration would be music to the ears of ByteDance, which is widely expected to list in Hong Kong this year. But its growing popularity and link to China has also caused some to worry. U.S. senators have warned that user data on TikTok could be sent to China and accessed by the Chinese government. TikTok has denied the accusation. U.S. Navy last month told its members to not install TikTok on their devices, according to media reports.

source https://techcrunch.com/2020/01/06/douyin-tiktok-app-in-china-hits-400-million-daily-active-users/

Facebook data misuse and voter manipulation back in the frame with latest Cambridge Analytica leaks

More details are emerging about the scale and scope of disgraced data company Cambridge Analytica’s activities in elections around the world — via a cache of internal documents that’s being released by former employee and self-styled whistleblower, Brittany Kaiser.

The now shut down data modelling company, which infamously used stolen Facebook data to target voters for President Donald Trump’s campaign in the 2016 U.S. election, was at the center of the data misuse scandal that, in 2018, wiped billions off Facebook’s share price and contributed to a $5BN FTC fine for the tech giant last summer.

However plenty of questions remain, including where, for whom and exactly how Cambridge Analytica and its parent entity SCL Elections operated; as well as how much Facebook’s leadership knew about the dealings of the firm that was using its platform to extract data and target political ads — helped by some of Facebook’s own staff.

Certain Facebook employees were referring to Cambridge Analytica as a “sketchy” company as far back as September 2015 — yet the tech giant only pulled the plug on platform access after the scandal went global in 2018.

Facebook CEO Mark Zuckerberg has also continued to maintain that he only personally learned about CA from a December 2015 Guardian article, which broke the story that Ted Cruz’s presidential campaign was using psychological data based on research covering tens of millions of Facebook users, harvested largely without permission. (It wasn’t until March 2018 that further investigative journalism blew the lid off the story — turning it into a global scandal.)

Former Cambridge Analytica business development director Kaiser, who had a central role in last year’s Netflix documentary about the data misuse scandal (The Great Hack), began her latest data dump late last week — publishing links to scores of previously unreleased internal documents via a Twitter account called @HindsightFiles. (At the time of writing Twitter has placed a temporary limit on viewing the account — citing “unusual activity”, presumably as a result of the volume of downloads it’s attracting.)

Since becoming part of the public CA story Kaiser has been campaigning for Facebook to grant users property rights over their data. She claims she’s releasing new documents from her former employer now because she’s concerned this year’s US election remains at risk of the same type of big-data-enabled voter manipulation that tainted the 2016 result.

“I’m very fearful about what is going to happen in the US election later this year, and I think one of the few ways of protecting ourselves is to get as much information out there as possible,” she told The Guardian.

“Democracies around the world are being auctioned to the highest bidder,” is the tagline clam on the Twitter account Kaiser is using to distribute the previously unpublished documents — more than 100,000 of which are set to be released over the coming months, per the newspaper’s report.

The releases are being grouped into countries — with documents to-date covering Brazil, Kenya and Malaysia. There is also a themed release dealing with issues pertaining to Iran, and another covering CA/SCL’s work for Republican John Bolton’s Political Action Committee in the U.S.

The releases look set to underscore the global scale of CA/SCL’s social media-fuelled operations, with Kaiser writing that the previously unreleased emails, project plans, case studies and negotiations span at least 65 countries.

A spreadsheet of associate officers included in the current cache lists SCL associates in a large number of countries and regions including Australia, Argentina, the Balkans, India, Jordan, Lithuania, the Philippines, Switzerland and Turkey, among others. A second tab listing “potential” associates covers political and commercial contacts in various other places including Ukraine and even China.

A UK parliamentary committee which investigated online political campaigning and voter manipulation in 2018 — taking evidence from Kaiser and CA whistleblower Chris Wylie, among others — urged the government to audit the PR and strategic communications industry, warning in its final report how “easy it is for discredited companies to reinvent themselves and potentially use the same data and the same tactics to undermine governments, including in the UK”.

“Data analytics firms have played a key role in elections around the world. Strategic communications companies frequently run campaigns internationally, which are financed by less than transparent means and employ legally dubious methods,” the DCMS committee also concluded.

The committee’s final report highlighted election and referendum campaigns SCL Elections (and its myriad “associated companies”) had been involved in in around thirty countries. But per Kaiser’s telling its activities — and/or ambitions — appear to have been considerably broader and even global in scope.

Documents released to date include a case study of work that CA was contracted to carry out in the U.S. for Bolton’s Super PAC — where it undertook what is described as “a personality-targeted digital advertising campaign with three interlocking goals: to persuade voters to elect Republican Senate candidates in Arkansas, North Carolina and New Hampshire; to elevate national security as an issue of importance and to increase public awareness of Ambassador Bolton’s Super PAC”.

Here CA writes that it segmented “persuadable and low-turnout voter populations to identify several key groups that could be influenced by Bolton Super PAC messaging”, targeting them with online and Direct TV ads — designed to “appeal directly to specific groups’ personality traits, priority issues and demographics”. 

Psychographic profiling — derived from CA’s modelling of Facebook user data — was used to segment U.S. voters into targetable groups, including for serving microtargeted online ads. The company badged voters with personality-specific labels such as “highly neurotic” — targeting individuals with customized content designed to pray on their fears and/or hopes based on its analysis of voters’ personality traits.

#CONSCIENTIOUS? As in do you care about your #children & talk about them on social media? Well, then your data has been categorized to say this is the message @AmbJohnBolton + #CambridgeAnalytica had for you #NorthCarolina #Hindsightis2020 https://t.co/yYYDm9UTNA pic.twitter.com/C0gHa1fdGG

— Hindsight is 2020 (@HindsightFiles) January 3, 2020

The process of segmenting voters by personality and sentiment was made commercially possible by access to identity-linked personal data — which puts Facebook’s population-scale collation of identities and individual-level personal data squarely in the frame.

It was a cache of tens of millions of Facebook profiles, along with responses to a personality quiz app linked to Facebook accounts, which was sold to Cambridge Analytica in 2014, by a company called GSR, and used to underpin its psychographic profiling of U.S. voters.

In evidence to the DCMS committee last year GSR’s co-founder, Aleksandr Kogan, argued that Facebook did not have a “valid” developer policy at the time, since he said the company did nothing to enforce the stated T&Cs — meaning users’ data was wide open to misappropriation and exploitation.

The UK’s data protection watchdog also took a dim view. In 2018 it issued Facebook with the maximum fine possible, under relevant national law, for the CA data breach — and warned in a report that democracy is under threat. The country’s information commissioner also called for an “ethical pause” of the use of online microtargeting ad tools for political campaigning.

No such pause has taken place.

Meanwhile for its part, since the Cambridge Analytica scandal snowballed into global condemnation of its business, Facebook has made loud claims to be ‘locking down’ its platform — including saying it would conduct an app audit and “investigate all apps that had access to large amounts of information”; “conduct a full audit of any app with suspicious activity”; and “ban any developer from our platform that does not agree to a thorough audit”.

However, close to two years later, there’s still no final report from the company on the upshot of this self ‘audit’.

And while Facebook was slapped with a headline-grabbing FTC fine on home soil, there was in fact no proper investigation; no requirement for it to change its privacy-hostile practices; and blanket immunity for top execs — even for any unknown data violations in the 2012 to 2018 period. So, ummm…

In another highly curious detail, GSR’s other co-founder, a data scientist called Joseph Chancellor, was in fact hired by Facebook in late 2015. The tech giant has never satisfactorily explained how it came to recruit one of the two individuals at the center of a voter manipulation data misuse scandal which continues to wreak hefty reputational damage on Zuckerberg and his platform. But being able to ensure Chancellor was kept away from the press during a period of intense scrutiny looks pretty convenient.

Aleksandr Kogan and Joseph Chancellor were deceptively harvesting/matching data with that personality quiz on Facebook as GSR for SCL. John Bolton PAC was the client. @chrisinsilico’s papers to DCMS committee have the emails about delivering the data to Bolton. https://t.co/NLagY2PWEG

— David Carroll https://techcrunch.com/2020/01/06/facebook-data-misuse-and-voter-manipulation-back-in-the-frame-with-latest-cambridge-analytica-leaks/

Twitter offers more support to researchers — to ‘keep us accountable’

Twitter has kicked off the New Year by taking the wraps off a new hub for academic researchers to more easily access information and support around its APIs — saying the move is in response to feedback from the research community.

The new page — which it’s called ‘Twitter data for academic researchers’ — can be found here.

It includes links to apply for a developer account to access Twitter’s APIs; details of the different APIs offered and links to additional tools for researchers, covering data integration and access; analysis; visualization; and infrastructure and hosting.

“Over the past year, we’ve worked with many of you in the academic research community. We’ve learned about the challenges you face, and how Twitter can better support you in your efforts to advance understanding of the public conversation,” the social network writes, saying it wants to “make it even easier to learn from the public conversation”.

Twitter is also promising “more enhancements and resources” for researchers this year.

It’s likely no accident the platform is putting a fresh lick of paint on its offerings for academics given that 2020 is a key election year in the U.S. — and concerns about the risk of fresh election meddling are riding high.

Tracking conversation flow on Twitter also still means playing a game of ‘bot or not’ — one that has major implications for the health of democracies. And in Europe Twitter is one of a number of platform giants which, in 2018, signed up to a voluntary Code of Practice on disinformation that commits it to addressing fake accounts and online bots, as well as to empowering the research community to monitor online disinformation via “privacy-compliant” access to platform data.

“At Twitter, we value the contributions of academic researchers and see the potential for them to help us better understand our platform, keeping us accountable, while helping us tackle new challenges through discoveries and innovations,” the company writes on the new landing page for researchers while also taking the opportunity to big up the value of its platform — claiming that “if it exists, it’s probably been talked about on Twitter”.

If Twitter lives up to its promises of active engagement with researchers and their needs, it could smartly capitalism on rival Facebook’s parallel missteps in support for academics.

Last year Facebook was accused of ‘transparency-washing’ with its own API for researchers, with a group of sixty academics slamming the ad archive API as as much a hinderance as a help.

Months later Facebook was still being reported to have done little to improve the offering.

source https://techcrunch.com/2020/01/06/twitter-offers-more-support-to-researchers-to-keep-us-accountable/

India’s ruling party accused of running deceptive Twitter campaign to gain support for a controversial law

Bharatiya Janata Party, the ruling party in India, has been accused of running a highly deceptive Twitter campaign to trick citizens into supporting a controversial law.

First, some background: The Indian government passed the Citizenship Amendment Act (CAA) last month that eases the path of non-Muslim minorities from the neighboring Muslim-majority nations of Afghanistan, Bangladesh and Pakistan to gain Indian citizenship.

But, combined with a proposed national register of citizens, critics have cautioned that it discriminates against minority Muslims in India and chips away at India’s secular traditions.

Over the past few weeks, tens of thousands of people in the country — if not more — have participated in peaceful protests across the nation against the law. The Indian government, which has temporarily cut down internet access and mobile communications in many parts of India to contain the protests, has so far shown no signs of withdrawing the law.

On Saturday, it may have found a new way to gain support for it, however.

India’s Home Minister Amit Shah on Thursday tweeted a phone number, urging citizens to place a call to that number in “support of the CAA law.”

Thousands of people in India today, many affiliated with the BJP party, began circulating that phone number on Twitter with the promise that anyone who places a call would be offered job opportunities, free mobile data, Netflix credentials, and even company with “lonely women.”

The story of CAA support, in four pictures… pic.twitter.com/ueLNmqDRr8

— Meghnad (@Memeghnad) January 4, 2020

Desperate times call for desperate measures…#CAA pic.twitter.com/zzMGDyMmPP

— SamSays (@samjawed65) January 4, 2020

This is absolutely fake. If you want free Netflix please use someone else's account like the rest of us. https://t.co/PHhwdA3sEI

— Netflix India (@NetflixIndia) January 4, 2020

Huffington Post India called the move latest “BJP ploy” to win support for its controversial law. BoomLive, a fact checking organization based in India, reported the affiliation of many of these people to the ruling party.

We have reached out to a BJP spokesperson and Twitter spokespeople for comment.

First time in 70 years that a legislation passed by Parliament needs huge rallies, promises of sex, jobs and Netflix accounts to drum up support.

— Rohini Singh (@rohini_sgh) January 4, 2020

If the allegations are true, this won’t be the first time BJP has used Twitter to aggressively promote its views. In 2017, BuzzFeed News reported that a number of political hashtags that appeared in the top 10 Twitter’s trends column in India were the result of organized campaigns.

Pratik Sinha, co-founder of fact-checking website Alt News, last year demonstrated how easy it was to manipulate many politicians in the country to tweet certain things after he gained accessed to a Google document of prepared statements and tinkered with the content.

Last month, snowfall in Kashmir, a highly sensitive region that hasn’t had internet connection for more than four months, began trending on Twitter in the U.S. It mysteriously disappeared after many journalists questioned how it made it to the list.

Very curious what is going on with this “trend” about a Kashmir, which hasn’t had internet since August. It’s not labeled an ad. What’s going on here @Jack? pic.twitter.com/KGF8kiPO0o

— Julia Carrie Wong (@juliacarriew) December 22, 2019

When we reached out, a Twitter spokesperson in India pointed TechCrunch to an FAQ article that explained how Trending Topics work. Nothing in the FAQ article addressed the question.

source https://techcrunch.com/2020/01/04/indias-ruling-party-accused-of-running-deceptive-twitter-campaign-to-gain-support-for-a-controversial-law/

Snapchat quietly acquired AI Factory, the company behind its new Cameos feature, for $166M

After acquiring Ukraine startup Looksery in 2015 to supercharge animated selfie lenses in Snapchat — arguably changing the filters game for all social video and photo apps — Snap has made another acquisition with roots in the country, co-founded by one of Looksery’s founders, to give a big boost to its video capabilities.

The company has acquired AI Factory, a computer vision startup that Snap had worked with to create Snapchat’s new Cameos animated selfie-based video feature, for a price believed to be in the region of $166 million.

The news was first reported by a Ukrainian publication AIN, and while I’m still waiting for a direct reply from Snap about the acquisition, I’ve had the news confirmed by another source close to the deal. Snap also confirmed the news to another publication, VC.ru (in Russian).

Victor Shaburov, the founder of Looksery who then went on to become Snap’s director of engineering — leaving in May 2018 to found and lead AI Factory — declined to provide a comment for this story.

Cameos, launched last month, lets you take a selfie, which is then automatically “animated” and inserted into a short video. The selection of videos, currently around 150, is created by Snap, with the whole concept not unlike the one underpinning ‘deepfakes’ — AI-based videos that look “real” but are actually things that never really happened.

Deepfake videos have been around for a while. But if your experience of that word has strong dystopian undertones, we now appear to be in a moment where consumer apps are tapping into the technology in a race for new — fun, lighthearted — features to attract and keep users. Just today, Josh reported that TikTok has secretly built a deepfake tool, too. I expect we’ll be hearing about Facebook’s newest deepfake tool in 3, 2, 1…

From what I understand, while AI Factory has offices in San Francisco, the majority of the team of around 70 is based out of Ukraine. Part of the team will relocate with the deal, and part will stay there.

Snap had also been an investor in AI Factory. Part of its early interest would have been because of the track record of the talent associated with the startup: lenses have been a huge success for Snap — 70% of its daily active users play with them, and they not only bring in new users, but increase retention and bring in revenues by way of sponsorships or users buying them — so creating new features to give users more ways to play around with their selfies is a good bet.

It’s not clear whether AI Factory will be developing a way to insert selfies into any video, or if the feature will be tied just to specific videos offered by Snap itself, or whether the videos will extend beyond the timing of a GIF. It’s also not clear what else AI Factory was working on: the company’s site is offline and there is very little information about the company beyond its mission to bring more AI-based imaging tools into mainstream apps and usage.

The company’s LinkedIn profile says that AI Factory “provide[s] multiple AI business solutions based on image and video recognition, analysis and processing,” so while the company will come under Snap’s wing, there may be scope for the team to build some of its technology into more innovative ways for businesses to use the Snap platform in the future, too.

We’ll update this post as we learn more, and if/when we hear back from Snap directly.

source https://techcrunch.com/2020/01/03/snapchat-quietly-acquired-ai-factory-the-company-behind-its-new-cameos-feature-for-166m/

ByteDance & TikTok have secretly built a Deepfakes maker

TikTok parent company ByteDance has teamed up with one of the most controversial apps to let you insert your face into videos starring someone else. TechCrunch has learned that ByteDance has developed an unreleased feature using life-like Deepfakes technology that the app’s code refers to as Face Swap. Code in both TikTok and its Chinese sister app Douyin asks users to take a multi-angle biometric scan of their face, then choose from a selection of videos they want to add their face to and share.

Users scan themselves, pick a video, and have their face overlaid on the body of someone in the clip with ByteDance’s new Face Swap feature

The Face Swap option was built atop the API of Chinese Deepfakes app Zao, which uses artificial intelligence to blend one person’s face into another’s body as they move and synchronize their expressions. Zao went viral in September despite privacy and security concerns about how users’ facial scans might be abused.

The Deepfakes feature, if launched in Douyin and TikTok, could create a more controlled environment where face swapping technology plus a limited selection of source videos  can be used for fun instead of spreading misinformation. It might also raise awareness of the technology so more people are aware that they shouldn’t believe everything they see online. But it’s also likely to heighten fears about what Zao and ByteDance could do with such sensitive biometric data — similar to what’s used to set up FaceID on iPhones. Zao was previously blocked by China’s WeChat for presenting “security risks”.

Several other tech companies have recently tried to consumerize watered-down versions of Deepfakes. The app Morphin lets you overlay a computerized rendering of your face on actors in GIFs. Snapchat offered a FaceSwap option for years that would switch the visages of two people in frame, or replace one on camera with one from your camera roll, and there are standalone apps that do that too like Face Swap Live. Then last month, TechCrunch spotted Snapchat’s new Cameos for inserting a real selfie into video clips it provides, though the results aren’t meant to look confusingly realistic.

But ByteDance’s teamup with Zao could bring convincingly life-like Deepfakes to TikTok and Douyin, two of the world’s most popular apps with over 1.5 billion downloads.

Zao in the Chinese iOS App Store

Hidden Inside TikTok and Douyin

TechCrunch received a tip about the news from Israeli in-app market research startup Watchful.ai. The company had discovered code for the Deepfakes feature in the latest version of TikTok’s and Douyin’s Android apps. Watchful.ai was able to activate the code in Douyin to generate screenshots of the feature, though it’s not currently available to the public.

First, users scan their face into TikTok. This also serves as an identity check to make sure you’re only submitting your own face so you can’t make unconsented Deepfakes of anyone else using an existing photo or a single shot of their face. By asking you to blink, nod, and open and close your mouth while in focus and proper lighting, Douyin can ensure you’re a live human and create a manipulable scan of your face that it can stretch and move to express different emotions or fill different scenes.

You’ll then be able to pick from videos ByteDance claims to have the rights to use, and it will replace the face of whoever’s in the clip with your own. You can then share or download the Deepfake video, though it will include an overlayed watermark the company claims will help distinguish the content as not being real.

Code in th apps reveals that the Face Swap feature relies on the Zao API. There are many references to this API including strings like:

zaoFaceParams
/media/api/zao/video/create
enter_zaoface_preview_page
isZaoVideoType
zaoface_clip_edit_page

Watchful also discovered unpublished updates to TikTok and Douyin’s terms of service that cover privacy and usage of the Deepfakes feature. Inside the US version of TikTok’s Android app, English text in the code explains the feature and some of its terms of use:

“Your facial pattern will be used for this feature. Read the Drama Face Terms of Use and Privacy Policy for more details. Make sure you’ve read and agree to the Terms of Use and Privacy Policy before continuing. 1. To make this feature secure for everyone, real identity verification is required to make sure users themselves are using this feature with their own faces. For this reason, uploaded photos can’t be used; 2. Your facial pattern will only be used to generate face-change videos that are only visible to you before you post it. To better protect your personal information, identity verification is required if you use this feature later. 3. This feature complies with Internet Personal Information Protection Regulations for Minors. Underage users won’t be able to access this feature. 4. All video elements related to this feature provided by Douyin have acquired copyright authorization.”

ZHEJIANG, CHINA – OCTOBER 18 2019 Two us senators have sent a letter to the us national intelligence agency saying TikTok could pose a threat to us national security and should be investigated. Visitors visit the booth of douyin(Tiktok) at the 2019 smart expo in hangzhou, east China’s zhejiang province, Oct. 18, 2019.- PHOTOGRAPH BY Costfoto / Barcroft Media (Photo credit should read Costfoto / Barcroft Media via Getty Images)

A longer terms of use and privacy policy was also found in Chinese within Douyin. Translated into English, some highlights from the text include:

• “The ‘face-changing’ effect presented by this function is a fictional image generated by the superimposition of our photos based on your photos. In order to show that the original work has been modified and the video generated using this function is not a real video, we will mark the video generated using this function. Do not erase the mark in any way.”

• “The information collected during the aforementioned detection process and using your photos to generate face-changing videos is only used for live detection and matching during face-changing. It will not be used for other purposes . . . And matches are deleted immediately and your facial features are not stored.”
  

• “When you use this function, you can only use the materials provided by us, you cannot upload the materials yourself. The materials we provide have been authorized by the copyright owner”.

• “According to the ‘Children’s Internet Personal Information Protection Regulations’ and the relevant provisions of laws and regulations, in order to protect the personal information of children / youths, this function restricts the use of minors”.

We reached out to TikTok and Douyin for comment regarding the Deepfakes feature, when it might launch, how the privacy of biometric scans are protected, the age limit, and the nature of its relationship with Zao. However, TikTok declined to answer those questions. Instead a spokesperson insisted that “after checking with the teams I can confirm this is definitely not a function in TikTok, nor do we have any intention of introducing it. I think what you may be looking at is something slated for Douyin – your email includes screenshots that would be from Douyin, and a privacy policy that mentions Douyin. That said, we don’t work on Douyin here at TikTok.” They later told TechCrunch that “The inactive code fragments are being removed to eliminate any confusion”, which implicitly confirms that Face Swap code was found in TikTok.

A Douyin spokesperson told TechCrunch that “Douyin has no cooperation with Zao” despite references to Zao in the code. They also denied that the Face Swap terms of service appear in TikTok despite TechCrunch reviewing code from the app showing those terms of service and the feature’s functionality.

This is suspicious, and doesn’t explain why code for the Deepfakes feature and special terms of service in English for the feature appear in TikTok, and not just Douyin where the app can already be activated and a longer terms of service was spotted. TikTok’s US entity has previously denied complying with censorship requests from the Chinese government in contradiction to sources who told the Washington Post and that TikTok did censor some political and sexual content at China’s behest.

It’s possible that the Deepfakes Face Swap feature never officially launches in China or the US. But it’s fully functional, even if unreleased, and demonstrates ByteDance’s willingness to embrace the controversial technology despite its reputation for misinformation and non-consensual pornography. At least it’s restricting the use of the feature by minors, only letting you face-swap yourself, and preventing users from uploading their own source videos. That avoid it being used to create dangerous misinformation Deepfakes like the one making House Speaker Nancy Pelosi seem drunk.

“It’s very rare to see a major social networking app restrict a new, advanced feature to their users 18 and over only” Watchful.ai co-founder and CEO Itay Kahana tells TechCrunch. “These deepfake apps might seem like fun on the surface, but they should not be allowed to become trojan horses, compromising IP rights and personal data, especially personal data from minors who are overwhelmingly the heaviest users of TikTok to date.”

TikTok has already been banned by the US Navy and ByteDance’s acquisition and merger of Musically into TikTok is under investigation by the Comittee On Foreign Investment In The United States. Deepfake fears could further heighten scrutiny.

With the proper safeguards, though, face-changing technology could usher in a new era of user generated content where the creator is always at the center of the action. It’s all part of a new trend of personalized media that could be big in 2020. Social media has evolved from selfies to Bitmoji to Animoji to Cameos and now consumerized Deepfakes. When there are infinite apps and videos and notifications to distract us, making us the star could be the best way to hold our attention.

source https://techcrunch.com/2020/01/03/tiktok-deepfakes-face-swap/