Tuesday, 22 May 2018

Backlink Blindspots: The State of Robots.txt

Posted by rjonesx.

Here at Moz we have committed to making Link Explorer as similar to Google as possible, specifically in the way we crawl the web. I have discussed in previous articles some metrics we use to ascertain that performance, but today I wanted to spend a little bit of time talking about the impact of robots.txt and crawling the web.

Most of you are familiar with robots.txt as the method by which webmasters can direct Google and other bots to visit only certain pages on the site. Webmasters can be selective, allowing certain bots to visit some pages while denying other bots access to the same. This presents a problem for companies like Moz, Majestic, and Ahrefs: we try to crawl the web like Google, but certain websites deny access to our bots while allowing that access to Googlebot. So, why exactly does this matter?

Why does it matter?

Graph showing how crawlers hop from one link to another

As we crawl the web, if a bot encounters a robots.txt file, they're blocked from crawling specific content. We can see the links that point to the site, but we're blind regarding the content of the site itself. We can't see the outbound links from that site. This leads to an immediate deficiency in the link graph, at least in terms of being similar to Google (if Googlebot is not similarly blocked).

But that isn't the only issue. There is a cascading failure caused by bots being blocked by robots.txt in the form of crawl prioritization. As a bot crawls the web, it discovers links and has to prioritize which links to crawl next. Let's say Google finds 100 links and prioritizes the top 50 to crawl. However, a different bot finds those same 100 links, but is blocked by robots.txt from crawling 10 of the top 50 pages. Instead, they're forced to crawl around those, making them choose a different 50 pages to crawl. This different set of crawled pages will return, of course, a different set of links. In this next round of crawling, Google will not only have a different set they're allowed to crawl, the set itself will differ because they crawled different pages in the first place.

Long story short, much like the proverbial butterfly that flaps its wings eventually leading to a hurricane, small changes in robots.txt which prevent some bots and allow others ultimately leads to very different results compared to what Google actually sees.

So, how are we doing?

You know I wasn't going to leave you hanging. Let's do some research. Let's analyze the top 1,000,000 websites on the Internet according to Quantcast and determine which bots are blocked, how frequently, and what impact that might have.

Methodology

The methodology is fairly straightforward.

  1. Download the Quantcast Top Million
  2. Download the robots.txt if available from all top million sites
  3. Parse the robots.txt to determine whether the home page and other pages are available
  4. Collect link data related to blocked sites
  5. Collect total pages on-site related to blocked sites.
  6. Report the differences among crawlers.

Total sites blocked

The first and easiest metric to report is the number of sites which block individual crawlers (Moz, Majestic, Ahrefs) while allowing Google. Most site that block one of the major SEO crawlers block them all. They simply formulate robots.txt to allow major search engines while blocking other bot traffic. Lower is better.

Bar graph showing number of sites blocking each SEO tool in robots.txt

Of the sites analyzed, 27,123 blocked MJ12Bot (Majestic), 32,982 blocked Ahrefs, and 25,427 blocked Moz. This means that among the major industry crawlers, Moz is the least likely to be turned away from a site that allows Googlebot. But what does this really mean?

Total RLDs blocked

As discussed previously, one big issue with disparate robots.txt entries is that it stops the flow of PageRank. If Google can see a site, they can pass link equity from referring domains through the site's outbound domains on to other sites. If a site is blocked by robots.txt, it's as though the outbound lanes of traffic on all the roads going into the site are blocked. By counting all the inbound lanes of traffic, we can get an idea of the total impact on the link graph. Lower is better.

According to our research, Majestic ran into dead ends on 17,787,118 referring domains, Ahrefs on 20,072,690 and Moz on 16,598,365. Once again, Moz's robots.txt profile was most similar to that of Google's. But referring domains isn't the only issue with which we should be concerned.

Total pages blocked

Most pages on the web only have internal links. Google isn't interested in creating a link graph — they're interested in creating a search engine. Thus, a bot designed to act like Google needs to be just as concerned about pages that only receive internal links as they are those that receive external links. Another metric we can measure is the total number of pages that are blocked by using Google's site: query to estimate the number of pages Google has access to that a different crawler does not. So, how do the competing industry crawlers perform? Lower is better.

Once again, Moz shines on this metric. It's not just that Moz is blocked by fewer sites— Moz is blocked by less important and smaller sites. Majestic misses the opportunity to crawl 675,381,982 pages, Ahrefs misses 732,871,714 and Moz misses 658,015,885. There's almost an 80 million-page difference between Ahrefs and Moz just in the top million sites on the web.

Unique sites blocked

Most of the robots.txt disallows facing Moz, Majestic, and Ahrefs are simply blanket blocks of all bots that don't represent major search engines. However, we can isolate the times when specific bots are named deliberately for exclusion while competitors remain. For example, how many times is Moz blocked while Ahrefs and Majestic are allowed? Which bot is singled out the most? Lower is better.

Ahrefs is singled out by 1201 sites, Majestic by 7152 and Moz by 904. It is understandable that Majestic has been singled out, given that they have been operating a very large link index for many years, a decade or more. It took Moz 10 years to accumulate 904 individual robots.txt blocks, and took Ahrefs 7 years to accumulate 1204. But let me give some examples of why this is important.

If you care about links from name.com, hypermart.net, or eclipse.org, you can't rely solely on Majestic.

If you care about links from popsugar.com, dict.cc, or bookcrossing.com, you can't rely solely on Moz.

If you care about links from dailymail.co.uk, patch.com, or getty.edu, you can't rely solely on Ahrefs.

And regardless of what you do or which provider you use, you can't links from yelp.com, who.int, or findarticles.com.

Conclusions

While Moz's crawler DotBot clearly enjoys the closest robots.txt profile to Google among the three major link indexes, there's still a lot of work to be done. We work very hard on crawler politeness to ensure that we're not a burden to webmasters, which allows us to crawl the web in a manner more like Google. We will continue to work more to improve our performance across the web and bring to you the best backlink index possible.

Thanks to Dejan SEO for the beautiful link graph used in the header image and Mapt for the initial image used in the diagrams.


Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!



source https://moz.com/blog/blindspots-in-the-link-graph

Monday, 21 May 2018

Progressive advocacy groups call on the FTC to ‘make Facebook safe for democracy’

A team of progressive advocacy groups, including MoveOn and Demand Progress, are asking the Federal Trade Commission to “make Facebook safe for democracy.” According to Axios, the campaign, called Freedom From Facebook, is set to launch a six-figure ad campaign on Monday that will run on Facebook, Instagram and Twitter, among other platforms.

The other advocacy groups behind the campaign are Citizens Against Monopoly, Content Creators Coalition, Jewish Voice for Peace, Mpower Change, Open Markets Institute and SumOfUs. Together they are calling on the FTC to “break up Facebook’s monopoly” by forcing it to spin-off Instagram, WhatsApp and Messenger into separate, competing companies. They also want the FTC to require interoperability so users can communicate across competing social networks and strengthen privacy regulations.

Freedom From Facebook’s site also includes an online petition and privacy guide that links to FB Purity and the Electronic Frontier Foundation’s Privacy Badger, browser extensions that help users streamline their Facebook ad preferences and block online trackers, respectively.

The FTC recently gained a new chairman after President Donald Trump’s pick for the position Joseph Simons was sworn in early this month, along with four new commissioners also nominated by Trump. Simons is an antitrust lawyer who has represented large tech firms like Microsoft and Sony. The FTC is currently investigating whether or not Facebook’s involvement with Cambridge Analytica violated a previous legal agreement it had with the commission, but many people are wondering if it and other federal agencies are capable of regulating tech companies, especially after many lawmakers seemed confused about how social media works during Facebook CEO Mark Zuckerberg’s congressional hearing last month.

Despite its data privacy and regulatory issues, Facebook is still doing well from a financial perspective. Its first-quarter earnings report showed strong user growth and revenue above Wall Street’s expectations.

TechCrunch has contacted Freedom From Facebook and Facebook for comment.



source https://techcrunch.com/2018/05/20/progressive-advocacy-groups-call-on-the-ftc-to-make-facebook-safe-for-democracy/

Are algorithms hacking our thoughts?

SEO Isn’t the Only Way to Drive Traffic. Here Are 6 Alternative Strategies.

As marketers, we invest a lot of our time and energy into SEO.

Considering that Google receives over 66,000 searches every second, we’d be stupid not to.

But when it comes down to it, Google and other search engines are just one of the many ways you can drive traffic to your website.

In fact, depending on your target audience and competition, Google may not even be your best traffic source.

Take Upworthy, for example.

This site, which has a reputation for sharing feel-good viral videos, gets almost 43% of their traffic from social and less than 19% from search.

Groups of consumers all use the web differently.

Some like to get their content from friends or influencers on social.

A few depend on their trusted newsletters to tell them what content they need to read.

Others use an alternative search engine like YouTube or even Facebook.

To prove that you don’t need Google to drive traffic to your website, here are six alternatives that can help boost your visits.



  1. Create a YouTube channel



  2. Start conversations on social



  3. Partner with influencers



  4. Take advantage of your email subscribers



  5. Provide assistance on forums and question sites



  6. Take advantage of guest posting

1. Create a YouTube channel

YouTube has 1.5 billion active users each month, making it the world’s second-largest social platform.

With 1.5 billion active users each month, YouTube is the world’s second-largest social platform.

active users on social platforms 2017

Over 30 million users log on to watch a total of 5 billion videos each and every day.

These numbers are massive.

However, these numbers still don’t rival Google, which processes about 3.5 billion searches per day.

So, how can you drive even a fraction of the traffic with YouTube that you could with Google?

Well, the beauty of YouTube is that it allows you to share videos.

While Google does display YouTube videos as part of its results lists (Google does, after all, own YouTube), the results aren’t quite the same.

In order to appeal and reach these video-lovers, you need to create YouTube content.

Because YouTube reaches more people from the ages of 18-49 than any cable network, this is particularly true if you’re marketing to younger generations.

youtube 18-49 age demographic reaches more than network or broadcast television

Gen Z’s love affair with YouTube runs especially deep.

In fact, 50% of Gen Z-ers told AdWeek they wouldn’t be able to live without the sharing site.

They also shared why.

For Gen Z-ers, YouTube is more than just a video-sharing site. Look at all of the things that they go to YouTube for:

generation z social media survey

If Gen Z’s love for videos continues as they grow older and start making new purchasing decisions, this may be a major game-changer for marketing in years to come.

Uploading videos to YouTube allows you to appear in front of customers who use the site like a traditional search engine.

Then, with appropriately-placed links, you can drive those users back to your website.

Let’s take a look at how GoPro’s YouTube channel does this.

As one of the most popular YouTube channels around, GoPro uses their high-quality videos to show what their product is capable of doing.

However, simply uploading videos doesn’t drive sales.

That’s why they also make it easy for customers to get from their YouTube channel to a product page.

Here, we have a video shot users shot with the GoPro HERO6:

GoPro hero6 YouTube video

Right within the first couple lines of the description, viewers have a link that will bring them straight to the product page for the GoPro HERO6.

Interested customers can check out what the new camera is capable of, then easily make their way over to the GoPro website to get additional details or make a purchase.

And this isn’t the only way GoPro pushes traffic to their website.

You can also find a link to their website and their social media profiles within their cover image.

gopro hero image

This gives YouTube users who land on the company’s channel an opportunity to get to their website quickly.

You can also add links to your website within your actual video content.

YouTube even offers features that allow users to create interactive video ads.

interactive video advertisement options in youtube

These features display either over the video itself or at the end of a video, prompting viewers to make the next move.

You’ll see from this chart that both call-to-action overlays and auto end screens help drive traffic to your website.

Here’s an example of a call-to-action overlay from a HubSpot video.

hubspot youtube video has link

YouTube features the link right within the middle of the video. HubSpot timed it perfectly so that it appears when the viewer has a thorough enough understanding of inbound marketing to start feeling enticed.

Now, let’s compare this to an auto end screen.

Here’s an example from The Tonight Show Starring Jimmy Fallon.

jimmy fallon youtube end screen

This presents the viewer with a number of different options when they’ve completed a video. It includes an offer to head to the NBC website to download the show’s app.

When using either of these interactive features, be sure to provide a clear and direct value to your watcher.

Also, keep in mind that you don’t want to plaster over your video with links to dozens of content pieces.

Instead, focus on just one or two that truly support the contents of your video.

2. Start conversations on social

If you do it correctly, social can rival Google as a traffic source.

According to Sprout Social, 48% of Millennials and 48% of Gen X-ers followed a brand on social media in Q1 of 2017.

which generations are following brands on social

This means you have less noise to cut through to get to your target audience.

Additionally, social provides you with an opportunity to engage and entertain – something you’ll struggle to do on a search engine.

When social users scroll through their timelines and news feeds, they’re looking for just about anything that will pique their interest.

If you’re capable of providing something high-quality and interesting, they’ll click through to your site to check out more.

However, it’s important to recognize how social is changing for brands.

In 2017, social accounted for 25.6% of all site visits.

social vs search referral traffic

But this number is actually down from past years.

From the same study, we can see that from about the end of 2013 to the beginning of 2017, social was outperforming search for driving traffic.

Unfortunately, the changes to the Facebook algorithm led to serious declines in organic reach on the platform.

Compared to the first half of 2017, Facebook saw about a 9% drop in social media referrals in the second half of the year.

social media referral web traffic

However, one social site’s loss is another’s gain.

While Facebook saw a large dip at the end of 2017, both Pinterest and Instagram saw some pretty significant jumps.

Regardless of the social networks you choose to target, you want to provide meaningful and valuable information.

Remember that users log onto social to engage with their friends – not to have someone sell them something.

You want to find ways to engage your user while they’re still on the platform.

Buzzfeed Tasty does an excellent job at this.

buzzfeed tasty facebook video

Within their Facebook post, they include it all.

They created a video to grab attention.

They have friendly messages that establish the viewer as their friend.

And they include not one but two links that can either push the user to buy or to check out the recipe from the video.

And they do all of that without making it feel overwhelming!

This tactic has gained the Tasty Facebook page serious attention – almost 95 million followers!

To replicate this for your own audience, know your audience and create social content that blends seamlessly into their timelines.

Another way to get a conversation started is to use Twitter threads.

Twitter threads allow you to share insights, tips, and thoughts that connect by replying to yourself – and they have the power to bring you viral attention.

Check out what happened to travel blogger Hey Ciara.

On January 1st, she started this thread.

cheap flights twitter thread

With this one thread alone, Ciara gained over 13,000 new Twitter followers, 6,000 new Instagram followers, and a 10x increase in her blog traffic.

Threads like these provide a distinct value to followers in a relatable way.

After sharing her secrets, Ciara drops a link to a blog post on her website containing more information.

ciara johnson tweets blog post

Though the engagement with the link is significantly less than the first post, it was still able to drive traffic.

She already established authority with her followers in the thread. They trust her and know that she has information worth sharing. That gives them a reason to click on the link.

Creating a Twitter thread typically takes minimal effort. However, you want to be prepared to keep the momentum going.

Know the direction you’re going to take before you begin and be ready to share images, videos, or links that support your claims.

3. Partner with influencers

There are a lot of scams on the web, making shoppers wary about who they choose to purchase from.

If they’ve never heard of a business, they’re not going to pull out their credit cards after seeing just one ad.

Before they’re ready to shop, they need to trust you.

Unfortunately, it takes time to build trust organical

source https://blog.kissmetrics.com/alternative-strategies-to-drive-traffic/

Where to watch Zuckerberg’s meeting with EU MEPs on Tuesday

The Facebook founder Mark Zuckerberg’s meeting with elected representatives of the European Union’s ~500 million citizens will be livestreamed after all, it was confirmed today.

MEPs had been angered by the original closed door format of the meeting, which was announced by the EU parliament’s president last week. But on Friday a majority of the political groups in the parliament had pushed for it to be broadcast online.

This morning president Antonio Tajani confirmed that Facebook had agreed to the 1hr 15 minute hearing being livestreamed.

A Facebook spokesperson also sent us this short statement today: “We’re looking forward to the meeting and happy for it to be live streamed.”

When is the meeting?

The meeting will take place on Tuesday May 22 at 18.15 to 19.30CET. If you want to tune in from the US the meeting is scheduled to start at 9.15PT /12.15ET.

Tajani’s announcement last week said it would start earlier, at 17.45CET, so the meeting appears to have been bumped on by half an hour. We’ve asked Facebook whether Zuckerberg will meet in private with the parliament’s Conference of Presidents prior to the livestream being switched on and will update this story with any response.

Where to watch it online?

According to Tajani’s spokesperson, the meeting will be broadcast on the EU parliament’s website. At the time of writing it’s not yet listed in the EPTV schedule — but we’re expecting it to be viewable here.

Who will be meeting with Zuckerberg?

The Facebook founder will meet with EU parliament president Tajani, along with leaders of the parliament’s eight political groups, and with Claude Moraes, the chair of the EU parliament’s Civil Liberties, Justice and Home Affairs (LIBE) committee.

It’s worth noting that the meeting is not a formal hearing, such as the sessions with Zuckerberg in the US Senate and Congress last month. Nor is it a full Libe committee hearing — discussions remain ongoing for Facebook representatives to meet with the full Libe committee at a later date.

What will Zuckerberg be asked about?

In the wake of the Cambridge Analytica data misuse scandal, MEPs are keen to discuss concerns related to social media’s impact on election processes with Zuckerberg.

Indeed, the impact of social media spread online disinformation is also the topic of an ongoing enquiry by the UK parliament’s DCMS committee which spent some five hours grilling Facebook’s CTO last month. Although Zuckerberg has thrice declined the committee’s summons — preferring to meet with EU parliamentarians instead.

Other topics on the agenda will include privacy and data protection — with Moraes likely to ask about how Facebook’s business model impacts EU citizens’ fundamental rights, and how EU regulations might need to evolve to keep pace, as he explained to us on Friday.

Some of the political group leaders are also likely to bring up concerns around freedom of expression as pressure in the region has ramped up on online platforms to get faster at policing hate speech.



source https://techcrunch.com/2018/05/21/where-to-watch-zuckerbergs-meeting-with-eu-meps-on-tuesday/

What Google's GDPR Compliance Efforts Mean for Your Data: Two Urgent Actions

Posted by willcritchlow

It should be quite obvious for anyone that knows me that I’m not a lawyer, and therefore that what follows is not legal advice. For anyone who doesn’t know me: I’m not a lawyer, I’m certainly not your lawyer, and what follows is definitely not legal advice.

With that out of the way, I wanted to give you some bits of information that might feed into your GDPR planning, as they come up more from the marketing side than the pure legal interpretation of your obligations and responsibilities under this new legislation. While most legal departments will be considering the direct impacts of the GDPR on their own operations, many might miss the impacts that other companies’ (namely, in this case, Google’s) compliance actions have on your data.

But I might be getting a bit ahead of myself: it’s quite possible that not all of you know what the GDPR is, and why or whether you should care. If you do know what it is, and you just want to get to my opinions, go ahead and skip down the page.

What is the GDPR?

The tweet-length version is that the GDPR (General Data Protection Regulation) is new EU legislation covering data protection and privacy for EU citizens, and it applies to all companies offering goods or services to people in the EU.

Even if you aren’t based in the EU, it applies to your company if you have customers who are, and it has teeth (fines of up to the greater of 4% of global revenue or EUR20m). It comes into force on May 25. You have probably heard about it through the myriad organizations who put you on their email list without asking and are now emailing you to “opt back in.”

In most companies, it will not fall to the marketing team to research everything that has to change and achieve compliance, though it is worth getting up to speed with at least the high-level outline and in particular its requirements around informed consent, which is:

"...any freely given, specific, informed, and unambiguous indication of the data subject's wishes by which he or she, by a statement or by a clear affirmative action, signifies agreement to the processing of personal data relating to him or her."

As always, when laws are made about new technology, there are many questions to be resolved, and indeed, jokes to be made:

But my post today isn’t about what you should do to get compliant — that’s specific to your circumstances — and a ton has been written about this already:

My intention is not to write a general guide, but rather to warn you about two specific things you should be doing with analytics (Google Analytics in particular) as a result of changes Google is making because of GDPR.

Unexpected consequences of GDPR

When you deal directly with a person in the EU, and they give you personally identifiable information (PII) about themselves, you are typically in what is called the "data controller" role. The GDPR also identifies another role, which it calls "data processor," which is any other company your company uses as a supplier and which handles that PII. When you use a product like Google Analytics on your website, Google is taking the role of data processor. While most of the restrictions of the GDPR apply to you as the controller, the processor must also comply, and it’s here that we see some potentially unintended (but possibly predictable) consequences of the legislation.

Google is unsurprisingly seeking to minimize their risk (I say it’s unsurprising because those GDPR fines could be as large as $4.4 billion based on last year’s revenue if they get it wrong). They are doing this firstly by pushing as much of the obligation onto you (the data controller) as possible, and secondly, by going further by default than the GDPR requires and being more aggressive than the regulation requires in shutting down accounts that infringe their terms (regardless of whether the infringement also infringes the GDPR).

This is entirely rational — with GA being in most cases a product offered for free, and the value coming to Google entirely in the aggregate, it makes perfect sense to limit their risks in ways that don’t degrade their value, and to just kick risky setups off the platform rather than taking on extreme financial risk for individual free accounts.

It’s not only Google, by the way. There are other suppliers doing similar things which will no doubt require similar actions, but I am focusing on Google here simply because GA is pervasive throughout the web marketing world. Some companies are even going as far as shutting down entirely for EU citizens (like unroll.me). See this Twitter thread of others.

Consequence 1: Default data retention settings for GA will delete your data

Starting on May 25, Google will be changing the default for data retention, meaning that if you don’t take action, certain data older than the cutoff will be automatically deleted.

You can read more about the details of the change on Krista Seiden’s personal blog (Krista works at Google, but this post is written in her personal capacity).

The reason I say that this isn’t strictly a GDPR thing is that it is related to changes Google is making on their end to ensure that they comply with their obligations as a data processor. It gives you tools you might need but isn’t strictly related to your GDPR compliance. There is no particular “right” answer to the question of how long you need to/should be/are allowed to keep this data stored in GA under the GDPR, but by my reading, given that it shouldn’t be PII anyway (see below) it isn’t really a GDPR question for most organizations. In particular, there is no particular reason to think that Google’s default is the correct/mandated/only setting you can choose under the GDPR.

Action: Review the promises being made by your legal team and your new privacy policy to understand the correct timeline setting for your org. In the absence of explicit promises to your users, my understanding is that you can retain any of this data you were allowed to capture in the first place unless you receive a deletion request against it. So while most orgs will have at least some changes to make to privacy policies at a minimum, most GA users can change back to retain this data indefinitely.

Consequence 2: Google is deleting GA accounts for capturing PII

It has long been against the Terms of Service to store any personally identifiable information (PII) in Google Analytics. Recently, though, it appears that Google has become far more diligent in checking for the presence of PII and robust in their handling of accounts found to contain any. Put more simply, Google will delete your account if they find PII.

It’s impossible to know for sure that this is GDPR-related, but being able if necessary to demonstrate to regulators that they are taking strict actions against anyone violating their PII-related terms is an obvious move for Google to reduce the risk they face as a Data Processor. It makes particular sense in an area where the vast majority of accounts are free accounts. Much like the previous point, and the reason I say that this is related to Google’s response to the GDPR coming into force, is that it would be perfectly possible to get your users’ permission to record their data in third-party services like GA, and fully comply with the regulations. Regardless of the permissions your users give you, Google’s GDPR-related crackdown (and heavier enforcement of the related terms that have been present for some time) means that it’s a new and greater risk than it was before.

Action: Audit your GA profile and implementation for PII risks:

  • There are various ways you can search within GA itself to find data that could be personally identifying in places like page titles, URLs, custom data, etc. (see these two excellent guides)
  • You can also audit your implementation by reviewing rules in tag manager and/or reviewing the code present on key pages. The most likely suspects are the places where people log in, take key actions on your site, give you additional personal information, or check out

Don’t take your EU law advice from big US tech companies

The internal effort and coordination required at Google to do their bit to comply even “just” as data processor is significant. Unfortunately, there are strong arguments that this kind of ostensibly user-friendly regulation which incurs outsize compliance burdens on smaller companies will cement the duopoly and dominance of Google and Facebook and enables them to pass the costs and burdens of compliance onto sectors that are already struggling.

Regardless of the intended or unintended consequences of the regulation, it seems clear to me that we shouldn’t be basing our own businesses’ (and our clients’) compliance on self-interested advice and actions from the tech giants. No matter how impressive their own compliance, I’ve been hugely underwhelmed by guidance content they’ve put out. See, for example, Google’s GDPR “checklist” — not exactly what I’d hope for:

Client Checklist: As a marketer we know you need to select products that are compliant and use personal data in ways that are compliant. We are committed to complying with the GDPR and would encourage you to check in on compliance plans within your own organisation. Key areas to think about: How does your organisation ensure user transparency and control around data use? Do you explain to your users the types of data you collect and for what purposes? Are you sure that your organisation has the right consents in place where these are needed under the GDPR? Do you have all of the relevant consents across your ad supply chain? Does your organisation have the right systems to record user preferences and consents? How will you show to regulators and partners that you meet the principles of the GDPR and are an accountable organisation?

So, while I’m not a lawyer, definitely not your lawyer, and this is not legal advice, if you haven’t already received any advice, I can say that you probably can’t just follow Google’s checklist to get compliant. But you should, as outlined above, take the specific actions you need to take to protect yourself and your business from their compliance activities.


Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!



source https://moz.com/blog/two-urgent-gdpr-actions

GDPR: What it Means for Google Analytics & Online Marketing

Posted by Angela_Petteys

If you’ve been on the Internet at all in the past few months, you’ve probably seen plenty of notices about privacy policy updates from one service or another. As a marketer, a few of those notices have most likely come from Google.

With the General Data Privacy Regulation (GDPR) set to go into effect on May 25th, 2018, many Internet services have been scrambling to get in compliance with the new standards — and Google is no exception. Given the nature of the services Google provides to marketers, GDPR absolutely made some significant changes in how they conduct business. And, in turn, some marketers may have to take steps to make sure their use of Google Analytics is allowable under the new rules. But a lot of marketers aren’t entirely sure what exactly GDPR is, what it means for their jobs, and what they need to do to follow the rules.

What is GDPR?

GDPR is a very broad reform that gives citizens who live in the European Economic Area (EEA) and Switzerland more control over how their personal data is collected and used online. GDPR introduces a lot of new rules and if you’re up for a little light reading, you can check out the full text of the regulation online. But here are a few of the most significant changes:

  • Companies and other organizations have to be more transparent and clearly state what information they’re collecting, what it will be used for, how they’re collecting it, and if that information will be shared with anyone else. They can also only collect information that is directly relevant for its intended use. If the organization collecting that information later decides to use it for a different purpose, they must get permission again from each individual.
  • GDPR also spells out how that information needs to be given to consumers. That information can no longer be hidden in long privacy policies filled with legal jargon. The information in disclosures needs to be written in plain language and “freely given, specific, informed, and unambiguous.” Individuals also have to take an action which clearly gives their consent to their information being collected. Pre-checked boxes and notices that rely on inaction as a way of giving consent will no longer be allowed. If a user does not agree to have their information collected, you cannot block them from accessing content based on that fact.
  • Consumers also have the right to see what information a company has about them, request that incorrect information be corrected, revoke permission for their data to be saved, and have their data exported so they can switch to another service. If someone decides to revoke their permission, the organization needs to not only remove that information from their systems in a timely manner, they also need to have it removed from anywhere else they’ve shared that information.
  • Organizations must also be able to give proof of the steps they’re taking to be in compliance. This can include keeping records of how people opt in to being on marketing lists and documentation regarding how customer information is being protected.
  • Once an individual’s information has been collected, GDPR sets out requirements for how that information is stored and protected. If a data breach occurs, consumers must be notified within 72 hours. Failing to comply with GDPR can come with some very steep consequences. If a data breach occurs because of non-compliance, a company can be hit with fines as high as €20 million or 4% of the company’s annual global revenue, whichever amount is greater.

Do US-based businesses need to worry about GDPR?

Just because a business isn’t based in Europe doesn’t necessarily mean they’re off the hook as far as GDPR goes. If a company is based in the United States (or elsewhere outside the EEA), but conducts business in Europe, collects data about users from Europe, markets themselves in Europe, or has employees who work in Europe, GDPR applies to them, too.

Even if you’re working with a company that only conducts business in a very specific geographic area, you might occasionally get some visitors to your site from people outside of that region. For example, let’s say a pizza restaurant in Detroit publishes a blog post about the history of pizza on their site. It’s a pretty informative post and as a result, it brings in some traffic from pizza enthusiasts outside the Detroit area, including a few visitors from Spain. Would GDPR still apply in that sort of situation?

As long as it’s clear that a company’s goods or services are only available to consumers in the United States (or another country outside the EEA), GDPR does not apply. Going back to the pizza restaurant example, the other content on their site is written in English, emphasizes their Detroit location, and definitely doesn’t make any references to delivery to Spain, so those few page views from Spain wouldn’t be anything to worry about.

However, let’s say another US-based company has a site with the option to view German and French language versions of pages, lets customers pay with Euros, and uses marketing language that refers to European customers. In that situation, GDPR would apply since they are more clearly soliciting business from people in Europe.

Google Analytics & GDPR

If you use Google Analytics, Google is your data processor and since they handle data from people all over the world, they’ve had to take steps to become compliant with GDPR standards. However, you/your company are considered the data controller in this relationship and you will also need to take steps to make sure your Google Analytics account is set up to meet the new requirements.

Google has been rolling out some new features to help make this happen. In Analytics, you will now have the ability to delete the information of individual users if they request it. They’ve also introduced data retention settings which allow you to control how long individual user data is saved before being automatically deleted. Google has set this to be 26 months as the default setting, but if you are working with a US-based company that strictly conducts business in the United States, you can set it to never expire if you want to — at least until data protection laws change here, too. It’s important to note that this only applies to data about individual users and events, so aggregate data about high-level information like page views won’t be impacted by this.

To make sure you’re using Analytics in compliance with GDPR, a good place to start is by auditing all the data you collect to make sure it’s all relevant to its intended purpose and that you aren’t accidentally sending any personally identifiable information (PII) to Google Analytics. Sending PII to Google Analytics was already against its Terms of Service, but very often, it happens by accident when information is pushed through in a page URL. If it turns out you are sending PII to Analytics, you’ll need to talk to your web development team about how to fix it because using filters in Analytics to block it isn’t enough — you need to make sure it’s never sent to Google Analytics in the first place.

PII includes anything that can potentially be used to identify a specific person, either on its own or when combined with another piece of information, like an email address, a home address, a birthdate, a zip code, or an IP address. IP addresses weren’t always considered PII, but GDPR classifies them as an online identifier. Don’t worry, though — you can still get geographical insights about the visitors to your site. All you have to do is turn on IP anonymization and the last portion of an IP address will be replaced with a zero, so you can still get a general idea of where your traffic is coming from, although it will be a little less precise.

If you use Google Tag Manager, IP anonymization is pretty easy. Just open your Google Analytics tag or its settings variable, choose “More Settings,” and select “Fields to Set.” Then, choose “anonymizeip” in the “Field Name” box, enter “true” in the “Value” box,” and save your changes.

If you don’t use GTM, talk to your web development team about editing the Google Analytics code to anonymize IP addresses.

Pseudonymous information like user IDs and transaction IDs are still acceptable under GDPR, but it needs to be protected. User and transaction IDs need to be alphanumeric database identifiers, not written out in plain text.

Also, if you haven’t already done so, don’t forget to take the steps Google has mentioned in some of those emails they’ve sent out. If you’re based outside the EEA and GDPR applies to you, go into your Google Analytics account settings and accept the updated terms of processing. If you’re based in the EEA, the updated terms have already been included in your data processing terms. If GDPR applies to you, you’ll also need to go into your organization settings and provide contact information for your organization.

Privacy policies, forms, & cookie notices

Now that you’ve gone through your data and checked your settings in Google Analytics, you need to update your site’s privacy policy, forms, and cookie notices. If your company has a legal department, it may be best to involve them in this process to make sure you’re fully compliant.

Under GDPR, a site’s privacy policy needs to be clearly written in plain language and answer basic questions like what information is being collected, why it’s being collected, how it’s being collected, who is collecting it, how it will be used, and if it will be shared with anyone else. If your site is likely to be visited by children, this information needs to be written simply enough for a child to be able to understand it.

Forms and cookie notices also need to provide that kind of information. Cookie consent forms with really vague, generic messages like, “We use cookies to give you a better experience and by using this site, you agree to our policy,” are not GDPR compliant.

GDPR & other types of marketing

The impact GDPR will have on marketers isn’t just limited to how you use Google Analytics. If you use some particular types of marketing in the course of your job, you may have to make a few other changes, too.

Referral deals

If you work with a company that does “refer a friend”-type promotions where a customer has to enter information for a friend to receive a discount, GDPR is going to make a difference for you. Giving consent for data to be collected is a key part of GDPR and in these sorts of promotions, the person being referred can’t clearly consent to their information being collected. Under GDPR, it is possible to continue this practice, but it all depends on how that information is being used. If you store the information of the person being referred and use it for marketing purposes, it would be a violation of GDPR standards. However, if you don’t store that information or process it, you’re OK.

Email marketing

If you’re an email marketer and already follow best industry standards by doing things like only sending messages to those who clearly opt in to your list and making it easy for people to unsubscribe, the good news is that you’re probably in pretty good shape. As far as email marketing goes, GDPR is going to have the biggest impact on those who do things that have already been considered sketchy, like buying lists of contacts or not making it clear when someone is signing up to receive emails from you.

Even if you think you’re good to go, it’s still a good time to review your contacts and double check that your European contacts have indeed opted into being on your list and that it was clear what they were signing up for. If any of your contacts don’t have their country listed or you’re not sure how they opted in, you may want to either remove them from your list or put them on a separate segment so they don’t get any messages from you until you can get that figured out. Even if you’re confident your European contacts have opted in, there’s no harm in sending out an email asking them to confirm that they would like to continue receiving messages from you.

Creating a double opt-in process isn’t mandatory, but it would be a good idea since it helps remove any doubt over whether or not a person has agreed to being on your list. While you’re at it, take a look at the forms people use to sign up to be on your list and make sure they’re in line with GDPR standards, with no pre-checked boxes and the fact that they’re agreeing to receive emails from you is very clear.

For example, here’s a non-GDPR compliant email signup option I recently saw on a checkout page. They tell you what they’re planning to send to you, but the fact that it’s a pre-checked box placed underneath the more prominent “Place Order” button makes it very easy for people to unintentionally sign up for emails they might not actually want.

Jimmy Choo, on the other hand, also gives you the chance to sign up for emails while making a purchase, but since the box isn’t pre-checked, it’s good to go under GDPR.

Marketing automation

As is the case with standard email marketing, marketing automation specialists will need to make sure they have clear consent from everyone who has agreed to be part of their lists. Check your European contacts to make sure you know how they’ve opted in. Also review the ways people can opt into your list to make sure it’s clear what, exactly, they’re signing up for so that your existing contacts would be considered valid.

If you use marketing automation to re-engage customers who have been inactive for a while, you may need to get permission to contact them again, depending on how long it has been since they last interacted with you.

Some marketing automation platforms have functionality which will be impacted by GDPR. Lead scoring, for example, is now considered a form of profiling and you will need to get permission from individuals to have their information used in that way. Reverse IP tracking also needs consent.

It’s also important to make sure your marketing automation platform and CRM system are set to sync automatically. If a person on your list unsubscribes and continues receiving emails because of a lapse between the two, you could get in trouble for not being GDPR compliant.

Gated content

A lot of companies use gated content, like free reports, whitepapers, or webinars, as a way to generate leads. The way they see it, the person’s information serves as the price of admission. But since GDPR prohibits blocking access to content if a person doesn’t consent to their information being collected, is gated content effectively useless now?

GDPR doesn’t completely eliminate the possibility of gated content, but there are now higher standards for collecting user information. Basically, if you’re going to have gated content, you need to be able to prove that the information you collect is necessary for you to provide the deliverable. For example, if you were organizing a webinar, you’d be justified in collecting email addresses since attendees need to be sent a link to join in. You’d have a harder time claiming an email address was required for something like a whitepaper since that doesn’t necessarily have to be delivered via email. And of course, as with any other form on a site, forms for gated content need to clearly state all the necessary information about how the information being collected will be used.

If you don’t get a lot of leads from European users anyway, you may want to just block all gated content from European visitors. Another option would be to go ahead and make that information freely available to visitors from Europe.

Google AdWords

If you use Google AdWords to advertise to European residents, Google already required publishers and advertisers to get permission from end users by putting disclaimers on the landing page, but GDPR will be making some changes to these requirements. Google will now be requiring publishers to get clear consent from individuals to have their information collected. Not only does this mean you have to give more information about how a person’s information will be used, you’ll also need to keep records of consent and tell users how they can opt out later on if they want to do so. If a person doesn’t give consent to having their information collected, Google will be making it possible to serve them non-personalized ads.

In the end

GDPR is a significant change and trying to grasp the full scope of its changes is pretty daunting. This is far from being a comprehensive guide, so if you have any questions about how GDPR applies to a particular client you’re working with, it may be best to get in touch with their legal department or team. GDPR will impact some industries more than others, so it’s best to get some input from someone who truly understands the law and how it applies to that specific business.


Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!



source https://moz.com/blog/gdpr-and-online-marketing