Yesterday, SEMrush released an updated version of their June 2017 ranking factors report, again placing Direct Traffic right at the top of the list – stating that its the most important ranking factor for Google.
You can (and should) download it here.
For those that may not be familiar with SEMrush, they provide an excellent online set of tools for both SEO market analysis and site specific crawling, reporting, and a fairly extensive keyword list with plenty of data.
I’m a big fan of theirs (despite not having used the tool for a number of years) and they consistently won “Best Tool” in my old industry reports.
They also fairly consistently win Best Tool in SEO awards ceremonies etc. Basically, they can be considered an authoritative source of SEO analysis.
Some Definitions to ensure we’re in alignment:
Ranking Factor: One of the considerations Google makes when deciding how to appropriately rank a page for a given search query.
To qualify as a ranking factor, the item in consideration must drive in some way the relevance of the page to the query, or the strength of the page itself, and not be merely correlative or coincidental.
Other good examples of ranking factor studies have been published by Backlinko, SearchMetrics, SISTRIX and many others.
Direct Traffic: Requests that your website receives from browsers without an http_referrer.
This could be direct type in traffic, or bookmarked URLs.
It could also be website referral traffic that has lost its referrer (for instance on https>http links), URL shorteners that haven’t passed a referrer ID, particularly when links are clicked through from a non-web browser app, like Tweetdeck…
Finally, increasingly I see Google stripping the http_referrer itself from organic traffic. That last one happens a lot more than most people realize, and the total % has been increasing month on month for some years now.
Let’s Assume Direct Traffic IS a Ranking Factor:
While I’m initially skeptical of the claim, it seems a whole lot of other SEO’s think that direct traffic may indeed be a ranking factor:
So lets think this through for a minute:
Q: Would Direct Traffic be a Good Quality Signal?
A: Sure, why not…. I guess sites that have a significant amount of direct type in traffic could be considered authoritative.
After all, if people are going straight to the site either by typing the URL in, or by bookmarking it, you could certainly ascertain that its serving a purpose, or is of high quality.
I certainly agree that this (direct type in traffic) might be valuable data for Google.
Q: Are Google Able to Get a Clean, Dependable Signal of ‘referrerless’ web traffic?
A: Also probably yes on this one.
They don’t need to rely on Google Analytics data either, since they both power a significant DNS service, are a large scale bandwidth provider through Google Fiber, have significant clickstream data through Google Chrome & other potential channels.
So, yes – I think they probably could get a decent sense of the number of requests without a referrer…
Although, thats no guarantee its actual type in traffic(!).
Q: Could Google Efficiently Parse this Data for Ranking Purposes?
A: I don’t see why not – once you can verify that there is direct traffic to a given URL, or root domain, you could apply it a score which updates periodically and is built into the overall site trust factor score. Also a pass in my mind.
So Why the Skepticism Then?
I’ve got a couple of reasons, that in my mind are deal breakers…
1) The technical definition of direct traffic simply being requests made without an http_referrer is far too vague a concept.
We’re not talking about direct type in traffic most of the time, rather:
- requests made from non web browsers
- many URL shorteners
- social media platforms and apps
- links with mismatched security protocols
- links shared across devices (desktop to mobile particularly)
This varied source of “direct traffic” invalidates the theory that direct type in is a dependable quality metric – while it may be one, extracting the actual direct type in or bookmarked traffic away from the other sources is not currently possible.
2) As a ranking factor, it would be exceptionally easy to game.
If all you need to do is strip out the referrer on all internal links to “fool” google into thinking its direct traffic, you could do this with a few lines of PHP code, or with server configuration, and lets not forget that HTML5 actually gives us the option to prevent referrers from being passed with the a rel=noreferrer tag.
3) The industry wide migration to HTTPS:
Given that by design, HTTPS secure pages do not pass a referrer to http pages on requests, it effectively blacks-out huge portions of the linking web.
That from a server standpoint would make much of this traffic appear to be direct, unless specific steps are taken to re-insert referrer data.
Why Do SEMrush Think it IS a Ranking Factor?
My best guess is that, with the high number of factors Google is considering these days (some say 200+, others say thousands – I’m not making an estimate personally, but its certainly more than the ones they go into in their report) direct correlation between any of them can be remarkably low.
We see this time and time again in everyone else’s ranking correlation studies – links by themselves are an obvious indicator of page strength, but this is only considered hand in hand with Google’s perceived relevancy of the content of a landing page, vs. the query the user made.
That’s to say that, a page with high relevancy to the query, can outrank one with lower relevancy even though it has a much greater number of inbound links, and the perceived authority (PageRank) of those links.
Once you start mixing in post processing factors following document retrieval (ie. user behavior, bounce rate vs SERP competitors, time on site, site speed, page weight, etc. etc. etc.) the picture gets very murky indeed.
I can totally see why “direct traffic” correlates highly with rankings, its because by design Google is sorting better results, and better websites, higher in their results.
These websites are likely to correlate with having higher traffic, and in turn will generate higher numbers of “direct hits” (without a referrer). It all fits together forming a complex, but predictable picture.
Another significant point here is the continued obfuscation of SEO traffic as direct, by Google themselves. Famously Groupon conducted an “experiment” (or total mess up, depending on who you ask) which demonstrated that a significant amount of their direct traffic was in fact, organic.
I’ve seen this time and time again, particularly on long tail queries to deep pages on massive sites. When you de-index those pages from Google, they cease to receive direct traffic – two things that should not correlate.
You may have your own opinions as to why this may be the case, but I don’t think its too far a stretch to imagine that slowly migrating organic traffic into direct for analytics purposes, causes SEO’s harm, by devaluing their work.
But maybe thats just me.
The True Test of an Algorithm
The best test of an algorithm, isn’t whether it can explain results, rather whether it can predict them.
This wouldn’t be too hard to test in the context of what we’re discussing here – all a large site would need to do would be to mask internal referrers at a server level, to make it appear that 100% of traffic navigating internally was in fact direct traffic.
If the theory put forward by SEMrush were true, that direct traffic is a ranking factor, then you would theoretically see rankings explode – if it were the single most important factor.
I’m not going to suggest to any of my clients that they do this, but you could theoretically build a fairly robust test.
Or, you know, just buy lots of traffic and point it through a redirector to strip the referrer….
…I’m sure that will appear on Fiverr some point in the next couple of weeks if SEMrush continue to push this theory forward 😉
All this is Just My Opinion Though: What Do You Think?
As it appears, around a third of SEO’s believe it is a ranking factor, I’d LOVE to have your comments below – don’t be shy! 😉
Depends on what “clickstream data” you use 😛
A few thoughts on the questions you laid out:
> Q: Would Direct Traffic be a Good Quality Signal?
Depends! Traffic can have a lot of different flavours. I can think of a couple:
– Is the traffic bought?
– Is it automated traffic?
For me, these wouldn not automatically classify as “quality”. Also, don’t forget, a site with a lot of direct traffic can be out of Google’s index. I’ve seen penalized websites, that have more direct traffic than better ranking ones. So, I wouldn’t blatantly say that direct traffic is an indicator of quality.
Obvisouly if you have a lot of qualified traffic, it could be a reflex of people wanting to reach your site.
> Q: Are Google Able to Get a Clean, Dependable Signal of ‘referrerless’ web traffic?
Agree with you, mostly. But we must remember that, even if Google used GA data, there are plenty of sites that don’t have GA implemented. Also, a lot of users use other browsers than Chrome. So, I’m not sure the data would be that reliable or accurately representative.
> Q: Could Google Efficiently Parse this Data for Ranking Purposes?
Same considerations as above.
I do agree with yout skepticism. I think they might be mistakenly calling “direct traffic” to branded traffic, which could be something a bit less biased, and that Google can see as a clearer intent… And, of course, let’s not forget the correlation != causation. If they see sites ranking high; that have lots of direct traffic; doesn’t mean this is what makes those sites rank high.
I had a look at the methodology they linked from the study page. They write they understant, as they write, “Correlation doesn’t mean causation”. But, the thing is, have they tested these “factors”? Have they purchased a domain/site and try to buy direct traffic to it? I don’t see any mention to testing these assumptions in their methodology.
That said, I believe there’s some kind of misalignement of concepts in their study. In order to be able to get a better assessment of what they meant, I would need to get a better clarification of what they understand as “direct traffic”. I don’t think it’s prudent, as a prominent tool provider, to be spreading something that can have a broad interpretation.
Anyway, these are my primary thoughts. This could lead to a much longer discussion, which I definitely would like to have with whomever decided on these ranking factors.
No. it’s freaking not. I can’t believe we’re even having this discussion. Direct traffic means somebody already knows about your brand. They’ve already been exposed to it. How’d they do that? likely through search. Direct traffic is an EFFECT of ranking well – but it’s also an effect of TV ads, radio, print, whatever exposes you. Sure it correlates, because nobody is going to type in the URL of a site they haven’t yet found.
We KEEP saying correlation != causation, then we write entire posts proving we don’t understand what we just said.
You’ve put together a nice argument here. I like the fact that you don’t automatically discount the idea of direct traffic being a factor. Asking questions as to whether they could get this data and use it reliably are excellent places to start.
At the end of the day I still feel that the conclusion they’ve reached is conflating a result with a ranking factor. I’ve made the analogy that the top success factor for a brick and mortar store might be the amount of foot traffic received. Yet, hiring a bunch of extras to show up at your store isn’t going to lead to success.
That cool flash mob that shows up every Tuesday to dance isn’t helping business.
To be fair, I believe I’ve seen SEMRush folks say that direct traffic is a sign of a site doing good marketing. Because in the case of foot traffic, it increases due to OTHER actions. So what SEMRush seems to be saying is that good marketing is a ranking factor.
But how would Google measure THAT?
And it dovetails with the issue of determining just what ‘direct traffic’ really means in an ecosystem that is regularly stripping referrers. What are we measuring exactly? How much dark social are we tracking here?
But what might good marketing do for a site? More social mentions? More links? Better links? Topical links? We keep doing these studies and treating links like some monolithic factor when I think there is broad agreement that all links are not equal. And they’re not the same in each vertical.
Could marketing increase the volume of branded search? Sure could. And could THAT be a factor that Google might use? Maybe. That would make more sense to me than a very amorphous metric such as direct traffic. In fact, you’d probably see a nice correlation between those two data points, but only one is more stable and transparent.
And didn’t Gary recently mention sentiment analysis as being a part of the algorithm. Now, I’m not sure anyone actually followed up and asked HOW exactly Google would measure that but it certainly points to alternate ways to determine authority.
I think that’s where things get murky for me.
Some authoritative sites just aren’t going to get a lot of direct traffic. People rarely go directly to Stack Overflow but it still does, you know, pretty darn well. So to think that Google wouldn’t rank that content because of low direct traffic … just doesn’t make sense.
That might be the problem with using such a small sample to conduct this type of analysis. I know 600,000 sounds like a lot but … it’s not.
I’m a sound believer that marketing is an important part of a successful organic effort. And the result of that is often more direct traffic. But that does not make it a factor. From the lack of definition in measurement to differences in business models that would increase or decrease this metric (think cars.com doesn’t see a bias based on their domain?), direct traffic remains a product of other activities that can be highly correlated with success in some verticals.
I could go on but I won’t.
Excellent analysis, AJ, along with Martin’s. I can add some clarification on Gary Illyes’s statements about Google monitoring brand mentions. I was at both conferences where this came up (State of Search and Pubcon) and had the opportunity to discuss it one-on-one with Gary.
First, he very much wants to distance himself from calling this “sentiment” analysis (although it sure sounds like it, which is probably why so many who heard him actually though he said that).
What he clarified with me is that this has nothing (directly) to do with ranking. Rather, Google “may” pay attention sometimes when a brand or entity is being mentioned a lot on the web. These mentions could be almost anywhere (social media, forums, blog comments, etc.). A lot of mentions of an entity in connection with a certain topic or area “could” tell Google that “perhaps” the site should be ranking for that topic. He also mentioned a lot of either positive or negative mentions could be a “signal” to Google, although he still shied away from calling it “sentiment analysis.”
Hope that helps!
The word I would use to describe putting out a whitepaper on rankings factors that labels Direct Traffic as most important is “embarrassing”.
First, we still don’t know the source of their “Direct Traffic” numbers (or, at least I have not received a response to my inquiry), but let’s assume it is from GA data they have acquired from their customer base.
We know with certainty that Google miscategorizes Organic Traffic as Direct Traffic (https://searchengineland.com/60-direct-traffic-actually-seo-195415). We know that traffic attribution models are broken when a browser is opened and closed, so traffic that is generated originally from a search click will result in a new direct-traffic impression if the browser is open and closed and the tab remains in place. We know that improved organic rankings increase traffic. Finally, we know that other ranking factors like links increase traffic to a page or site, which too would suffer from the same open-close browser issue for extra direct-traffic impressions.
Ultimately, we have strong reason to believe that the causal arrow is not Direct Traffic -> Rankings but, instead, Rankings->Direct Traffic or Links->Rankings && Links->Direct Traffic.
I have to be careful now that I work for a competitor, but candidly, I feel like this is a classic example of throwing math at data and providing little to no human insight. The good folks at SEMRush clearly put a lot of effort in data analysis and in presentation (the report is beautiful) but skipped the most important part – interpretation of results. To be honest, I can’t imagine ever including Direct Traffic as a factor in the first place knowing how confounding a variable it is.
Yes! This was exactly my point the last time they pushed out study a few months ago, Russ.
I feel that they’ve spent a lot of time preparing for people be skeptical on their methodology (it’s been the common response each time I note the issue is around the data analysis interpretation) and having to explain how it works (which to be fair, is unique and different in a positive way).
The larger problem I’ve had with their data analysis is that it leaves out the possibilities that could feed into direct traffic. Reminds me of the situation where Facebook Likes were correlated with ranking higher a number of years ago and the issue again stemmed from not accounting for what factors could be merged into a basic 1-1 correlation. With this study, I feel they’ve done a better job to account for multiple factors that aren’t directly related with direct traffic, but not when accounting for what things could be the real cause of their data showing direct traffic.
Another reason not to transition to HTTPS… I jest.
The more I think of this, it could be a shrewd move by SemRush to push the discussion around their report. I doubt it would have had the coverage it received if they had led with content / links as the main ranking factor – although that isn’t to detract from the quality of the report. A little controversy does no harm in the promotion stakes.
What next? The stunning conclusion that having a website in the first place is a direct ranking factor?
“Direct” traffic is a very murky pool & always has been. Bookmarks, type-ins, stripped referrers, “don’t know, count it as direct”. IMO it’s too messy to extract really meaningful data you can use directly.
Like social, good, natural, direct traffic is potentially indicative of a good website, but it’s not a pre-requisite. At best, direct traffic is an indirect signal & good, natural, direct traffic has a tendency to cause natural bumps in other more direct signals.
Since Vince, Google has a bit of a skew towards “good” or well-known websites, or websites a user might expect to see in response to a query (don’t forget some users use search as a bookmark), but it’s not an absolute given. Once you have that known-website skew, websites with good, natural, direct traffic are likely to have a tendency to surface more often than unknowns, but ++direct traffic !~= ++rankings.
Agreed with the majority of the post, particularly that’s it’s not a major factor. Two things that come to mind while reading:
1 – I think it is more than reasonable to assume that Google monitors Chrome type in traffic, from users who display normal user patterns (as opposed to virtual terminals spun up on AWS or other faked browser usage), and assume that it may factor in to the algorithm somewhere. I have zero data to support it as anything other than a possible input, wouldn’t call it a major factor, and it’s obviously a minority slice of all direct traffic on the web.
2 – We’ve observed scenarios where offline coverage like a TV commercial or magazine feature causes a sustained bump in people searching for the brand – and we see a simultaneous rankings lift on non branded keywords. A lot of it shows up as direct, likely from type in traffic, but a lot of it also shows up as branded organic searches, which I think is the more likely source of the non branded traffic increase. It would be easy for a casual SEO to see that spike, not notice the branded search lift as the more likely factor, and attribute some of those results to the direct traffic itself. Not correct, but understandable why many would conclude it’s a factor.
I presented on this at Ungagged 3 years ago and is included in a 2012 patent by a Google engineer named Naveneet Panda.
Change “direct traffic” to Navigational Queries + Brand Queries (type in traffic plus brand = reviews, brand + competitors, etc) and you have, IMO, the foundation of Panda 1.0, which basically divided the amount of organic traffic you have “earned” by the the amount of traffic you get from generic keywords.
Here is the relevant text from the patent.
“The score is determined from quantities indicating user actions of seeking out and preferring particular sites and the resources found in particular sites.
A site quality score for a particular site can be determined by computing a ratio of a numerator that represents user interest in the site as reflected in user queries directed to the site and a denominator that represents user interest in the resources found in the site as responses to queries of all kinds
I’m going to defend SEMrush here. Cards on the table, they sponsor some of my events and, like you Martin, I’m a big fan of their tool.
Do I think direct traffic is a ranking factor? No.
Do quite a few other people? Clearly, from Martin’s Twitter poll, yes. I can’t believe that’s entirely down to SEMrush’s study either.
Maybe it was a PR stunt; maybe there are confounding factors (OK, ‘probably’) – the organic being lumped into ‘direct’ is one that hadn’t occurred to me til I read it here; and maybe they’re just showing what they found from their data, sharing it and sparking discussion. Maybe all of the above.
What it has done is get a lot of ‘SEO brains’ – like the author and commentors above (great job Martin!) to throw their light/knowledge on this and, hopefully, make a lot of people (like me) a little wiser.
Worth bearing in mind that there was a *lot* of other good stuff in the report. I know this was the ‘headline’ but let’s not throw the baby out with the bathwater.
Is the SEO world a better, slightly more informed place than it was before this report? I’d argue – yes.
Yep. Agree. I don’t buy it.
In most cases, a large chunk of what shows as direct traffic is not really direct traffic. And it would be way too easy to game.
I always liked the idea that certain “ranking factors” could actually be used in a qualitative way. That is – the page looks good. It has the right primary signals. Then the secondary signals are used to kind of double check the primary factors.
Maybe direct, social shares, brand search volume etc – all these things that are not the big factors (in what is inevitably thousands of smaller factors that differ wildly in weighting and application) are used in some other way to simply double check that the primary factors can be trusted.
I await the barrage of emails offering direct traffic for SEO purposes….
Marcus
Go to Fiverr and you can buy tons of direct traffic, so yeah the off putting factor would be that it’s easy to game, I’ve ordered one of these before just to see what happens, and you can often specify where it comes from location wise, the regularity of visits and all sorts of factors,so it’s just not a trust worthy metric.
Is it a claim that it’s a ranking factor for your whole site ie to put it crudely total direct visits = more authoritative and trusted domain, or is it on a page by page basis? Like from an eComm perspective I may have a particular product that I have in mind, search it, find a perfect page, buy the product and I’ll never visit that site again because it only deals in that particular product, but that doesn’t mean it’s a bad site as it’s provided a great experience and done exactly what it’s intended.
(Direct) is also a bin for everything that’s not correctly attributed, so there’s all sorts of mess in there. I just don’t believe it can be a ranking factor,.
I would have no issue with this study if they had not specifically stated that it was not a correlation analysis. This implies that direct traffic is causal, which is obviously complete nonsense.
Studies like this, when not clearly explained, do more harm than good IMHO
Rankings cause direct traffic. Both immediately and in the longer term.
Traffic of any kind causes links. We know what links do. But that’s a secondary effect.
Traffic of any kind causes brand and domain search to increase. Could this be a factor in something somewhere? Sure.
Anybody else got a plausible idea why this isn’t “tail wags dog?”
I think we’re on the same page, Martin. Nice analysis – and very even-handed. I think there’s a lot more correlation in this “study” than is healthy and frankly, I’m embarrassed for SEMRush for the result. Direct traffic can certainly contribute indirectly to rankings, but I’m very skeptical of it being part of Google’s ranking algorithms. Too easy to game being a big reason.
I have to admit when I saw that in their study, I decided to ignore it as bad science. I really respect the SEMRush folks, and their tool, but it makes no sense at all for Google to measure direct traffic and use it as a ranking factor. I won’t even bother to defend the statement.
However, it should be no surprise that their is a very strong correlation here. Sites with strong brands, and strong marketing, are going to get more links, and have more conversations taking place about them on the web. So the correatlion makes a great deal of sense. But, that does not make it a ranking factor.
Let me clarify Gary’s comments a little further. Part of the discussion occurred during my joint keynote with him at Pubcon.
Mentions on web sites CAN influence rankings. They do not act the same way as links. What Google does do, is that they look at the context of the mentions to capture additional relevance signals about a page / site. E.g. if you sell books, but don’t have a page on used history books (or talk about it on your site), but a lot of people discuss you in the context of selling used history books, you might still to rank for it.
So, back to direct traffic. Clearly, a great deal of correlation there makes sense, especially in the top 10 of the results. That’s what we have going on here.
From what we’re seeing, direct traffic is clearly a ranking factor.
Gary