Categories

Authors

Pages

Recent posts

Recent comments

Archives

The Impact of Google Encrypted Search

Wednesday, 19 October 2011 @ 22:58, by Bret Gundersen

Let’s start with a summary of what you’ve been reading. In an effort to make “search more secure,” Google is encrypting more search queries through SSL. Searchers signed-in to a Google account are redirected to https://www.google.com (notice the s) and (here’s the rub) search keywords are stripped from the referrer [correction follows] when a natural keyword is clicked. The typed keyword is hidden from you and your site optimization systems (that’s us).

What’s going to change? Natural search keywords and search engines will be under-reported in SiteCatalyst. If you’ve implemented organic search integration in SearchCenter you will see a reduction in organic search term traffic. However, SearchCenter paid keyword data is not affected.

Google engineer Matt Cutts estimates that search encryption will impact less than 10% of organic search queries. To date we have seen that 0.5% to 2% of Google searches are affected, though we expect that number to increase. As a percent of total search traffic, we’ve seen a 0.3% to 1.5% impact.

Rather than hope the 0.5%, 2%, or 10% ratio applies to your site, follow these steps to measure the impact directly:

1. Run the Traffic Sources > Referrers report using the Instance metric

Referrers Report

2. Select the current day in the date selector

3. Filter the report to show referrers from major search engines by adding this to the search box: google.com/search OR google.com/url OR search.yahoo.com OR bing.com. Some tips:

  • Add non-US domain endings, like google.co.uk, where appropriate
  • Don’t forget to capitalize the OR operator
  • In SiteCatalyst 14, this search may take a few minutes

Major Search Engine Filter

4. Note the total in the bottom-right corner of the page (I drop it into Excel, but your memory may be better than mine)

Total

5. Filter the report to encrypted search referrers by searching for google.com/url AND q=& (again, capitalization matters for the AND operator)

Encrypted Search Filter

6. Divide the second total by the first to estimate the change to total search traffic caused by encrypted search.

Because the google.com domain is used for so much more than search, SiteCatalyst only counts a “search” when there is a visible keyword. This prevents mail.google.com from being counted as a search engine.  Unfortunately, this will also prevent Google encrypted searches from being counted as search traffic. As a result, the following SiteCatalyst reports will be impacted:

  • Referrer Types: Search Engine traffic will decrease in favor of traffic for “Other Web Sites”
  • Search Keywords (All, Paid and Natural): Metrics associated with keywords will decrease in favor of “None.”
  • Search Engines (All, Paid and Natural): Metrics associated with Google search engines will decrease in favor of “None.”

We are currently evaluating changes to the processing platform to improve reporting of encrypted search traffic, though there is no way to uncover the original search query.

Impact on Landing Page Optimization

Google search encryption will affect retargeting and landing page adaptation if site content is customized based on the query parameter. However, because search encryption does not apply to all data, there should be an actionable sample of data for both paid and organic search marketers to perform conversion analysis and landing page optimization (especially since Bing and Yahoo do not offer encrypted search options).

For more information, Search Engine Land has a written a detailed post about Google’s recent search encryption changes here: http://searchengineland.com/google-to-begin-encrypting-searches-outbound-clicks-by-default-97435

The esteemed Chris Haleua from the SearchCenter product management team contributed heavily to this post (in other words, if you don’t like it, blame him).

UPDATE (2): Starting Tuesday, 8 Thursday, 10 November 2011, you will see the term “Keyword Unavailable” in the Search Keywords (All and Natural), Marketing Channel Detail and SearchCenter Organic to Paid Keyword reports. Additionally, any loss in Natural Search attribution for Google in the Search Engines report will return to normal levels. This change will not be retroactive. The gradual decrease you’ve noticed in the Search Engines, Marketing Channel, and Referrer Types reports will jump back up by 2-4% 14-20% (according to most customers I’ve spoken with) starting Thursday afternoon, Pacific Time.

UPDATE 3: The changes are live. You will now see “Keyword Unavailable” in keyword reports and Search channel and referrer types will return to normal levels. If you’re using the Channel Manager plugin, you will need to use an updated version.

We are still investigating the status of Unified Sources DB VISTA rules, and hope to have an update on those soon.

Thank you for the comments on this topic, both here and in the Ideas Exchange.

  • on  
  • Bookmark at  

respond to post

responses

  1. Jonathan Kay 2011/10/20 @ 4:02 am

    Very useful post - thanks.

    Happy to share that I’m seeing a 2.5% impact for one of my UK clients when looking at google.com traffic to their .co.uk site (for this morning). Google hasn’t yet made the change to google.co.uk
  2. Hugh W 2011/10/20 @ 5:12 am

    Is there going to be an update to sitecatalyst so that visits satisfying the conditions in point 5 can be correctly attributed in the search engines and referrer types report? The lack of a keyword shouldn’t impact the ability to identify these visits as from Google organic search as there is still sufficient information in the referrer to differentiate these from google non-search (such as iGoogle).
  3. Jeff 2011/10/20 @ 7:41 am

    Keep in mind that for larger sites, you may run into the unique referrer URL limit of 25k per day - https://omniture-help.custhelp.com/app/answers/detail/a_id/1043/kw/referrer
  4. Bret Gundersen 2011/10/20 @ 9:46 am

    Hugh, you’re right. There is enough information in the referrer to identify this as natural search and we are currently evaluating a SiteCatalyst update. We could, for example, just set the keyword value to “Encrypted Search” when it’s empty. I’m interested in your thoughts on what would make the most sense to users.
  5. Paulsen 2011/10/20 @ 11:45 am

    The guys over at Keystone (Kevin Rogers) have developed a plugin to do just that from what I understand. Check it out here:

    http://www.keystonesolutions.com/community/2011/10/tracking-google-secure-search-in-sitecatalyst/

    But it would be nice to have a native solution.
  6. Sylvain 2011/10/20 @ 12:23 pm

    Hi,
    If Google change only impacts Natural Search, why is it impacting Omniture Paid Search Keywords and Engines report?
    Thank you,
    Sylvain
  7. Chris Haleua 2011/10/20 @ 2:09 pm

    @Paulsen Thanks for the proactive work on that plugin. A native solution is being considered and we will make sure to update the blog once we know more.

    @Sylvain The Paid search reports should not be impacted because they should always have a query value. Can you tell us more about how your report is being impacted?
  8. John Hunter 2011/10/20 @ 6:47 pm

    Very interesting. I am surprised the % of “lost data” will be so low.

    The article you reference is very interesting. It is good that, at least, google webmaster central will continue to show all keyword data. It is sad to lose nay keyword data though - it is very interesting.

    Setting to “encrypted search” does seem much better than nothing.
  9. Nate Orshan 2011/10/24 @ 1:43 pm

    Cheers Bret & Chris for the explanation. So far for the past week I’m seeing a 2.8% to 3.5% loss of recorded Google organic KW (i.e., “instances of Google searches with a blank ‘q’ parameter” divided by “total instances of Google searches”) across a couple of sites. Not the end of the world…but definitely disturbing, especially given that “Big G” could turn off the referrer spigot completely on a whim.

    I’d also recommend Danny Sullivan’s “Google Puts A Price On Privacy” ( http://searchengineland.com/google-puts-a-price-on-privacy-98029 ) to get a sense of the righteous outrage directed at Google’s hypocrisy for starting to cut off organic keyword data to site publishers in the name of protecting user privacy…but not doing likewise for AdWords keyword data.
  10. Emily Hill 2011/10/24 @ 4:45 pm

    @Bret … weighing in on Hugh’s feedback. Yes, identifying it as Encrypted Search bucket would be GREAT. (and even better than Google Analytic’s naming of the “Not Provided” bucket, which would surely elicit many questions from users of “What’s the difference between ‘None’ and ‘Not Provided’ in the Omniture search reports?)

    I know our SEM team works very hard for every 1-2% of organic search traffic they can grow, so this is a much needed enhancement from that perspective even if we can’t see the actual search terms.

    Hugh logged this on Ideas Exchange, so anyone who agrees - please go Promote it here: http://ideas.omniture.com/t5/Adobe-Idea-Exchange-for-Omniture/Google-logged-in-searches/idi-p/5723
  11. Scott 2011/10/25 @ 7:24 am

    Great post, thanks for being proactive in explaining the implications to us. We use the Unified Sources DB VISTA rule for our referral traffic, does this change also impact that? If so are you also considering a revision to that, similar to the native solution you’re exploring for SC? Thanks.
  12. Bret Gundersen 2011/10/25 @ 9:03 am

    Thanks for the comments, everyone. I’ll update this post as soon as we have more information about the change to how we’re reporting searches.

    Jeff, very astute comment about 25k referrer limit for large sites. The good news is that this limit is applied after the day is complete. If you limit your time period to the current day, you’ll pull from the full data set.
  13. Vedha 2011/11/2 @ 11:53 am

    Did something happen with the way these filters are set up? Until Sunday a client’s encrypted searches were between 0.6%-1.7% which spiked to 6% on Monday and 11% on Tuesday. As of 2 Nov, 1:30pm ESP we were still trending at 11%. I want to make sure this methodology still works right before hitting the panic button.
  14. Bret Gundersen 2011/11/2 @ 4:53 pm

    Vedha,

    There have been no changes to how the filters work in SiteCatalyst. Just make sure you’re doing your analysis on the current day’s referrers, to avoid running into the unique URL limit Jeff mentioned.
  15. Tad Miller 2011/11/4 @ 7:21 am

    Seeing 19% of Google Searches missing keyword data as of November 4. So much for “single digit percentages” being effected.
  16. Scott Calise 2011/11/7 @ 11:39 am

    I just want to be sure that the updates you are making will also be made for those who use the Unified Sources DB Vista Rule. We do not use the standard referral reports, but exclusively rely on this Vista rule.

    Can someone please confirm?
    Thanks,
    Scott
  17. Bret Gundersen 2011/11/8 @ 12:37 pm

    Scott, we are still evaluating whether and how to update existing DB VISTA rules, but I will update this post when I have more information.
  18. Franklin 2011/11/16 @ 2:03 pm

    Does the 11/10/11 “fix” impact EVAR2 reports?
  19. Kevin Rogers 2011/12/11 @ 3:27 pm

    Has there been anything found out yet about the impact to the Unified Sources VISTA rule?