QUIC in the wild

Update: This post has been edited to correct errors of fact about developer tools and APIs with respect to QUIC. Also, the original post implied that Google might be using QUIC in an anti-competitive manner. We regret the errors, and we apologize for the implication.

Thanks to Yan Zhu of Brave; also, our sincere thanks to Ilya Grigorik, Eric Lawrence, Mike West, and Andrew Howden of Google for their assistance.

Back-story: Brave users reported ads getting past our ad-blocking shields in previous Chromium versions, beginning with reports of ads displaying on YouTube.com on October 11, 2016. uBlock Origin users had reported similar bugs. We discovered during testing that disabling QUIC seemed to stop these ads. As a result, we pushed an update to disable QUIC in Brave on January 25, 2017. This update appears to have temporarily abated the incoming bug reports about ads getting past our shields.

We’ve concluded our investigation of YouTube ads getting past ad blockers, and can confirm that ad blocking with QUIC re-enabled operates as expected in Chrome 58 with uBlock Origin.

We would like to end on a positive note about QUIC. It realizes significant speedups, due to round trip reductions and other advantages, over the HTTPS/TCP/IP alternative. As the post details, we suggest others in ad tech become familiar with it to benefit from the same advantages Google is seeing. 05/10/17 – 3:45pm PDT

When we inspected web page traffic via chrome://net-internals, we discovered that QUIC requests were and still are being used for a majority of Google’s ad domains

What is QUIC?

Originally announced in 2013, QUIC (Quick UDP Internet Connections) is an experimental network protocol, which runs on top of the UDP protocol and is usually requested through port 443 with an Alternative Service HTTP request header flag (example: alt-svc:quic="googleads.g.doubleclick.net:443"). A QUIC working group has been established to standardize the protocol; QUIC is currently still considered experimental.

From a high level, QUIC requests pack several round trips into a single, one-way request, including the security (TLS) handshake. Google’s diagram from their 2015 blog post on the topic helps illustrate the round trip savings:

QUIC is enabled by default in Google’s Chrome browser and underlying Chromium open source browser code. As of March 2017, Chrome accounts for 58.9% of users browsing the web.
Brave blocks QUIC requests. QUIC is an opt-in feature in Opera, and is currently not available in other Firefox, Edge, and Safari. HTTPS requests containing the alt-svc: quic=":443" response header fall back to traditional TCP connections in other browsers, or when QUIC fails in Chrome.
QUIC use has not flown completely under the radar. Google’s April 2015 blog post on QUIC mentioned that roughly 50% of Google traffic was being requested from Chrome/Chromium with QUIC. Missing from the announcement and other QUIC documentation is any mention of QUIC usage in Google’s Doubleclick ad requests, or in Google Analytics tracking requests.

How can I review QUIC sessions and requests?

QUIC runs on top of the UDP protocol. QUIC requests are often made through the same port (443) that is used for TCP requests. Aside from some corporate firewalls that block UDP requests by their protocol number, making QUIC requests to port 443 helps requests get through firewalls configured to allow TCP requests to that port, independent of the protocol number in the IP header. This is a clever hack that dramatically reduces adoption friction, by avoiding the need for firewall reconfiguration in most cases.

Chrome shows QUIC requests in Chrome Developer Tools. If you want to distinguish between QUIC and HTTP in the Network interface, right click to select and reveal the Protocol column.
QUIC requests are visible in the chrome:// internal browser URLs, and of course in WireShark and other lower-level packet sniffing tools.

Tips for observing QUIC traffic

The easiest way to capture and observe QUIC traffic in detail is within the chrome://net-internals interface. Some chrome://net-internalsshortcuts are included below for reference.

To view the QUIC settings and session connections within the dedicated QUIC panel, enter the following address in the URL bar in Chrome: chrome://net-internals#quic
To view QUIC session requests in the Events panel, enter the following address in the URL bar in Chrome: chrome://net-internals/#events&q=type:QUIC_SESSION
To access the alt-svc panel to view domains that contain the quic :443, Alternative Service HTTP response header, enter the following address in the URL bar in Chrome: chrome://net-internals/#alt-svc
To export a JSON log that includes QUIC request traffic, enter the following address in the URL bar in Chrome: chrome://net-internals/#export
The HTTP/2 and SPDY indicator Chrome extension provides a shortcut to the chrome://net-internals Events panel.
Within the Network section of the Chrome Developer Tools, users can right click to reveal the Protocol column, which will show http/2+quic/36 as the request protocol. This column is not displayed with default settings in Developer Tools.

To disable QUIC

Enter chrome://flags/ into the URL bar in Chrome.
Locate the Experimental QUIC protocol flag.
Click to expand the drop down menu and select Disabled

QUIC and Google ad requests

When we inspected web page traffic via chrome://net-internals, we discovered that QUIC requests were and still are being used for a majority of Google’s ad domains, including domains involved with bidding such as the one below:

14419: QUIC_SESSION

bid.g.doubleclick.net  
Start Time: 2017-04-20 00:28:09.220

t= 95972 [st=     0] +QUIC_SESSION  [dt=340129+]  
--> cert_verify_flags = 6
--> host = "bid.g.doubleclick.net"
--> port = 443
--> privacy_mode = false
--> require_confirmation = false

t= 96028 [st=    56]    QUIC_CHROMIUM_CLIENT_STREAM_SEND_REQUEST_HEADERS  
--> :authority: bid.g.doubleclick.net
:method: GET
:scheme: https
accept: */*  
accept-encoding: gzip, deflate, sdch, br  
accept-language: en-US,en;q=0.8  
cookie:id=224683e7072a008a||t=1492673205|et=730|cs=002213fd48a93fd86aff9b0172; IDE=AHWqTUnVMhtPcKMXwHTn_nS5yFVKx1XEjcVJo18Kg9O3XAc2HbLk0EBHtQ; DSID=NO_DATA  
referer: https://tpc.googlesyndication.com/safeframe/1-0-7/html/container.html  
user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36  
--> quic_priority = 1
--> quic_stream_id = 5

Primary Google domains observed opening QUIC sessions

Google domains that made requests within QUIC sessions

Alternate Service Mappings

https://pagead2.googlesyndication.com  
quic googleads.g.doubleclick.net:443, expires 2017-05-20 00:33:40,quic :443, expires 2017-05-20 00:33:40

https://tpc.googlesyndication.com  
quic :443, expires 2017-05-20 00:33:22

https://s0.2mdn.net  
quic :443, expires 2017-05-20 00:33:22

https://cm.g.doubleclick.net  
quic googleads.g.doubleclick.net:443, expires 2017-05-20 00:33:22,quic :443, expires 2017-05-20 00:33:22

https://www.google.com  
quic :443, expires 2017-05-20 00:33:22

https://adx.g.doubleclick.net  
quic googleads.g.doubleclick.net:443, expires 2017-05-20 00:33:22,quic :443, expires 2017-05-20 00:33:22

https://securepubads.g.doubleclick.net  
quic :443, expires 2017-05-20 00:33:22

https://ad.doubleclick.net  
quic :443, expires 2017-05-20 00:33:19

https://googleads4.g.doubleclick.net  
quic googleads.g.doubleclick.net:443, expires 2017-05-20 00:33:03,quic :443, expires 2017-05-20 00:33:03

https://content.googleapis.com  
quic :443, expires 2017-05-20 00:33:02

https://apis.google.com  
quic :443, expires 2017-05-20 00:33:01

https://www.googletagservices.com  
quic googleads.g.doubleclick.net:443, expires 2017-05-20 00:29:26,quic :443, expires 2017-05-20 00:29:26

https://accounts.google.com  
quic :443, expires 2017-05-20 00:33:01

https://www.google-analytics.com  
quic :443, expires 2017-05-20 00:33:00

https://ade.googlesyndication.com  
quic googleads.g.doubleclick.net:443, expires 2017-05-20 00:32:58,quic :443, expires 2017-05-20 00:32:58

https://googleads.g.doubleclick.net  
quic googleads.g.doubleclick.net:443, expires 2017-05-20 00:32:54,quic :443, expires 2017-05-20 00:32:54

https://www.gstatic.com  
quic :443, expires 2017-05-20 00:29:55

https://s1.2mdn.net  
quic :443, expires 2017-05-20 00:32:43

https://stats.g.doubleclick.net  
quic :443, expires 2017-05-20 00:32:40

https://pubads.g.doubleclick.net  
quic :443, expires 2017-05-20 00:32:06

https://static.doubleclick.net  
quic :443, expires 2017-05-20 00:32:07

https://www.youtube.com  
quic :443, expires 2017-05-20 00:32:07

https://video-ad-stats.googlesyndication.com  
quic :443, expires 2017-05-20 00:32:07

https://s.ytimg.com  
quic :443, expires 2017-05-20 00:32:06

https://google-analytics.com  
quic :443, expires 2017-05-20 00:32:03

https://imasdk.googleapis.com  
quic :443, expires 2017-05-20 00:32:00

https://fonts.googleapis.com  
quic :443, expires 2017-05-20 00:31:55

https://ssl.google-analytics.com  
quic :443, expires 2017-05-20 00:31:58

https://www.googletagmanager.com  
quic :443, expires 2017-05-20 00:31:55

https://fonts.gstatic.com  
quic :443, expires 2017-05-20 00:31:55

https://p4-fbm4tfy4du3vk-rsg77dtzm53vwr6k-if-v6exp3-v4.metric.gstatic.com  
quic :443, expires 2017-05-20 00:30:57

https://ssl.gstatic.com  
quic :443, expires 2017-05-20 00:27:03

https://bid.g.doubleclick.net  
quic :443, expires 2017-05-20 00:31:19

https://p4-fbm4tfy4du3vk-rsg77dtzm53vwr6k-854535-i2-v6exp3.ds.metric.gstatic.com  
quic :443, expires 2017-05-20 00:31:08

https://p4-fbm4tfy4du3vk-rsg77dtzm53vwr6k-854535-i1-v6exp3.v4.metric.gstatic.com  
quic :443, expires 2017-05-20 00:31:08

https://ajax.googleapis.com  
quic :443, expires 2017-05-20 00:31:02

https://www.googleadservices.com  
quic googleads.g.doubleclick.net:443, expires 2017-05-20 00:27:45,quic :443, expires 2017-05-20 00:27:45

https://plus.google.com  
quic :443, expires 2017-05-20 00:27:25

https://clients2.google.com  
quic :443, expires 2017-05-20 00:26:24

Examples of Doubleclick QUIC requests

Example of TCP a HTTP GET Request URL with the alt-svc: quic="443" HTTP response header

What does this mean for the ad ecosystem?

It would be helpful for Doubleclick to clarify how they’re using QUIC for ads and tracking. With DoubleClick’s dominance in the ad market, publishers and advertisers need to have a clear understanding and documentation regarding which protocols are being used for ad and tracking requests.

We want to emphasize that we’re fans of QUIC as a protocol, and we hope it is widely adopted. But from our contacts in ad tech, it appears that QUIC is not understood or used yet. While Doubleclick is not obligated to share its QUIC usage details, we hope that it will do so to increase QUIC adoption across the ad-tech ecosystem.