Repository navigation
dns-over-https DoH #509
Description
Normally a DNS resolver would be something you plug in at a different level, however for DoH you need an http client already and for some optimizations (eg using cloudflare DoH to only access the Cloudflare web should only require one connection) you need to be using the same machinery for your DoH and regular web requests
As such httpx should implement DoH directly
Activity
sethmlarson commentedon Nov 6, 2019
DoH can be implemented as a middleware via capturing the request URL domain name and rewrite it to an IP address that's been resolved via DoH. I agree we should implement it once and distribute it via a package.
lovelydinosaur commentedon Nov 6, 2019
That's a great call!
graingert commentedon Nov 6, 2019
It would also be good to support both the Chromium style lookup current DNS-over-UDP provider IP in table approach, and the Firefox everything via Cloudflare approach, and allow a custom DoH provider
sethmlarson commentedon Nov 6, 2019
Do you have a link to Chromium's approach? I haven't heard of that.
graingert commentedon Nov 6, 2019
@sethmlarson https://blog.chromium.org/2019/09/experimenting-with-same-provider-dns.html
graingert commentedon Nov 6, 2019
The table is enumerated here: https://www.chromium.org/developers/dns-over-https
graingert commentedon Nov 6, 2019
Whatever resolver system is used to resolve the initial DoH server should also be pluggable, eg aiodns etc
lovelydinosaur commentedon Nov 6, 2019
Also: I’d really like to see this sort of thing tackled as third party middleware, rather than including it in the core package.
If we have third party packages that later become sufficiently proven and well used, then we can take a call on including directly in the package itself.
toppk commentedon Nov 13, 2019
It's a little bit tricky if your are doing TLS, as the http hostname often must stay as the fqdn for certificate validation reasons.
I do think that plugable name resolution could be useful in several use cases:
I currently do (4) and (5) in requests and (3) in httpx. For requests I had to monkey patch inside urllib3 (urllib3.util.connection.create_connection, if you're curious), so I'd be happy to see a sufficient middleware api in httpx for these use cases. Note that for cases 2 and 3, it's not just name resolution but also port selection.
Is there a middleware/plugin system inside of httpx currently that shows how such a system might work? I wasn't planning on porting my requests hacks just yet, but this might be fun.
lovelydinosaur commentedon Nov 27, 2019
Closing this as currently-out-of-scope.
Feasible that we might want to cover this in the core package at some point in the future, but for the foreseeable future let's instead push for making it possible in a third party package via #345