My book Mastering Modern Payments is out now!
Integrate Stripe into your Rails app in no time flat. Payments, subscriptions, marketplaces, and more than 100 examples.
Frequently I come across confusion with domain names. Why doesn't my website work? Why is this stupid thing broken, everything I try fails, I just want it to work!! Invariably the question asker either doesn't know what DNS is or doesn't understand how something fundamental works. More generally, people think that DNS is scary or complicated. This article is an attempt at quelling that fear. DNS is easy once you understand a few basic concepts.
What is DNS
First things first. DNS stands for Domain Name System. Fundamentally it's a globally distributed key value store. Servers around the world can give you the value associated with a key, and if they don't know they'll ask other servers for the answer.
That's it. That's all there is to it. You (or your web browser) ask for the value associated with the key
www.example.com and get back
Basic Exploration and Fundamental Types
The great thing about the DNS is that it's completely public and open so it's easy to poke around. Let's do a little exploring, starting with this domain,
petekeen.net which I am hosting on a machine named
web01.bugsplat.info. Note that you can run all of these examples from an OS X or linux command line.
First, let's look at a simple domain name to IP address mapping:
$ dig web01.bugsplat.info
dig command is a veritable Swiss Army knife for querying DNS servers and we'll be using it quite a bit. Here's the first part of the response:
; <<>> DiG 9.7.6-P1 <<>> web01.bugsplat.info ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 51539 ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
There's only one interesting thing in here. We asked for one record and got exactly one respose. Here's the question we asked:
;; QUESTION SECTION: ;web01.bugsplat.info. IN A
dig defaults to asking for
A stands for address and is one of the basic fundamental types of records in the DNS. An
A record holds exactly one
IPv4 address. There's an equivalent record for
IPv6 addresses named
AAAA. Next, let's look at the answer our DNS server gave us:
;; ANSWER SECTION: web01.bugsplat.info. 300 IN A 18.104.22.168
This says the host
web01.bugsplat.info. has exactly one
300 is called the
TTL value, or time to live. It's the number of seconds that this record can be cached before it needs to be checked again. The
IN component stands for
Internet and is meant to disambiguate between the various types of networks that the DNS historically was responsible for. You can read about those in IANA's DNS Parameters document (thanks for the correction, mcmatterson!)
The rest of the response tells you things about the response itself:
;; Query time: 20 msec ;; SERVER: 192.168.1.1#53(192.168.1.1) ;; WHEN: Fri Jul 19 20:01:16 2013 ;; MSG SIZE rcvd: 56
Specifically, it tells you how long it took for your server to respond, what that server's IP address is (
192.168.1.1), what port
dig asked (
53, the default DNS port), when the query completed, and how many bytes the response contained.
As you can see, there's an awful lot going on in a single DNS query. Every time you open a web page your browser makes literally dozens of these queries to resolve the web host, all of the hosts where external resources like images and scripts are located, etc. Every single resource involves at least one DNS query, which would involve an awful lot of traffic if DNS wasn't designed to be heavily cached.
What you probably can't see, however, is that the DNS server at
192.168.1.1 contacted a whole chain of other servers in order to answer that simple question of what address does
web01.bugsplat.info map to. Let's run a trace to see all of the servers that
dig would have to contact if they weren't already cached:
$ dig +trace web01.bugsplat.info ; <<>> DiG 9.7.6-P1 <<>> +trace web01.bugsplat.info ;; global options: +cmd . 137375 IN NS l.root-servers.net. . 137375 IN NS m.root-servers.net. . 137375 IN NS a.root-servers.net. . 137375 IN NS b.root-servers.net. . 137375 IN NS c.root-servers.net. . 137375 IN NS d.root-servers.net. . 137375 IN NS e.root-servers.net. . 137375 IN NS f.root-servers.net. . 137375 IN NS g.root-servers.net. . 137375 IN NS h.root-servers.net. . 137375 IN NS i.root-servers.net. . 137375 IN NS j.root-servers.net. . 137375 IN NS k.root-servers.net. ;; Received 512 bytes from 192.168.1.1#53(192.168.1.1) in 189 ms info. 172800 IN NS c0.info.afilias-nst.info. info. 172800 IN NS a2.info.afilias-nst.info. info. 172800 IN NS d0.info.afilias-nst.org. info. 172800 IN NS b2.info.afilias-nst.org. info. 172800 IN NS b0.info.afilias-nst.org. info. 172800 IN NS a0.info.afilias-nst.info. ;; Received 443 bytes from 22.214.171.124#53(126.96.36.199) in 1224 ms bugsplat.info. 86400 IN NS ns-1356.awsdns-41.org. bugsplat.info. 86400 IN NS ns-212.awsdns-26.com. bugsplat.info. 86400 IN NS ns-1580.awsdns-05.co.uk. bugsplat.info. 86400 IN NS ns-911.awsdns-49.net. ;; Received 180 bytes from 188.8.131.52#53(184.108.40.206) in 239 ms web01.bugsplat.info. 300 IN A 220.127.116.11 bugsplat.info. 172800 IN NS ns-1356.awsdns-41.org. bugsplat.info. 172800 IN NS ns-1580.awsdns-05.co.uk. bugsplat.info. 172800 IN NS ns-212.awsdns-26.com. bugsplat.info. 172800 IN NS ns-911.awsdns-49.net. ;; Received 196 bytes from 18.104.22.168#53(22.214.171.124) in 15 ms
The DNS is arranged in a hierarchy. Remember how
dig inserted a single
. after the hostname we asked for before,
web01.bugsplat.info? Well, that
. is pretty important and stands for the root of the hierarchy. The root DNS servers are run by various companies and governments around the world. Originally there were only a handful of these servers but as the Internet has grown more have been added, so that now there are notionally 13. Each one of these servers, however, has dozens or hundreds of physical machines hiding behind a single IP.
So, at the top of the trace we see the root servers, each represented by an
NS record. An
NS record maps a domain name, in this case the root, to a DNS server. When you register a domain name with a registrar like Namecheap or Godaddy they create
NS records for you.
In the next block you can see that
dig randomly picked one of the root server responses and asked it for the
web01.bugsplat.info. Which root server? Let's ask!
$ dig -x 126.96.36.199 ; <<>> DiG 9.8.3-P1 <<>> -x 188.8.131.52 ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 2862 ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0 ;; QUESTION SECTION: ;241.5.5.192.in-addr.arpa. IN PTR ;; ANSWER SECTION: 241.5.5.192.in-addr.arpa. 3261 IN PTR f.root-servers.net.
-x flag tells
dig to do a reverse lookup on the given IP address. The DNS responds with a
PTR record which maps an IP with a hostname, in this case
Getting back to our original query, the
F root server responded with another set of
NS servers, this time the ones responsible for the
info top level domain.
dig asks one of these servers for the
A record for
web01.bugsplat.info, gets back another set of
NS servers, and then asks one of those servers for the
A record for
web01.bugsplat.info. and finally receives an actual answer. (thanks for the corrections, colmmacc!)
Whew! That would be a heck of a lot of traffic, except that almost all of these entries are cached for a long time by every server in the chain. Your computer caches too, as does your browser. Most of the time DNS resolution will never touch the root servers because their IP addresses hardly ever change. The top level domains
org, etc, are also generally heavily cached.
There are a few other types that you should be aware of. The first is
MX, which maps a domain name to one or more email servers. Email is so important to the functioning of the Internet that it gets its own record type. Here are the
MX records for
$ dig petekeen.net mx ; <<>> DiG 9.7.6-P1 <<>> petekeen.net mx ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 18765 ;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0 ;; QUESTION SECTION: ;petekeen.net. IN MX ;; ANSWER SECTION: petekeen.net. 86400 IN MX 60 web01.bugsplat.info. ;; Query time: 272 msec ;; SERVER: 192.168.1.1#53(192.168.1.1) ;; WHEN: Fri Jul 19 20:33:43 2013 ;; MSG SIZE rcvd: 93
Note that an
MX record points at a name and not an IP address.
The other record type that you should be familiar with is
CNAME which stands for Canonical Name and maps one name onto another. Let's look at the response we get for a
$ dig www.petekeen.net ; <<>> DiG 9.7.6-P1 <<>> www.petekeen.net ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 16785 ;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0 ;; QUESTION SECTION: ;www.petekeen.net. IN A ;; ANSWER SECTION: www.petekeen.net. 86400 IN CNAME web01.bugsplat.info. web01.bugsplat.info. 300 IN A 184.108.40.206 ;; Query time: 63 msec ;; SERVER: 192.168.1.1#53(192.168.1.1) ;; WHEN: Fri Jul 19 20:36:58 2013 ;; MSG SIZE rcvd: 86
The first thing to notice is that we get back two answers. The first says that
www.petekeen.net maps to
web01.bugsplat.info. The second gives the
A record for that server. One way to think about a
CNAME is as an alias for another domain name.
Why CNAME is Messed Up
CNAMEs are incredibly useful, but they have one very important gotcha: if there a
CNAME exists for a particular name, that is the only record allowed for that name. No
NS, no nothing. This is because the DNS substitutes the
CNAME's target for its own value, so every record valid for the target is also valid for the
CNAME. This is why you can't have a
CNAME on a root domain like
petekeen.net, because you generally have to have other records for that domain like
Querying Other Servers
Let's say for sake of argument that you messed up a DNS configuration. You think you've fixed the problem, but you don't want to wait for the cache to expire to see. With
dig you can actually query one of a number of public DNS servers instead of your default server like this:
$ dig www.petekeen.net @220.127.116.11
@ symbol followed by an IP address or hostname tells
dig to query that server on the default DNS port. I use this a lot to query Google's public DNS servers or Level 3's sort-of-public servers at
In this last section we'll talk about some common situations that web developers find themselves in.
Redirect bare domain to www
Almost always you'll want to redirect a bare domain like
www.iskettlemanstillopen.com. Registrars like Namecheap and DNSimple call this a URL Redirect. In Namecheap you would set up a URL Redirect like this:
@ stands for the root domain
iskettlemanstillopen.com. Let's look at the
A record for that domain:
$ dig iskettlemanstillopen.com ;; QUESTION SECTION: ;iskettlemanstillopen.com. IN A ;; ANSWER SECTION: iskettlemanstillopen.com. 500 IN A 18.104.22.168
That IP is owned by Namecheap and is running a small web server that just serves up an HTTP-level redirect to
$ curl -I iskettlemanstillopen.com curl -I iskettlemanstillopen.com HTTP/1.1 302 Moved Temporarily Server: nginx Date: Fri, 19 Jul 2013 23:53:21 GMT Content-Type: text/html Connection: keep-alive Content-Length: 154 Location: http://www.iskettlemanstillopen.com/
CNAME to Heroku or Github
Notice in the screenshot above that there's a second row defining a
CNAME. In this case
www.iskettlemanstillopen.com maps to an application running on Heroku. You'll have to set up Heroku with a similar domain mapping, of course:
$ heroku domains === warm-journey-3906 Domain Names warm-journey-3906.herokuapp.com www.iskettlemanstillopen.com
Github is similar, except that the mapping lives in a file called
CNAME at the root of your pages, as described in their documentation.
Most DNS servers allow you to set up DNS wildcards. For example, I have a wildcard
CNAME set up for
*.web01.bugsplat.info that maps to
web01.bugsplat.info. That way I can host arbitrary things on
web01 and not have to create new DNS entries for them every time:
$ dig randomapp.web01.bugsplat.info ;; QUESTION SECTION: ;randomapp.web01.bugsplat.info. IN A ;; ANSWER SECTION: randomapp.web01.bugsplat.info. 300 IN CNAME web01.bugsplat.info. web01.bugsplat.info. 15 IN A 22.214.171.124
Hopefully this gives you a good beginning understanding of what DNS is and how to about exploring and verifying your configuration. Just remember that you can always ask the DNS questions and generally get back answers. The Internet standards (RFCs) that define DNS are:
There are a few more interesting RFCs as well, including 4034 about a standard named
DNSSEC and 5321 which talks about DNS as it relates to email. These are all fascinating reads if you want more background information
This article is featured in Hacker Monthly issue 42.