add a managment command "domains" to check the domains, notify owner of domain, update admin docs

if a nameserver is not reachable or does not answer queries for the domain, the domain is flagged as not available (and also as not public).

also added catching of PeerBadKey exception.

docs/admin.rst

@@ -305,6 +305,8 @@ it runs as the same user as the nsupdate.info wsgi application::
     0  3 * * * $HOME/env/bin/python $HOME/env/bin/django-admin.py clearsessions
     # clear outdated registrations:
     30 3 * * * $HOME/env/bin/python $HOME/env/bin/django-admin.py cleanupregistration
+    # check whether the domain nameservers are reachable / answer queries:
+    0  4 * * * $HOME/env/bin/python $HOME/env/bin/django-admin.py domains --check --notify-user
 
 
 Dealing with abuse
@@ -342,6 +344,25 @@ While abuse_blocked is set, the service won't accept updates for this host.
 The user can see the ABUSE-BLOCKED status on the web interface, but can not
 change the flag.
 
+Dealing with badly configured domains
+-------------------------------------
+
+In the regular jobs example in the previous section,
+django-admin.py domains --check --notify-user means that we'll check all
+domains that are currently flagged as available.
+
+We query the nameserver configured for the domain and check if it answers a
+SOA query for this domain. If we can't reach the nameserver or it does not
+answer the query, we flag the domain as not available. We also flag it as
+not public (this only is a change if it was public before).
+If --notify-user is given, we notify the owner of the domain by email if we
+flag the domain as not available. Owner in this context means: the user who
+added the domain to our service.
+
+Please note that we can not check whether the nameserver accepts dynamic
+updates for the domain. The dns admin could have set arbitrary restrictions
+on this and we do not know them. So if you have a domain configured with the
+service, please make sure that dynamic updates really work.
 
 Database contents
 -----------------

nsupdate/main/dnstools.py

@@ -318,6 +318,10 @@ def update_ns(fqdn, rdtype='A', ipaddr=None, action='upd', ttl=60):
         logger.error("PeerBadSignature - shared secret mismatch? zone: %s" % (origin, ))
         set_ns_availability(domain, False)
         raise DnsUpdateError("PeerBadSignature")
+    except dns.tsig.PeerBadKey:
+        logger.error("PeerBadKey - shared secret mismatch? zone: %s" % (origin, ))
+        set_ns_availability(domain, False)
+        raise DnsUpdateError("PeerBadKey")
     except dns.tsig.PeerBadTime:
         logger.error("PeerBadTime - DNS server did not like the time we sent. zone: %s" % (origin, ))
         set_ns_availability(domain, False)

nsupdate/management/commands/domains.py

@@ -0,0 +1,105 @@
+"""
+dealing with domains (Domain records in our database)
+"""
+
+import dns.resolver
+
+from optparse import make_option
+
+from django.core.management.base import BaseCommand
+from django.core.mail import send_mail
+from django.db import transaction
+
+from nsupdate.main.models import Domain
+from nsupdate.main.dnstools import FQDN, query_ns, NameServerNotAvailable
+
+
+MSG = """\
+Your domain: %(domain)s (comment: %(comment)s)
+
+Issue: The nameserver of the domain is not reachable and was set to not available
+       (and also to not public, in case it was public).
+
+Explanation:
+You created the domain on our service and entered a primary nameserver IP for it.
+We tried to query that nameserver but it either was not reachable or it did not
+answer queries for this domain.
+
+Resolution:
+If you really want that domain to work and you really control that nameserver:
+
+1. fix the nameserver so it responds to queries for this domain
+2. make sure the nameserver is reachable from us
+3. make sure it accepts dynamic updates for hosts in this domain
+4. make sure it uses the same secret as configured on the service
+5. set the domain to "available" on the service
+6. check if the "public" flag is correctly
+
+Alternatively, if you do not use the domain with our service, delete the
+domain entry, so it is removed from our database. This will also remove all
+hosts that were added to this domain (if any).
+"""
+
+
+def check_dns(domain):
+    """
+    checks if the nameserver is reachable and answers queries for the domain.
+
+    note: we can't reasonably check for dynamic updates as the dns admin might
+    have put restrictions on which hosts are allowed to be updated.
+
+    :param domain: domain name
+    :return: available status
+    """
+    fqdn = FQDN(host=None, domain=domain)
+    try:
+        query_ns(fqdn, 'SOA')
+        queries_ok = True
+    except (dns.resolver.Timeout, dns.resolver.NoNameservers,
+            dns.resolver.NXDOMAIN, dns.resolver.NoAnswer, NameServerNotAvailable):
+        # note: currently the domain is also set to unavailable as a
+        # side effect in query_ns()
+        queries_ok = False
+    return queries_ok
+
+
+class Command(BaseCommand):
+    help = 'deal with domains'
+
+    option_list = BaseCommand.option_list + (
+        make_option('--check',
+                    action='store_true',
+                    dest='check',
+                    default=False,
+                    help='check whether nameserver for domain is reachable and answers queries',
+        ),
+        make_option('--notify-user',
+                    action='store_true',
+                    dest='notify_user',
+                    default=False,
+                    help='notify the user by email when domain gets flagged as unavailable',
+        ),
+    )
+
+    def handle(self, *args, **options):
+        check = options['check']
+        notify_user = options['notify_user']
+        with transaction.atomic():
+            for d in Domain.objects.all():
+                if check and d.available:
+                    domain = d.name
+                    comment = d.comment
+                    creator = d.created_by
+                    available = check_dns(domain)
+                    if not available:
+                        d.available = False  # see comment in check_dns()
+                        d.public = False
+                        if notify_user:
+                            from_addr = None  # will use DEFAULT_FROM_EMAIL
+                            to_addr = creator.email
+                            subject = "issue with your domain %(domain)s" % dict(domain=domain)
+                            msg = MSG % dict(domain=domain, comment=comment)
+                            send_mail(subject, msg, from_addr, [to_addr], fail_silently=True)
+                        msg = "setting unavailable flag for domain %s (created by %s)\n" % (domain, creator, )
+                        self.stdout.write(msg)
+                    d.save()