Differences

This shows you the differences between two versions of the page.

--- pfsense:dns:troubleshooting:slow_dns_response [2021/01/31 16:01] – created peter
+++ pfsense:dns:troubleshooting:slow_dns_response [2021/02/01 09:24] (current) – peter
@@ Line 1: / Line 1: @@
 ====== PFSense - DNS - Troubleshooting - Slow DNS Response ======
+Various things to check.
+<WRAP info>
+**CONCLUSION:**  Unbound restarts each time a DHCP lease gets initiated or renewed.
+Ideally, Unbound should not restart but simply needs to refresh.  Unbound needs to be fixed.
+If pfBlockerNG is being used, this could mean upwards of 60 seconds of downtime whilst it reloads any blocklists.
+</WRAP>
+----
+===== Change Firewall Maximum Table Entries =====
+Navigate to **System -> Advanced > Firewall & Nat**:
+  * Firewall Maximum Table Entries:  2000000
+<WRAP info>
+**NOTE:**  If this figure is too low, it will result in slowness.
+The higher this value, the more memory it will use, so be careful setting this too high on systems with low memory.
+</WRAP>
+----
+===== Monitor the Status =====
+Navigate to **Status -> Monitoring**.
+  * Click the **Settings** Icon.
+{{:pfsense:dns:troubleshooting:pfsense_-_status_-_monitoring_-_settings.png?800|}}
+----
+  * Left Axis:  **Quality**.
+  * Right Axis:  **None**.
+  * Click **Update Graphs**.
+<WRAP info>
+**NOTE:**  This will show the status of a continuous ping to your default gateway over time.
+</WRAP>
+{{:pfsense:dns:troubleshooting:pfsense_-_status_-_monitoring_-_settings_-_default_-_settings.png?800|}}
+----
+Check if there is any recent slowness being reported:
+{{:pfsense:dns:troubleshooting:pfsense_-_status_-_monitoring_-_settings_-_default_-_interactive_graph.png?800|}}
+{{:pfsense:dns:troubleshooting:pfsense_-_status_-_monitoring_-_settings_-_default_-_data_summary.png?800|}}
+----
+===== Check what interfaces are running =====
+Navigate to **Diagnostics -> Command Prompt**.
+<code bash>
+pciconf -lv | grep -A1 -B3 network
+</code>
+returns:
+<code bash>
+em0@pci0:1:0:0:	class=0x020000 card=0x00008086 chip=0x150c8086 rev=0x00 hdr=0x00
+    vendor     = 'Intel Corporation'
+    device     = '82583V Gigabit Network Connection'
+    class      = network
+    subclass   = ethernet
+em1@pci0:2:0:0:	class=0x020000 card=0x00008086 chip=0x150c8086 rev=0x00 hdr=0x00
+    vendor     = 'Intel Corporation'
+    device     = '82583V Gigabit Network Connection'
+    class      = network
+    subclass   = ethernet
+em2@pci0:3:0:0:	class=0x020000 card=0x00008086 chip=0x150c8086 rev=0x00 hdr=0x00
+    vendor     = 'Intel Corporation'
+    device     = '82583V Gigabit Network Connection'
+    class      = network
+    subclass   = ethernet
+em3@pci0:4:0:0:	class=0x020000 card=0x00008086 chip=0x150c8086 rev=0x00 hdr=0x00
+    vendor     = 'Intel Corporation'
+    device     = '82583V Gigabit Network Connection'
+    class      = network
+    subclass   = ethernet
+em4@pci0:5:0:0:	class=0x020000 card=0x00008086 chip=0x150c8086 rev=0x00 hdr=0x00
+    vendor     = 'Intel Corporation'
+    device     = '82583V Gigabit Network Connection'
+    class      = network
+    subclass   = ethernet
+em5@pci0:6:0:0:	class=0x020000 card=0x00008086 chip=0x150c8086 rev=0x00 hdr=0x00
+    vendor     = 'Intel Corporation'
+    device     = '82583V Gigabit Network Connection'
+    class      = network
+    subclass   = ethernet
+</code>
+<WRAP info>
+**NOTE:**  This shows Intel NICs.
+Intel usually works well with pfSense.
+If on the otherhand this was not Intel, this could be a cause for the slowness.
+</WRAP>
+----
+===== Check DNS Resolver Caching Figures =====
+Navigate to **Diagnostics -> Command Prompt**.
 <code bash>
@@ Line 8: / Line 124: @@
 <code bash>
-total.num.queries=1297
+total.num.queries=11060
 total.num.queries_ip_ratelimited=0
-total.num.cachehits=1026
+total.num.cachehits=10669
-total.num.cachemiss=271
+total.num.cachemiss=391
-total.num.prefetch=96
+total.num.prefetch=342
-total.num.expired=88
+total.num.expired=295
-total.num.recursivereplies=271
+total.num.recursivereplies=438
-msg.cache.count=1552
+msg.cache.count=2073
-rrset.cache.count=3277
+rrset.cache.count=4222
-infra.cache.count=3255
+infra.cache.count=6734
-key.cache.count=132
+key.cache.count=174
 </code>
+<WRAP info>
+**NOTE:**  This combines the output of:
+  * unbound-control -c /var/unbound/unbound.conf stats_noreset | grep total.num
+  * unbound-control -c /var/unbound/unbound.conf stats_noreset | grep cache.count
+Output is:
+  * **num.queries**:  number of queries received by thread.
+  * **num.cachehits**:  number of queries that were successfully answered using a cache lookup.
+  * **num.cachemiss**:  number of queries that needed recursive processing.
+  * **num.prefetch**:  number of cache prefetches performed.
+    * This number is included in cachehits, as the original query had the unprefetched answer from cache, and resulted in recursive processing, taking a slot in the requestlist.
+    * Not part of the recursivereplies (or the histogram thereof) or cachemiss, as a cache response was sent.
+  * **num.zero_ttl**:  number of replies with ttl zero, because they served an expired cache entry.
+  * **num.recursivereplies**:  The number of replies sent to queries that needed recursive processing.
+    * Could be smaller than **threadX.num.cachemiss** if due to timeouts no replies were sent for some queries.
+  * **msg.cache.count**:  The number of items (DNS replies) in the message cache.
+  * **rrset.cache.count**:  The number of RRsets in the rrset cache. This includes rrsets used by the messages in the message cache, but also delegation information.
+  * **infra.cache.count**:  The number of items in the infra cache.
+    * These are IP addresses with their timing and protocol support information.
+  * **key.cache.count**:  The number of items in the key cache.
+    * These are DNSSEC keys, one item per delegation point, and their validation status.
+</WRAP>
+----
+===== Check System Logs for the DNS Resolver =====
+Navigate to **Status -> System Logs -> System -> DNS Resolver**.
+Check if the DNS Resolver is restarting when this issue occurs.
+{{:pfsense:dns:troubleshooting:pfsense_-_status_-_system_logs_-_system_-_dns_resolver_-_restart_of_unbound.png?800|}}
+<WRAP info>
+**NOTE:**  This shows Unbound restarting!
+See:  https://forum.netgate.com/topic/115482/frequent-unbound-restarts/67
+It seems that Unbound restarts when the WAN DHCP lease gets renewed.
+  * Compare the DNS and DHCP logs will probably show there is a correlation.
+  * Every time a Remote Client connection is initiated or stopped, Unbound has to restart.
+    * The DHCP code is currently sending a HUP signal to the DNS Resolver, which Unbound takes as needing to do a restart.
+    * Ideally, when a new host enters the DNS local zone because it gets a DHCP release the correct thing to do is reload that zone, not restart the DNS service.  Unbound needs to be fixed.
+  * If pfBlockerNG is being used, this could mean upwards of 60 seconds of downtime whilst it reloads any blocklists.
+A temporary workaround is not to use DHCP.
+  * For every DHCP entry, you could set and maintain, at the bottom of the page, the **DHCP Static Mappings for this Interface**.
+  * On the **Status -> DHCP Leases** page you can also choose what lease you want to add as a Static lease.
+Other options to try:
+  * Reduce the number of pfBlockerNG blocks.
+  * This should speed up the restart of Unbound.
+</WRAP>
+----
+===== References =====
+https://forum.netgate.com/topic/115482/frequent-unbound-restarts/67