Differences

This shows you the differences between two versions of the page.

--- pfsense:network:configure_gigabit_speed_networking [2020/11/30 10:49] – created peter
+++ pfsense:network:configure_gigabit_speed_networking [2021/01/06 09:36] (current) – [Edit the bootloader] peter
@@ Line 1: / Line 1: @@
 ====== PFSense - Network - Configure Gigabit Speed Networking ======
+Some systems may struggle to get full gigabit.
+There are many factors that can influence this:
+  * pfSense by design is able to use only 1 core per connection.  This limitation still exists, however single-core performance has considerably improved.
+  * Routers rarely open just one connection, so a single connection is rarely a bottleneck in the real world.
+  * Web browsers opens about 8 TCP connections per website, Torrent clients open hundreds of connections, Netflix opens multiple TCP connections when streaming video, etc.
+Many modern NICs have multiple transmit and receive queues, being able to work simultaneously on many connections.
+  * Intel I210-AT Network Interfaces have 4 transmit and 4 receive queues, and can simultaneously handle 4 connections.
+  * I211-AT Network Interfaces have 2 transmit/receive queues, but is still good enough to deliver 1Gbit on pfSense when more one than one connection is used.
+With some fine tuning, pfSense can take advantage of this and route at 1Gbit when using more than one connection.
+----
+===== Configure pfSense =====
+Navigate to **System -> Advanced -> Networking**.
+In **Network Interfaces**, at the bottom:
+  * Hardware Checksum Offloading:  **Not Checked**.
+  * Hardware TCP Segmentation Offloading  **Not Checked**.
+  * Hardware Large Receive Offloading  **Not Checked**.
+----
+===== Edit the bootloader =====
+Edit some settings from the shell.
+SSH to the box or connect with the serial cable.
+Edit **/boot/loader.conf.local** (you may need to create it, if it doesn't exist) and insert the following settings:
+<file bash /boot/loader.conf>
+# agree with Intel license terms
+legal.intel_ipw.license_ack=1
+legal.intel_iwi.license_ack=1
+# This is the magic. If you don't set this, queues won't be utilized properly.
+# Allow multiple processes for receive/transmit processing.
+hw.igb.rx_process_limit="-1"
+hw.igb.tx_process_limit="-1"
+# More settings to play with below.  Not strictly necessary.
+# Force NIC to use 1 queue (Don't really need this).
+# hw.igb.num_queues=1
+# Give enough RAM to network buffers (default is usually OK).
+# kern.ipc.nmbclusters="1000000"
+#net.pf.states_hashsize=2097152
+#hw.igb.rxd=4096
+#hw.igb.txd=4096
+#net.inet.tcp.syncache.hashsize="1024"
+#net.inet.tcp.syncache.bucketlimit="100"
+</file>
+After saving this file, reboot the router to apply it.
+----
+===== Test =====
+Run some tests to verify that your settings worked properly.
+The easiest way it to use **iperf3** with multiple connections, where one device is on the LAN and the other one in the internet.
+On the server run the following command:
+<code bash>
+iperf3 -s
+</code>
+On your LAN run this command:
+<code bash>
+iperf3 -c SERVER_IP_HERE -P 4
+</code>
+If everything went well, you should be seeing about 940Mbit/s throughput, similar to the snippet below:
+<code bash>
+- - - - - - - - - - - - - - - - - - - - - - - - -
+[  5]  43.00-44.00  sec  56.1 MBytes   470 Mbits/sec    0    481 KBytes
+[  7]  43.00-44.00  sec  55.7 MBytes   468 Mbits/sec    0    438 KBytes
+[SUM]  43.00-44.00  sec   112 MBytes   938 Mbits/sec    0
+- - - - - - - - - - - - - - - - - - - - - - - - -
+[  5]  44.00-45.00  sec  56.4 MBytes   473 Mbits/sec    0    481 KBytes
+[  7]  44.00-45.00  sec  56.1 MBytes   470 Mbits/sec    0    438 KBytes
+[SUM]  44.00-45.00  sec   112 MBytes   943 Mbits/sec    0
+- - - - - - - - - - - - - - - - - - - - - - - - -
+[  5]  45.00-46.00  sec  56.1 MBytes   470 Mbits/sec    0    481 KBytes
+[  7]  45.00-46.00  sec  55.6 MBytes   466 Mbits/sec    0    438 KBytes
+[SUM]  45.00-46.00  sec   112 MBytes   936 Mbits/sec    0
+- - - - - - - - - - - - - - - - - - - - - - - - -
+[  5]  46.00-47.00  sec  57.7 MBytes   484 Mbits/sec    0    481 KBytes
+[  7]  46.00-47.00  sec  55.0 MBytes   461 Mbits/sec    0    438 KBytes
+[SUM]  46.00-47.00  sec   113 MBytes   945 Mbits/sec    0
+- - - - - - - - - - - - - - - - - - - - - - - - -
+[  5]  47.00-48.00  sec  55.2 MBytes   463 Mbits/sec    0    481 KBytes
+[  7]  47.00-48.00  sec  55.8 MBytes   468 Mbits/sec    0    438 KBytes
+[SUM]  47.00-48.00  sec   111 MBytes   931 Mbits/sec    0
+</code>
+If everything went well, you should be seeing about 940Mbit/s throughput.
+----