This is an old revision of the document!

Networking - Buffer Bloat

Ideally you want to see A to A+'s for bufferbloat.

Test result showing a letter grade worse than a B, probably indicate you have bufferbloat.

That means the device at your bottleneck link (most likely your router) is letting bulk traffic (uploads/downloads) interfere with (and slow down) your time-sensitive traffic (gaming, Skype, Facetime, etc.)

If not then tune.

Test for Bufferbloat

The DSL Reports Speed Test makes accurate measurements of the download and upload speeds along with the latency during the test.

1. Start a ping to google.com. You’ll see a series of lines, one per ping, typically with times in the 20-100 msec range.

2. Run a speed test simultaneously. To do this, start one of the speed test services below:

http://fast.com - (fast.com now tests for latency under load but we’d like more folk to check their results against our quick test)
http://speedtest.net
http://testmy.net
http://speedof.me

3. Watch the ping times while the speed test is running. If the times jump up when uploading or downloading, then your router is probably bloated.

Change the Q size

IMPORTANT: The bandwidth settings for the limiters need to be set to the upload/download speed of your internet connection.

It is important to keep in mind that what you are doing in pfSense is setting up a rate limiter.

If you set the numbers lower than your connection will allow, you'll get a great buffer bloat score but you'll slow your network throughput to whatever value you chose.

If you set the number too high, the rate limiter wont come into play and you'll be subjected to the same performance and buffer bloat you had prior to making the changes.

The idea is to let pfSense do the rate limiting closer to home.

Letting your provider do it for you increases latency… and that's what we are really trying to avoid.

So to minimize latency, a recommendation is to first do a speed test to find out what your connection is capable of, then set the bandwidth of the limiters in pfSense to those numbers.

Adjust the queue size adds another dimension to the optimizations.

Someday your connection might receive a speed upgrade and you may forgot to adjust the limiter to make use of it!

Instructions

Create Limiters

Navigate to Firewall → Traffic Shaper → Limiters.

1.) Create "Out" limiter

    Tick "Enable"
    Name: FQ_CODEL_OUT
    Bandwidth: 96907 Kbit/s
    Mask: None
    Queue Management Algorithm: Tail Drop
    Scheduler: FQ_CODEL
        target: 5
        interval: 100
        quantum: 300
        limit: 10240
        flows: 20480
    Click Save/Apply Changes

2.) Add "Out" queue

    Tick "Enable"
    Name: fq_codel_out_q
    Mask: None
    Queue Management Algorithm: Tail Drop
    Click Save/Apply Changes

3.) Create "In" limiter

    Tick "Enable"
    Name: FQ_CODEL_IN
    Bandwidth: 83886 Kbit/s
    Mask: None
    Queue Management Algorithm: Tail Drop
    Scheduler: FQ_CODEL
        target: 5
        interval: 100
        quantum: 300
        limit: 10240
        flows: 20480
    Click Save/Apply Changes

4.) Add "In" queue

    Tick "Enable"
    Name: fq_codel_in_q
    Mask: None
    Queue Management Algorithm: Tail Drop
    Click Save/Apply Changes

Create Floating Rules

Add quick pass floating rule to handle ICMP traceroute. This rule matches ICMP traceroute packets so that they are not matched by the WAN-Out limiter rule that utilizes policy routing. Policy routing breaks traceroute.

    Action: Pass
    Quick: Tick Apply the action immediately on match.
    Interface: WAN
    Direction: out
    Address Family: IPv4
    Protocol: ICMP
    ICMP subtypes: Traceroute
    Source: any
    Destination: any
    Description: policy routing traceroute workaround
    Click Save

2.) Add quick pass floating rule to handle ICMP echo-request and echo-reply. This rule matches ping packets so that they are not matched by the limiter rules. See bug 9024 for more info.

    Action: Pass
    Quick: Tick Apply the action immediately on match.
    Interface: WAN
    Direction: any
    Address Family: IPv4
    Protocol: ICMP
    ICMP subtypes: Echo reply, Echo Request
    Source: any
    Destination: any
    Description: limiter drop echo-reply under load workaround
    Click Save

3.) Add a match rule for incoming state flows so that they're placed into the FQ-CoDel in/out queues

    Action: Match
    Interface: WAN
    Direction: in
    Address Family: IPv4
    Protocol: Any
    Source: any
    Destination: any
    Description: WAN-In FQ-CoDel queue
    Gateway: Default
    In / Out pipe: fq_codel_in_q / fq_codel_out_q
    Click Save

4.) Add a match rule for outgoing state flows so that they're placed into the FQ-CoDel out/in queues

    Action: Match
    Interface: WAN
    Direction: out
    Address Family: IPv4
    Protocol: Any
    Source: any
    Destination: any
    Description: WAN-Out FQ-CoDel queue
    Gateway: WAN_DHCP
    In / Out pipe: fq_codel_out_q / fq_codel_in_q
    Click Save/Apply Changes