Skip to main content

Linux QoS Configuration Using TC

In this Document we will go through QoS configuration example using Linux Traffic Control (TC).

Required Linux Modules​

root@dent-1:/usr/src/linux-5.16.9# lsmod
Module                  Size  Used by
sch_htb                36864  1

QoS Configuration Example​

The requirement in this example as follows:

  1. PC1 should have 100 mbps upload speed to the Server.
  2. PC2 should have 75 mbps upload speed to the Server.
  3. PC3 should have 25 mbps upload speed to the Server.

We will use Hierarchy Token Bucket (HTB) classfull QDISC, this classful QDISC allows us to create different classes with different speed limits and attach them to the same interface's QDISC.

Topology​

topology1

Configuration​

Configuration steps:

  • Create HTB QDISC on link enp0s12 with handle-id 8003:.
  • Create 3 child Classes for that QDISC with:
    • classid 8003:1 with rate limit = 100mbps.
    • classid 8003:2 with rate limit = 75mbps.
    • classid 8003:1 with rate limit = 25mbps.
  • Create a Flower Filter for the QDISC with three rules to classify the traffic:
    • Rule 10 that match PC1 ip (192.168.1.1/32) and set the classid to 8003:1.
    • Rule 20 that match PC2 ip (192.168.1.2/32) and set the classid to 8003:2.
    • Rule 30 that match PC3 ip (192.168.1.3/32) and set the classid to 8003:3.
tip

In this Example we attached the three classes directly to the QDISC, however we can create a Top-Level parent class to limit the overall speed of the link to 200, and then create child class of that parent class, and each class will have its own limit, also it's recommended to set the overall limit to a value less than the interface speed to make sure that to keep the enqueueing in the QDISC level instead of interface HW level.

qdiscs-iproute2
  qdisc enp0s12 root 
    handle 8003:
    qdisc-kind htb
    exit
  exit
classes-iproute2
  class enp0s12 8003: 8003:1 
    qdisc-options htb rate 100mbit
    exit
  class enp0s12 8003: 8003:2 
    qdisc-options htb rate 75mbit
    exit
  class enp0s12 8003: 8003:3 
    qdisc-options htb rate 25mbit
    exit
  exit
tc-filters-iproute2
  qdisc-filter enp0s12 8003: 
    rule 10 
      filter flower match src_ip 192.168.1.1/32
      filter flower classid 8003:1
      exit
    rule 20 
      filter flower match src_ip 192.168.1.2/32
      filter flower classid 8003:2
      exit
    rule 30 
      filter flower match src_ip 192.168.1.3/32
      filter flower classid 8003:3
      exit
   exit
info

There are other parameters in the classes that we can use to control the traffic flow like:

  • priority: to give different enqueue priorities.
  • ceil: Maximum rate at which a class can send, if its parent has bandwidth to spare. Defaults to the configured rate, which implies no borrowing
  • burst: Amount of bytes that can be burst at ceil speed, in excess of the configured rate. Should be at least as high as the highest burst of all children.

Verify​

We will test the speed using iperf3 with the server (192.168.1.10) as iperf server. Before the QoS config, all three PCs have around 200+mbps upload speed.

  • PC3 rate before the Config:
root@PC3:~# iperf3 -c 192.168.1.10
Connecting to host 192.168.1.10, port 5201
[  5] local 192.168.1.3 port 41860 connected to 192.168.1.10 port 5201
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   261 MBytes   219 Mbits/sec  314             sender
[  5]   0.00-10.04  sec   259 MBytes   216 Mbits/sec                  receiver

  • PC3 after the QoS config:

root@PC3:~# iperf3 -c 192.168.1.10
Connecting to host 192.168.1.10, port 5201
[  5] local 192.168.1.3 port 58770 connected to 192.168.1.10 port 5201   
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  29.9 MBytes  25.0 Mbits/sec    0             sender
[  5]   0.00-10.56  sec  28.8 MBytes  22.9 Mbits/sec                  receiver


  • PC2 after QoS config:
root@PC2:~# iperf3 -c 192.168.1.10
Connecting to host 192.168.1.10, port 5201
[  5] local 192.168.1.2 port 52900 connected to 192.168.1.10 port 5201    
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  88.0 MBytes  73.9 Mbits/sec  691             sender
[  5]   0.00-10.18  sec  85.6 MBytes  70.6 Mbits/sec                  receiver

  • PC1 after QoS:
root@PC1:~# iperf3 -c 192.168.1.10
Connecting to host 192.168.1.10, port 5201
[  5] local 192.168.1.1 port 38994 connected to 192.168.1.10 port 5201
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   114 MBytes  95.4 Mbits/sec  1068             sender
[  5]   0.00-10.19  sec   112 MBytes  91.9 Mbits/sec                  receiver

tip

To have more accurate rate limiting it's recommended to create top-level Class with the overall rate limit, and then attach the child classes to the top-level class instead of attaching them directly to the QDISC.

Troubleshooting​

  • We can check the class statistics to check the drops and over limits:
root@dent-1:~# tc -s class show dev enp0s12
class htb 8003:10 root prio 0 rate 100Mbit ceil 100Mbit burst 1600b cburst 1600b 
 Sent 122884864 bytes 81193 pkt (dropped 0, overlimits 10886 requeues 0) 
 backlog 0b 0p requeues 0
 lended: 11101 borrowed: 0 giants: 0
 tokens: 1917 ctokens: 1917

class htb 8003:20 root prio 0 rate 75Mbit ceil 75Mbit burst 1584b cburst 1584b 
 Sent 190621351 bytes 125948 pkt (dropped 0, overlimits 17333 requeues 0) 
 backlog 0b 0p requeues 0
 lended: 17495 borrowed: 0 giants: 0
 tokens: 2546 ctokens: 2546

class htb 8003:30 root prio 0 rate 25Mbit ceil 25Mbit burst 1600b cburst 1600b 
 Sent 33907347 bytes 22418 pkt (dropped 0, overlimits 6864 requeues 0) 
 backlog 0b 0p requeues 0
 lended: 6893 borrowed: 0 giants: 0
 tokens: 7670 ctokens: 7670