Traffic Shaping/Control

Introduction

Starting with Version 2.5.5, Shorewall has builtin support for traffic shaping and control. Before this version, the support was quite limited. You were able to use your own tcstart script (and you still are), but besides the tcrules file it was not possible to define classes or queuing disciplines inside the Shorewall config files.

The support for traffic shaping and control still does not cover all options available (and especially all algorithms that can be used to queue traffic) in the Linux kernel but it should fit most needs. If you are using your own script for traffic control and you still want to use it in the future, you will find information on how to do this, later in this document. But for this to work, you will also need to enable traffic shaping in the kernel and Shorewall as covered by the next sections.

Linux traffic shaping and control

This section gives a brief introduction of how controlling traffic with the Linux kernel works. Although this might be enough for configuring it in the Shorewall configuration files, we strongly recommend that you take a deeper look into the Linux Advanced Routing and Shaping HOWTO. At the time of writing this, the current version is 1.0.0.

Since kernel 2.2 Linux has extensive support for controlling traffic. You can define different algorithms that are used to queue the traffic before it leaves an interface. The standard one is called pfifo and is (as the name suggests) of the type First In First out. This means, that it does not shape anything, if you have a connection that eats up all your bandwidth, this queuing algorithm will not stop it from doing so.

For Shorewall traffic shaping we use two algorithms, one is called HTB (Hierarchical Token Bucket) and SFQ (Stochastic Fairness Queuing). SFQ is easy to explain: it just tries to track your connections (tcp or udp streams) and balances the traffic between them. This normally works well. HTB allows you to define a set of classes, and you can put the traffic you want into these classes. You can define minimum and maximum bandwidth settings for those classes and order them hierarchically (the less prioritized classes only get bandwidth if the more important have what they need). Shorewall builtin traffic shaping allows you to define these classes (and their bandwidth limits), and it uses SFQ inside these classes to make sure, that different data streams are handled equally.

If you are running Shorewall-shell or if you are running Shorewall-perl 4.1.5 or earlier:

You can only shape outgoing traffic. The reason for this is simple, the packets were already received by your network card before you can decide what to do with them. So the only choice would be to drop them which normally makes no sense (since you received the packet already, it went through the possible bottleneck (the incoming connection). The next possible bottleneck might come if the packet leaves on another interface, so this will be the place where queuing might occur. So, defining queues for incoming packets is not very useful, you just want to have it forwarded to the outgoing interface as fast as possible.
There is one exception, though. Limiting incoming traffic to a value a bit slower than your actual line speed will avoid queuing on the other end of that connection. This is mostly useful if you don't have access to traffic control on the other side and if this other side has a faster network connection than you do (the line speed between the systems is the bottleneck, e.g. a DSL or Cable Modem connection to your provider's router, the router itself is normally connected to a much faster backbone). So, if you drop packets that are coming in too fast, the underlying protocol might recognize this and slow down the connection. TCP has a builtin mechanism for this, UDP has not (but the protocol over UDP might recognize it , if there is any).
The reason why queuing is bad in these cases is, that you might have packets which need to be prioritized over others, e.g. VoIP or ssh. For this type of connections it is important that packets arrive in a certain amount of time. For others like HTTP downloads, it does not really matter if it takes a few seconds more.
If you have a large queue on the other side and the router there does not care about QoS or the QoS bits are not set properly, your important packets will go into the same queue as your less time critical download packets which will result in a large delay.

If you are running Shorewall-perl 4.1.6 or later:

You can shape incoming traffic through use of an Intermediate Frame Block (IFB) device. See below. But beware: using an IFB can result in queues building up both at your ISPs router and at your own.

This is not to say that you cannot shape download traffic, regardless of which Shorewall release you are running.

If you wish to shape downloads, you can always configure traffic shaping on your firewall's local interface. An example appears below.
Again, however, this can result in queues building up both at your ISPs router and at your own.

You shape and control outgoing traffic by assigning the traffic to classes. Each class is associated with exactly one network interface and has a number of attributes:

PRIORITY - Used to give preference to one class over another when selecting a packet to send. The priority is a numeric value with 1 being the highest priority, 2 being the next highest, and so on.
RATE - The minimum bandwidth this class should get, when the traffic load rises. Classes with a higher priority (lower PRIORITY value) are served even if there are others that have a guaranteed bandwidth but have a lower priority (higher PRIORITY value).
CEIL - The maximum bandwidth the class is allowed to use when the link is idle.
MARK - Netfilter has a facility for marking packets. Packet marks have a numeric value which is limited in Shorewall to the values 1-255. You assign packet marks to different types of traffic using entries in the /etc/shorewall/tcrules file.

One class for each interface must be designated as the default class. This is the class to which unmarked traffic (packets to which you have not assigned a mark value in /etc/shorewall/tcrules) is assigned.

Netfilter also supports a mark value on each connection. You can assign connection mark values in /etc/shorewall/tcrules, you can copy the current packet's mark to the connection mark (SAVE), or you can copy the connection mark value to the current packet's mark (RESTORE). For more information, see this article.

Linux Kernel Configuration

You will need at least kernel 2.4.18 for this to work, please take a look at the following screenshot for what settings you need to enable. For builtin support, you need the HTB scheduler, the Ingress scheduler, the PRIO pseudoscheduler and SFQ queue. The other scheduler or queue algorithms are not needed.

This screen shot shows how I configured QoS in a 2.6.16 Kernel:

And here's my recommendation for a 2.6.21 kernel:

Enable TC support in Shorewall

You need this support whether you use the builtin support or whether you provide your own tcstart script.

To enable the builtin traffic shaping and control in Shorewall, you have to do the following:

Set TC_ENABLED to "Internal" in /etc/shorewall/shorewall.conf. Setting TC_ENABLED=Yes causes Shorewall to look for an external tcstart file (See a later section for details).
Setting CLEAR_TC parameter in /etc/shorewall/shorewall.conf to Yes will clear the traffic shaping configuration during Shorewall [re]start and Shorewall stop. This is normally what you want when using the builtin support (and also if you use your own tcstart script)
The other steps that follow depend on whether you use your own script or the builtin solution. They will be explained in the following sections.

Using builtin traffic shaping/control

Shorewall's builtin traffic shaping feature provides a thin layer on top of the ingress qdesc, HTB and SFQ. That translation layer allows you to:

Define HTB classes using Shorewall-style column-oriented configuration files.
Integrate the reloading of your traffic shaping configuration with the reloading of your packet-filtering and marking configuration.
Assign traffic to HTB classes by TOS value.
Assign outgoing TCP ACK packets to an HTB class.
Assign traffic to HTB classes based on packet mark value.

Warning

Shorewall's builtin traffic shaping feature is limited to ten (10) devices.

Those few features are really all that builtin traffic shaping/control provides; consequently, you need to understand HTB and Linux traffic shaping as well as Netfilter packet marking in order to use the facility. Again, please see the links at top of this article.

For defining bandwidths (for either devices or classes) please use kbit or kbps (for Kilobytes per second) and make sure there is NO space between the number and the unit (it is 100kbit not 100 kbit). Using mbit, mbps or a raw number (which means bytes) could be used, but note that only integer numbers are supported (0.5 is not valid).

To properly configure the settings for your devices you need to find out the real up- and downstream rates you have. This is especially the case, if you are using a DSL connection or one of another type that do not have a guaranteed bandwidth. Don't trust the values your provider tells you for this; especially measuring the real download speed is important! There are several online tools that help you find out; search for "dsl speed test" on google (For Germany you can use arcor speed check). Be sure to choose a test located near you.

/etc/shorewall/tcdevices

This file allows you to define the incoming and outgoing bandwidth for the devices you want traffic shaping to be enabled. That means, if you want to use traffic shaping for a device, you have to define it here.

Columns in the file are as follows:

INTERFACE - Name of interface. Each interface may be listed only once in this file. You may NOT specify the name of an alias (e.g., eth0:0) here; see FAQ #18. You man NOT specify wildcards here, e.g. if you have multiple ppp interfaces, you need to put them all in here! With Shorewall versions prior to 3.0.8 and 3.2.0 Beta 8, the device named in this column must exist at the time that Shorewall is started, restarted or refreshed. Beginning with Shorewall 3.0.8 and 3.2.0 Beta 8, Shorewall will determine if the device exists and will only configure the device if it does exist. If it doesn't exist, the following warning is issued:
WARNING: Device <device name> not found -- traffic-shaping configuration skipped
Shorewall assigns a sequential interface number to each interface (the first entry in /etc/shorewall/tcdevices is interface 1, the second is interface 2 and so on) Beginning with Shorewall-perl 4.1.6, you can explicitly specify the interface number by prefixing the interface name with the number and a colon (":"). Example: 1:eth0.
IN-BANDWIDTH - The incoming Bandwidth of that interface. Please note that when you use this column, you are not traffic shaping incoming traffic, as the traffic is already received before you could do so. This Column allows you to define the maximum traffic allowed for this interface in total, if the rate is exceeded, the excess packets are dropped. You want this mainly if you have a DSL or Cable Connection to avoid queuing at your providers side. If you don't want any traffic to be dropped set this to a value faster than your interface maximum rate (or to 0 (zero), if you are running Shorewall 3.2.6 or later).
To determine the optimum value for this setting, we recommend that you start by setting it significantly below your measured download bandwidth (20% or so). While downloading, measure the ping response time from the firewall to the upstream router as you gradually increase the setting.The optimal setting is at the point beyond which the ping time increases sharply as you increase the setting.
OUT-BANDWIDTH - Specify the outgoing bandwidth of that interface. This is the maximum speed your connection can handle. It is also the speed you can refer as "full" if you define the tc classes. Outgoing traffic above this rate will be dropped.
OPTIONS (Added in Shorewall-perl 4.1.4) — A comma-separated list of options from the following list:
classify
If specified, classification of traffic into the various classes is done by CLASSIFY entries in /etc/shorewall/tcrules or by entries in /etc/shorewall/tcfilters. No MARK value will be associated with classes on this interface.
REDIRECTED INTERFACES (Added in Shorewall-perl 4.1.6) — Entries are appropriate in this column only if the device in the INTERFACE column names a Intermediate Frame Block (IFB). It lists the physical interfaces that will have their input shaped using classes defined on the IFB. Neither the IFB nor any of the interfaces listed in this column may have an IN-BANDWIDTH specified. You may specify zero (0) or a dash ("-:) in the IN-BANDWIDTH column.
IFB devices automatically get the classify option.

Example 1.

Suppose you are using PPP over Ethernet (DSL) and ppp0 is the interface for this. The device has an outgoing bandwidth of 500kbit and an incoming bandwidth of 6000kbit

#INTERFACE    IN-BANDWITH      OUT-BANDWIDTH
ppp0           6000kbit         500kbit

/etc/shorewall/tcclasses

This file allows you to define the actual classes that are used to split the outgoing traffic.

INTERFACE - Name of interface. Users of Shorewall-perl 4.1.6 or later may also specify the interface number. Must match the name (or number) of an interface with an entry in /etc/shorewall/tcdevices. If the interface has the classify option in /etc/shorewall/tcdevices, then the interface name or number must be followed by a colon and a class number. Examples: eth0:1, 4:9. Class numbers must be unique for a given interface.
MARK - The mark value which is an integer in the range 1-255. You define these marks in the tcrules file, marking the traffic you want to go into the queuing classes defined in here. You can use the same marks for different Interfaces. You must specify "-' in this column if the device specified in the INTERFACE column has the classify option in /etc/shorewall/tcdevices.
RATE - The minimum bandwidth this class should get, when the traffic load rises. Please note that first the classes which equal or a lesser priority value are served even if there are others that have a guaranteed bandwidth but a lower priority. If the sum of the RATEs for all classes assigned to an INTERFACE exceed that interfaces's OUT-BANDWIDTH, then the OUT-BANDWIDTH limit will not be honored.
CEIL - The maximum bandwidth this class is allowed to use when the link is idle. Useful if you have traffic which can get full speed when more important services (e.g. interactive like ssh) are not used. You can use the value "full" in here for setting the maximum bandwidth to the defined output bandwidth of that interface.
PRIORITY - you have to define a priority for the class. packets in a class with a higher priority (=lesser value) are handled before less prioritized ones. You can just define the mark value here also, if you are increasing the mark values with lesser priority.
OPTIONS - A comma-separated list of options including the following:
- default - this is the default class for that interface where all traffic should go, that is not classified otherwise.
  Note
  defining default for exactly one class per interface is mandatory!
- tos-<tosname> - this lets you define a filter for the given <tosname> which lets you define a value of the Type Of Service bits in the ip package which causes the package to go in this class. Please note, that this filter overrides all mark settings, so if you define a tos filter for a class all traffic having that mark will go in it regardless of the mark on the package. You can use the following for this option: tos-minimize-delay (16) tos-maximize-throughput (8) tos-maximize-reliability (4) tos-minimize-cost (2) tos-normal-service (0)
  Note
  Each of this options is only valid for one class per interface.
- tcp-ack - if defined causes an tc filter to be created that puts all tcp ack packets on that interface that have an size of <=64 Bytes to go in this class. This is useful for speeding up downloads. Please note that the size of the ack packets is limited to 64 bytes as some applications (p2p for example) use to make every package an ack package which would cause them all into here. We want only packets WITHOUT payload to match, so the size limit. Bigger packets just take their normal way into the classes.
  Note
  This option is only valid for class per interface.

/etc/shorewall/tcrules

Important

Unlike rules in the shorewall-rules(5) file, evaluation of rules in this file will continue after a match. So the final mark for each packet will be the one assigned by the LAST tcrule that matches.

The fwmark classifier provides a convenient way to classify packets for traffic shaping. The “/etc/shorewall/tcrules” file is used for specifying these marks in a tabular fashion. For an in-depth look at the packet marking facility in Netfilter/Shorewall, please see this article.

Normally, packet marking occurs in the PREROUTING chain before any address rewriting takes place. This makes it impossible to mark inbound packets based on their destination address when SNAT or Masquerading are being used. You can cause packet marking to occur in the FORWARD chain by using the MARK_IN_FORWARD_CHAIN option in shorewall.conf.

Columns in the file are as follows:

MARK or CLASSIFY - MARK specifies the mark value is to be assigned in case of a match. This is an integer in the range 1-255. This value may be optionally followed by “:” and either “F”, “P” or "T" to designate that the marking will occur in the FORWARD, PREROUTING or POSTROUTING chains respectively. If this additional specification is omitted, the chain used to mark packets will be determined as follows:
- If the SOURCE is $FW[:<address>], then the rule is inserted in the OUTPUT chain.
- Otherwise, the chain is determined by the setting of the MARK_IN_FORWARD_CHAIN option in shorewall.conf.
Note
The "T" qualifier was added in Shorewall version 3.3.6 and is not available in earlier versions.
Normally, the mark is applied to the packet. If you follow the mark value with ":" and "C", then the mark is applied to the connection. "C" can be combined with "F", "P" or "T" to designate that the connection should be marked in a particular chain (e.g., "CF", "CP", "CT").
There are additional special values available:
1. RESTORE[/mask] -- restore the packet's mark from the connection's mark using the supplied mask if any. Your kernel and iptables must include CONNMARK support.
  As above, may be followed by :P, :F or :T.
2. SAVE[/mask] -- save the packet's mark to the connection's mark using the supplied mask if any. Your kernel and iptables must include CONNMARK support.
  As above, may be followed by :P, :F or :T.
3. CONTINUE Don't process any more marking rules in the table.
  As above, may be followed by :P, :F or :T.
4. COMMENT (Added in Shorewall version 3.3.3) -- the rest of the line will be attached as a comment to the Netfilter rule(s) generated by the following entries. The comment will appear delimited by "/* ... */" in the output of shorewall show mangle
  To stop the comment from being attached to further rules, simply include COMMENT on a line by itself.
To use CLASSIFY, your kernel and iptables must include CLASSIFY target support. In that case, this column contains a classification (classid) of the form <major>:<minor> where <major> and <minor> are integers. Corresponds to the 'class' specification in these traffic shaping modules:
atm
cbq
dsmark
pfifo_fast
htb
prio
With Shorewall versions prior to 3.2.3, classify rules are always placed in the POSTROUTING chain. Beginning with Shorewall 3.2.3, classification occurs in the POSTROUTING chain except when the SOURCE contains $FW[:<address>] in which case, the classify action takes place in the OUTPUT chain. When used with the builtin traffic shaper, the <major> class is the interface number and the <minor> class is either a) the MARK value of the class preceded by the number "1" (MARK value 1 is <minor> class 11, MARK value 22 is <minor> class 122, and so on) or b) The class number (if the classify option was specified in for the interface /etc/shorewall/interfaces)
SOURCE - Source of the packet. A comma-separated list of interface names, IP addresses, MAC addresses and/or subnets for packets being routed through a common path. List elements may also consist of an interface name followed by ":" and an address (e.g., eth1:192.168.1.0/24). For example, all packets for connections masqueraded to eth0 from other interfaces can be matched in a single rule with several alternative SOURCE criteria. However, a connection whose packets gets to eth0 in a different way, e.g., direct from the firewall itself, needs a different rule.
Accordingly, use $FW in its own separate rule for packets originating on the firewall. In such a rule, the MARK column may NOT specify either ":P" or ":F" because marking for firewall-originated packets always occurs in the OUTPUT chain.
MAC addresses must be prefixed with "~" and use "-" as a separator.
Example: ~00-A0-C9-15-39-78
DEST - Destination of the packet. Comma separated list of IP addresses and/or subnets. If your kernel and iptables include iprange match support, IP address ranges are also allowed. List elements may also consist of an interface name followed by ":" and an address (e.g., eth1:192.168.1.0/24). If the MARK column specifies a classification of the form <major>:<minor> then this column may also contain an interface name.
PROTO - Protocol - Must be "tcp", "udp", "icmp", "ipp2p", "ipp2p:udp", "ipp2p:all" a number, or "all". "ipp2p" requires ipp2p match support in your kernel and iptables.
PORT(S) - Destination Ports. A comma-separated list of Port names (from /etc/services), port numbers or port ranges; if the protocol is "icmp", this column is interpreted as the destination icmp-type(s).
If the protocol is ipp2p, this column is interpreted as an ipp2p option without the leading "--" (example "bit" for bit-torrent). If no PORT is given, "ipp2p" is assumed.
This column is ignored if PROTOCOL = all but must be entered if any of the following field is supplied. In that case, it is suggested that this field contain "-"
CLIENT PORT(S) - (Optional) Port(s) used by the client. If omitted, any source port is acceptable. Specified as a comma-separate list of port names, port numbers or port ranges.
USER/GROUP (Added in Shorewall version 1.4.10) - (Optional) This column may only be non-empty if the SOURCE is the firewall itself. When this column is non-empty, the rule applies only if the program generating the output is running under the effective user and/or group. It may contain :
[!][<user name or number>]:[<group name or number>][+<program name>]
The colon is optional when specifying only a user.
Examples:
```
joe     #program must be run by joe
:kids   #program must be run by a member of the 'kids' group
!:kids  #program must not be run by a member of the 'kids' group
+upnpd  #program named upnpd (This feature was removed from Netfilter in kernel version 2.6.14).
```

TEST - Defines a test on the existing packet or connection mark. The rule will match only if the test returns true. Tests have the format [!]<value>[/<mask>][:C]

Where:

! Inverts the test (not equal)

<value> aterface.

REDIRECTED INTERFACES (Added in Shorewall-perl 4.1.6) — Entries are appropriate in this column only if the device in the INTERFACE column names a Intermediate Frame Block (IFB). It lists the physical interfaces that will have their input shaped using classes defined on the IFB. Neither the IFB nor any of the interfaces listed in this column may have an IN-BANDWIDTH specified. You may specify zero (0) or a dash ("-:) in the IN-BANDWIDTH column.

IFB devices automatically get the classify option.

Example 1.

Suppose you are using PPP over Ethernet (DSL) and ppp0 is the interface for this. The device has an outgoing bandwidth of 500kbit and an incoming bandwidth of 6000kbit

#INTERFACE    IN-BANDWITH      OUT-BANDWIDTH
ppp0           6000kbit         500kbit

/etc/shorewall/tcclasses

This file allows you to define the actual classes that are used to split the outgoing traffic.

INTERFACE - Name of interface. Users of Shorewall-perl 4.1.6 or later may also specify the interface number. Must match the name (or number) of an interface with an entry in /etc/shorewall/tcdevices. If the interface has the classify option in /etc/shorewall/tcdevices, then the interface name or number must be followed by a colon and a class number. Examples: eth0:1, 4:9. Class numbers must be unique for a given interface.
MARK - The mark value which is an integer in the range 1-255. You define these marks in the tcrules file, marking the traffic you want to go into the queuing classes defined in here. You can use the same marks for different Interfaces. You must specify "-' in this column if the device specified in the INTERFACE column has the classify option in /etc/shorewall/tcdevices.
RATE - The minimum bandwidth this class should get, when the traffic load rises. Please note that first the classes which equal or a lesser priority value are served even if there are others that have a guaranteed bandwidth but a lower priority. If the sum of the RATEs for all classes assigned to an INTERFACE exceed that interfaces's OUT-BANDWIDTH, then the OUT-BANDWIDTH limit will not be honored.
CEIL - The maximum bandwidth this class is allowed to use when the link is idle. Useful if you have traffic which can get full speed when more important services (e.g. interactive like ssh) are not used. You can use the value "full" in here for setting the maximum bandwidth to the defined output bandwidth of that interface.
PRIORITY - you have to define a priority for the class. packets in a class with a higher priority (=lesser value) are handled before less prioritized ones. You can just define the mark value here also, if you are increasing the mark values with lesser priority.
OPTIONS - A comma-separated list of options including the following:
- default - this is the default class for that interface where all traffic should go, that is not classified otherwise.
  Note
  defining default for exactly one class per interface is mandatory!
- tos-<tosname> - this lets you define a filter for the given <tosname> which lets you define a value of the Type Of Service bits in the ip package which causes the package to go in this class. Please note, that this filter overrides all mark settings, so if you define a tos filter for a class all traffic having that mark will go in it regardless of the mark on the package. You can use the following for this option: tos-minimize-delay (16) tos-maximize-throughput (8) tos-maximize-reliability (4) tos-minimize-cost (2) tos-normal-service (0)
  Note
  Each of this options is only valid for one class per interface.
- tcp-ack - if defined causes an tc filter to be created that puts all tcp ack packets on that interface that have an size of <=64 Bytes to go in this class. This is useful for speeding up downloads. Please note that the size of the ack packets is limited to 64 bytes as some applications (p2p for example) use to make every package an ack package which would cause them all into here. We want only packets WITHOUT payload to match, so the size limit. Bigger packets just take their normal way into the classes.
  Note
  This option is only valid for class per interface.

/etc/shorewall/tcrules

Important

Unlike rules in the shorewall-rules(5) file, evaluation of rules in this file will continue after a match. So the final mark for each packet will be the one assigned by the LAST tcrule that matches.

The fwmark classifier provides a convenient way to classify packets for traffic shaping. The “/etc/shorewall/tcrules” file is used for specifying these marks in a tabular fashion. For an in-depth look at the packet marking facility in Netfilter/Shorewall, please see this article.

Normally, packet marking occurs in the PREROUTING chain before any address rewriting takes place. This makes it impossible to mark inbound packets based on their destination address when SNAT or Masquerading are being used. You can cause packet marking to occur in the FORWARD chain by using the MARK_IN_FORWARD_CHAIN option in shorewall.conf.

Columns in the file are as follows:

MARK or CLASSIFY - MARK specifies the mark value is to be assigned in case of a match. This is an integer in the range 1-255. This value may be optionally followed by “:” and either “F”, “P” or "T" to designate that the marking will occur in the FORWARD, PREROUTING or POSTROUTING chains respectively. If this additional specification is omitted, the chain used to mark packets will be determined as follows:
- If the SOURCE is $FW[:<address>], then the rule is inserted in the OUTPUT chain.
- Otherwise, the chain is determined by the setting of the MARK_IN_FORWARD_CHAIN option in shorewall.conf.
Note
The "T" qualifier was added in Shorewall version 3.3.6 and is not available in earlier versions.
Normally, the mark is applied to the packet. If you follow the mark value with ":" and "C", then the mark is applied to the connection. "C" can be combined with "F", "P" or "T" to designate that the connection should be marked in a particular chain (e.g., "CF", "CP", "CT").
There are additional special values available:
1. RESTORE[/mask] -- restore the packet's mark from the connection's mark using the supplied mask if any. Your kernel and iptables must include CONNMARK support.
  As above, may be followed by :P, :F or :T.
2. SAVE[/mask] -- save the packet's mark to the connection's mark using the supplied mask if any. Your kernel and iptables must include CONNMARK support.
  As above, may be followed by :P, :F or :T.
3. CONTINUE Don't process any more marking rules in the table.
  As above, may be followed by :P, :F or :T.
4. COMMENT (Added in Shorewall version 3.3.3) -- the rest of the line will be attached as a comment to the Netfilter rule(s) generated by the following entries. The comment will appear delimited by "/* ... */" in the output of shorewall show mangle
  To stop the comment from being attached to further rules, simply include COMMENT on a line by itself.
To use CLASSIFY, your kernel and iptables must include CLASSIFY target support. In that case, this column contains a classification (classid) of the form <major>:<minor> where <major> and <minor> are integers. Corresponds to the 'class' specification in these traffic shaping modules:
atm
cbq
dsmark
pfifo_fast
htb
prio
With Shorewall versions prior to 3.2.3, classify rules are always placed in the POSTROUTING chain. Beginning with Shorewall 3.2.3, classification occurs in the POSTROUTING chain except when the SOURCE contains $FW[:<address>] in which case, the classify action takes place in the OUTPUT chain. When used with the builtin traffic shaper, the <major> class is the interface number and the <minor> class is either a) the MARK value of the class preceded by the number "1" (MARK value 1 is <minor> class 11, MARK value 22 is <minor> class 122, and so on) or b) The class number (if the classify option was specified in for the interface /etc/shorewall/interfaces)
SOURCE - Source of the packet. A comma-separated list of interface names, IP addresses, MAC addresses and/or subnets for packets being routed through a common path. List elements may also consist of an interface name followed by ":" and an address (e.g., eth1:192.168.1.0/24). For example, all packets for connections masqueraded to eth0 from other interfaces can be matched in a single rule with several alternative SOURCE criteria. However, a connection whose packets gets to eth0 in a different way, e.g., direct from the firewall itself, needs a different rule.
Accordingly, use $FW in its own separate rule for packets originating on the firewall. In such a rule, the MARK column may NOT specify either ":P" or ":F" because marking for firewall-originated packets always occurs in the OUTPUT chain.
MAC addresses must be prefixed with "~" and use "-" as a separator.
Example: ~00-A0-C9-15-39-78
DEST - Destination of the packet. Comma separated list of IP addresses and/or subnets. If your kernel and iptables include iprange match support, IP address ranges are also allowed. List elements may also consist of an interface name followed by ":" and an address (e.g., eth1:192.168.1.0/24). If the MARK column specifies a classification of the form <major>:<minor> then this column may also contain an interface name.
PROTO - Protocol - Must be "tcp", "udp", "icmp", "ipp2p", "ipp2p:udp", "ipp2p:all" a number, or "all". "ipp2p" requires ipp2p match support in your kernel and iptables.
PORT(S) - Destination Ports. A comma-separated list of Port names (from /etc/services), port numbers or port ranges; if the protocol is "icmp", this column is interpreted as the destination icmp-type(s).
If the protocol is ipp2p, this column is interpreted as an ipp2p option without the leading "--" (example "bit" for bit-torrent). If no PORT is given, "ipp2p" is assumed.
This column is ignored if PROTOCOL = all but must be entered if any of the following field is supplied. In that case, it is suggested that this field contain "-"
CLIENT PORT(S) - (Optional) Port(s) used by the client. If omitted, any source port is acceptable. Specified as a comma-separate list of port names, port numbers or port ranges.
USER/GROUP (Added in Shorewall version 1.4.10) - (Optional) This column may only be non-empty if the SOURCE is the firewall itself. When this column is non-empty, the rule applies only if the program generating the output is running under the effective user and/or group. It may contain :
[!][<user name or number>]:[<group name or number>][+<program name>]
The colon is optional when specifying only a user.
Examples:
```
joe     #program must be run by joe
:kids   #program must be run by a member of the 'kids' group
!:kids  #program must not be run by a member of the 'kids' group
+upnpd  #program named upnpd (This feature was removed from Netfilter in kernel version 2.6.14).
```

TEST - Defines a test on the existing packet or connection mark. The rule will match only if the test returns true. Tests have the format [!]<value>[/<mask>][:C]

Where:

! Inverts the test (not equal)

<value> aterface.

REDIRECTED INTERFACES (Added in Shorewall-perl 4.1.6) — Entries are appropriate in this column only if the device in the INTERFACE column names a Intermediate Frame Block (IFB). It lists the physical interfaces that will have their input shaped using classes defined on the IFB. Neither the IFB nor any of the interfaces listed in this column may have an IN-BANDWIDTH specified. You may specify zero (0) or a dash ("-:) in the IN-BANDWIDTH column.

IFB devices automatically get the classify option.

Example 1.

Suppose you are using PPP over Ethernet (DSL) and ppp0 is the interface for this. The device has an outgoing bandwidth of 500kbit and an incoming bandwidth of 6000kbit

#INTERFACE    IN-BANDWITH      OUT-BANDWIDTH
ppp0           6000kbit         500kbit

/etc/shorewall/tcclasses

This file allows you to define the actual classes that are used to split the outgoing traffic.

INTERFACE - Name of interface. Users of Shorewall-perl 4.1.6 or later may also specify the interface number. Must match the name (or number) of an interface with an entry in /etc/shorewall/tcdevices. If the interface has the classify option in /etc/shorewall/tcdevices, then the interface name or number must be followed by a colon and a class number. Examples: eth0:1, 4:9. Class numbers must be unique for a given interface.
MARK - The mark value which is an integer in the range 1-255. You define these marks in the tcrules file, marking the traffic you want to go into the queuing classes defined in here. You can use the same marks for different Interfaces. You must specify "-' in this column if the device specified in the INTERFACE column has the classify option in /etc/shorewall/tcdevices.
RATE - The minimum bandwidth this class should get, when the traffic load rises. Please note that first the classes which equal or a lesser priority value are served even if there are others that have a guaranteed bandwidth but a lower priority. If the sum of the RATEs for all classes assigned to an INTERFACE exceed that interfaces's OUT-BANDWIDTH, then the OUT-BANDWIDTH limit will not be honored.
CEIL - The maximum bandwidth this class is allowed to use when the link is idle. Useful if you have traffic which can get full speed when more important services (e.g. interactive like ssh) are not used. You can use the value "full" in here for setting the maximum bandwidth to the defined output bandwidth of that interface.
PRIORITY - you have to define a priority for the class. packets in a class with a higher priority (=lesser value) are handled before less prioritized ones. You can just define the mark value here also, if you are increasing the mark values with lesser priority.
OPTIONS - A comma-separated list of options including the following:
- default - this is the default class for that interface where all traffic should go, that is not classified otherwise.
  Note
  defining default for exactly one class per interface is mandatory!
- tos-<tosname> - this lets you define a filter for the given <tosname> which lets you define a value of the Type Of Service bits in the ip package which causes the package to go in this class. Please note, that this filter overrides all mark settings, so if you define a tos filter for a class all traffic having that mark will go in it regardless of the mark on the package. You can use the following for this option: tos-minimize-delay (16) tos-maximize-throughput (8) tos-maximize-reliability (4) tos-minimize-cost (2) tos-normal-service (0)
  Note
  Each of this options is only valid for one class per interface.
- tcp-ack - if defined causes an tc filter to be created that puts all tcp ack packets on that interface that have an size of <=64 Bytes to go in this class. This is useful for speeding up downloads. Please note that the size of the ack packets is limited to 64 bytes as some applications (p2p for example) use to make every package an ack package which would cause them all into here. We want only packets WITHOUT payload to match, so the size limit. Bigger packets just take their normal way into the classes.
  Note
  This option is only valid for class per interface.

/etc/shorewall/tcrules

Important

Unlike rules in the shorewall-rules(5) file, evaluation of rules in this file will continue after a match. So the final mark for each packet will be the one assigned by the LAST tcrule that matches.

The fwmark classifier provides a convenient way to classify packets for traffic shaping. The “/etc/shorewall/tcrules” file is used for specifying these marks in a tabular fashion. For an in-depth look at the packet marking facility in Netfilter/Shorewall, please see this article.

Normally, packet marking occurs in the PREROUTING chain before any address rewriting takes place. This makes it impossible to mark inbound packets based on their destination address when SNAT or Masquerading are being used. You can cause packet marking to occur in the FORWARD chain by using the MARK_IN_FORWARD_CHAIN option in shorewall.conf.

Columns in the file are as follows:

MARK or CLASSIFY - MARK specifies the mark value is to be assigned in case of a match. This is an integer in the range 1-255. This value may be optionally followed by “:” and either “F”, “P” or "T" to designate that the marking will occur in the FORWARD, PREROUTING or POSTROUTING chains respectively. If this additional specification is omitted, the chain used to mark packets will be determined as follows:
- If the SOURCE is $FW[:<address>], then the rule is inserted in the OUTPUT chain.
- Otherwise, the chain is determined by the setting of the MARK_IN_FORWARD_CHAIN option in shorewall.conf.
Note
The "T" qualifier was added in Shorewall version 3.3.6 and is not available in earlier versions.
Normally, the mark is applied to the packet. If you follow the mark value with ":" and "C", then the mark is applied to the connection. "C" can be combined with "F", "P" or "T" to designate that the connection should be marked in a particular chain (e.g., "CF", "CP", "CT").
There are additional special values available:
1. RESTORE[/mask] -- restore the packet's mark from the connection's mark using the supplied mask if any. Your kernel and iptables must include CONNMARK support.
  As above, may be followed by :P, :F or :T.
2. SAVE[/mask] -- save the packet's mark to the connection's mark using the supplied mask if any. Your kernel and iptables must include CONNMARK support.
  As above, may be followed by :P, :F or :T.
3. CONTINUE Don't process any more marking rules in the table.
  As above, may be followed by :P, :F or :T.
4. COMMENT (Added in Shorewall version 3.3.3) -- the rest of the line will be attached as a comment to the Netfilter rule(s) generated by the following entries. The comment will appear delimited by "/* ... */" in the output of shorewall show mangle
  To stop the comment from being attached to further rules, simply include COMMENT on a line by itself.
To use CLASSIFY, your kernel and iptables must include CLASSIFY target support. In that case, this column contains a classification (classid) of the form <major>:<minor> where <major> and <minor> are integers. Corresponds to the 'class' specification in these traffic shaping modules:
atm
cbq
dsmark
pfifo_fast
htb
prio
With Shorewall versions prior to 3.2.3, classify rules are always placed in the POSTROUTING chain. Beginning with Shorewall 3.2.3, classification occurs in the POSTROUTING chain except when the SOURCE contains $FW[:<address>] in which case, the classify action takes place in the OUTPUT chain. When used with the builtin traffic shaper, the <major> class is the interface number and the <minor> class is either a) the MARK value of the class preceded by the number "1" (MARK value 1 is <minor> class 11, MARK value 22 is <minor> class 122, and so on) or b) The class number (if the classify option was specified in for the interface /etc/shorewall/interfaces)
SOURCE - Source of the packet. A comma-separated list of interface names, IP addresses, MAC addresses and/or subnets for packets being routed through a common path. List elements may also consist of an interface name followed by ":" and an address (e.g., eth1:192.168.1.0/24). For example, all packets for connections masqueraded to eth0 from other interfaces can be matched in a single rule with several alternative SOURCE criteria. However, a connection whose packets gets to eth0 in a different way, e.g., direct from the firewall itself, needs a different rule.
Accordingly, use $FW in its own separate rule for packets originating on the firewall. In such a rule, the MARK column may NOT specify either ":P" or ":F" because marking for firewall-originated packets always occurs in the OUTPUT chain.
MAC addresses must be prefixed with "~" and use "-" as a separator.
Example: ~00-A0-C9-15-39-78
DEST - Destination of the packet. Comma separated list of IP addresses and/or subnets. If your kernel and iptables include iprange match support, IP address ranges are also allowed. List elements may also consist of an interface name followed by ":" and an address (e.g., eth1:192.168.1.0/24). If the MARK column specifies a classification of the form <major>:<minor> then this column may also contain an interface name.
PROTO - Protocol - Must be "tcp", "udp", "icmp", "ipp2p", "ipp2p:udp", "ipp2p:all" a number, or "all". "ipp2p" requires ipp2p match support in your kernel and iptables.
PORT(S) - Destination Ports. A comma-separated list of Port names (from /etc/services), port numbers or port ranges; if the protocol is "icmp", this column is interpreted as the destination icmp-type(s).
If the protocol is ipp2p, this column is interpreted as an ipp2p option without the leading "--" (example "bit" for bit-torrent). If no PORT is given, "ipp2p" is assumed.
This column is ignored if PROTOCOL = all but must be entered if any of the following field is supplied. In that case, it is suggested that this field contain "-"
CLIENT PORT(S) - (Optional) Port(s) used by the client. If omitted, any source port is acceptable. Specified as a comma-separate list of port names, port numbers or port ranges.
USER/GROUP (Added in Shorewall version 1.4.10) - (Optional) This column may only be non-empty if the SOURCE is the firewall itself. When this column is non-empty, the rule applies only if the program generating the output is running under the effective user and/or group. It may contain :
[!][<user name or number>]:[<group name or number>][+<program name>]
The colon is optional when specifying only a user.
Examples:
```
joe     #program must be run by joe
:kids   #program must be run by a member of the 'kids' group
!:kids  #program must not be run by a member of the 'kids' group
+upnpd  #program named upnpd (This feature was removed from Netfilter in kernel version 2.6.14).
```

TEST - Defines a test on the existing packet or connection mark. The rule will match only if the test returns true. Tests have the format [!]<value>[/<mask>][:C]

Where:

! Inverts the test (not equal)

<value> aterface.

REDIRECTED INTERFACES (Added in Shorewall-perl 4.1.6) — Entries are appropriate in this column only if the device in the INTERFACE column names a Intermediate Frame Block (IFB). It lists the physical interfaces that will have their input shaped using classes defined on the IFB. Neither the IFB nor any of the interfaces listed in this column may have an IN-BANDWIDTH specified. You may specify zero (0) or a dash ("-:) in the IN-BANDWIDTH column.

IFB devices automatically get the classify option.

Example 1.

Suppose you are using PPP over Ethernet (DSL) and ppp0 is the interface for this. The device has an outgoing bandwidth of 500kbit and an incoming bandwidth of 6000kbit

#INTERFACE    IN-BANDWITH      OUT-BANDWIDTH
ppp0           6000kbit         500kbit

/etc/shorewall/tcclasses

This file allows you to define the actual classes that are used to split the outgoing traffic.

INTERFACE - Name of interface. Users of Shorewall-perl 4.1.6 or later may also specify the interface number. Must match the name (or number) of an interface with an entry in /etc/shorewall/tcdevices. If the interface has the classify option in /etc/shorewall/tcdevices, then the interface name or number must be followed by a colon and a class number. Examples: eth0:1, 4:9. Class numbers must be unique for a given interface.
MARK - The mark value which is an integer in the range 1-255. You define these marks in the tcrules file, marking the traffic you want to go into the queuing classes defined in here. You can use the same marks for different Interfaces. You must specify "-' in this column if the device specified in the INTERFACE column has the classify option in /etc/shorewall/tcdevices.
RATE - The minimum bandwidth this class should get, when the traffic load rises. Please note that first the classes which equal or a lesser priority value are served even if there are others that have a guaranteed bandwidth but a lower priority. If the sum of the RATEs for all classes assigned to an INTERFACE exceed that interfaces's OUT-BANDWIDTH, then the OUT-BANDWIDTH limit will not be honored.
CEIL - The maximum bandwidth this class is allowed to use when the link is idle. Useful if you have traffic which can get full speed when more important services (e.g. interactive like ssh) are not used. You can use the value "full" in here for setting the maximum bandwidth to the defined output bandwidth of that interface.
PRIORITY - you have to define a priority for the class. packets in a class with a higher priority (=lesser value) are handled before less prioritized ones. You can just define the mark value here also, if you are increasing the mark values with lesser priority.
OPTIONS - A comma-separated list of options including the following:
- default - this is the default class for that interface where all traffic should go, that is not classified otherwise.
  Note
  defining default for exactly one class per interface is mandatory!
- tos-<tosname> - this lets you define a filter for the given <tosname> which lets you define a value of the Type Of Service bits in the ip package which causes the package to go in this class. Please note, that this filter overrides all mark settings, so if you define a tos filter for a class all traffic having that mark will go in it regardless of the mark on the package. You can use the following for this option: tos-minimize-delay (16) tos-maximize-throughput (8) tos-maximize-reliability (4) tos-minimize-cost (2) tos-normal-service (0)
  Note
  Each of this options is only valid for one class per interface.
- tcp-ack - if defined causes an tc filter to be created that puts all tcp ack packets on that interface that have an size of <=64 Bytes to go in this class. This is useful for speeding up downloads. Please note that the size of the ack packets is limited to 64 bytes as some applications (p2p for example) use to make every package an ack package which would cause them all into here. We want only packets WITHOUT payload to match, so the size limit. Bigger packets just take their normal way into the classes.
  Note
  This option is only valid for class per interface.

/etc/shorewall/tcrules

Important

Unlike rules in the shorewall-rules(5) file, evaluation of rules in this file will continue after a match. So the final mark for each packet will be the one assigned by the LAST tcrule that matches.

The fwmark classifier provides a convenient way to classify packets for traffic shaping. The “/etc/shorewall/tcrules” file is used for specifying these marks in a tabular fashion. For an in-depth look at the packet marking facility in Netfilter/Shorewall, please see this article.

Normally, packet marking occurs in the PREROUTING chain before any address rewriting takes place. This makes it impossible to mark inbound packets based on their destination address when SNAT or Masquerading are being used. You can cause packet marking to occur in the FORWARD chain by using the MARK_IN_FORWARD_CHAIN option in shorewall.conf.

Columns in the file are as follows:

MARK or CLASSIFY - MARK specifies the mark value is to be assigned in case of a match. This is an integer in the range 1-255. This value may be optionally followed by “:” and either “F”, “P” or "T" to designate that the marking will occur in the FORWARD, PREROUTING or POSTROUTING chains respectively. If this additional specification is omitted, the chain used to mark packets will be determined as follows:
- If the SOURCE is $FW[:<address>], then the rule is inserted in the OUTPUT chain.
- Otherwise, the chain is determined by the setting of the MARK_IN_FORWARD_CHAIN option in shorewall.conf.
Note
The "T" qualifier was added in Shorewall version 3.3.6 and is not available in earlier versions.
Normally, the mark is applied to the packet. If you follow the mark value with ":" and "C", then the mark is applied to the connection. "C" can be combined with "F", "P" or "T"