Load Balancing Persistent

Quick Start for Impatient

Configuration export from the gateway router:

'''/ ip address'''
add address=192.168.0.1/24 network=192.168.0.0 broadcast=192.168.0.255 interface=Local
add address=10.111.0.2/24 network=10.111.0.0 broadcast=10.111.0.255 interface=wlan2
add address=10.112.0.2/24 network=10.112.0.0 broadcast=10.112.0.255 interface=wlan1

'''/ ip firewall mangle'''
add chain=prerouting src-address-list=odd in-interface=Local action=mark-connection \
new-connection-mark=odd passthrough=yes
add chain=prerouting src-address-list=odd in-interface=Local action=mark-routing \
new-routing-mark=odd passthrough=no
add chain=prerouting src-address-list=even in-interface=Local action=mark-connection \
new-connection-mark=even passthrough=yes
add chain=prerouting src-address-list=even in-interface=Local action=mark-routing \
new-routing-mark=even passthrough=no
add chain=prerouting in-interface=Local connection-state=new nth=1,1,0 \
action=mark-connection new-connection-mark=odd passthrough=yes
add chain=prerouting in-interface=Local action=add-src-to-address-list \
address-list=odd address-list-timeout=1d connection-mark=odd passthrough=yes
add chain=prerouting in-interface=Local connection-mark=odd action=mark-routing \
new-routing-mark=odd passthrough=no
add chain=prerouting in-interface=Local connection-state=new nth=1,1,1 \
action=mark-connection new-connection-mark=even passthrough=yes
add chain=prerouting in-interface=Local action=add-src-to-address-list \
address-list=even address-list-timeout=1d connection-mark=even passthrough=yes
add chain=prerouting in-interface=Local connection-mark=even action=mark-routing \
new-routing-mark=even passthrough=no

'''/ ip firewall nat'''
add chain=srcnat connection-mark=odd action=src-nat to-addresses=10.111.0.2 \
to-ports=0-65535
add chain=srcnat connection-mark=even action=src-nat to-addresses=10.112.0.2 \
to-ports=0-65535

'''/ ip route'''
add dst-address=0.0.0.0/0 gateway=10.111.0.1 scope=255 target-scope=10 routing-mark=odd
add dst-address=0.0.0.0/0 gateway=10.112.0.1 scope=255 target-scope=10 routing-mark=even
add dst-address=0.0.0.0/0 gateway=10.112.0.1 scope=255 target-scope=10

[edit] Explanation

First we give a code snippet and then explain what it actually does.
[edit] IP Addresses

/ ip address
add address=192.168.0.1/24 network=192.168.0.0 broadcast=192.168.0.255 interface=Local
add address=10.111.0.2/24 network=10.111.0.0 broadcast=10.111.0.255 interface=wlan2
add address=10.112.0.2/24 network=10.112.0.0 broadcast=10.112.0.255 interface=wlan1

The router has two upstream (WAN) interfaces with the addresses of 10.111.0.2/24 and 10.112.0.2/24. The LAN interface has the name "Local" and IP address of 192.168.0.1/24.
[edit] Mangle

/ ip firewall mangle
add chain=prerouting src-address-list=odd in-interface=Local action=mark-connection \
new-connection-mark=odd passthrough=yes
add chain=prerouting src-address-list=odd in-interface=Local action=mark-routing \
new-routing-mark=odd

All traffic from customers having their IP address previously placed in the address list "odd" is instantly marked with connection and routing marks "odd". Afterwards the traffic is excluded from processing against successive mangle rules in prerouting chain.

/ ip firewall mangle
add chain=prerouting src-address-list=even in-interface=Local action=mark-connection \
new-connection-mark=even passthrough=yes
add chain=prerouting src-address-list=even in-interface=Local action=mark-routing \
new-routing-mark=even

Same stuff as above, only for customers having their IP address previously placed in the address list "even".

/ ip firewall mangle
add chain=prerouting in-interface=Local connection-state=new nth=1,1,0 \
action=mark-connection new-connection-mark=odd passthrough=yes
add chain=prerouting in-interface=Local action=add-src-to-address-list \
address-list=odd address-list-timeout=1d connection-mark=odd passthrough=yes
add chain=prerouting in-interface=Local connection-mark=odd action=mark-routing \
new-routing-mark=odd passthrough=no

First we take every second packet that establishes new session (note connection-state=new), and mark it with connection mark "odd". Consequently all successive packets belonging to the same session will carry the connection mark "odd". Note that we are passing these packets to the second and third rules (passthrough=yes). Second rule adds IP address of the client to the address list to enable all successive sessions to go through the same gateway. Third rule places the routing mark "odd" on all packets that belong to the "odd" connection and stops processing all other mangle rules for these packets in prerouting chain.

/ ip firewall mangle
add chain=prerouting in-interface=Local connection-state=new nth=1,1,1 \
action=mark-connection new-connection-mark=even passthrough=yes
add chain=prerouting in-interface=Local action=add-src-to-address-list \
address-list=even address-list-timeout=1d connection-mark=even passthrough=yes
add chain=prerouting in-interface=Local connection-mark=even action=mark-routing \
new-routing-mark=even passthrough=no

These rules do the same for the remaining half of the traffic as the first three rules for the first half of the traffic.

The code above effectively means that each new connection initiated through the router from the local network will be marked as either "odd" or "even" with both routing and connection marks.

The above works fine. There are however some situations where you might find that the same IP address is listed under both the ODD and EVEN scr-address-lists. This behavior causes issues with apps that require persistent connections. A simple remedy for this situation is to add the following statement to your mangle rules:

add chain=prerouting in-interface=Local connection-state=new nth=1,1,1 \
src-address-list=!odd action=mark-connection new-connection-mark=even \
passthrough=yes

This will ensure that the new connection will not already be part of the ODD src-address-list. You will have to do the same for the ODD mangle rule thus excluding IP's already part of the EVEN scr-address-list.
[edit] NAT

/ ip firewall nat
add chain=srcnat connection-mark=odd action=src-nat to-addresses=10.111.0.2 \
to-ports=0-65535
add chain=srcnat connection-mark=even action=src-nat to-addresses=10.112.0.2 \
to-ports=0-65535

All traffic marked "odd" is being NATted to source IP address of 10.111.0.2, while traffic marked "even" gets "10.112.0.2" source IP address.
[edit] Routing

/ ip route
add dst-address=0.0.0.0/0 gateway=10.111.0.1 scope=255 target-scope=10 routing-mark=odd
add dst-address=0.0.0.0/0 gateway=10.112.0.1 scope=255 target-scope=10 routing-mark=even

For all traffic marked "odd" (consequently having 10.111.0.2 translated source address) we use 10.111.0.1 gateway. In the same manner all traffic marked "even" is routed through the 10.112.0.1 gateway.

/ ip route
add dst-address=0.0.0.0/0 gateway=10.112.0.1 scope=255 target-scope=10

Finally, we have one additional entry specifying that traffic from the router itself (the traffic without any routing marks) should go to 10.112.0.1 gateway.

source:http://wiki.mikrotik.com/wiki/Load_Balancing_Persistent

Two gateways failover with load balancing

Route

According to the examples above, you have:

/ ip route
add dst-address=0.0.0.0/0 gateway=10.111.0.1 scope=255 target-scope=10 routing-mark=odd
add dst-address=0.0.0.0/0 gateway=10.112.0.1 scope=255 target-scope=10 routing-mark=even
add dst-address=0.0.0.0/0 gateway=10.112.0.1 scope=255 target-scope=10

Now you have to change these lines to:

/ ip route
add dst-address=0.0.0.0/0 gateway=10.111.0.1 scope=255 target-scope=10 routing-mark=odd check-gateway=ping
add dst-address=0.0.0.0/0 gateway=10.112.0.1 scope=255 target-scope=10 routing-mark=even check-gateway=ping
add dst-address=0.0.0.0/0 gateway=10.112.0.1 scope=255 target-scope=10
add dst-address=0.0.0.0/0 gateway=10.111.0.1 scope=255 target-scope=10 distance=2

If ping fails to 10.111.0.1, then all traffic marked odd go's to the gateway 10.112.0.1, the oposite is also true.

All local traffic go's to the 10.112.0.1 as it's distance is smaller, if 10.112.0.1 fails, then 10.111.0.1 takes over.

The router pings gateway every 10 seconds and if to consecutive pings to the gateway fail, the route is considered dead. So, then testing keep in mind, that gateway failure is detected in 20 to 30 seconds.
[edit] NAT

/ ip firewall nat
add chain=srcnat connection-mark=odd action=src-nat to-addresses=10.111.0.2 \
to-ports=0-65535 comment="" disabled=no
add chain=srcnat connection-mark=even action=src-nat to-addresses=10.112.0.2 \
to-ports=0-65535 comment="" disabled=no

change to:

/ip firewall nat
add chain=srcnat src-address=192.168.0.0/24 action=masquerade

source:http://wiki.mikrotik.com/wiki/Two_gateways_failover_with_load_balancing

Routing Questions

Question: How does /ip route check-gateway work?

check-gateway sends pings every 10 seconds. if two successive pings fail, the gateway is considered dead.
Question: I have one /24 network advertised to two BGP peers using "/routing bgp networks" facility. How do I advertise a higher path cost to one of the peers?

You have to change the way you are redistributing your network, as filters are not applied to routes advertised from "/routing bgp networks". In most cases the network is connected directly to your router, so it's enough to set BGP instance to redistribute directly connected routes:

/routing bgp instance set default redistribute-connected=yes

To filter out all other connected networks except the needed one, create a routing filter for the BGP instance,

/routing filter add invert-match=yes prefix=10.0.0.0/24 action=discard name=InstanceOutFilter

then set filter "InstanceOutFilter" as the out-filter for "default" BGP instance.

/routing bgp instance set default out-filter=InstanceOutFilter

To communicate a lower preference value (higher path cost) to one of the peers, you have to prepend your AS number multiple times to the BGP AS_PATH attribute

/routing filter add prefix=10.0.0.0/24 set-bgp-prepend=4 name=Peer1OutFilter
/routing bgp peer set Peer1 out-filter=Peer1OutFilter

Question: I have a /22 (say 10.0.0.0/22) assigned IP space, split internally down into /30's, /28's, etc. Is it possible just to announce the /22 space via BGP with routing-test package?

Yes, it is possible. Do the following:

1. add an empty bridge interface:

/interface bridge add name=loopback

2. assign a /22 address to the bridge interface:

/ip address add address=10.0.0.1/22 interface=loopback

3. create a routing filter that filters out all prefixes except the /22 one

/routing filter add invert-match=yes prefix=10.0.0.0/22 prefix-length=22 action=discard name=myfilter

4. set filter "myfilter" as the out-filter for "default" BGP instance

/routing bgp instance set default out-filter=myfilter

Question: How to blackhole a network?

There are two ways to blackhole a network. First, you can do this manually by adding a blackhole route to the routing table, for example, to blackhole a 10.0.0.0/8 network, issue the following command:

/ip route add dst-address=10.0.0.0/8 kernel-type=blackhole

Routing filters are the other mean to blackhole a network. To create a routing filter that automatically blackholes all prefixes in 10.0.0.0/8 in the BGP feed, issue the following command:

/routing filter add prefix=10.0.0.0/8 prefix-length=8-32 set-kernel-type=blackhole chain=myfilter

Question: How to filter out the default route from outgoing BGP advertisements?

Assuming you have a static default route that is redistributed because redistribute-static parameter is set to yes, do the following:

/routing filter add chain=myfilter prefix=0.0.0.0/0 action=discard

Then set myfilter as the out-filter for BGP instance

/routing bgp instance set default out-filter=myfilter

Load Balancing over Multiple Gateways

The typical situation where you got one router and want to connect to two ISPs:

Of course, you want to do load balancing! There are several ways how to do it. Depending on the particular situation, you may find one best suited for you.

Policy Routing based on Client IP Address

If you have a number of hosts, you may group them by IP addresses. Then, depending on the source IP address, send the traffic out through Gateway #1 or #2. This is not really the best approach, giving you perfect load balancing, but it's easy to implement, and gives you some control too.

Let us assume we use for our workstations IP addresses from network 192.168.100.0/24. The IP addresses are assigned as follows:

• 192.168.100.1-127 are used for Group A workstations
• 192.168.100.128-253 are used for Group B workstations
• 192.168.100.254 is used for the router.

All workstations have IP configuration with the IP address from the relevant group, they all have network mask 255.255.255.0, and 192.168.100.254 is the default gateway for them. We will talk about DNS servers later.

Now, when we have workstations divided into groups, we can refer to them using subnet addressing:

• Group A is 192.168.100.0/25, i.e., addresses 192.168.100.0-127
• Group B is 192.168.100.128/25, i.e., addresses 192.168.100.128-255

If you do not understand this, take the TCP/IP Basics course,
or, look for some resources about subnetting on the Internet!

We need to add two IP Firewall Mangle rules to mark the packets originated from Group A or Group B workstations.

For Group A, specify

• Chain prerouting and Src. Address 192.168.100.0/25
• Action mark routing and New Routing Mark GroupA.

It is a good practice to add a comment as well. Your mangle rules might be interesting for someone else and for yourself as well after some time.

For Group B, specify

• Chain prerouting and Src. Address 192.168.100.128/25
• Action mark routing and New Routing Mark GroupB

All IP traffic coming from workstations is marked with the routing marks GroupA or GroupB. We can use these marks in the routing table.

Next, we should specify two default routes (destination 0.0.0.0/0) with appropriate routing marks and gateways:

This thing is not going to work, unless you do masquerading for your LAN! The simplest way to do it is by adding one NAT rule for Src. Address 192.168.100.0/24 and Action masquerade:

Test the setup by tracing the route to some IP address on the Internet!

From a workstation of Group A, it should go like this:

C:\>tracert -d 8.8.8.8

Tracing route to 8.8.8.8 over a maximum of 30 hops

1     2 ms     2 ms     2 ms  192.168.100.254
2    10 ms     4 ms     3 ms  10.1.0.1
...

From a workstation of Group B, it should go like this:

C:\>tracert -d 8.8.8.8

Tracing route to 8.8.8.8 over a maximum of 30 hops

1     2 ms     2 ms     2 ms  192.168.100.254
2    10 ms     4 ms     3 ms  10.5.8.1
...

You can specify the DNS server for workstations quite freely, just make it can be reached (test it by tracing the route to DNS server's IP address)!

Source : http://wiki.mikrotik.com/wiki/Load_Balancing_over_Multiple_Gateways