OpenWRT conditional hostname routing using MWAN3 and ipset

5 months ago ยท 9 min read

Linksys WRT 1900 AC V2

Network plan

My main requirement is to route the internet traffic to specific WAN connection conditionally depending on it's hostname. For example, If someone try to access facebook.com, twitter.com or youtube.com etc. I want those requests to be routed via a specific WAN (or internet router) and all the other request should routed through 2nd internet connection (WAN).

Second requirement was to monitor and collect stats about my home local network usage and persist them for later analysis. One of the motivational factor for collecting usage data was,there are rumors in our country that some Internet Service Providers(ISPs) charge for more data than what the user has actually consumed :)

Below is my network setup:

Network Setup

Diagram sources: lucid & excalidraw

Hardware

I'm using Linksys WRT 1900AC V2 (Cobra) router with OpenWRT latest firmware(v19.07.5). But you can use any OpenWRT compatible device to configure this. There are some OpenWRT like firmwares i:e dd-wrt & ๐Ÿ… tomato

OpenWRT Multiple WAN configuration

To begin with , You need to setup ssh on the OpenWRT router and access the router as root user ssh -v [email protected]. Most of the configurations can be done via the web UI but, It's easy to track the changes when configuring via command line, I used to take backups using scp and edit the config files in laptop and push them back to router via scp.

OpenWRT has a mwan3 package which allows you to configure various rules and policies on WAN connections. I have use mwan3 to configure the conditional load balancing/routing to achieve my above requirement.

To install the mwan3 package use

opkg update
opkg install mwan3

After installing mwan3 keep it aside and configure the IPSet, and later we can refer the IPSet names and create mwan3 config file at once.

To configure dynamic hostname base WAN routing you need to do 3 things

  1. create a ipset

    ipset -N twitter hash:ip

    in the above twitter is the name of ipset hash:ip is a template kind of thing, When router get it's first ping request. Following command will list all the ip sets

    ipset list

    To get information about a specific ip set

    ipset -L twitter

    Where twitter is the IP set name, If system has resolved IPs for the name , Then those IPs will be shown under Members parameter

    [email protected]:~# ipset -L twitter
    Name: twitter
    Type: hash:ip
    Revision: 4
    Header: family inet hashsize 1024 maxelem 65536
    Size in memory: 416
    References: 1
    Number of entries: 9
    Members:
    69.195.179.128
    69.195.182.128
    69.195.165.128
    104.244.42.2
    104.244.42.194
    104.244.42.66
    69.195.177.128
    209.237.193.128
    104.244.42.130

    Important thing to note here is, When you run above command ip set will be added to the memory, and they don't get persisted. Hence restarting the router will clear all the added ipset. In other words ipset rules are saved in memory, upon a reboot they will be lost. Because of that I have created a startup script (source code)

    /etc/init.d/autoipset

    to add all the ip sets I want when starting up the router. Instructions are given in above gist source on how to add the startup script. For more info about how to write init script in OpenWRT follow this official doc

  2. Update dnsmasq.conf with ipset name and dns name

    What is dnsmasq

    Open the

    /etc/dnsmasq.conf

    in router, and add ipset entries as below

    ipset=/.twitter.com/twitter

    here, AFAIK twitter after the last slash is the ipset name, Following is a sample dnsmasq.conf configuration content

    # /etc/dnsmasq.conf
    # Change the following lines if you want dnsmasq to serve SRV
    # records.
    # You may add multiple srv-host lines.
    # The fields are <name>,<target>,<port>,<priority>,<weight>
    
    # A SRV record sending LDAP for the example.com domain to
    # ldapserver.example.com port 289
    #srv-host=_ldap._tcp.example.com,ldapserver.example.com,389
    
    ipset=/.facebook.com/facebook
    ipset=/.instagram.com/instagram
    ipset=/.fbcdn.net/fbcdn
    ipset=/.twitter.com/twitter
    ipset=/.twimg.com/twimg

    after updating the config file run

    service dnsmasq restart

    to refresh the dnsmasq Note here: In my case default dnsmasq tool coming with the OpenWRT installation does not worked and I had to uninstall the pre-installed dnsmasq and install dnsmasq-full to get this work!

  3. Now IP sets are ready, So final thing is to configure the mwan3 image

    If you are new to mwan3 there is a really good documentation in OpenWRT docs,covering all the aspects.

    At this point, better to restart the router, and ping to those host name that you have configured above i:e

        ping twitter.com

    If you are curious about IPs and DNS, When you ping you will get one of following IPs at the time you execute the ping command

    whois -h whois.radb.net -- '-i origin AS13414' | grep ^route

    If you wonder how it resolve all the IPs , Google search about ASN Lookup, I also initially though that how to find all the IPs belongs to a particular domain and i might need to add them to routing table or something like that, But it's not required here. ok that is for your information about AS codes and IPs , now run the

        ipset -L twitter

    command and check whether the member IP have resolved Configuring the mwan3 can be done via the UI too. But i will put the configuration elements here for convenient. I assume that you are somewhat familiar with OpenWRT and mwan3 configurations, Hence will directly jump into mwan3 config. mwan3 config is located in

    /etc/config/mwan3

    and following is a sample configuration

        config rule 'facebook'
            option proto 'all'
            option family 'ipv4'
            option sticky '1'
            option ipset 'facebook'
            option use_policy 'testp'

    In above rule config, option sticky: Is important to set true in UI or '1' in the config option ipset: Is the IP set name created with ipset -N {name} and the ipset={domain}/{name} in the config file option use_policy: is the policy name which have the WAN as a member that you want to forward the request match with the above ipset (i:e facebook.com requests go to Dialog 4G WAN) Change the above config accordingly and put it in the mwan3 config at the top (will get the priority from configs in top to bottom) image Finally restart the mwan3 service

    mwan3 restart

    and check the mwan3 status

    mwan3 status

    or check the status from the web UI image it should show the above added new rul in Active ipv4 user rules: section

Verifying the changes

Router

Login to the router via ssh and check the following outputs

  • IPset members
ipset -L {name}

run the above command and check whether any member IPs have resolved

  • mwan3 configs
mwan3 status

run above command and check whether the newly added rule is listed in the active ipv4 user rules section

  • Check dnsmasq configuration syntax using below command
dnsmasq --test

External device (Laptop)

traceroute facebook.com

execute the above command and check from which WAN connection or hope the request been routed, Is it routing via the expected WAN connection

Monitoring traffic

This is about monitoring the home network traffic using the OpenWRT, This is a different topic than the above discussed topic.

image

I have used luciappstatistics package in OpenWRT, It provide good coverage of parameters, from Network interfaces, to memory & CPU usages and there are lot more supported in collectd. This package gives more similar graph outputs this is available in LERAN traffic graphs. So i believe this luci_app_statistics package give more professional looking stats in OpenWRT routers.

The default rrd configuration store the monitoring data in the temporary directory (/tmp/) hence the graph data get cleared when router is restarted. In the above doc , they have pointed to some scripts and docs which explains how to backup the data to an external device plugged into router's USB interface. Only thing lacking there is how to use existing vFat/Fat32 formatted pen drive to backup the data. So I had to google and collect the information from various sources, and though of organizing them here. To get vFat/FAT32 partition mounted you need following package in the router

opkg install dosfstools

And then simply follow the instruction in usb-installing and usb-drives To take a backup every X (i:e 6 hours) interval, You can use this script This sheduling is done using the corntabs

Schedule WiFi and switch WAN usages timely

To edit the crontab type

crontab -e

and following scheduled commands will switch off the wifi radios at mid night and switch on at 5AM morning. And also it will switch the MWAN3 policies at mid night and restore back at 8AM morning(That's where peak time start for SLT connections)

0 */6 * * * /etc/init.d/rrdbackup backup
0 8 * * * uci set mwan3.all.use_policy='Dialog4G_Only'; uci commit; mwan3 restart
0 0 * * * uci set mwan3.all.use_policy='SLT_Only'; uci commit; mwan3 restart
0 0 * * * /sbin/wifi down
0 5 * * * /sbin/wifi up

References