Mesh/Relay setup
This page is outdated and is kept for our communal scrapbook. Please use https://github.com/sudomesh/exitnode for a more up-to-date info.
Relay nodes are Internet connected hosts that allow mesh nodes to talk to each other using batman-adv over the Internet. Mesh nodes connect to relay nodes using a type of layer 2 tunnel called L2TP. The software used to establish these tunnels is called tunneldigger and was developed by wlan slovenija. A relay node is usually a VPS or a dedicated server with a good amount of available bandwidth and cpu.
L2TP tunnels are unencrypted. They are used primarily because they are layer 2 (so they can be used for batman-adv) and are part of the Linux kernel. They code does not take up much space on the space-limited mesh nodes and does not use much cpu.
Configuring a relay node involves the following steps:
- Ensure that you have a compatible operating system
- Get a static IP assigned from sudo mesh (or use the test IP)
- Configure B.A.T.M.A.N. Advanced
- Configure Tunneldigger Broker
Operating system
Relays must run Linux, since Linux-specific kernel modules are involved.
This guide is intended for Debian 7 (Wheezy). If you are running something else, see the following.
Ubuntu
It seems that some versions (at least version 12.10) don't have any way of installing the required L2TP modules.
To check if your version of Ubuntu has the L2TP modules, run:
sudo aptitude install apt-file sudo apt-file update apt-file search l2tp_netlink
If you get no results, or if the only result you get is a package called "user-mode-linux", then your version of Ubuntu does not work with tunneldigger :(
Debian 6 (Squeeze)
If you are running Debian 6, then you should upgrade to Debian 7. There is a comprehensive guide here. The short version is:
sudo aptitude update && sudo aptitude safe-upgrade
Edit your /etc/apt/sources.list from e.g:
deb http://ftp.us.debian.org/debian squeeze main deb http://security.debian.org/ squeeze/updates main
To:
deb http://ftp.us.debian.org/debian wheezy main deb http://security.debian.org/ wheezy/updates main
Now upgrade your system:
sudo aptitude update && sudo apt-get dist-upgrade
Notice that it is now recommended to use apt-get instead of aptitude for the dist-upgrade.
Follow the instructions on screen and reboot.
After rebooting you should be running a 3.x kernel. Check which version you're running with:
uname -a
If you are running a 2.x version, then you may be able to solve the problem by editing /etc/grub/menu.lst and finding lines like:
title vmlinuz-2.6.32-5-amd64 root (hd0,0) kernel /boot/vmlinuz-2.6.32-5-amd64 console=hvc0 root=/dev/xvda1 ro initrd /boot/initrd.img-2.6.32-5-amd64
and commenting them out like so:
#title vmlinuz-2.6.32-5-amd64 # root (hd0,0) # kernel /boot/vmlinuz-2.6.32-5-amd64 console=hvc0 root=/dev/xvda1 ro # initrd /boot/initrd.img-2.6.32-5-amd64
and rebooting.
If you encounter any other problems, refer to the official guide.
Other distros
You are on your own, but if you figure out how to get batman-adv and tunneldigger working on a different distro, then please edit this wiki.
Getting an assigned static IP
For testing purposes you can use the IP 10.42.254.254
If you want others to use your relay node, you should contact the sudo mesh group for a static IP assignment, either at info@sudomesh.org or by showing up to a weekly meeting or hacknight. Please include the following information:
- Relay node IP and port(s)
- Contact info for node administrator (at least an email address)
- How much bandwidth does the relay node provide? Is there a monthly limit?
In the future you will also have to agree to the terms of the Network Commons License, which has not yet been defined.
batman-adv
You should already have the batman_adv module. Try loading it:
sudo modprobe batman_adv
If there are no errors, you have the module.
Add batman_adv at the end of /etc/modules to make it auto-load on bootup:
batman_adv
Install the batctl utility:
sudo aptitude install batctl
Set a static IP for the bat0 interface by adding the following to /etc/network/interfaces:
iface bat0 inet static address 10.42.254.254 netmask 255.0.0.0
If you have already received a static IP assignment from sudo mesh, then use that in place of 10.42.254.254.
Tunneldigger broker
Install the broker
You must have the following kernel modules available:
- l2tp_core
- l2tp_eth
- l2tp_netlink
To check:
modprobe l2tp_core modprobe l2tp_eth modprobe l2tp_netlink
If there are no errors, you have the modules. If there are errors, then you should install the linux-image-extra package for your kernel. E.g:
sudo aptitude install linux-image-extra-3.5.0-36-generic
Use apt-cache search linux-image-extra to find the package for you kernel version (hint: You can find your kernel version using: uname -a)
It seems that some Ubuntu version do not include the l2tp_* modules in any packages. See the section on Ubuntu further up this page.
Now add the module names at the end of /etc/modules so they are automatically loaded on boot:
l2tp_core l2tp_eth l2tp_netlink
Now install some required packages:
sudo aptitude install iproute bridge-utils libnetfilter-conntrack3 python-dev libevent-dev ebtables python-pip git
Download tunneldigger with git if you have not already done so:
cd git clone https://github.com/wlanslovenija/tunneldigger.git sudo mv tunneldigger /opt cd /opt sudo chown root.root -R tunneldigger cd tunneldigger/broker
Now install the additional required python packages:
sudo pip install -r requirements.txt
If it doesn't work, change the line:
construct==2.0.6
to:
construct==2.06
You may have to remove the python-netfilter line in requirements.txt and install it manually:
git clone --depth=1 https://github.com/jlaine/python-netfilter.git cd python-netfiler sudo python setup.py install
If you have a python-netfilter package available to install via apt, then it is likely too old. Don't use it.
If you have any problems with pip, try upgrading it:
sudo pip install --upgrade pip
You may have to log out and log back in for pip to work after this
Configure the broker
Open l2tp_broker.cfg.
You will want to change the "address=" line to your public IP address, and "interface=" to your public ethernet interface. E.g:
address=93.184.216.119 interface=eth0
You may also want to change the ports line. If you're running a DNS server on the same host then you'll want to remove port 53 from the list of ports.
Change the "session.up=" line to:
session.up=/opt/tunneldigger/broker/scripts/up_hook.sh
Now create the up_hook.sh file:
cd /opt/tunneldigger/broker/scripts sudo touch up_hook.sh sudo chmod 755 up_hook.sh
Edit up_hook.sh to contain the following:
#!/bin/sh
INTERFACE="$3"
ifconfig $INTERFACE up
batctl if add $INTERFACE
if [ `cat /sys/class/net/bat0/operstate` != "up" ]; then
ifconfig bat0 <relay_mesh_ip> netmask 255.0.0.0 up
fi
Where <relay_mesh_ip> is your assigned mesh ip address (or 10.42.254.254 for testing).
Now you're ready to run the broker.
Run the broker
You can run the broker using:
sudo ./l2tp_broker.py l2tp_broker.cfg
To run the broker as a daemon and automatically on system startup, see the sections below.
Test the broker
To test the broker you need to install batman-adv and tunneldigger client on a different system. Ensure you have the L2TP modules installed and loaded on your client system. And ensure you have the batctl package installed and the batman_adv module loaded.
Install tunneldigger client requirements on you client system:
sudo aptitude install libnl-dev build-essential git
Download tunneldigger on client system:
cd git clone https://github.com/wlanslovenija/tunneldigger.git
Compile tunneldigger client:
cd tunneldigger make
Ensure that the tunneldigger broker is running on your relay node and run the tunneldigger client:
sudo ./l2tp_client -f -u foo -l <ip_of_your_relay_node>:<53> -i l2tp
The tunneldigger client should inform you when the tunnel has been established. This is what you'd expect to see:
l2tp-client: Performing broker selection... l2tp-client: Selected 192.157.221.200:53 as the best broker. l2tp-client: Tunnel successfully established.
If you use the -L option for l2tp_client, you may get an error from the broker like:
RTNETLINK answers: Invalid argument
Don't worry about it. The broker is attempting to clear out a sometimes non-existent traffic control rule using the command:
/sbin/tc qdisc del dev <tunnel_iface> root handle 1: htb default 0
This doesn't harm anything, but if you want to fix the behavior, look in traffic_control.py in the reset function.
After the tunnel is created, the interface l2tp0 should appear. Check with:
ifconfig l2tp0
Give it an IP in the 10.0.0.0/8 range, e.g:
sudo ifconfig l2tp0 10.254.254.254 netmask 255.0.0.0 up
Then add the tunnel interface to batman:
sudo batctl if add l2tp0
Now verify that batman is able to communicate over the tunnel:
batctl o
You should see output like:
[B.A.T.M.A.N. adv 2011.4.0, MainIF/MAC: l2tp1001/96:ea:1d:9a:e9:6c (bat0)] Originator last-seen (#/255) Nexthop [outgoingIF]: Potential nexthops ... 06:15:6d:72:05:21 0.064s (255) de:c9:2f:79:66:04 [ l2tp1001]: de:c9:2f:79:66:04 (255) de:c9:2f:79:66:04 0.476s (251) de:c9:2f:79:66:04 [ l2tp1001]: de:c9:2f:79:66:04 (251)
You may see only one entry in the list, but you should see at least one. You can run "batctl o" on the relay server as well and it should show one entry.
If you see at least one entry, that means you have layer 2 communication working between the client and server over the l2tp tunnel and that batman-adv is configured correctly.
Now check if layer 3 communication is also working. From the client:
ping 10.42.254.254
(or if you have are using an assigned static IP, ping that IP instead)
If you can ping, then everything is working correctly!
Configure the broker to auto-start
To configure the broker to start on boot, download the init script and place it in /etc/init.d:
cd wget https://github.com/sudomesh/tunneldigger/raw/master/broker/scripts/tunneldigger-broker.init.d_alt sudo mv tunneldigger-broker.init.d_alt /etc/init.d/tunneldigger sudo chmod 755 /etc/init.d/tunneldigger
Edit the script to ensure that the BROKER_DIR is set to the directory where tunneldigger broker resides on your system.
Now configure the script to start and stop when your system boots up and shuts down:
sudo update-rc.d tunneldigger defaults
Reboot your system and verify that tunneldigger starts correctly.
Securing your system
Now that you're running a piece of the mesh infrastructure, you may want to ensure that your server stays reasonably secure. We recommend that you set up automatic security updates as described in Mesh/Server_security this guide.