Mesh/Firmware

This page has been marked as stale, as it is outdated to the point of providing significant misinformation. Please update it before linking other pages here.

For now, please see the firmware repo on GitHub.

For the new ToDo see Software ToDo.

ToDo

Issues for version 0.3

Issues for later versions:

• Basic admin web UI for node database that let's you log in and list current nodes.
• fake captive portal working
• service browser working
• hardware watchdog working on ar71xx
• support for manually configured non-openwrt extender nodes
  • Allow user to switch ethernet ports into "extender node (manual)" mode
  • Only one network (either adhoc or open) can be extended using this mode. The user must be able to select which.
  • Set up port forwarding from home node to each manually configured extender node for web UI access, e.g. 1443 to 443 on extender node 1, etc.
    • This can be always-on for e.g. up to 16 ports without causing problems since it is only for traffic destined for the home node IP.
    • There should be links to the extender node web UIs from the home node web UI.
• Apply for internet-routable IPv6 subnet and give each node their own subnet + hand out IPv6 addresses to clients
• extender node firmware working on old atheros chipset
• Remote firmware updates working
  • Use either DNS-SD to list and pick an update server at random or IPv6 multi-homing?
• Remote automated root password reset (via h.sudomesh.org)
• IPv6 support for extender nodes
• Figure out how to legally use lower 5 ghz frequencies and ensure fancy back-off features are working
• Run OpenVPN on exit node.

In the future:

• Support TDMA on Linux (Adri is working on FreeBSD support, maybe we can port).

Stuff we are working on

Ranked from most to least important

InternetIsDownRedirect

When the node doesn't have internet access, it will redirect traffic to our mesh hosted Splash Page.

We need something hosted on the node that can check if it has access to the internet. There's a bit of an issue where certain OSes won't connect to APs that don't have internet access. Juul will look into building a hack that properly manages these requests and redirects them to our node-hosted site.

InternetIsDownRedirect may also have to fake the expected captive portal detection responses? We need to figure out if android/iOS/Mac/Windows will connect to a wifi that does not have internet access.

Status: Implemented except for OS-specific captive portal requests.

Splash page (Garden Gnome)

We can capture OS specific probes in order to specifically redirect captive portal requests without affecting any other network traffic.

Chris (snake_wrangler) was working on it and calling it the Garden Gnome. maxb picked up the idea and ran with it in nodejs: https://github.com/sudomesh/garden-gnome

Features:

• Brief info on the mesh
• Link to our website?

Status:

maxb has implemented it! We need a tiny bit of polishing, especially for corner case browsers.

Extender node firmware

See the Mesh/Technical Overview to learn about home nodes and extender nodes.

With the extender node firmware we're targeting a variety of hardware (if not chipsets), and we don't want to run any sort of makenode after the firmware has been flashed.

We're doing this by creating a /file/etc/uci-defaults script which will run first boot and can set configs depending on the "board" type: uci-defaults script gist

Mesh Protocol

BATMAN-adv was the protocol that we had assumed we'd be able to use. Unfortunately, it looks as though it won't support tunneling over the internet, which is one of the primary features we had been hoping to implement. See our mailing list convo.

We're now using babel. It's been looking pretty good for our particular applications. There are some security/trust issues that we'd like to investigate further - see http://lists.alioth.debian.org/pipermail/babel-users/2015-April/001973.html for some ideas that babel might incorporate in the future.

Status: Ready, but can be improved

Multiple virtual network interfaces with their own SSIDs

• The Mesh interface: An ad-hock mode, unprotected interface for the mesh nodes to talk to each other with Babel handling routing, ssid: pplsopen-node2node
• The Open interface: An access-point-mode, unprotected interface, for non-mesh devices to connect to the mesh, ssid: peoplesopen.net
• The Private interface: An access-point mode, private interface with WPA2, for the people who own the nodes, ssid: Decided by node owner

Traffic on the private interface should be completely separated from traffic on the non-private interfaces unless a client connected to the private interface requests an IP on the mesh.

Maybe the last one is optional because some people may not need that feature (they already have another access point and they want to keep it), but then how do people administrate the router?

In order to serve a secure web admin config to home users, we'll probably always serve 3 APs with one private WPA encrypted home link so that users can access their admin page.

Status: Implemented

Web admin interface

Development information should be put in Web Admin Dev. This section can remain a wish-list.

A very simple one-page interface. It should do at least the following:

• Display some set of user statistics

Ideally we could list/graph the number of people who have associated with your mesh node.

We could also just list/graph the up/down data of people who have been using the mesh.

LightSquid (used by pfSense)

• Set location, name, description.

But do you want to know the location centrally as well so that you can display nodes on the map? Will people enter this information twice or will you pull this information from nodes and then display on the map? Same for name and description. I would suggest that information is stored only once. In your case on the node itself. So probably you can then pull this information through nodewatcher scripts on nodes and then display nodes the map. Just really should not require people to enter or maintain information on two places because it desyncs very fast. Mitar (talk) 22:20, 24 July 2013 (PDT)

• Let people select how much bandwidth they share.

They always share 100% when they're not using the connection themselves.

This works if people are using their private SSIDs on the node. But if the node is connected to their existing home network you might not easily configure such sharing. But maybe there is a way to detect that host network is free and can limits can be increased. Mitar (talk) 22:20, 24 July 2013 (PDT)

Do any ISPs have bandwidth caps around here? If so, let people specify how many MB to share per month.

Maybe also a button for temporary increase limits (make them more restrictive) which are then after some time automatically restored.

• Let people change the admin password and the private wifi wpa2 password.

Probably private SSID as well.

• Donate / "buy routers as presents for your friends"-button.

One idea we had (but this is probably better for splash screen) is "adopt a node". Where a neighbor who uses a node a lot and depends on the node can donate some money to keep it up, but can then give a nickname or avatar to the node. Or something. Mitar (talk) 22:20, 24 July 2013 (PDT)

Status: Maxb implemented a luci-based ui. We decided we hate luci, so we're going to use the ubus uhttpd rpc protocol with a jsonrpc front-end. It's a bit copy-pasta'd from the openwireless router.

Source here:

https://github.com/sudomesh/sudowrt-luci2-webclient

Remote Updates

Once we deploy nodes, we'll want a way to update them as appropriate. The already built node configurator operates along similar lines, but we'd need to do some tweaking in order to make it work on the mesh. Also, we'd want to give the users the options to turn remote updates off. A somewhat decentralized system would be nice as well.

Watchdog script

Node tests itself to see if it has connectivity, etc and resets itself if necessary. OpenWrt supports the hardware watchdog on our PicoStations without any additional hacking, yay!

By default the hardware watchdog will automatically hard-reset the AP if /dev/watchdog is not written to at least once every 60 seconds. A Lua library has been written to interface with the batman-adv kernel module through the batctl command line utility. We need to identify a list of conditions that require a hard-reset and work them into the Lua watchdog script in the openwrt-firmware repository.

The Freifunk group has an awesome watchdog setup, details: http://wiki.freifunk.net/Kamikaze/LuCI/Watchdog

list of possible reset conditions: high sustained load, cron goes down, sshd goes down.

nodewatcher watchdog

Potential use of Quilt to update nodes.

Quality of Service (QoS)

We want those who own a node to decide how much internet they share with the network. This software allows users to shape their bandwidth based on type. There's an paper regarding layer 7 traffic shaping too.

Our supported hardware needs a very lightweight software, which is why we've been using tc (traffic control). It only allows the users to determine how much internet they share with the network.

Complete Distributions

These have firewall and network management tools included with the distribution.

• pfSense - a widely used firewall distribution, but there are most definitely difficulties with it.
• Zentyal - a firewall distribution with easy to use graphical interface.
• m0n0wall - a lightweight firewall distribution meant for embedded systems.

Packages

These are tools often used in network management distributions.

• netfilter/iptables - a set of hooks inside the Linux kernel that allows kernel modules to register callback functions with the network stack.
• iproute2 - a collection of utilities for controlling TCP / IP networking and traffic control.
• l7-filter (p2p filtering) - identifies packets based on application layer data. It classifies packets to be used with a bandwidth shaper.
• ipp2p (p2p filtering) - identifies peer-to-peer (P2P) data in IP traffic.
• Suricata - a high performance network intrusion detection system (IDS), intrusion prevention system (IPS), and network security monitoring engine.
• ipfirewall (ipfw) - a freeBSD firewall that uses netdummy.
• netdummy - a freeBSD traffic shaper and bandwidth manager.
• ipfw-classifyd - an application layer classifier for ipfw firewall for freeBSD.

Virtual Private Network (VPN)

The firmware should tunnel all internet traffic from the mesh through a VPN server, unless this feature is specifically disabled. This should not be a single server, as that would be a single point of failure.

• TunnelDigger - a lightweight tunneling client/server.
• OpenMesher - another option, but not ideal because of memory constraints on embedded systems.

Here is our Network Topology.

Mesh VPuN

If the mesh does not see any other nodes (and maybe even if it does?), and it has internet, then it should connect to another node or two over a VPuN server. The easy solution is to use the same VPuN servers as for the internet connection.

Network Topology

Status: Implemented

Location and status reporting

Something that reports location and status when polled. I think we can probably get away with using snmp v1.

We developed this format and easy to publish status data from nodes for our nodewatcher. OpenWrt packages are here. Mitar (talk) 22:02, 11 July 2013 (PDT)

Nice to have:

• Status info: How many nodes is your node connected to. Is the internet link working.
• An "I don't know what my internet bandwidth is, test it for me"-function.
• Usage statistics (so people can see how many people they helped get internet!)

This is the most important thing! Mitar (talk) 22:20, 24 July 2013 (PDT)

You should add as well graphs on how much bandwidth was consumed by the node. This is useful when hosts see that their Internet is slow and believe that it was because of the node. Then they can check and see if it is really node (which often is not) or maybe just ISP has problems. Important because people like to attribute issues they have to nodes they don't understand. Mitar (talk) 22:20, 24 July 2013 (PDT)

• Let people put up a bit of info about their node / house / co-op, on a simple web page that people can access only if they're connected to that node. It could be shown as part of the splash page.

Status: Waiting for nodewatcher project to finish

Intelligent Wifi Channel Switching

It would be nice to be able to have the network intelligently determine channels but we don't want a node changing channel if it means other nodes connected to it will have to change as well so it may make sense to only set the channel once on first boot based on which channels have other nodes.

IPv6 support

We should have IPv6 support, but I am ok with launching the mesh with only IPv4 and adding in IPv6 later. We can do without IPv6 but not without IPv4 (Juul (talk))

Stuff the firmware could have

DNS server

Each node could run its own (caching) DNS server.

For now, if you're logged into the private network on a node, going to http://my.node will take you to the web admin interface

Status:

Implemented web admin URL, but no caching DNS server yet.

RSSI Testing and Logging

At intervals, the nodes could conduct RSSI tests and log them with some way to compare and visualize signal strengths over time.

Caching web proxy

We could use Polipo to improve people's browsing experience. Not sure how much cpu and memory this would need. We may not be able to run it on the routers with less than 32 MB ram (e.g. the Bullet 2 HPs).

Maxb has installed polio as a transparent caching proxy on a picostation 2HP. It improved web browsing significantly! However, it sorta breaks "net neutrality". Also - it'll be irrelevant for https and it might break some things (webdav?).

I wonder if you could (as an end user?) enable/disable this kinda proxying? On splash page? We would need MAC, but it would only get logged on the mesh node itself (not network wide....)

Block ads and tracking

We could use e.g. Polipo with the sources from both adblock plus and ghostery. If we implement this, it should be an optional (default off) feature that you can select on the splash page, with a "remember this" that remembers either using a cookie or using your MAC (but then we'd be logging people's MAC addresses :-S). The block should probably be time-limited (e.g. 30 days).

Now that we're running more powerful devices, I wonder if we could actually run this kind of blocking on the actual mesh nodes. That way we could do a "remember this" MAC and only store it on the mesh node itself (NOT network-wide).

Compatible devices

We're going to target ar71xx devices with decent storage + processing speed + memory for "router board" nodes.

Right now we're using for "router board" nodes: WD MyNet N750 and/or TP-Link TL-WDR4300 The MyNets are no longer in production, but the TL-WDR4300 are the exact same board, just with external antennas.

Then we are using extender nodes which plug into the home node and just bridge/forward traffic. This could theoretically be any router which supports openwrt. We've begun by targeting a variety of ubiquiti long-distance outdoor radios (Nanostation M line, Bullet M line, Picostation M line, Nanobridge/beam/etc).

We would also like to support non-openwrt routers (like Ubiquiti Airfiber and Nanobeam 802.11ac devices) but these will only be able to extend one of the three networks (Mesh, Public or Private) and will need to use WDS mode to do so (at least for Mesh).