Oh look: It's another very long email from juul!
I've been working on the config for the N750 + antenna-node configuration
and here are my thoughts so far.
We should _not_ let the LAN ports be one big ethernet interface for use
both as a wired version of open0 _and_ a way to connect antenna-nodes. We
never want to bridge two interfaces with attached antenna-nodes but
treating the multiple LAN ethernet interfaces as one interface is
effectively the same as bridging.
Example scenario: You have to nanobridges on your roof linking you to two
parts of the mesh. They are both plugged into LAN ports on your N750 and
since the LAN ports are treated as one interface then the two nanobridges
are able to communicate on layer 2. The nodes at the houses to which you
are connecting with your nanobridges have the same setup. Now we
effectively have the spanning tree protocol as our mesh protocol instead of
It was Alex who pointed this out when we were talking about bridging but
for some reason I hadn't connected the dots and recognized that the same is
true when telling the built-in switch to treat the interfaces as one.
This means that each physical port on the N750 that we wish to connect to
an antenna-node must be its own interface and should have a /32 netmask.
These interfaces can still have the same node IP as open0, etc, without any
I suggest we allocate two of the four ports for this purpose until we can
make something that intelligently changes the config when an antenna-node
has been connected.
We now have two remaining LAN ports that can act as a single interface. We
could then bridge this LAN interface to open0. However, we want to avoid
channel interference when sending from/to open0 but there is no reason we
should try to avoid channel interference for traffic coming from the wired
interfaces. If we bridge them we cannot treat them differently, so we
should not bridge.
It is not clear if babel channel diversity also takes into account channel
information for manually published routes such as the open0 route we
currently publish with this rule:
redistribute if open0 metric 128
I'll have to look at the source code to check. If the functionality is not
there then it should be fairly easy to add.
Aaaanyway: Since we don't want to bridge open0 and LAN, this complicates
things because we then need two subnets per node (otherwise we have the
same /26 on both LAN and open0, which is not going to work). Less than a
/26 on either interface gets iffy, so it seems like we'll need to have two
/26 subnets per node now.
One last but important thing I realized: If the antenna-nodes have their
wifi and ethernet interfaces bridged, then we will have problems. Imagine
the following setup:
(N750 A) ------ (nanobridge A) ~~~~ (nanobridge B) ----- (N750 B)
Where "-----" means ethernet and "~~~~" means wifi.
If both nanobridge A and nanobridge B are simply bridging their ethernet
and wifi interfaces, then we have the following problems:
1. If e.g. nanobridge A sends a DHCP request, then it will be received by
both N750 A and N750 B and they will have no reliable way of know if the
request was sent by "their" nanobridge or the remote nanobridge.
2. When managing e.g. nanobridge A via the N750 A web admin interface it
will be impossible for nanobridge A to know whether it should grant admin
access to N750 A or N750 B.
I can think of only two solutions to this:
1. Pre-configure all antenna-nodes with static IPs and knowledge of which
N750 node is their parent.
2. Don't bridge the ethernet and wifi interfaces on antenna-nodes and
instead run babel on them.
I like solution 2 much better since it's easier for both us and node
operators. Not sure if we'll see a performance hit if we don't bridge. We
have a few nanobridges though so we could easily test this.
What do y'all say?
PS: Obviously babel channel diversity doesn't apply to antenna nodes since
babel sees them as ethernet interfaces but since they are mostly
directional and far away from the N750 it is fine to treat them as having
no interference, which is the default for ethernet interfaces anyway.
PPS: Did you know that the default STP delay when adding an interface to a
bridge is apparently 30 seconds? Not sure if OpenWRT has a different
default or maybe uses RSTP (rapid spanning tree protocol) to deal with
this, but if not then it is something to be aware of in order to now go
insane when troubleshooting. From:
I've been trying to reach out to folks and show them people's open, like
at a couple of conferences I just spoke at here on the east coast, but
we have no website.
It's not terribly crucial, but is there a timeline on restoring it? It'd
be great to know so that I don't keep trying to pull it up in front of a
bunch of people :)
Thanks everyone!! Hope all is well
I'm running a little late, but it does look like I successfully compiled an
ar71xx binary of notdchpclient and notdhcpserver. I've attached the ipks to
this email in case anyone is super anxious to get running with them,
otherwise I'll be at sudo a little after 8.
With help from Jake and Emborg I think I now have an idea how to go about
making the garden nodes do what we want.
We'll use the ESP8266-07 version which has a number of useful features over
the cheaper versions:
* A single ADC input (cheaper versions have none)
* Many more GPIO pins
* Shielded wifi chips
* Both ceramic on-board antenna and u.fl connector (not that we really need
And it's still only $3.34 including shipping:
We'll then use a 4051 chip to switch between eight different analog inputs.
These are dirt cheap (about 13 cents apiece if you buy 100):
Selecting analog input will take up three GPIO pins, but we have plenty.
We'll use three one-dollar solar garden lights:
We will connect them such that their batteries are hooked up in series,
providing a maximum of 3.6 volts total for the ESP8266 (which happens to be
the maximum it can handle). If we're worried about too much voltage then
that can be dealt with easily, and the minimum voltage is 1.7 volts, which
makes things easy.
The ADC is 0-1 volts, so we'll use a voltage divider to read the voltage of
the three series-coupled batteries.
The ESP8266 has a deep sleep feature where it basically uses no power. It
can be programmed to wake up after custom amount of elapsed time. We'll
program it to wake up e.g. every ten minutes, measure the voltage, then
shut down if there is not enough. If there is enough voltage it will read
sensor data, connect over wifi, report data, and go back into deep sleep
We will measure soil humidity using a capacitance measurement which
requires two GPIO pins and two analog inputs. Treehearder already has the
code for this, it just has to be ported from an Atmel attiny to the
ESP8266. The sensor consists of two long pieces of conductive metal wrapped
in thin non-conductive insulation (two long pieces of iron in heat-shrink
tube should be good) and a couple of simple support components.
Measuring door opening/closing is still up for grabs. I'm not sure how to
do this reliable on any type of door/gate in a way that will be reliable
and won't break. On top of that it's not clear whether we'd need to check
door status more frequently than every few minutes but if we do then we'll
need some way to save "the door was opened" such that the ESP8266 will know
about it when it wakes up. This could be done with a simple flip-flop/latch
For directional antennas we can make a laser-cuttable Yagi template (using
online template-generators and some inkscape foo), cut and hot-glue copper
elements onto the template and fit the assembled yagi into a piece of PVC
pipe and seal with hot glue. It probably makes sense to fit the ESP8266
inside of that tube as well and cover it in hot glue to avoid
condensation-related failure. Jake suggested that we simply solder the
ESP8266 board straight onto the yagi active element.
All of this being said, I'm 100% for mcgyvering the first prototype using
whatever is fast and easy!
Today April and I spoke with Tim and Chris about our
problems with obtaining an uplink. The tl;dr version is that
we have two possibilities for circumventing the last-mile
shakedown: BART and radio links.
You probably don't need to read the rest. :)
First off, the prices quoted to us for a 1Gbit were fairly
normal. They are due to the last-mile being neither competitive
nor transparent. Zayo may still be an option, but getting the
building wired for fiber may be a huge ordeal and it still might
be very expensive.
The only workaround may be to find an alternative, cheaper
route to a data center; IP transit is cheap these days and
getting cheaper. There are two sources of problems
here: transport (the connection from our network to a
data center) and data center internals.
The data centers we have available to us are: "365 Main St."
in downtown Oakland, Evocative in Emeryville, and then
things further away such as Hurricane Electric. Unfortunately,
the AT&T CO near the Omni is probably not useful to us.
There's a whole lot of cost and complexity involved in getting into a
data center and it would be nice to avoid paying for co-location.
Transport is tricky. We learned that BART can provide transport
(not just dark fiber leases), but we don't know where we can
connect to them physically, so an additional radio or fiber link
may be necessary. Don't know how much it'll cost.
The other intriguing option is a two-hop radio link (up to the
hills and then down to a data center). We discussed using
licensed bands for this, and it seems feasible with some
perseverance. Again, don't know much it'll cost.
We also discussed my idea of attaching fiber to poles, which
seems to be feasible in theory but probably prohibitively expensive
That's a graph of delay times for a "cloud" server to reach the omni
network. The results are pretty awesomely clear. This translates to real
improvement in network performance at the omni. I imagine that most people
have already experienced this :-)
Anyhow - just a thanks to everyone who got this together - just the very
first step on improving connectivity in the east bay!