Oh look: It's another very long email from juul!
I've been working on the config for the N750 + antenna-node configuration
and here are my thoughts so far.
We should _not_ let the LAN ports be one big ethernet interface for use
both as a wired version of open0 _and_ a way to connect antenna-nodes. We
never want to bridge two interfaces with attached antenna-nodes but
treating the multiple LAN ethernet interfaces as one interface is
effectively the same as bridging.
Example scenario: You have to nanobridges on your roof linking you to two
parts of the mesh. They are both plugged into LAN ports on your N750 and
since the LAN ports are treated as one interface then the two nanobridges
are able to communicate on layer 2. The nodes at the houses to which you
are connecting with your nanobridges have the same setup. Now we
effectively have the spanning tree protocol as our mesh protocol instead of
babel.
It was Alex who pointed this out when we were talking about bridging but
for some reason I hadn't connected the dots and recognized that the same is
true when telling the built-in switch to treat the interfaces as one.
This means that each physical port on the N750 that we wish to connect to
an antenna-node must be its own interface and should have a /32 netmask.
These interfaces can still have the same node IP as open0, etc, without any
problems.
I suggest we allocate two of the four ports for this purpose until we can
make something that intelligently changes the config when an antenna-node
has been connected.
We now have two remaining LAN ports that can act as a single interface. We
could then bridge this LAN interface to open0. However, we want to avoid
channel interference when sending from/to open0 but there is no reason we
should try to avoid channel interference for traffic coming from the wired
interfaces. If we bridge them we cannot treat them differently, so we
should not bridge.
It is not clear if babel channel diversity also takes into account channel
information for manually published routes such as the open0 route we
currently publish with this rule:
redistribute if open0 metric 128
I'll have to look at the source code to check. If the functionality is not
there then it should be fairly easy to add.
Aaaanyway: Since we don't want to bridge open0 and LAN, this complicates
things because we then need two subnets per node (otherwise we have the
same /26 on both LAN and open0, which is not going to work). Less than a
/26 on either interface gets iffy, so it seems like we'll need to have two
/26 subnets per node now.
One last but important thing I realized: If the antenna-nodes have their
wifi and ethernet interfaces bridged, then we will have problems. Imagine
the following setup:
(N750 A) ------ (nanobridge A) ~~~~ (nanobridge B) ----- (N750 B)
Where "-----" means ethernet and "~~~~" means wifi.
If both nanobridge A and nanobridge B are simply bridging their ethernet
and wifi interfaces, then we have the following problems:
1. If e.g. nanobridge A sends a DHCP request, then it will be received by
both N750 A and N750 B and they will have no reliable way of know if the
request was sent by "their" nanobridge or the remote nanobridge.
2. When managing e.g. nanobridge A via the N750 A web admin interface it
will be impossible for nanobridge A to know whether it should grant admin
access to N750 A or N750 B.
I can think of only two solutions to this:
1. Pre-configure all antenna-nodes with static IPs and knowledge of which
N750 node is their parent.
2. Don't bridge the ethernet and wifi interfaces on antenna-nodes and
instead run babel on them.
I like solution 2 much better since it's easier for both us and node
operators. Not sure if we'll see a performance hit if we don't bridge. We
have a few nanobridges though so we could easily test this.
What do y'all say?
PS: Obviously babel channel diversity doesn't apply to antenna nodes since
babel sees them as ethernet interfaces but since they are mostly
directional and far away from the N750 it is fine to treat them as having
no interference, which is the default for ethernet interfaces anyway.
PPS: Did you know that the default STP delay when adding an interface to a
bridge is apparently 30 seconds? Not sure if OpenWRT has a different
default or maybe uses RSTP (rapid spanning tree protocol) to deal with
this, but if not then it is something to be aware of in order to now go
insane when troubleshooting. From:
http://www.linuxfoundation.org/collaborate/workgroups/networking/bridge#Doe…
--
marc/juul
Hi Mitar (and anyone else really)
I'm trying to figure out how to disable just 802.11b but not 802.11g. I
know how to enable greenfield mode, but not how to just disable 802.11b
Also, a few other questions:
If I enable short guard interval, what will that do for 802.11g? Do you
recommend using it for the public "give wifi to the people" access points
in an urban environment where there will still be some older 802.11g
clients?
Would you enable 40 mhz mode for these public access points as well? If I
understand correctly then they will still be able to use 20 mhz for 802.11g
devices and 40 mhz for 802.11n devices. Correct?
If I have a 1x1 802.11n node, like a Bullet M2, will it actually help to
enable TX-STBC or RX-STBC1 ? My understanding is that RX-STBC1 will still
give an improvement but that TX-STBC will do nothing (or maybe even reduce
performance?) if you have only one transmit antenna. Is this correct?
I'm not sure I understand what DSSS_CCK-40 does? It enables support for
this mode, but under what circumstances will it actually be used?
--
marc/juul
I thought I understood how route selection works: Most specific route is
always selected.
So if you have a route for 10.0.0.0/8 and another for 10.0.0.0/24 then the
/24 will be selected when the destination falls within the /24
This works as expected.
However, if I add a /32 route e.g. 10.0.0.42/32 then it gets prioritized
_lower_ than the /8 and /24
Why does this happen and is there anything I can do about it?
Example:
> ip route add 10.0.0.42/32 src 10.0.0.1 dev eth0
> ip route add 10.0.0.0/24 src 10.0.0.1 dev eth0
> ip route
10.0.0.0/24 dev eth0 proto kernel scope link src 10.0.0.1
10.0.0.0/8 dev eth0 proto kernel scope link src 10.0.0.1
10.0.0.42 dev eth0 proto kernel scope link src 10.0.0.1
Aaaa! Why is the /32 all the way at the bottom?
In other news: Made some progress on extender node stuff tonight. I decided
to use VLANs to put both the open and adhoc networks on the extender nodes
and have updated the configs for tp-link and extender node accordingly.
--
marc/juul
Hey so I'm trying to debug some slightly strange tunneldigger behaviour and
thought I'd check to see if anyone here has any thoughts.
This page shows ping times to a few mesh nodes from a VPS monitor server:
http://192.241.217.196/smokeping/smokeping.cgi?target=Mesh
Both MaxbMyNet1 and MaxbMyNet2 show a consistent increase in ping times
starting Monday (5-25-15) at like 11am or so.
MaxbMyNet1 has a direct ethernet connection to the internet and is
tunnelling to the exit server, while MaxbMyNet2 does not have any ethernet
connection and is instead connecting to the internet through MaxbMyNet1.
If I ssh into MaxbMyNet1, I can see that the l2tp0 tunnel is correctly
setup and that tunneldigger seems to be working correctly:
root@my:~# ps | grep tunneldigger
9538 root 5296 S /usr/bin/tunneldigger -u Sudomesh-MyNet-2 -i
l2tp0 -t 1 -b 104.236.181.226 8942 -L 20000kbit -s /opt/mesh/tunnel_hook -I
eth0.1
root@my:~# ip addr show l2tp0
18: l2tp0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1438 qdisc htb state
UNKNOWN group default qlen 1000
link/ether da:d8:46:b7:d7:9b brd ff:ff:ff:ff:ff:ff
inet 100.64.3.1/32 scope global l2tp0
valid_lft forever preferred_lft forever
inet6 fe80::d8d8:46ff:feb7:d79b/64 scope link
valid_lft forever preferred_lft forever
root@my:~# ip addr show eth0.1
11: eth0.1@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue
state UP group default
link/ether 00:90:a9:0b:73:cb brd ff:ff:ff:ff:ff:ff
inet 192.168.13.37/24 brd 192.168.13.255 scope global eth0.1
valid_lft forever preferred_lft forever
inet 192.168.0.102/24 brd 192.168.0.255 scope global eth0.1
valid_lft forever preferred_lft forever
inet6 fe80::290:a9ff:fe0b:73cb/64 scope link
valid_lft forever preferred_lft forever
Even more strangely, I can ping the world-routable IP of the exit server
and get back ping times consistent with the lower line of the graph:
root@my:~# ping 104.236.181.226
PING 104.236.181.226 (104.236.181.226): 56 data bytes
64 bytes from 104.236.181.226: seq=0 ttl=52 time=14.670 ms
64 bytes from 104.236.181.226: seq=1 ttl=52 time=14.264 ms
64 bytes from 104.236.181.226: seq=2 ttl=52 time=13.241 ms
64 bytes from 104.236.181.226: seq=3 ttl=52 time=13.949 ms
64 bytes from 104.236.181.226: seq=4 ttl=52 time=13.626 ms
64 bytes from 104.236.181.226: seq=5 ttl=52 time=18.133 ms
64 bytes from 104.236.181.226: seq=6 ttl=52 time=13.531 ms
And if I manually specify ping packets to go over the eth0.1 interface and
NOT the l2tp0 interface they have low ping times:
root@my:~# ping -I eth0.1 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: seq=0 ttl=55 time=21.834 ms
64 bytes from 8.8.8.8: seq=1 ttl=55 time=16.872 ms
64 bytes from 8.8.8.8: seq=2 ttl=55 time=19.764 ms
64 bytes from 8.8.8.8: seq=3 ttl=55 time=17.265 ms
64 bytes from 8.8.8.8: seq=4 ttl=55 time=16.989 ms
64 bytes from 8.8.8.8: seq=5 ttl=55 time=18.188 ms
However, if I ping over the tunnel and through the exit server I get the
slower times:
root@my:~# ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: seq=0 ttl=56 time=28.958 ms
64 bytes from 8.8.8.8: seq=1 ttl=56 time=29.211 ms
64 bytes from 8.8.8.8: seq=2 ttl=56 time=28.965 ms
64 bytes from 8.8.8.8: seq=3 ttl=56 time=29.022 ms
And then, weirdly, restarting tunneldigger on the MyNet seems to have fixed
it (look for the new line that will proably start around 16:00 on Monday
which will be at the lower time).
Thoughts? I'll keep taking a look at it, and it's possible it has something
to do with our up hook on the exit server which adds the new l2tp interface
to babel, but wanted to put it out there in case anyone had any ideas.
Max
So I built the tunneldigger package that Alex was working on. He just
simply added an option for binding to a specific interface when attempting
to create a tunnel. I built it and tested it on the two MyNets in my house
(one with a direct ethernet connection to the internet and one without) and
they are both working exactly the way I expected! Hooray!
I had to make a couple changes to our nodewatcher-firmware-packages (the
repo for openwrt package makefiles for wlanslovenija code) and do a few
other sort of repo architectural things. It shouldn't break anything, but
technically the sudowrt-firmware repo will need a rebuild. Should be an
excellent opportunity to test out the "rebuild" script eh?
For Tunneldigger we're still going to need a more permanent fix when we
want to use a domain name instead of IP addresses. The DNS lookup is what
was causing us all of that churn, so we'll want to re-write it to avoid
that particular step.
Lemme know if I missed anything?
Thanks!
Max
It's 5 am and I'm just about to sleep
I've been tweaking sudowrt-firmware and makenode and I now have a working
setup with a tp-link wdr3500 running the home node firmware and a bullet m2
running the extender node firmware that are talking to each other with
notdhcp correctly without any manual configration after flashing.
I took the liberty of adding max's ssh pub key (same one as sudoroom.org)
to the extender node firmware since there's really no other way to get into
it during development. Alex feel free to add your own as well.
The home nodes have a "for development only" IP of 172.22.0.1 and the
extender nodes have 172.22.0.2
SO! Just some notdhcp hook script wrangling left and then I'll get the new
web gui finalized enough that it can replace the luci gui, and maybe
someone can package it into sudowrt-packages? Then that's it! Ready to go
for a usable 0.2.
Oh btw, I was wondering how we should parcel up the eternet port. Here's
one option:
Port 0: The mesh
Port 1: The private network
Port 2: An extender node
Port 3: An extender node
Or we could do port 0 and 1 on the mesh both? Whaddaya think? We can make
them dymanically switchable from the web gui in next version.
--
marc/juul
Hey so is anyone planning on being at sudo this Sunday? There are probably
some things we can do, though it will depend a bit on whether Will will be
thre or not.
I went to the ham store, but they definitely didn't have any affordable
mounts that fit our purposes. That being said, I actually think it'd be
pretty trivial to make a mount ourselves with a bit of angle iron and a few
bolts. I don't imagine we'll have the time this Sunday to do that, but if
anyone were to be excited about it, some group could go in that direction.
There is plenty of cabling and setting up routers to do, though. Who's
coming with me....?
I changed the default IP in sudowrt-firmware and makenode
All instances of 192.168.13.37 have been changed to 172.22.0.1
Netmask is still /24
It was getting annoying to work with the routers when on a 192.168.0,0/16
network (such as the one in sudo room).
--
marc/juul
Meshmap is one of the last things remaining on the old server. Both
Nodeshot and its Django backend are running code from 2012 with local
modifications. Can't update any single thing without making the rest
incompatible.
Anyone else wanna take this one? :)
