26 Jun
2015
26 Jun
'15
4:37 p.m.
As I am rolling out a bunch of new babel nodes, I decided to get a
cluster (2 nanos and a pico) up in the lab, where I have good
connectivity to the rest of the network, to replace an aging cluster
by the pool.
So I booted it up and configured it for the right channels and a new
set of ip addresses... didnt have good LED support at all (RSSI does
not seem to do anything)...
I got blinkenlights to sort of work, and they were lit up, kind of
solid, for some reason... [1]
...people started wandering by to complain about the network...
naturally I didnt notice because I was even closer to the exit points
than anyone else...
...to discover that I was offering the shortest path to the exit
nodes, and thus had bypassed the two existing ~50mbit links into lab
links that were located indoors and going through a thousand+ meters
of trees... that was barely doing a megabit with 800+ms of delay.
(channel diversity not working did not help either)
After that experience, I decided that I would make the firmware for
unconfigured nodes export a 512 metric, and use a high rxcost until
they were fully configured AND in place. I might disable ipv4 entirely
in favor of the autoconfigured ula openwrt has, and just start
configuring stuff based on the appearance of new ulas in the network.
[1] if you come up with a useful LED config for nanostations and
picostations, let me know.
--
Dave Täht
worldwide bufferbloat report:
http://www.dslreports.com/speedtest/results/bufferbloat
And:
What will it take to vastly improve wifi for everyone?
https://plus.google.com/u/0/explore/makewififast
26 Jun
26 Jun
5:02 p.m.
New subject: [Babel-users] a cautionary note on setting up new babel nodes
...to discover that I was offering the shortest path
to the exit
nodes, and thus had bypassed the two existing ~50mbit links into lab
links that were located indoors and going through a thousand+ meters
of trees... that was barely doing a megabit with 800+ms of delay.
What was the issue? No multicast loss on the link? I find that difficult
to believe.
(channel diversity not working did not help either)
Not sure about that -- if computing the channel number fails, babeld will
assume that the link interferes with all other links, so the misconfigured
link should be disfavoured.
I'm sure you already know that, but until I get around to rewriting the
channel determination code for recent "we don't break userspace" Linux,
you can work around it with manual configuration:
interface wlan42 channel 42
After that experience, I decided that I would make the
firmware for
unconfigured nodes export a 512 metric,
Good plan, although I'd make it 32000. This way it'll be easy to spot an
unconfigured node in BabelWeb.
Note that if you do that, you should apply it at the metric level, not at
the
-- Juliusz
10:04 p.m.
New subject: [Babel-users] a cautionary note on setting up new babel nodes
On Fri, Jun 26, 2015 at 5:02 PM, Juliusz Chroboczek
<jch(a)pps.univ-paris-diderot.fr> wrote:
When you gradually engineer a wireless network to where it can do
50mbits or more, packet loss at the lowest rates becomes slight,
particularly when your stations have no interference (clear channels),
and your APs output a watt (picostations), or are rated for 50km
(nanostations) - even going through heavy foliage, I had long
suspected that most of my packet loss had become congestive in nature.
Most of the network is engineered not to find a best route
interactively, but to fall back to other nodes when one fails (hangs,
or is shut off). For example the longest path is always to the 2Ghz
radios (which are also used as APs), usually going through a
nanostation m5 over ethernet, to the next "hop" nanostations, always
on different channels, with the 2ghz radios having a higher rxcost
configured and diversity on, and *no* backbone per se to the main exit
points over 2ghz. (till this accident. I had only added ethernet to
the lab a few weeks before). So they (usually) just serve as a backup
to the 5ghz directional radios, and only carry traffic to stations and
leaf nodes.
In other news the nodes I am replacing (with exactly the same
hardware, recycled) are 3+ years old. With every improvement in
software, they got faster and more reliable, when I started I was
lucky to get 5mbit end to end (7 hops worst case, no diversity, and
bridged), to 12 (diversity with a bad link), to 20+ (when aggregation
started to work in adhoc), to the latest build where I am seeing as
much as 70mbit over 1000 meters of foliage.... (and wont know til I
deploy how well that worst link will do...)
It turned out I wasn't even running the final fq_codel on one critical node:
root@pool-pole2-omni:~# uname -a
Linux pool-pole2-omni 3.3.8 #1 Wed Sep 19 09:48:30 PDT 2012 mips GNU/Linux
tc -s qdisc show dev wlan0 # 6 hours of traffic
...
qdisc nfq_codel 30: parent 1:3 limit 1000p flows 1024 quantum 1000
target 5.0ms interval 100.0ms ecn
Sent 2265181234 bytes 1601340 pkt (dropped 3685, overlimits 0 requeues 85466)
backlog 0b 0p requeues 85466
maxpacket 0 drop_overlimit 0 new_flow_count 22621 ecn_mark 29
new_flows_len 0 old_flows_len 0
Still, I take great joy from seeing at least some drops and marks, on
wifi, and a ton of FQ (requeues). It needs MORE! :)
Yes, I will manually configure for this deployment. It would be
important to get right for battlemesh.
I will also check to see if it is working elsewhere on older systems.
(in linux´s defense, nl80211 was the future api for the last 4 years
and the other deprecated)
...to discover
that I was offering the shortest path to the exit nodes,
and thus had bypassed the two existing ~50mbit links into lab links that
were located indoors and going through a thousand+ meters of trees... that
was barely doing a megabit with 800+ms of delay.
What was the issue? No multicast loss on the link? I find that difficult
to believe. (channel diversity not working did not help
either)
Not sure about that -- if computing the channel number fails, babeld will
assume that the link interferes with all other links, so the misconfigured
link should be disfavoured.
I'm sure you already know that, but until I get around to rewriting the
channel determination code for recent "we don't break userspace" Linux, you
can work around it with manual configuration: interface wlan42 channel 42
Yes.
After that experience, I decided that I would
make the firmware for
unconfigured nodes export a 512 metric,
Good plan, although I'd make it 32000. This way it'll be easy to spot an
unconfigured node in BabelWeb.
Note that if you do that, you should apply it at the metric level, not at
the
I will kill the rxcost thing universally on the entire deployement
this week, however that is mildly simpler to set in the current
openwrt config file. I think there is no way to set a "default"
exported metric?
-- Juliusz
--
Dave Täht
worldwide bufferbloat report:
http://www.dslreports.com/speedtest/results/bufferbloat
And:
What will it take to vastly improve wifi for everyone?
https://plus.google.com/u/0/explore/makewififast
3772
days inactive
3773
days old
2 comments
2 participants
participants (2)
-
Dave Taht -
Juliusz Chroboczek