Oh look: It's another very long email from juul!
I've been working on the config for the N750 + antenna-node configuration
and here are my thoughts so far.
We should _not_ let the LAN ports be one big ethernet interface for use
both as a wired version of open0 _and_ a way to connect antenna-nodes. We
never want to bridge two interfaces with attached antenna-nodes but
treating the multiple LAN ethernet interfaces as one interface is
effectively the same as bridging.
Example scenario: You have to nanobridges on your roof linking you to two
parts of the mesh. They are both plugged into LAN ports on your N750 and
since the LAN ports are treated as one interface then the two nanobridges
are able to communicate on layer 2. The nodes at the houses to which you
are connecting with your nanobridges have the same setup. Now we
effectively have the spanning tree protocol as our mesh protocol instead of
babel.
It was Alex who pointed this out when we were talking about bridging but
for some reason I hadn't connected the dots and recognized that the same is
true when telling the built-in switch to treat the interfaces as one.
This means that each physical port on the N750 that we wish to connect to
an antenna-node must be its own interface and should have a /32 netmask.
These interfaces can still have the same node IP as open0, etc, without any
problems.
I suggest we allocate two of the four ports for this purpose until we can
make something that intelligently changes the config when an antenna-node
has been connected.
We now have two remaining LAN ports that can act as a single interface. We
could then bridge this LAN interface to open0. However, we want to avoid
channel interference when sending from/to open0 but there is no reason we
should try to avoid channel interference for traffic coming from the wired
interfaces. If we bridge them we cannot treat them differently, so we
should not bridge.
It is not clear if babel channel diversity also takes into account channel
information for manually published routes such as the open0 route we
currently publish with this rule:
redistribute if open0 metric 128
I'll have to look at the source code to check. If the functionality is not
there then it should be fairly easy to add.
Aaaanyway: Since we don't want to bridge open0 and LAN, this complicates
things because we then need two subnets per node (otherwise we have the
same /26 on both LAN and open0, which is not going to work). Less than a
/26 on either interface gets iffy, so it seems like we'll need to have two
/26 subnets per node now.
One last but important thing I realized: If the antenna-nodes have their
wifi and ethernet interfaces bridged, then we will have problems. Imagine
the following setup:
(N750 A) ------ (nanobridge A) ~~~~ (nanobridge B) ----- (N750 B)
Where "-----" means ethernet and "~~~~" means wifi.
If both nanobridge A and nanobridge B are simply bridging their ethernet
and wifi interfaces, then we have the following problems:
1. If e.g. nanobridge A sends a DHCP request, then it will be received by
both N750 A and N750 B and they will have no reliable way of know if the
request was sent by "their" nanobridge or the remote nanobridge.
2. When managing e.g. nanobridge A via the N750 A web admin interface it
will be impossible for nanobridge A to know whether it should grant admin
access to N750 A or N750 B.
I can think of only two solutions to this:
1. Pre-configure all antenna-nodes with static IPs and knowledge of which
N750 node is their parent.
2. Don't bridge the ethernet and wifi interfaces on antenna-nodes and
instead run babel on them.
I like solution 2 much better since it's easier for both us and node
operators. Not sure if we'll see a performance hit if we don't bridge. We
have a few nanobridges though so we could easily test this.
What do y'all say?
PS: Obviously babel channel diversity doesn't apply to antenna nodes since
babel sees them as ethernet interfaces but since they are mostly
directional and far away from the N750 it is fine to treat them as having
no interference, which is the default for ethernet interfaces anyway.
PPS: Did you know that the default STP delay when adding an interface to a
bridge is apparently 30 seconds? Not sure if OpenWRT has a different
default or maybe uses RSTP (rapid spanning tree protocol) to deal with
this, but if not then it is something to be aware of in order to now go
insane when troubleshooting. From:
http://www.linuxfoundation.org/collaborate/workgroups/networking/bridge#Doe…
--
marc/juul
Hey just wanted to mention that I was having an issue flashing the MyNets
with the web UI. I would select a firmware and then select "upload" and
instead of redirecting to the next page where it shows a countdown until
the device is supposed to be ready, the page just sat and attempted to
reload with no effect.
I don't really know why the hell it might be doing this, but it does seem
that using a private browsing window works. Maybe some weird session
thing.....
Anyways - now that I've got the firmware on this device I'm probably not
going to want to flash it again a bunch of times just to test if this
works, but if anyone else wants to check in and let us know if that seems
to work that'd be great.
Thanks!
Max
This seems like an interesting but potentially solvable inefficiency:
https://github.com/sudomesh/makenode/issues/9
Does someone have interest in looking into babel to identify what would be
needed to ensure that the same radio is not used for both inbound and
outbound when relaying (in situations where this is viable) ?
Since most nodes in the network will be dual-radio or better, then this
feature might really make a big difference in the long run, since the delay
incurred for each hop can be greatly reduced when you can receive and
transmit at the same time.
marc/juul
Hey Jenny,
I really wish you wouldn't have publicly walked back April's statement like
this. A number of people have different ideas for the next directions for
this project, and I think one of the things we've realized lately is that
when they don't align, we rely on eachother's good judgement and
communication in order to not shout eachother down. In fact, I don't really
agree with your statement:
Namely, we're not intending to create a free-as-in-beer, fast, high-speed
> network, but rather the focus is on cultivating a community of participants
> sharing their existing bandwidth with a public, open network and developing
> local applications and services- all built by and for the people with
> resiliency in mind.
I, for one, do intend in part to create a fast high-speed network which
would allow folks to join with a sliding scale membership. I don't really
think that anything in the grant proposal that April wrote up fundamentally
contradicts our values or motives, so I'd prefer that if you have your own
constructive additions you voice them without attempting to cancel out
anyone else's.
Thanks
On Fri, Mar 13, 2015 at 12:30 PM, Jenny Ryan <tunabananas(a)gmail.com> wrote:
> Hi Laura!
>
> Please use Marc's reply for the update - the grant application isn't a
> totally accurate portrayal of our goals, current progress and next steps.
> Namely, we're not intending to create a free-as-in-beer, fast, high-speed
> network, but rather the focus is on cultivating a community of participants
> sharing their existing bandwidth with a public, open network and developing
> local applications and services- all built by and for the people with
> resiliency in mind.
>
> Thanks for putting this together!
>
> On 03/13/2015 11:23 AM, Laura Turiano wrote:
> > Thanks Marc and April. This is so exciting and you are really being
> > thoughtful about the build out and testing.
> > Laura
> >
> > On 3/13/15 7:07 AM, April Glaser wrote:
> > > Hi Laura,
> > >
> > > To clarify, the $40 node isn't $40 a month for an Internet connection.
> > > It's the price of the antenna set up to connect to the network.
> > >
> > > So we're still working out the details on how we will distribute
> > > access to the network. Some parts of Oakland still report that less
> > > than 50% of residents have access to a reliable Internet connection.
> > > In those areas we might experiment with other methods of distribution.
> > > To that end, we're doing outreach to local organizations and the like
> > > to discuss how to best expand the network.
> > >
> > > Here's a description of our work that we used recently to apply for a
> > > grant, in case this also helps.
> > >
> > > *What's your project? What's below is about 30 words over. Can someone
> > > tighten it up?*
> > > We are a homegrown, community-owned network in Oakland, California
> > > working toprovide free Internet access at faster speeds than
> > > traditional providers.
> > > The People's OpenNetworkis dedicated to the idea that our community
> > > must have a central role in theoperation of ourcommunications
> > > networks. That is why we are building our own free network that
> > > provides high-speed, open access to the global Internet, while hosting
> > > local applications and services crafted by and for users in the East
> Bay.
> > > We are committed to universal, equitable, and unfettered access, free
> > > of unwanted surveillance and censorship. People's Openuses Sudo Mesh
> > > firmware, a free software project developed by
> > > volunteersenablingrobust, non-hierarchical mesh networks. The firmware
> > > provides a simple way for users to share a configurable portion of
> > > their Internet connection with the network – made relatively safe
> > > because the traffic from the shared connection is notassociated with
> > > the donor's IP address. Sudo Mesh is distinct from other opensource
> > > firmwaresbecause we prioritize both sharing andprivacy.
> > > Every aspect of People's Open is participatory, and every week we host
> > > three open meetings, including firmware development and community
> > > organizing. We're collaborating with local organizations and our
> > > diverse neighbors to co-create the network.
> > > *What assumptions will you test? *
> > > We hope to establish that a small-scale community-run network can
> > > provideservices currently assumed to be the province of large,
> > > top-down Internet providers. Specifically, we are exploring how local
> > > media and applications can be developed on such a network tobenefit
> > > local users. Examples of services include a local bulletin board,
> > > grassroots journalism outlets, local Voice-over-IP, archives,
> > > community asset maps, and Internet radio, all hosted on our local
> > > network.
> > > Instead of paid-subscribers, our goal is to have active participants.
> > > To that end, we currently offer training and hold open hack-nights at
> > > Oakland community hackerspaceSudoroom. Topics range from cryptography
> > > and network administration to antenna design and firmware hacking. We
> > > are actively designing our network through conversations with
> > > community partners, like Media Alliance, AspirationTech,
> > > ICSI/UCBerkeley, and others as part of our long-term outreach strategy.
> > > Access to the Internet is a human right, and we oppose practices that
> > > corner users into paying exorbitant rates to get online. Our network
> > > offers a free connection to anyone within range or willing to host a
> > > node. We challenge the idea that users need to trade personal data to
> > > engage with their community online, and People's Open encourages our
> > > neighbors to support other free software projects.
> > > *Who is the audience/user of this project? How will they be impacted? *
> > > The current model of Internet distribution in the East Bay isn't
> > > working for everyone. Many neighborhoods continue to report that over
> > > fifty-percent of residents lack a reliable home Internet connection.
> > > People's Open is partnering with community anchors, like churches,
> > > neighborhood gardens, schools, small businesses, and libraries, to
> > > mount antennas in underserved neighborhoods.
> > > This is a community network, and we're working with our neighbors to
> > > build and maintain it collectivity. We meet with local leaders and
> > > invite our neighbors to participate, fostering collective expertise
> > > and helping to ensure sustainability of the network.
> > > We are building a captive portal that directs users tolocal
> > > applications, such asa community calendar, grassroots media, maps, and
> > > bulletin boards. Working directly with activist groups to co-design
> > > trustworthy platforms, we're exploring ways to host local social media
> > > and digital classrooms. We are also in conversation with branch
> > > libraries and social service organizations about hosting information
> > > directories on the network. People’s Open is a grassroots media
> > > project, and we want to help meet the information needs of our
> > > communities.
> > > In sum, we provide a faster connection to the global Internet than a
> > > traditional residential ISP, while strengthening our community’s
> > > relationship with technologies that we depend on everyday.
> > >
> > > *What have you made so far? *
> > > For the past year and a half, most of our work has gone into
> > > developing and testing our firmware, which is a heavily modified
> > > version of OpenWRT. Our sources are on GitHub
> > > (https://github.com/sudomesh/) <https://github.com/sudomesh/%29>and
> > > are available for other communities wishing to create a similar
> > > network. We are finally at a point where we can begin to offer a
> > > reliable networking service.
> > > We have also deployed two testbed networks, one in Omni Commons, a
> > > giant community center which houses Sudoroom, and one in West Oakland,
> > > consisting of routers running our firmware.
> > > People's Open also has a long-term outreach and communications
> > > strategy, with the goal that the network will bloom and remain
> > > responsive to our communities. Our outreach strategy focuses on three
> > > tiers: large organizational partners, community anchors like churches
> > > and small businesses, and neighborhood mapping. In working with
> > > existing and trusted community groups, we hope to invite their
> > > networks to join and participate.
> > > Finally, we have a dedicated team in the project for the long-haul. We
> > > have three open hack-nights a week at Sudoroom, a well-known community
> > > center, where anyone is welcome to get involved at every level of the
> > > project. We frequently welcome new participants and keep our website
> > > up-to-date.
> > >
> > >
> > >
> > >
> > > On 3/13/15 3:04 AM, Marc Juul wrote:
> > >> On Thu, Mar 12, 2015 at 3:13 PM, Laura Turiano<scylla(a)riseup.net>
> > >> wrote:
> > >>
> > >>> Hello meshers,
> > >>>
> > >>> I'm writing an update for Oaklandish about the Omni and would like to
> > >>> include info about progress on the mesh network. Can someone tell me
> > >>> how
> > >>> many nodes have been installed, any other accomplishments, what are
> the
> > >>> next steps, etc.?
> > >>>
> > >> Hi. Here are my thoughts. Other mesh folk, please correct or expand
> > >> as you
> > >> please.
> > >>
> > >> Next week we're activating a test network at the Omni to test our
> indoor
> > >> nodes in an apartment-complex-like setting, as well as a small six
> node
> > >> high-speed roof-to-roof network in west Oakland. We have been working
> > >> on a
> > >> new, friendlier, web admin interface as well as features that will
> allow
> > >> people to start out with an entry-level ~$40 node and upgrade their
> > >> coverage later by adding rooftop or street-facing nodes without any
> > >> extra
> > >> configuration.
> > >>
> > >> Over the next month or so we'll be stress-testing our two networks and
> > >> completing these new features. The next phase will be a beta release
> > >> where
> > >> we invite the adventurous to adopt nodes. I would like to see a beta
> > >> test
> > >> network with maybe 50-100 node locations. The beta test will be less
> > >> about
> > >> testing the technology (though there will be some of that) and more
> > >> about
> > >> understanding the problems and opportunities that arise when a diverse
> > >> group of people with diverse skill-sets have to run their own
> > >> network. If
> > >> the network is to succeed as it grows, then it cannot rely only on the
> > >> small group of volunteers that make up sudo mesh. We're going to have
> to
> > >> figure out how to communicate to node operators that this is not a
> > >> traditional ISP with a support line. Instead it is rather like a
> > >> community
> > >> garden where everyone helps out to make it succeed and we want so
> figure
> > >> out how to best facilitate that cooperation. During this phase we'll
> > >> also
> > >> be finalizing our automation tools for receiving orders for new nodes,
> > >> automatically configuring those nodes and shipping them out. Once
> we're
> > >> comfortable that everything is ready for a rapidly expanding network
> the
> > >> next phase might take the form of a large crowdfunding campaign where
> > >> people can get nodes as perks.
> > >>
> > >
> >
> >
>
>
> --
>
> Jenny
> http://jennyryan.net
> http://sudomesh.org
> http://thevirtualcampfire.org
> http://technomadic.tumblr.com
>
> `~`~`~`~`~`~`~`~`~`~`~`~`~`~`~`~`~`~`
> "Technology is the campfire around which we tell our stories."
> -Laurie Anderson
>
> "Storytelling reveals meaning without committing the error of defining it."
> -Hannah Arendt
>
> "To define is to kill. To suggest is to create."
> -Stéphane Mallarmé
> ~`~`~`~`~`~`~`~`~`~`~`~`~`~`~`~`~`~`
>
It seems that there is now preliminary support for nanobeam devices in
OpenWRT:
https://dev.openwrt.org/ticket/16796
So that's nice, but the second to last comment says that speeds are much
lower than for stock firmware. It could be that this person just didn't
configure his node correctly or it could be that this is the case for all
ubiquiti devices.
Lately we've been talking about the possibility of using non-openwrt
routers for point to point bridged links. We've heard from a few people
that performance of ubiquiti M5 gear for point to point links is basically
the same in OpenWRT and stock firmware.
I think we should test that ourselves. I'm ordering some nanobridges and
nanostations right now and we should do a few tests with point to point and
point to multipoint using openwrt and stock firmware.
It'd be nice to have benchmarks on our wiki. I can't find any existing
benchmarks online?
--
marc/juul
Right now makenode has configs that are trying to handle differences in
chipsets, frequency, etc. in a way that's agnostic of the actual router
model. That's cool, but maybe more trouble than it's worth (at least for
now).
Unless there are objections I'm going to change it so we have one
configuration per router model. I want us to get to a point, very soon,
where we have the following working and separate configurations:
* Nanobridge M5
** Bridging only
* Nanostation M5
** Bridging only (on both ethernet ports)
* Nanostation M2
** Bridging only (on both ethernet ports)
* Western Digital My Net N600/N750
** Full sudowrt firmware with the 5 GHz radio only having
pplsopen.net-node2node
The My Net has five ethernet ports. I'm thinking we can configure them like
so:
Port 1: To home DSL/Cable Internet connection (if any)
Port 2: Connect your local gear to the mesh (like peoplesopen.net ssid but
wired)
Port 3: Connect a Nanostation M2 which is basically like adding an antenna.
You'd use this to extend your coverage out onto the street (or you could
just add an actual antenna if that's feasible given distance between
routers and coaxial loss).
Port 4 and 5: Connect rooftop Nanobridge M5s or Nanostation M5s.
In the future we should consider making it possible to easily reconfigure
these, both at initial configuration time and through an "advanced" tab in
the GUI, but for now I think it's much easier to have ports dedicated to
each operation.
Here's how we'd use this in different scenarios:
# We're just relaying off someone's rooftop
We'd have two or more Nano(bridge/station) M5s on the roof and they'd just
connect to each-other. They're already bridging so no extra configuration
is needed.
# Someone just wants to be part of the mesh without paying a lot
They get a My Net N600 and hook it into their internet, if any.
# Someone wants a rooftop link and wants to use the mesh in their house
They get a My Net N600 and hook it into their internet, if any, and they
hook their rooftop-mounted Nano(bridge/station) into port 4 or 5 of the My
Net.
# Previous scenario but with more street-level coverage
They additionally hook in a Nanostation M2 to port 3 of the My Net or they
hook in one or more external antennas to the My Net using the internal u.fl
connectors. The second solution is definitely not something we should
encourage for people who aren't already comfortable with that level of DIY.
-----
One possible downside to this setup is that all of the CPU-intensive stuff
is handled by the My Net router and the other (very capable) devices are
just bridging. I think the solution is to accept this less-than-perfect
solution for now and rely on whatever future router we decide to use as a
My Net replacement to have a much faster CPU (this is probably a safe bet).
I should mention that Max and Adri today discovered that CPU _is_ a huge
limiting factor on the Picostation 2 routers, so if we put too much load on
the (granted much much faster, maybe 7-8 times faster) newer devices it may
become an issue again.
We just ordered five Western Digital My Net N750 routers for use in the
Omni test network, and I think the N600 version (same, but non-gigabit and
cheaper) is a good candidate for home-routers that we can give to people.
Thoughts?
--
marc/juul
On Thu, Mar 5, 2015 at 11:34 AM, yar <yardenack(a)gmail.com> wrote:
> It's at maybe the worst possible time (6pm tonight) but there's
> nothing we can do about it. We'll try to make it as quick & smooth as
> possible.
all done!
