Mesh/MeshApps

From Sudo Room
Revision as of 13:18, 13 May 2014 by Jwentwistle (talk | contribs) (added filesystem information)
Jump to navigation Jump to search

What sort of services would we like to integrate with the network? See also about local server choices.

Services

The network should provide secure communication services (email, contacts, calendar, phone) to create more incentive to be on the network. The services can be integrated into a user account distributed database with master-master replication and low latency (ex. Cassandra). These services will use existing open source projects.

Storage / File Systems

There are several options for a distributed file system and each have there advantages and disadvantages. The distributed file system can be used to offer file storage or reduce latency across the network.

  • Lustre - Used by sixty percent of the top 100 websites.
    Performance: Amazingly fast! I can assert that Lustre can serve a lot of streams and that encoding speed is not affected by accessing files via Lustre.
    POXIS compatibility: Very good!. No need to modify applications to use luster.
    Replication, Load Balancing and Fail Over: Very bad!. For replication load balancing we and fail over we need to rely on other software such as virtual IPs and DRDB.
    Installation: The worst!. Impossible to install by mere mortals. Requires a very specific combination of kernel, lustre patches and tweaks to get it working. And current luster patches usually work with old kernels that are incompatible with new hardware/software.
  • MogileFS
    Performance: Good for small files but not usable for medium to large files. This is mostly due to HTTP overhead since all files are send/receive via HTTP requests that encode all data in base64 adding a 33% overhead to each file.
    POXIX compatibility is non existent. All applications require to be modified to use mogilefs that renders it useless for streaming/encoding since most streaming servers and encoding tools do not understand MogileFS protocol.
    Replication and failover out of the box and load balancing can be implemented in the application by accessing more than one tracker at a time.
    Installation: Relatively easy and ready to use packages exist in most distributions. The only difficulty I found was setting the database master-slave to eliminate the single point of failure.
  • GlusterFS
    Performance: Very bad for streaming. I cannot reach more than a few Mbps in a 10Gbps network. Clients and Server CPU skyrockets on heavy writes. For encoding works because the CPU is saturated before the network and I/O.
    POXIS: Almost compatible. The tools I use can access gluster mounts as normal folders in disk but in some edge cases things start causing problems. Check gluster mailing lists and you will see there are a lot of problems.
    Replication, Failover and Load balancing: The best! if they actually worked. Gluster is very new and it has a lot of bugs and performance problems.
    Installation is too easy. The management command line is amazing and setting replicated, striped and distributed volumes among several servers can not be any easier.

([http://stackoverflow.com/questions/916902/lustre-gluster-or-mogilefs-for-video-storage-encoding-and-streaming 

source of above information])

XtreemFS - needs more research.

Databases and Servers

  • Cassandra - a decentralized database that has benchmarked with the lowest latencies and has master-master replication.
  • CouchDB - I can tell you that it quickly becomes complicated to write multi-user app with CouchDB. Decent access control means that you need a database per user and likely a few extra databases. You then have to set all of these databases up to synchronize between each other which is further complicated by the limitations on what information is available by the limitations on how synchronization rules and filters can be written. There seem to be no good tools for managing/visualizing these complex relationships so you end up having to roll your own.
  • PouchDB - a javascript based couchDB.
  • Scuttlebutt - a data synchronization protocol, or a peer-to-peer replicatable data structure.
  • TahoeLAFS - a decentralized database.
  • LevelDB - a fast and lightweight key/value database library.
  • MongoDB - an open-source document database, a NoSQL database. Written in C++.
  • Node.js - an HTTP server written in javascript.


  • Radicale - a CalDAV (calendar) and CardDAV (contacts) server. (python)
  • Village Telco - is an initiative to build low-cost community telephone network hardware and software that can be set up in minutes anywhere in the world. No mobile phone towers or land lines are required.


  • MaidSafe - a secure communications platform (charges money)
  • OpenMAMA - a high performance middleware agnostic messaging API that interfaces with a variety of message oriented middleware systems
  • an encrypted asynchronous messaging system to replace email
  • a module to have collaborative documents, voting, and groups (or sub-groups)
  • a module for peer to peer file sharing.
  • a module for object-orientated tagging, shareable feeds.
  • a module for feeds and posts, such as Trsst.

Utilities and Misc.

Mapping


Personas

  • Curator [ Individual who cares about data being right and available for the community ]
  • Analyzer [ Wants to visualize data in context, overlays of data, how data changes ]
  • Organizer/Activist [ wants to organize people around an issue, actionable outputs ]
  • Consumer [ gets info from the system, but doesn’t give back ]
  • Broadcaster/Reporter [ radio, streams, news, users/organizations who push into the system a lot ]

Use Cases

Leisure, Treasure Hunt, Animal Spotting, History / Science Plotting , Art/Design/Drawing, Mapping Secret/Weird stuff, Expression, Emotion

  • Jordan is a 36-year old scientist and queer activist. Occasionally, he travels north to a 'radical fairy' retreat center on over 80 acres of land. He's undertaking a project to map out the diverse and undocumented biology and wildlife in this sacred area. For this, he would like a tool that would enable him to automatically map the coordinates, attach a picture, and include a comment about what it is, as well as the ability to add to social media (Facebook & Twitter).

Bulletin Board / Local Information sharing (jobs, events, etc) (people to people)

  • Martin is a 35-year old Latino man who runs a website for displaying new job postings and opportunities for diverse, underserved populations in Oakland (eg; youth, veterans, and formerly incarcerated). Martin would like to easily transpose this data onto a searchable map that he could populate daily with new opportunities and programs.

Sensors: hardware-based [inputs & outputs]

  • Naomi is a Midweatern 30-year old environmental activist concerned about the impact of fracking on the quality of water in her county. She's organizing a campaign to encourage fellow concerned homeowners to monitor changes and contaminants in air and water quality, seismic activity, radiation, and other potential environmental impacts of fracking.

Evidence Based Citizenship (crowdsourced data/reporting for an institution to solve, neighborhood to be aware of) (people to institutions)

  • Alyssa is a 35-year old African-American woman working for the city government. Passionate about diversifying and expanding civic participation, she's been participating with Open Oakland to make civic data more accessible. She's working on an educational campaign for creating short videos explaining civic issues and how local government works.

Political Crisis Readiness (people to people, against institutions)

Contexts

Community Asset Mapping / Archiving

  • Kiara is a 34-year-old African-American community organizer and activist interested in mapping the various organizations addressing digital divide issues, as well as available resources such as public computers, free training and education, and free wifi spots.
  • Molly is a 26 year old community organizer working on Oakland Wiki, a website editable by anyone and dedicated to everything Oakland. She recently announced the launch of an oral history project, the end product of which would ideally be mapped onto specific areas and landmarks.

Community Hub (communications / sensors / input + output APIs )

  • Ali is a 28-year old Iraqi-American and works to catalyze community organizations in the Middle East. He is currently collecting stories from hackerspaces in the form of comics, as well as a 'Sister Spaces' project to partner hackerspaces for sharing insights and infrastructure challenges.
  • Luke is a 33-year-old Australian data and open government geek. Among the mappable projects he's working on with Open Oakland are storm drains (the Adopt-A-Drain project), crime data, vacant and blighted properties, and organizations and resources addressing the digital divide.

Organizing Tool (activists, security matters)

  • Mateo is a 32-year old Latino father living in the San Antonio neighborhood in Oakland, CA. He is interested in the capabilities of a mesh network for streaming Creative Commons-licensed content to his neighbors (possibly using an Asterisk server) and providing a means for connecting the community both digitally and physically through an online neighborhood bulletin board. He has a strong interest in reaching out to local businesses with an 'everybody wins' model of free wifi through a mesh, with a map-based splash page for communicating with others in the network.

Crisis Readiness

  • Jen is a 30-year-old Asian-American technologist and community organizer in East Oakland. Passionate about DIY education and grassroots community-building, she's interested in mapping DIY communities as well as available resources for crisis readiness (such as relief centers, tools, food, water and shelter).
  • Alexis is a 29-year old African-American technologist interested in the potential for mesh networking to expand internet access in underserved areas and creating a resilient communications network for crisis readiness. She is especially passionate about issues surrounding access to technology, and would like to see underserved communities participating in putting themselves on the map.

Play, games, fun uses

Retrieved from "https://sudoroom.org/mediawiki/index.php?title=Mesh/MeshApps&oldid=8024"