289
edits
Difference between revisions of "Mesh/MeshApps"
added filesystem information
(adds link to local servers page.) |
Jwentwistle (talk | contribs) (added filesystem information) |
||
| Line 2: | Line 2: | ||
=Services= | =Services= | ||
The network ''should'' provide secure communication services (email, contacts, calendar, phone) to create more incentive to be on the network. The services can be integrated into a user account distributed database with master-master replication and low latency (ex. [http://www.datastax.com/resources/whitepapers/benchmarking-top-nosql-databases Cassandra]). | The network ''should'' provide secure communication services (email, contacts, calendar, phone) to create more incentive to be on the network. The services can be integrated into a user account distributed database with master-master replication and low latency (ex. [http://www.datastax.com/resources/whitepapers/benchmarking-top-nosql-databases Cassandra]). These services will use existing open source projects. | ||
==Storage / File Systems== | |||
There are several options for a distributed file system and each have there advantages and disadvantages. The distributed file system can be used to offer file storage or reduce latency across the network. | |||
* [http://opensfs.org/lustre/ Lustre] - Used by sixty percent of the top 100 websites. | |||
*: Performance: Amazingly fast! I can assert that Lustre can serve a lot of streams and that encoding speed is not affected by accessing files via Lustre. | |||
*: POXIS compatibility: Very good!. No need to modify applications to use luster. | |||
*: Replication, Load Balancing and Fail Over: Very bad!. For replication load balancing we and fail over we need to rely on other software such as virtual IPs and DRDB. | |||
*: Installation: The worst!. Impossible to install by mere mortals. Requires a very specific combination of kernel, lustre patches and tweaks to get it working. And current luster patches usually work with old kernels that are incompatible with new hardware/software. | |||
* [https://code.google.com/p/mogilefs/ MogileFS] | |||
*: Performance: Good for small files but not usable for medium to large files. This is mostly due to HTTP overhead since all files are send/receive via HTTP requests that encode all data in base64 adding a 33% overhead to each file. | |||
*: POXIX compatibility is non existent. All applications require to be modified to use mogilefs that renders it useless for streaming/encoding since most streaming servers and encoding tools do not understand MogileFS protocol. | |||
*: Replication and failover out of the box and load balancing can be implemented in the application by accessing more than one tracker at a time. | |||
*: Installation: Relatively easy and ready to use packages exist in most distributions. The only difficulty I found was setting the database master-slave to eliminate the single point of failure. | |||
* [http://www.gluster.org/ GlusterFS] | |||
*: Performance: Very bad for streaming. I cannot reach more than a few Mbps in a 10Gbps network. Clients and Server CPU skyrockets on heavy writes. For encoding works because the CPU is saturated before the network and I/O. | |||
*: POXIS: Almost compatible. The tools I use can access gluster mounts as normal folders in disk but in some edge cases things start causing problems. Check gluster mailing lists and you will see there are a lot of problems. | |||
*: Replication, Failover and Load balancing: The best! if they actually worked. Gluster is very new and it has a lot of bugs and performance problems. | |||
*: Installation is too easy. The management command line is amazing and setting replicated, striped and distributed volumes among several servers can not be any easier. | |||
([http://stackoverflow.com/questions/916902/lustre-gluster-or-mogilefs-for-video-storage-encoding-and-streaming | |||
source of above information]) | |||
[http://www.xtreemfs.org/ XtreemFS] - needs more research. | |||
==Databases and Servers== | |||
* [https://cassandra.apache.org/ Cassandra] - a decentralized database that has benchmarked with the lowest latencies and has master-master replication. | |||
* [https://couchdb.apache.org/ CouchDB] - I can tell you that it quickly becomes complicated to write multi-user app with CouchDB. Decent access control means that you need a database per user and likely a few extra databases. You then have to set all of these databases up to synchronize between each other which is further complicated by the limitations on what information is available by the limitations on how synchronization rules and filters can be written. There seem to be no good tools for managing/visualizing these complex relationships so you end up having to roll your own. | |||
* [http://pouchdb.com/ PouchDB] - a javascript based couchDB. | |||
* [https://github.com/dominictarr/scuttlebutt Scuttlebutt] - a data synchronization protocol, or a peer-to-peer replicatable data structure. | |||
* [https://tahoe-lafs.org/trac/tahoe-lafs TahoeLAFS] - a decentralized database. | |||
* [http://code.google.com/p/leveldb/ LevelDB] - a fast and lightweight key/value database library. | |||
* [http://www.mongodb.org/ MongoDB] - an open-source document database, a NoSQL database. Written in C++. | |||
* [http://nodejs.org/ Node.js] - an HTTP server written in javascript. | |||
*[http://radicale.org/ Radicale] - a CalDAV (calendar) and CardDAV (contacts) server. (python) | *[http://radicale.org/ Radicale] - a CalDAV (calendar) and CardDAV (contacts) server. (python) | ||
*[http://villagetelco.org Village Telco] - is an initiative to build low-cost community telephone network hardware and software that can be set up in minutes anywhere in the world. No mobile phone towers or land lines are required. | *[http://villagetelco.org Village Telco] - is an initiative to build low-cost community telephone network hardware and software that can be set up in minutes anywhere in the world. No mobile phone towers or land lines are required. | ||
*[http://maidsafe.net/ MaidSafe] - a secure communications platform (charges money) | *[http://maidsafe.net/ MaidSafe] - a secure communications platform (charges money) | ||
| Line 36: | Line 75: | ||
* How about [http://www.frontlinesms.com/ Frontline SMS]! | * How about [http://www.frontlinesms.com/ Frontline SMS]! | ||
* [https://www.docker.io/ Docker] - an open-source project to easily create lightweight, portable, self-sufficient containers from any application. | * [https://www.docker.io/ Docker] - an open-source project to easily create lightweight, portable, self-sufficient containers from any application. | ||
=Mapping= | =Mapping= | ||