Introduction

Introduction

To use commercially available hardware along with open source software to create a reliable and redundant Neighbourhood wireless network using mesh routing technologies.
These neighbourhood meshes may be interconnected using routing or VPN over a connection such as the Air-Stream core network or a third party network such as the Internet.
By connecting members directly to other people in their local area, we create a much more community based network. We then have an opportunity to encourage people in the same community to interact and work together, whether it be for gaming, organising local events or sharing stories on the local history. These are all possible on the Internet but are often lost amongst the massive amount of data - in a local network the content becomes more personal.

Aims

• Mass producable (easily sourced components)
• Low price
• Compact and professionally presented unit package
• Minimal end user configuration
• Well presented documentation for the end user
• Quality documentation for the technical side
• Network design allowing for large scalability (subnet allocations and network backbones)
• Reliable end user security (so that home LANs are not at risk by joining the network)

Advantages

• Easily expand the network - a new node could be added in a matter of minutes.
• Reduce the likelihood of a tree or building blocking a user’s network access (as multiple paths for them to access the network).
• Roaming within a subnet would become possible.
• Adding a new node to the mesh will improve redundancy rather than simply dropping shared bandwidth.
• Encourages interactivity between local communities - much simpler than on the Internet due to its scale. This could develop into a working relationship with local councils - both in potential grants and in additional site locations.
• Increases the size of the network - there is a law stating a network is as useful as the square of its users.
• Having a larger membership base can only help in negotiating new sites for the core Air-Stream network.
• Helps to move Air-Stream into the more mainstream community - community wireless becomes accessible to those without extensive networking experience.
• Allows the end user network to grow more independently of Air-Stream allowing it to
focus on developing the core network.
• The ability to access the Air-Stream network, and thus also your home network from a drastically increased number of locations in the city.
• Greater node density allows for faster connections (such as 802.11g) between nodes, since each node is much closer than in the current client/ap setup.

Example Applications

A number of uses the network could be put to. This list will grow much larger with time.
Some of these are also possible on the current Air-Stream network, but would benefit from having many more members interacting on the network.
• Much larger user base leads to many more people available for gaming. Tournaments and championship ladders could be setup.
• Roaming around a subnet using a laptop or PDA.
• VoIP network becoming much larger and the ability to use wireless VoIP handsets.
• Local content able to be streamed across the network - for example a fair or other community event.
• A network of webcams located around the city.
• Local historical societies and schools working with interactive mapping software to create content based on their local area.

Overview of mesh

Overview of a wireless mesh

(Base on an entry from wikipedia entry or similar, and include diagrams).

Software Implementation

Software on WRT unit

OLSR running routing between devices
• Vpn to allow access to LAN side of device (off by default)
• Iptables to filter access to LAN (very strong defaults)
• Httpd to provide basic stats and user info
• Minimal uniform splash page – eg basic contact info for the node and what you can find on this network segment (list of services this user offers on their lan)
• A full httpd would be running elsewhere on this user’s network
• Linked into the OLSR plugin to provide OLSR network stats?
• Frottle or similar program to address the hidden node problem – research more extensively (eg melbwireless and openwrt forums)
• SSHd to allow for secure remote access to the device.

Software on additional network devices

Laptops and other portable devices would run the OLSR client, or hopefully get a DHCP assigned from the AP.
Captive portalling on the AP Would be sweet also if possible.

Logical (Networking) Implementation

Routing Protocol

OLSR is to be used as the routing protocol.
• Linux, Windows, Mac OS X and BSD (?) clients available.
• Open Source and well documented thus fits well in with other software (eg quagga and *nix) already used on the network.
• Documented support for OpenWRT and has been deployed on this platform internationally (eg www.freifunk.net).
• Would also be able to be installed on the WRAP boards.
• Able to be installed on a Linux/BSD platform so we are not limited in our choice of platform for the Zone Server.

Routing into and out of mesh - Static Workaround

A solution to the issue of getting the two dynamic routing protocols (OLSR and BGP/OSPF) to talk to each other is shown in the below diagram.

Routing into and out of mesh - Dynamic Solution

A better solution would be to integrate the kernel routing so that Quagga dynamically receives the information about the mesh, rather than there simply being a static route set on the Zone Server.
This will be preferable when there are multiple routes into and out of the mesh (which is optimal for redundancy).

Network Hardware

Business Node Routers

The WRAP boards would be well suited for business nodes and more technical orientated members.
• They are able to have two wireless cards onboard (as two Mini-PCI slots) making them suitable for the “supernode” type of setup.
• These would be preferable for business nodes as they are a more flexible platform than the WRT54G, in terms of storage space and OS support.

Client Node Routers

The Linksys WRT54G devices are ideally suited for the home nodes.
• They are low cost and easily available in shops, so sourcing the unit will not be an issue.
• They have a well developed Linux based OS (OpenWRT) which has support for a large number of packages.
• They have a proven track record in a number of networks internationally, both community based and commercial. Wildcat Wireless in the US currently have 30 of these devices running successfully in a mesh setup.

Antenna

The standard antenna to be used in the network would be a 4 slot waveguide. In use to date it has been shown potential in being able to meet our requirements.

Power over Ethernet (PoE)

Power over Ethernet takes advantage of the fact that only two of the four pairs in a Cat5 network cable are used for carrying network data. The other two pairs are left unused and can therefore be used to carry power to the device. This is possible since we are only dealing with low power levels, and saves us having to run an additional cable to the device for power.
This makes the total install simpler and neater.

Mounting hardware

Additional hardware needed to mount the antenna and weatherproof box, and also for other items such as the cable runs.

Weatherproofing unit

Need to decide whether to go for a professional solution (and if so which one) or whether to investigate using PVC downpipe sealed for the casing of the device.
Also need to consider whether to mount the unit as is inside the case or whether to remove the PCB board from the manufacturer’s casing.
Of course temperature and humidity considerations need to also be taken into account.+

Additional Hardware Options

Weather Station
• Weather devices
• May be possible to interface some basic weather measurement data to the serial interface of either the WRT54G board or the WRAP units.
• This would then allow us to plot city wide maps based on data collected from our own weather units.
Webcams
It would also be nice if we were able to interface the WRAP boards with USB webcams, providing another feature on the network.

Network Management and Visualisation

Httpinfo

The HttpInfo plugin for OLSR will be set up so that each node’s routing information is available to the rest of the network.

Dotdraw

The DotDraw plugin will be setup on the Zone Server to provide a live visual overview of the network.

Authentication

Introduction

• The network we will be building, with fast access and wide coverage, will be appealing to a number of non members. Authentication will likely be required to keep the available bandwidth for our members.
It is certainly not aiming to restrict the availability of the network to the community, but helping to ensure that those who wish the participate on the network are able to do so without having their access limited by people taking advantage of their generosity.
• If current solutions do not meet our needs then a possible solution is as follows.

Proposed solution

Step by step scenario:
1. User Joe walks up to a mesh member’s house with his laptop in hand. (Is the laptop running OLSR or just wishing for a NAT connection?)
2. Joe’s laptop is handed out an address (eg 10.0.0.100) via the mesh DHCP server.
3. Joe logs onto the zone server (which is a high performance machine capable of running apache, mysql etc) and provides his login details. This is possible since port 80 (and any other relevant port - eg for ssl) is automatically permitted by each mesh router.
4. If login accepted, the zone server then adds his ip to the relevant database. It also adds it to a web page visible to the whole network. Eg 10.0.0.1/accept.txt would then contain 10.0.0.100. This way other routers can check the authenticated user list.
5. Each router will periodically (eg every 5 minutes) check this list and update their local allow cache accordingly (eg just replace should be sufficient). If the router receives a packet from an ip address not in its list, it will then check the zone server’s allow list. When the accept table is updated then the iptables firewall is also updated by the relevant shell script.
6. When Joe has finished his session he can go back to the mesh web page and click a logout button which would remove his entry from the accept.txt file.

Additional Comments

• What do we want to do in the even a Zone Server is down - will this break the mesh
15
network’s operation or will it be so infrequent that it is not a major concern?
• Some devices which may not be able to log onto a web page (such as a VoIP handset) may need to be auto added. A solution to this would be to auto forward all VoIP ports (subject to abuse). This also solves the legal issue of providing access to the emergency numbers.

Testing

Air-Stream Open Day Testing

Set up a number of WRT54G routers running OLSR around the event.
The OLSR client could then be provided on a number of cds, allowing laptop users to install the client and test out the network.
Gives a good chance to test scalability and actually using the OLSR application, and encourage people there to interact with the network, rather than just looking at it.
Set up one of the x86 machines so that it can do live visualisation - so we can have a screen running all day that shows the network topology live.
Need to test out the x86 installs, configure the visualisation software and set up the WRT54G devices with the clients. Also need to source a few more WRT devices from members to use for the day.

Rooftop Trials

These will be carried out to establish the real world performance of antenna combinations.
A combination of antennas may also be used such as:
End A (WRT)
End B (Laptop)
4 Slot waveguide
Internal PCMCIA Card
8 Slot waveguide
4 Slot Waveguide
8 dBi Omni
8 Slot Waveguide
8 dBi Omni
1
5dBi Dish
Cantenna

Trial network

It is likely that a trial mesh network will be carried out in the suburbs of Daw Park/Melrose Park and a number of surrounding suburbs.

Plotting Coverage

• Nodes can be plotted on a 2D metropolitan map to provide a view of the physical layout of the network.
• A GPS unit could be used with a laptop to provide a rough idea of real world network coverage.

Additional Questions

These will be need to be answered in future revisions of this document.
• Can the software run in the proposed hardware?
• How easy will it be to deploy and configure the software – see German group who offer a firmware compile based on web page form entries for settings.
• How scalable is it to our needs – would 20-30 nodes in a suburb be fine? – see overseas uses
• What IP ranges are we to use?
• Routing issues?
• Can we add backbones from one side of the mesh to the other to help distribute traffic (eg on a different channel and acting solely as a backbone) – will OLSR scale to this?
• What antennas to use – waveguides are cheap and easy to mass produce and horizontal polarization is preferable but is a 4 slot going to perform ok as 8 slots are quite large.
• How badly will trees kill our signal or conversely will there be too much noise from other devices to make it unusable?
• We would ideally want 250m – 500m coverage for a decent network initially to get the critical mass going.
• Will 54mbps work over some links (eg 802.11g going to run over <1km links alright?)
• Can client connect not running OLSR (routing issues – use NAT in this case – check more in mailing lists)
• Vertical spread of the 4 slot waveguide – eg can someone with one on their roof use their laptop in their backyard to access their home lan?
• How good is OLSR – eg will it meet our needs – check with German etc trials
• Possible to have multiple paths out of the network? How would this go with the routing mixes… eg one supernode connected to Pasadena and another connected to Melrose Park, both could offer a default route out of the network…. Could be an issue that we will just have to bear with later. Preferable to have multiple routes out but cannot expect optimal path selection talking across OLSR and BGP/OSPF. Could be resolved in later
17 1km OLSR releases or could this come from the kernel routing tables on the gateway? Either way an additional hop through the core Air-Stream network is unlikely to cause any hassles and would be better than 8 hops through the wireless mesh, not to mention redundancy if one gateway were to be unavailable.
• How can we recognize local businesses which contribute by funding/hosting a node?
o aim for ads on the zone server page
o this would then mean the zone server would be the portal site for the nodes, eg this is their network home page rather than the main AS page. The AS page news could be fed into this site by RSS or whatever.
• Network updates and information over RSS?
• How will DHCP flow to a home user on the LAN side of the WRT? (check up on mailing list regarding all nodes needing to either run the OLSR client or they will not get routing info and thus must run through NAT) - eg run NAT as per normal home users with a shared DSL modem, and for more advanced users wishing to run a server can simply run the OLSR client which adds them directly into the Neighbourhood mesh subnet. This solves the problem of having to allocate subnets for every home lan and then deal with routing - simplifies each site's configuration
• How to then allow the user to select to use NAT (eg PDA, Voip phone) or OLSR (eg laptop wishing for roaming)?
• What happens when the zone DHCP server goes down - do we want a high redundancy network or is this a tolerable issue in the time it takes to fix the problem? Maybe allocate each of the static mesh nodes (eg those with waveguides on roofs) an ip via the website when they join up so they wont go down with the DHCP server.
• Can each home LAN have a seperate subnet or will that break the OLSR routing?
• How to handle authentication? (answered later)

Implementing the mesh

Overview

Our implementation of the mesh will be similar to the reference mesh model discussed above.
However given the geographical size and low population density of Adelaide, implementing a city wide wireless mesh would be very demanding on network resources - the routing tables would be immense and sending a packet from one side of the city to the other could involve over 100 hops!
Therefore this project will aim for a larger number of smaller meshes, consisting of rougly one to three suburbs in size. Some applications may extend this further, such as offering a single mesh to allow for seamless roaming down Belair Road (which would cover 5+ suburbs).
Smaller meshes mean that each mesh is kept to a managable level, whilst still offering the advantages of using mesh technologies.
Each of these smaller meshes would then use a “Zone Server” to interface with the core Air-Stream network. This is the gateway from the mesh to the rest of the network.

The Zone Server

The core of the local neighbourhood mesh is the Zone Server. This is the machine that would currently be the local access point, with backbones to remote APs located elsewhere in the network. This machine will handle authentication and provide a link to remote networks. It will also host services such as a local irc, IM and http server, which would be interlinked to other similar servers elsewhere on the network. By localizing these services we provide redundancy so that if a backbone were to fail, the local mesh could still function autonomously. This is currently not the case in a number of wireless networks, where if a backbone were to go down then the local users have no provided services to use.
Is it possible to have multiple zone servers (eg can we have multiple paths out of the mesh?) This would be optimal for redundancy in a network otherwise we have a single point of failure which may isolate the mesh.
• Hosts interlinked IRC and Jabber servers
• At reliable site – preferably business or experienced member
• Uses a more high powered card and waveguide combination mounted high etc to cover as much as possible of the local area. This ensures access to backbone is faster and less network demanding.
• Try to concentrate content locally rather than traffic traversing backbones needlessly.
• Approach community friendly local businesses to see if they would be interested in hosting a zone server or a supernode.

Current Local Wireless Design

Currently a local wireless network consists of an access point with an omnidirectional antenna to which nearby nodes connect to using directional antennas. This situation is fine for users with line of sight to the access point but means that those who are unfortunate enough to have a tree or building in the way are unable to get any access to the network.
This can be seen in the below example.

The mesh network acts by connecting the nodes to each other. This means that total bandwidth may be reduced, especially when packets are crossing from one side of the network to the other. The compromise is that nodes that previously were unable to connect are now able to be a part of the network, and every new node added expands the network’s coverage, rather than simply reducing the total available bandwidth (as would happen in a spoke network). Also packets sent between two adjacent or nearby nodes can go direct without tying up all the network’s bandwidth.

Hardware Setup Diagrams

Node Client

The Node client is the basic setup to be deployed to the “typical” home or business user wishing to connect into the mesh. It consists of a Linksys WRT unit running OLSR on OpenWRT as described later on. It is connected to a waveguide which is used to interconnect with other nearby units. A Power over Ethernet (see later on for description) module could be created to complete the package.

Supernode Client

A “supernode” is simply a Node client with a backbone connected. This backbone is used to help distribute traffic over the network. It would be most effective when used to connect from one side to the other. This way traffic traversing the whole network would have a single hop route to use, rather than having to pass through numerous routers. This will help to free up the network traffic.
This type of setup would be ideal for the more technical members (such as the majority of the current membership base) and on business setups.
A variation on the supernode idea would be to use two 180 degree waveguides with two radios rather than a 360 degree waveguide and single radio. This would effectively double the output signal and allow the two antennas to be mounted independently (eg on the sides of a chimney rather than on top) which may be beneficial in a number of situations.
The second device would not need to run OLSR as the WRT unit could simply add another interface (VLAN) to its OLSR routing config. This opens the way for a high powered unit such as the Senao AP for distance links or the Minitar 802.11g (providing they can do Ad-Hoc) units to be used for shorter links.

Zone Server (synch up with section previously describing zone server)

The Zone server hardware is simply a router box that is more powerful than a Wrt unit. This is to allow for more complex services to be run. It is connected to a waveguide in the same manner as a standard Client Node. A backbone connects the Zone Server to the core Air-Stream network. Additional configuration is therefore needed to ensure routing between the two interfaces is seamless. (See section on Routing for further information).

Zone Server with a VPN Link

In some situations the Zone Server may be unable to connect directly to another Zone Server using wireless, and thus a VPN tunnel using the Internet or another network service may be an option. This would work in a similar fashion to a direct backbone. Details on configuring the VPN tunnel can be found at the Darwin based website: www.the-mesh.org

Links and conclusion

Useful Links

www.olsr.org
http://www.linux.org/docs/ldp/howto/OLSR-IPv6-HOWTO/index.html
www.openwrt.org
www.freifunk.net
www.the-mesh.org

Conclusion