General Networking Infomation

Networks provide a means of connecting two or more computers in order to share resources, such as files and printers, and for communication through websites, email, chat and video.

The underlying systems used by computers to communicate, operate regardless of the medium in which they physically travel. Cables, telephone lines, radio signals such as used by Air-Stream and even infrared light beams, can be used to link computers in a network.

BGP (Border Gateway Protocol)

The Border Gateway Protocol (BGP) is an inter-autonomous system routing protocol. An autonomous system is a network or group of networks under a common administration and with common routing policies. BGP is used to exchange routing information for the Air-Stream WAN and is the protocol used between our Access Points Routers (APR).

BGP is a very robust and scalable routing protocol, as evidenced by the fact that BGP is the main routing protocol employed on the Internet today. To achieve scalability, BGP uses many route parameters, called attributes, to define routing policies and maintain a stable routing environment.

In addition to the BGP attributes, classless inter-domain routing (CIDR) is used to reduce the size of the routing tables. For example, assume that an APR such as Julia Farr owns the IP address block 10.123.x.x, this block would consist of 256 address blocks 10.123.0.x through to 10.123.255.x. If the APR assigns each block to a member without CIDR it would be necessary for the APR to advertise 256 address blocks to its peers. But CIDR can subnet the address space and advertise only one block 10.123.x.x which is much smaller as it is rendered obsolete by CIDR, allowing a significant reduction in the BGP routing tables.

BGP Attributes
Routes learned via BGP have associated properties that are used to determine the best route to a destination when multiple paths exist to a particular destination eg where an APR has multiple backbones. These properties are referred to as BGP attributes and include:

Weight
Local preference
Multi-exit discriminator
Origin
AS_path
Next hop
Community

Weight Attribute
Weight is an attribute that is local to a router. The weight attribute is not advertised to neighbouring routers. If the router learns about more than one route to the same destination, the route with the highest weight will be preferred and will be installed in the IP routing table.

Local Preference Attribute
The local preference attribute is used to prefer an exit point from the local autonomous system (AS). Unlike the weight attribute, the local preference attribute is propagated throughout the local AS. If there are multiple exit points from the AS, the local preference attribute is used to select the exit point for a specific route.

Multi-Exit Discriminator Attribute
The multi-exit discriminator (MED) is used as a suggestion to an external AS regarding the preferred route. The term suggestion is used because the external AS that is receiving the MEDs may be using other BGP attributes for route selection.

Origin Attribute
The origin attribute indicates how BGP learned about a particular route. The origin attribute can have one of three possible values:

IGP is the route interior to the originating AS. This value is set when the network router configuration command is used to inject the route into BGP.
EGP is the route learned via the Exterior Border Gateway Protocol (EBGP).
Incomplete is where the origin of the route is unknown or learned in some other way.

AS path Attribute
When a route advertisement passes through an autonomous system, the AS number is added to an ordered list of AS numbers that the route advertisement has traversed.

Next-Hop Attribute
The EBGP next-hop attribute is the IP address that is used to reach the advertising router. For EBGP peers, the next-hop address is the IP address of the connection between the peers. For IBGP, the EBGP next-hop address is carried into the local AS, as illustrated in
Therefore, it is important to have an IGP running in the AS to propagate next-hop routing information.

Community Attribute
The community attribute provides a way of grouping destinations, called communities, to which routing decisions (such as acceptance, preference, and redistribution) can be applied. Route maps are used to set the community attribute. Predefined community attributes are:

no-export—Do not advertise this route to EBGP peers.
no-advertise—Do not advertise this route to any peer.
internet—Advertise this route to the Internet community; all routers in the network belong to it.

BGP could possibly receive multiple advertisements for the same route from multiple sources. BGP selects only one path as the best path. When the path is selected, BGP puts the selected path in the IP routing table and propagates the path to its neighbours. BGP uses the following criteria, in the order presented, to select a path for a destination:

If the path specifies a next hop that is inaccessible, drop the update.
Prefer the path with the largest weight.
If the weights are the same, prefer the path with the largest local preference.
If the local preferences are the same, prefer the path that was originated by BGP running on this router.
If no route was originated, prefer the route that has the shortest AS path.
If all paths have the same AS path length, prefer the path with the lowest origin type (where IGP is lower than EGP, and EGP is lower than incomplete).
If the origin codes are the same, prefer the path with the lowest MED attribute.
If the paths have the same MED, prefer the external path over the internal path.
If the paths are still the same, prefer the path through the closest IGP neighbour.
Prefer the path with the lowest IP address, as specified by the BGP router ID.

Sample Config Files (for Quagga/Zebra)
Basic settings for a quagga BGP setup:

zebra.conf:
-------------------------
hostname blah                # your hostname
password blah                # your password
enable password blah
interface rl0                # your interfaces
interface wi0
interface wi1
line vty
-------------------------

bgpd.conf:
-------------------------
hostname blah
password blah
enable password blah
router bgp 655xx            # your AS
network 10.x.x.x/27            # 
network 10.y.y.y/30            # your networks
network 10.z.z.z/30            #
neighbor 10.y.y.y remote-as 655yy    # peer1
neighbor 10.z.z.z remote-as 655zz    # peer2
line vty
-------------------------

Good resources for BGP routing are:

Quagga an OpenSource Routing Standard that runs on Linux and *BSD

Cisco's Border Gateway Protocol Page

BGP on OpenWRT form martybugs

BGP on Quagga

Integrating the Wireless LAN with your existing home network

So you have made the effort of putting an antenna onto your roof, made the connection to your local AP and got either a coax run to a wireless card in your computer or a ethernet run back from the waterproofed AP on your roof. How do you make sure that all the other machines in your house can use this network too?

There is no single answer to this question that is correct, it all depends on your level of knowledge or how much effort you want to go to.

NAT: Network address translation

The internet gateway: typically a hardware ADSL router

One of the most simple ways of getting the rest of your computers on the network is by using network address translation. YOu are probably already using it to get a number of computers on your home LAN connected to the internet concurrently. How this works is you allow one machine on the gateway to the internet to translate from a real world IP to a private address range typically 192.168.x.y. At first I used the private range 192.168.0.0/24 which mean that all the IPs 192.168.0.0 to 192.168.0.255 could be used by my home network and connected to the internet thought my real bigpond address 144.136.177.192, using windows this can be achieve using internet connection sharing.

The wireless gateway: typically a machine with two network interfaces, one to the LAN one to the air-stream mesh.

Using the same philosphy the machine connected to the air-stream AP has its routed wireless mesh address 10.121.0.2 but it also has an internal LAN address in my example 192.168.0.3. It uses the same principal that the internet gateway router usings by sharing this one address among the other machines in my private range.

For this to work we need to do a couple of things. If my internet gateway is 192.168.0.1 then the default route for all my LAN machines will be 192.168.0.1 eg IP 192.168.0.4 netmask 255.255.255.0 gateway 192.168.0.1 DNS 61.9.224.15 (or whatever your ISP suggests). What we can do is put a static route in the machine at 192.168.0.1 to send all traffic bound for the Air-Stream wireless lan (10.96.0.0/11, 10.96.0.0 netmask 255.224.0.0) to be diverted to 192.168.0.3 which has a connection to teh rest fo the air-stream wan.

In the same way that NAT is applied to the internet connection it is also applied to the wireless LAN connection.

The reason NAT provides you with a certain amount of security as you are hiding on an address range that other cannot access but only connection that are requested by you are allowed through.

Dynamic Routing

In order to keep track of all the different ways that traffic can flow around the air-stream mesh a type of advanced routing known as dynamic routing. Where all the parameters regarding the topology of the network are given to a network daemon and it works out the best route to take based on these parameters. As the network grows it becomes more reliant on the dynamic routing to find alternative routes, this would need to be updated by hand on a number of routers if it were to be done by hand.

Border Gateway Protocol (BGP)
The type of routing that is used by Air-Stream is called border gateway protocol, it allows for filtering of routes and instant updates that OSPF and RIP do not allow.

/27 Routed Network using BGP
This allows us to make use of the 32 IPs all network users are entitled to!

If you are technically inclined the idea of using NAT may seems like a waste, as one of the advantages of being part of the WAN is that we have lots of IPs for everybody, unlike the internet which has largely run out of IP real estate.

If you want to advertise your subnet yourself you will need to run the network suite zebra or quagga (preffed) on a Unix, Linux or BSD operating system. Or if you have access to a windows 2003 server it can also run these protocols as can Cisco IOS based routers.

Alternatively you may be able to make arrangements with your AP administrator that they can keep the routing records as part of thier AP and you will only need to prepare a couple of static routes on your end.

Sorry if this is a bit of a glossary more details about dynamic routing can be found on www.quagga.net

Firewalling
Once your machines are routed onto the wireless network you may want to think about firewalling to keep your computers in and to keep unwanted packets out. This should all be done on the same machine that is doing the routing.

Sam Hodge

It is preferable that people be as self sufficent as possible, because the network administrators are under enough load as it is.

If all of this routing stuff is of interest to you I suggest you offer to lend the network administrators a hand or write a brief how-to on the Air-Stream site.

LDAP

The aim of the LDAP service is to centralize user information for access by different network services e.g. Domain Controllers, Email Severs, Web Servers and other complient services.

It works like a centralised directory service allowing both servers and users to lookup and search user details. LDAP Servers require very low system resources and can be synchronized with other LDAP Servers to offer very high levels of reliability.

LDAP stands for Lightweight Directory Access Protocol. A language based on the X.500 directory standard that allows clients and servers to communicate. The LDAP server allows access to the hierarchical structure of an active directory.

Managing Collections of Sub-Networks

When a router receives a packet of information (as it does thousands of times per second) it inspects the IP Address of that packet, and based on that, can make decisions on which physical path to send it through.

This process is known as routing the packet. Special software protocols have been created to support the routing of network traffic between subnetworks. These protocols operate with ‘smart’ algorithms that can adapt the flow of network traffic when problems occur. Protocols such as Border Gateway Protocol (BGP) and are widely used across the Internet today.

The algorithms employed by these protocols provide great network stability ensuring that if one network connection is lost, the router can quickly adapt to send data through an alternate network path. Consequently, the greater the number of network connections that are available, the greater the reliability and bandwidth for its users.

Managing Networked Computers

Computers on a network communicate with each other using a special numbered addressing system. This is known as Internet Protocol (or IP). Each computer has a unique IP Address, which is four sets of numbers from 0-255, separated by a decimal point. This gives a total possible address space from 0.0.0.0 to 255.255.255.255. For example: 203.152.23.202 is an Internet IP Address.

IP addresses can be grouped into smaller logical segments of the entire address space, which can then be assigned to computers in a LAN. Breaking up IP Address space into smaller sub-networks is known as Subnetting.

This allows for logical control of data between LANs, and maps well onto the physical network design outlined.

Open WRT and OLSR Working Group

A group has beem formed by Air-Stream members to work with interested code developers and every day people to look into practical ways to use OpenWRT able wireless devices and OLSR to build a very low cost grass roots community wireless networks.

There is no need to be a Air-Stream Member as anyone can participate, to join trials of the mesh network, all you need is a OpenWRT able wireless device , volenteer and participate.

Meshdoc

Hi!

This will be the eventual home of the more formal documentation of all things meshy.
For now the project is under heavy development over at mesh.air-stream.org

Please check there, and if you wish to assist your input is welcomed.
Once the content has been discussed further it will be written up and added to this section of the website.

Cheers,
Simon

Introduction

To use commercially available hardware along with open source software to create a reliable and redundant Neighbourhood wireless network using mesh routing technologies.
These neighbourhood meshes may be interconnected using routing or VPN over a connection such as the Air-Stream core network or a third party network such as the Internet.
By connecting members directly to other people in their local area, we create a much more community based network. We then have an opportunity to encourage people in the same community to interact and work together, whether it be for gaming, organising local events or sharing stories on the local history. These are all possible on the Internet but are often lost amongst the massive amount of data - in a local network the content becomes more personal.

Aims

• Mass producable (easily sourced components)
• Low price
• Compact and professionally presented unit package
• Minimal end user configuration
• Well presented documentation for the end user
• Quality documentation for the technical side
• Network design allowing for large scalability (subnet allocations and network backbones)
• Reliable end user security (so that home LANs are not at risk by joining the network)

Advantages

• Easily expand the network - a new node could be added in a matter of minutes.
• Reduce the likelihood of a tree or building blocking a user’s network access (as multiple paths for them to access the network).
• Roaming within a subnet would become possible.
• Adding a new node to the mesh will improve redundancy rather than simply dropping shared bandwidth.
• Encourages interactivity between local communities - much simpler than on the Internet due to its scale. This could develop into a working relationship with local councils - both in potential grants and in additional site locations.
• Increases the size of the network - there is a law stating a network is as useful as the square of its users.
• Having a larger membership base can only help in negotiating new sites for the core Air-Stream network.
• Helps to move Air-Stream into the more mainstream community - community wireless becomes accessible to those without extensive networking experience.
• Allows the end user network to grow more independently of Air-Stream allowing it to
focus on developing the core network.
• The ability to access the Air-Stream network, and thus also your home network from a drastically increased number of locations in the city.
• Greater node density allows for faster connections (such as 802.11g) between nodes, since each node is much closer than in the current client/ap setup.

Example Applications

A number of uses the network could be put to. This list will grow much larger with time.
Some of these are also possible on the current Air-Stream network, but would benefit from having many more members interacting on the network.
• Much larger user base leads to many more people available for gaming. Tournaments and championship ladders could be setup.
• Roaming around a subnet using a laptop or PDA.
• VoIP network becoming much larger and the ability to use wireless VoIP handsets.
• Local content able to be streamed across the network - for example a fair or other community event.
• A network of webcams located around the city.
• Local historical societies and schools working with interactive mapping software to create content based on their local area.

Overview of mesh

Overview of a wireless mesh

(Base on an entry from wikipedia entry or similar, and include diagrams).

Software Implementation

Software on WRT unit

OLSR running routing between devices
• Vpn to allow access to LAN side of device (off by default)
• Iptables to filter access to LAN (very strong defaults)
• Httpd to provide basic stats and user info
• Minimal uniform splash page – eg basic contact info for the node and what you can find on this network segment (list of services this user offers on their lan)
• A full httpd would be running elsewhere on this user’s network
• Linked into the OLSR plugin to provide OLSR network stats?
• Frottle or similar program to address the hidden node problem – research more extensively (eg melbwireless and openwrt forums)
• SSHd to allow for secure remote access to the device.

Software on additional network devices

Laptops and other portable devices would run the OLSR client, or hopefully get a DHCP assigned from the AP.
Captive portalling on the AP Would be sweet also if possible.

Logical (Networking) Implementation

Routing Protocol

OLSR is to be used as the routing protocol.
• Linux, Windows, Mac OS X and BSD (?) clients available.
• Open Source and well documented thus fits well in with other software (eg quagga and *nix) already used on the network.
• Documented support for OpenWRT and has been deployed on this platform internationally (eg www.freifunk.net).
• Would also be able to be installed on the WRAP boards.
• Able to be installed on a Linux/BSD platform so we are not limited in our choice of platform for the Zone Server.

Routing into and out of mesh - Static Workaround

A solution to the issue of getting the two dynamic routing protocols (OLSR and BGP/OSPF) to talk to each other is shown in the below diagram.

Routing into and out of mesh - Dynamic Solution

A better solution would be to integrate the kernel routing so that Quagga dynamically receives the information about the mesh, rather than there simply being a static route set on the Zone Server.
This will be preferable when there are multiple routes into and out of the mesh (which is optimal for redundancy).

Network Hardware

Business Node Routers

The WRAP boards would be well suited for business nodes and more technical orientated members.
• They are able to have two wireless cards onboard (as two Mini-PCI slots) making them suitable for the “supernode” type of setup.
• These would be preferable for business nodes as they are a more flexible platform than the WRT54G, in terms of storage space and OS support.

Client Node Routers

The Linksys WRT54G devices are ideally suited for the home nodes.
• They are low cost and easily available in shops, so sourcing the unit will not be an issue.
• They have a well developed Linux based OS (OpenWRT) which has support for a large number of packages.
• They have a proven track record in a number of networks internationally, both community based and commercial. Wildcat Wireless in the US currently have 30 of these devices running successfully in a mesh setup.

Antenna

The standard antenna to be used in the network would be a 4 slot waveguide. In use to date it has been shown potential in being able to meet our requirements.

Power over Ethernet (PoE)

Power over Ethernet takes advantage of the fact that only two of the four pairs in a Cat5 network cable are used for carrying network data. The other two pairs are left unused and can therefore be used to carry power to the device. This is possible since we are only dealing with low power levels, and saves us having to run an additional cable to the device for power.
This makes the total install simpler and neater.

Mounting hardware

Additional hardware needed to mount the antenna and weatherproof box, and also for other items such as the cable runs.

Weatherproofing unit

Need to decide whether to go for a professional solution (and if so which one) or whether to investigate using PVC downpipe sealed for the casing of the device.
Also need to consider whether to mount the unit as is inside the case or whether to remove the PCB board from the manufacturer’s casing.
Of course temperature and humidity considerations need to also be taken into account.+

Additional Hardware Options

Weather Station
• Weather devices
• May be possible to interface some basic weather measurement data to the serial interface of either the WRT54G board or the WRAP units.
• This would then allow us to plot city wide maps based on data collected from our own weather units.
Webcams
It would also be nice if we were able to interface the WRAP boards with USB webcams, providing another feature on the network.

Network Management and Visualisation

Httpinfo

The HttpInfo plugin for OLSR will be set up so that each node’s routing information is available to the rest of the network.

Dotdraw

The DotDraw plugin will be setup on the Zone Server to provide a live visual overview of the network.

Authentication

Introduction

• The network we will be building, with fast access and wide coverage, will be appealing to a number of non members. Authentication will likely be required to keep the available bandwidth for our members.
It is certainly not aiming to restrict the availability of the network to the community, but helping to ensure that those who wish the participate on the network are able to do so without having their access limited by people taking advantage of their generosity.
• If current solutions do not meet our needs then a possible solution is as follows.

Proposed solution

Step by step scenario:
1. User Joe walks up to a mesh member’s house with his laptop in hand. (Is the laptop running OLSR or just wishing for a NAT connection?)
2. Joe’s laptop is handed out an address (eg 10.0.0.100) via the mesh DHCP server.
3. Joe logs onto the zone server (which is a high performance machine capable of running apache, mysql etc) and provides his login details. This is possible since port 80 (and any other relevant port - eg for ssl) is automatically permitted by each mesh router.
4. If login accepted, the zone server then adds his ip to the relevant database. It also adds it to a web page visible to the whole network. Eg 10.0.0.1/accept.txt would then contain 10.0.0.100. This way other routers can check the authenticated user list.
5. Each router will periodically (eg every 5 minutes) check this list and update their local allow cache accordingly (eg just replace should be sufficient). If the router receives a packet from an ip address not in its list, it will then check the zone server’s allow list. When the accept table is updated then the iptables firewall is also updated by the relevant shell script.
6. When Joe has finished his session he can go back to the mesh web page and click a logout button which would remove his entry from the accept.txt file.

Additional Comments

• What do we want to do in the even a Zone Server is down - will this break the mesh
15
network’s operation or will it be so infrequent that it is not a major concern?
• Some devices which may not be able to log onto a web page (such as a VoIP handset) may need to be auto added. A solution to this would be to auto forward all VoIP ports (subject to abuse). This also solves the legal issue of providing access to the emergency numbers.

Testing

Air-Stream Open Day Testing

Set up a number of WRT54G routers running OLSR around the event.
The OLSR client could then be provided on a number of cds, allowing laptop users to install the client and test out the network.
Gives a good chance to test scalability and actually using the OLSR application, and encourage people there to interact with the network, rather than just looking at it.
Set up one of the x86 machines so that it can do live visualisation - so we can have a screen running all day that shows the network topology live.
Need to test out the x86 installs, configure the visualisation software and set up the WRT54G devices with the clients. Also need to source a few more WRT devices from members to use for the day.

Rooftop Trials

These will be carried out to establish the real world performance of antenna combinations.
A combination of antennas may also be used such as:
End A (WRT)
End B (Laptop)
4 Slot waveguide
Internal PCMCIA Card
8 Slot waveguide
4 Slot Waveguide
8 dBi Omni
8 Slot Waveguide
8 dBi Omni
1
5dBi Dish
Cantenna

Trial network

It is likely that a trial mesh network will be carried out in the suburbs of Daw Park/Melrose Park and a number of surrounding suburbs.

Plotting Coverage

• Nodes can be plotted on a 2D metropolitan map to provide a view of the physical layout of the network.
• A GPS unit could be used with a laptop to provide a rough idea of real world network coverage.

Additional Questions

These will be need to be answered in future revisions of this document.
• Can the software run in the proposed hardware?
• How easy will it be to deploy and configure the software – see German group who offer a firmware compile based on web page form entries for settings.
• How scalable is it to our needs – would 20-30 nodes in a suburb be fine? – see overseas uses
• What IP ranges are we to use?
• Routing issues?
• Can we add backbones from one side of the mesh to the other to help distribute traffic (eg on a different channel and acting solely as a backbone) – will OLSR scale to this?
• What antennas to use – waveguides are cheap and easy to mass produce and horizontal polarization is preferable but is a 4 slot going to perform ok as 8 slots are quite large.
• How badly will trees kill our signal or conversely will there be too much noise from other devices to make it unusable?
• We would ideally want 250m – 500m coverage for a decent network initially to get the critical mass going.
• Will 54mbps work over some links (eg 802.11g going to run over <1km links alright?)
• Can client connect not running OLSR (routing issues – use NAT in this case – check more in mailing lists)
• Vertical spread of the 4 slot waveguide – eg can someone with one on their roof use their laptop in their backyard to access their home lan?
• How good is OLSR – eg will it meet our needs – check with German etc trials
• Possible to have multiple paths out of the network? How would this go with the routing mixes… eg one supernode connected to Pasadena and another connected to Melrose Park, both could offer a default route out of the network…. Could be an issue that we will just have to bear with later. Preferable to have multiple routes out but cannot expect optimal path selection talking across OLSR and BGP/OSPF. Could be resolved in later
17 1km OLSR releases or could this come from the kernel routing tables on the gateway? Either way an additional hop through the core Air-Stream network is unlikely to cause any hassles and would be better than 8 hops through the wireless mesh, not to mention redundancy if one gateway were to be unavailable.
• How can we recognize local businesses which contribute by funding/hosting a node?
o aim for ads on the zone server page
o this would then mean the zone server would be the portal site for the nodes, eg this is their network home page rather than the main AS page. The AS page news could be fed into this site by RSS or whatever.
• Network updates and information over RSS?
• How will DHCP flow to a home user on the LAN side of the WRT? (check up on mailing list regarding all nodes needing to either run the OLSR client or they will not get routing info and thus must run through NAT) - eg run NAT as per normal home users with a shared DSL modem, and for more advanced users wishing to run a server can simply run the OLSR client which adds them directly into the Neighbourhood mesh subnet. This solves the problem of having to allocate subnets for every home lan and then deal with routing - simplifies each site's configuration
• How to then allow the user to select to use NAT (eg PDA, Voip phone) or OLSR (eg laptop wishing for roaming)?
• What happens when the zone DHCP server goes down - do we want a high redundancy network or is this a tolerable issue in the time it takes to fix the problem? Maybe allocate each of the static mesh nodes (eg those with waveguides on roofs) an ip via the website when they join up so they wont go down with the DHCP server.
• Can each home LAN have a seperate subnet or will that break the OLSR routing?
• How to handle authentication? (answered later)

Implementing the mesh

Overview

Our implementation of the mesh will be similar to the reference mesh model discussed above.
However given the geographical size and low population density of Adelaide, implementing a city wide wireless mesh would be very demanding on network resources - the routing tables would be immense and sending a packet from one side of the city to the other could involve over 100 hops!
Therefore this project will aim for a larger number of smaller meshes, consisting of rougly one to three suburbs in size. Some applications may extend this further, such as offering a single mesh to allow for seamless roaming down Belair Road (which would cover 5+ suburbs).
Smaller meshes mean that each mesh is kept to a managable level, whilst still offering the advantages of using mesh technologies.
Each of these smaller meshes would then use a “Zone Server” to interface with the core Air-Stream network. This is the gateway from the mesh to the rest of the network.

The Zone Server

The core of the local neighbourhood mesh is the Zone Server. This is the machine that would currently be the local access point, with backbones to remote APs located elsewhere in the network. This machine will handle authentication and provide a link to remote networks. It will also host services such as a local irc, IM and http server, which would be interlinked to other similar servers elsewhere on the network. By localizing these services we provide redundancy so that if a backbone were to fail, the local mesh could still function autonomously. This is currently not the case in a number of wireless networks, where if a backbone were to go down then the local users have no provided services to use.
Is it possible to have multiple zone servers (eg can we have multiple paths out of the mesh?) This would be optimal for redundancy in a network otherwise we have a single point of failure which may isolate the mesh.
• Hosts interlinked IRC and Jabber servers
• At reliable site – preferably business or experienced member
• Uses a more high powered card and waveguide combination mounted high etc to cover as much as possible of the local area. This ensures access to backbone is faster and less network demanding.
• Try to concentrate content locally rather than traffic traversing backbones needlessly.
• Approach community friendly local businesses to see if they would be interested in hosting a zone server or a supernode.

Current Local Wireless Design

Currently a local wireless network consists of an access point with an omnidirectional antenna to which nearby nodes connect to using directional antennas. This situation is fine for users with line of sight to the access point but means that those who are unfortunate enough to have a tree or building in the way are unable to get any access to the network.
This can be seen in the below example.

The mesh network acts by connecting the nodes to each other. This means that total bandwidth may be reduced, especially when packets are crossing from one side of the network to the other. The compromise is that nodes that previously were unable to connect are now able to be a part of the network, and every new node added expands the network’s coverage, rather than simply reducing the total available bandwidth (as would happen in a spoke network). Also packets sent between two adjacent or nearby nodes can go direct without tying up all the network’s bandwidth.

Hardware Setup Diagrams

Node Client

The Node client is the basic setup to be deployed to the “typical” home or business user wishing to connect into the mesh. It consists of a Linksys WRT unit running OLSR on OpenWRT as described later on. It is connected to a waveguide which is used to interconnect with other nearby units. A Power over Ethernet (see later on for description) module could be created to complete the package.

Supernode Client

A “supernode” is simply a Node client with a backbone connected. This backbone is used to help distribute traffic over the network. It would be most effective when used to connect from one side to the other. This way traffic traversing the whole network would have a single hop route to use, rather than having to pass through numerous routers. This will help to free up the network traffic.
This type of setup would be ideal for the more technical members (such as the majority of the current membership base) and on business setups.
A variation on the supernode idea would be to use two 180 degree waveguides with two radios rather than a 360 degree waveguide and single radio. This would effectively double the output signal and allow the two antennas to be mounted independently (eg on the sides of a chimney rather than on top) which may be beneficial in a number of situations.
The second device would not need to run OLSR as the WRT unit could simply add another interface (VLAN) to its OLSR routing config. This opens the way for a high powered unit such as the Senao AP for distance links or the Minitar 802.11g (providing they can do Ad-Hoc) units to be used for shorter links.

Zone Server (synch up with section previously describing zone server)

The Zone server hardware is simply a router box that is more powerful than a Wrt unit. This is to allow for more complex services to be run. It is connected to a waveguide in the same manner as a standard Client Node. A backbone connects the Zone Server to the core Air-Stream network. Additional configuration is therefore needed to ensure routing between the two interfaces is seamless. (See section on Routing for further information).

Zone Server with a VPN Link

In some situations the Zone Server may be unable to connect directly to another Zone Server using wireless, and thus a VPN tunnel using the Internet or another network service may be an option. This would work in a similar fashion to a direct backbone. Details on configuring the VPN tunnel can be found at the Darwin based website: www.the-mesh.org

Links and conclusion

Useful Links

www.olsr.org
http://www.linux.org/docs/ldp/howto/OLSR-IPv6-HOWTO/index.html
www.openwrt.org
www.freifunk.net
www.the-mesh.org

Conclusion

OpenWRT and Quagga For Dynamic Router

Adding Quagga to OpenWRT for Dynamic Routing

Read up on http://quagga.net

Get your AS number for your AP and neighbours from the committee

Install Quagga

ipkg install quagga
ipkg install quagga-zebra
ipkg install quagga-bgpd

Configure Quagga

vi /etc/quagga/zebra.conf

vi /etc/quagga/bgpd.conf

Add PID files

touch /var/bgpd.pid
chmod 777 /var/bgpd.pid
touch /var/zebra.pid
chmod 777 /var/zebra.pid

Check Init files

/etc/init.d/S35quagga

OpenWRT Kamikaze Build

Building your own OpenWRT firmware from SVN: Kamikaze

Using OpenWRT from the the downloads.openwrt.org means you are bound to a slow release and testing cycle: White Russian RC6 is the most current release at time of writing

This means that you are stuck with an older kernel 2.4.30 and older builds of the software that is based on that kernel

Also the configuration and management system is a generation behind

The alternative is to use Kamikaze, this is downloaded from a developers site

Anybody with some basic programming skills and some linux experience will have no trouble building the source directly from the Subversion Repository

Take this guide as a glossary, refer to https://dev.openwrt.org/ as your bible

Target build Linux Operating System

You will need a linux operating system that is installed rather than a live cd as you will need to write to disk and have permissions to build software

I reccomend Gentoo Linux, but Ubuntu and Fedora Core are good forgiving Linux Operating Systems for a new user

SVN client

In order to download the source from the repository you will need svn

Use your package management system to install svn

http://subversion.tigris.org/project_packages.html

Standard Command line tools: wget, gunzip, make

I would be surprised if your linux install didnt have this but avoid a barebones linux like slackware which may lack some of the more standard command line tools

Working gcc C++ compiler

Although the OpenWrt creates a tool chain, this tool chain needs to be compiled itself, before it can be compiled for the target platform

This may mean choosing a "Development" suite of tools for your linux operating system

But I would imagine this would be installed already if you are used to compling your own software

Disk Requirements

Make sure you have about 3 gigabytes of free disk space

So if you are putting this on a spare system dig out a 20Gb drive

Bandwidth Requirements

Im not sure of the exact amount, but this will download a lot of data, this requires a broadband connection, without a excess cost scheme

The total amount of data is less than a gigabyte but the disk usage is a lot higher due to decompression

Also the download isnt over once the svn download is done, there will be further wget downloads

SVN download of source

Now you have your svn client installed, open a shell, navigate to a location on the disks with 3GB of free space and paste the following command

svn co https://svn.openwrt.org/openwrt/trunk/

Menuconfig

Now in the same folder issue the command below

make menuconfig

Intially you can choose the hardware platform you are building for ie broadcom xxxxx for kernel 2.4.32 or broadcom xxxx for kernel 2.6.17

This will allow you to choose what will be put into the firmware you want to make
Anything that is chosen as a [M] will be installed via ipkg, or will be a kernel module
Anything that is chosen as a [*] will be complied into the kernel or will be in the firmware itself
Anything that is chosen as [ ] will not be complied

This is the time to choose that Atheros based kernel for madwifi that you always wanted to run 802.11a under OpenWRT

watch for debug messages as they will help you solve dependancy issues

Build tool chain

Once you have locked in all your choices, you simply have to build the firmware ready to install

Unforetunately you are developing for a system that isnt x86 like the system you are compiling on for this you need a special complier called a toolchain

The developers are OpenWRT have taken care of this and it is part of the make file to compile the tool chain for you

make

Initally there will be a massive download and compile cycle to build all the complier tool chains for the various ulibc firmware for the different CPU types you have chosen

Just hang in there and wait, it might be worth doing this on your fastest system if you are in a hurry

Wait

Build firmware image

Once the tool chain is built, it can go ahead and compile all the kernel, kernel modules, and software (busybox etc) you need for your openwrt device

Wait

Build additional packages

Some packages are chosen by default and they will be compiled and bundled into the firmware or packed into thier own .ipkg files ready to be installed

as outline on the https://dev.openwrt.org page:

Note: Kamikaze only contains the essential set of packages, extra packages can be found in the 'Kamikaze packages' directory above. To use these packages simply symlink them into Kamikaze's package directory.

So if you want quagga, Kismet, olsr etc, you will need to symlink the directories and thier dependancies and hope like hell all the .ipkg files will fit into the flash

you also need to point to a web server where you will host the .ipkg files this will be in the /etc/ipkg.conf on the new install

Wait

Output

find the

bin

folder and install it onto your openwrt device, and give yourself a leet pat on the back

Celebrate

Physical Network Structures

Groups of computers can be connected in many different configurations.

A simple Local Area Network (LAN) consists of a few computers, which are directly connected to each other.

This allows for high-speed communication between the participating machines, but offers little flexibility for adding more clients. Networks like this are generally confined to a relatively small area, such as a home, business or office.

As more clients are added to the LAN, we require more efficient control of network traffic. This is achieved through the deployment of devices such as Switches and Routers.

A network Switch (S) operates like a central hub through which several computers can communicate at high speed. Computers can send out network broadcasts to inform all other computers of their presence, and high speed peer-to-peer communication is maintained.

Routers can be used to interconnect several LANs together, forming a Wide Area Network (WAN). A Router (R) is used at each connection point between the LANs to control and monitor the passage of data.

The use of Routers allows confinement of network broadcasts to the Local Network only, instead of traversing the entire network, and reducing overall efficiency.

The flow control and management of data as it traverses a physical network is known as Routing. Perhaps the best example of this network structure is the Internet, built upon these same principles; it is a worldwide collection of interconnected.

Wireless Client Ethernet Bridge

There are two network configurations possible for a client host or network when using an external wireless device. The most simple is a bridged configuration where the hosts on the members network are bridged directly to the wireless network and take IPs from the subnet of remote access point. The other type is a routed configuration in which the members network is given a seperate subnet that is routed onto the network. This page deals with devices capable of bridging (but not necessarily routing.)

Unlike internal wireless devices such as PCI, Mini-PCI, PCMCIA and USB adapters, an important feature required when using the alternative wireless Ethernet device is the ability to bridge or clone the Mac address of the device it is connected to allow seemless packet forwarding between the wireless network and the members network or computer.

The aim being to connect the Mac addresses of the router and the members without wireless Ethernet device(s) Mac address used in the network.

All ethernet wirless devices that are designed to bridge will work correctly for most client setups.

However the confussion lies when members purchase an AP which works in both AP mode and client/bridge mode such as the Classic Minitar "the ideal" but remember that this is only the ideal if you intend that at a later stage to use it in other modes.

In most cases an AP that works in both modes are capable of connecting using only one IP address. However unless the AP, when operating in client mode can bridge correctly or can clone the members Mac address, it will not forward packets correctly and disallow the use of other subnets.

Some examples of bridges and APs that could be used as a Wireless Client Ethernet Bridge:

Linksys WRT54G / WRT54GL Router AP only, but can run in Bridge if setup with openwrt (b/g) $108.00
Linksys WET54G Wireless to Ethernet bridge only (b/g) $148.50
Linksys WGA54G Wireless to Ethernet bridge only (b/g) $134.00
Linksys WET54GS5 Wireless to Ethernet Bridge with 5-port Switch (b/g) $149.75
Senao NL-2611 CB3 Deluxe AP/Bridge (b) $190.00
Senao NL-3054 CB3 Plus Deluxe AP/Bridge/Client (b/g) $225.00
Netgear WGE101 Wireless to Ethernet bridge Only (b/g) $139.00
Dlink DWL900+ Wireless Access Point AP/Bridge/Client/Repeater (b/b+) $134.99
Dlink DWL-810+ Wireless to Ethernet bridge only (b/b+) $134.95
"Freenet Antennas" UltraWAP 802.11 b/g AP/Bridge/Client/Repeater $99.00 (90 mW TX) to $170.00 (200 mW TX)
NetComm NP5410 Wireless to Ethernet bridge only (b/g) $155.76
Edimax EW-7206APg Wireless Access Point AP/Bridge/Client/Repeater (b/g) $100

Most 802.11g devices will operate in 802.11b automatically but should be set to 802.11b only in the firmware to improve performance.

Note: Air-Stream is not a supplier of equipment and prices listed are only listed as a guide only. Please visit member discounts for some of our preferred suppliers.

Some definitions:

Access Point Mode
Access Point mode is used to connect to one or more wireless clients. Wireless clients can only communicate to APs in Access Point mode.

Access Point Client / Wireless Client Mode
AP Client or Wireless Client mode allows the Access Point to effectively become a wireless client to another AP. In essence the AP has now become a wireless adapter card. You would use this mode to allow the device to communicate with an existing AP. Wireless cards will not communicate with access points in AP Client / Wireless Client mode.

Point-to-Point / Wireless Bridge
Point-to-Point / Wireless Bridge mode allows the Access Point to communicate with another Access Point capable of point-to-point bridging. However, be aware that most manufacturers use proprietary settings when enabling bridging mode in the Access Point. A typical scenario for this selection is connecting two routers at Mac level through a wireless connection (ie mac3 and mac2 in the diagram above)

Point-to-Multipoint / Multi-point Bridge
Point-to-Multi-point / Multi-point Bridge mode is the same as Point-to-point / Wireless Bridge mode, however this mode allows you to use more than two Access Points.

Repeater Mode
As a Wireless Repeater, the Access Point extends the range of the wireless network by repeating the wireless signal of the remote AP (Access Point). The Ethernet MAC address of the remote AP is required for the Access Point to act as a wireless range extender. The repeater appears to the remote AP as just another client, so this mode does not require any special proprietary bridging protocols.

Air-Stream IP Addressing

Air-Stream uses private addressing from the 10/8 range utilising 10.96.0.0/11, this is designed to allow interconnection into wireless networks around Australia should the day ever come.

These are:

VIC - 10.0.0.0/11
WA - 10.32.0.0/11
NT - 10.64.0.0/12
ACT - 10.80.0.0/12
SA - 10.96.0.0/11
TAS - 10.128.0.0/11
QLD - 10.160.0.0/11
NSW - 10.192.0.0/11
NZ - 10.224.0.0/11

General rule of thumb is that an Access Point will be assigned 1x/27 for the Nodes hanging off it, 1x/27 for the Local LAN at the access point if applicable and each node will be assigned a single address from the Access Points /27 pool to which they then can route their very own /27 to if they wish.

In some circumstances an AP may require additional IP’s in which case another /27 is made available or a single /26 can be provisioned, it is unlikely and not very smart to run more than 30 nodes off a single Radio so the above would apply only to AP’s with multiple radios offering node access.

The majority of the Air-Stream backbones are run on /30’s

The Addressing scheme has been designed to allow scalability while also allowing good use of VLSM(Variable Length Subnet Masking) in the future to summarize route tables, each area is designated an inner and outer segment, inner being within 20km of the CBD and outer being further than 20km.

Glossary:

/27 is 30 usable IP’s, a /26 is 62 usable IP’s and a /30 is 2 usable IP’s.
An Access Point is the Radio that nodes terminate on, would almost always go into a router at the same location.
A Node is an end users radio that is acting as a client to the Access Point.

This is how IPs are allocated, based on Adelaide CBD being the centre:

Section	Access Point Nodes	Back Bone	Member Subnets
North(In)	10.121.128.0/18	10.125.128.0/18	10.106.0.0/16
North(Out)	10.121.192.0/18	10.125.192.0/18	10.107.0.0/16
North-East(In)	10.122.0.0/18	10.126.0.0/18	10.108.0.0/16
North-East(Out)	10.122.64.0/18	10.126.64.0/18	10.109.0.0/16
East(In)	10.122.128.0/18	10.126.128.0/18	10.110.0.0/16
East(Out)	10.122.192.0/18	10.126.192.0/18	10.111.0.0/16
South-East(In)	10.123.0.0/18	10.127.0.0/18	10.112.0.0/16
South-East(Out)	10.123.64.0/18	10.127.64.0/18	10.113.0.0/16
South(In)	10.123.128.0/18	10.127.128.0/18	10.114.0.0/16
South(Out)	10.123.192.0/18	10.127.192.0/18	10.115.0.0/16
South-West(In)	10.120.0.0/18	10.124.0.0/18	10.100.0.0/16
South-West(Out)	10.120.64.0/18	10.124.64.0/18	10.101.0.0/16
West(In)	10.120.128.0/18	10.124.128.0/18	10.102.0.0/16
West(Out)	10.120.192.0/18	10.124.192.0/18	10.103.0.0/16
North-West(In)	10.121.0.0/18	10.125.0.0/18	10.104.0.0/16
North-West(Out)	10.121.64.0/18	10.125.64.0/18	10.105.0.0/16

Air-Stream's Routing

Air-Stream primarily uses BGP for routing around the domain, some AP owners choose to run another protocol such as OSPF/RIP locally then redistribute into the BGP network, this is fine.

There are filters run on the Air-Stream operated routers to control routing updates and what can and can’t be routed, it’s generally fairly open.

Each AP is assigned its own Private AS (Autonomous System) and AP owners have full control of their own AS in regards to what goes in and out. Initally we started off with OSPF but ran into multicast and topology issues, BGP allowed us to work around this while providing added control over the network so it was a win win situation.

As previously mentioned we like to summarise addresses where possible at the routers and have a good design in place to allow for huge growth. We are able to provide BGP templates for people to use as configs, it really isn’t that hard and configuration isn’t overly advanced for the nodes, about the most interesting thing is that they run in a route reflector client mode due to the topology and iBGP.

QoS
We have QoS(Quality of Service) in place on the Air-Stream routers giving precedence to traffic seen to be important by us(the committee) and the members, we are always open to new suggestions of what should be given more precedence, it’s not the easiest thing to implement when you are going between cisco,linux and bsd routers around the network but it seems to work quite well.