Kronosnet
|
AN!Wiki :: Kronosnet |
What is Kronosnet
First, the homepage.
Kronosnet is a new network topology designed by Fabio M. Di Nitto and Federico Simoncelli to provide highly adaptable, redundant and secure networking over both high and low-speed links.
In a sense, kronosnet is similar to a VPN in that connections between nodes are on a common subnet. Unlike traditional VPNs though, that run on a point to point or point to multipoint setup, kronosnet is designed to run multipoint to multipoint, and can use up to eight links between each node. Multiple kronosnet interfaces can be configured on any given node. While operating, the kronosnet daemon, kronosnetd, will keep track of the latency between links. This allows for the fastest link to always be chosen. Link failures are automatically accounted for and recovery is done automatically.
This means that, so long as at least one link is up, communication to the host will work, regardless of what happens to the underlying network links!
Where It's At
As of now, kronosnet is alpha. That is, now and then, branches will work to an extent, but the program is in heavy development and subject to rapid and radical change.
These docs will cover some planned features. When unimplemented but planned features are discussed, the target release version will be show as a superscript.
The Most Basic
The most basic setup possible would be two servers, each with two network interfaces, running kronosnet. A slightly more realistic setup would be three interfaces each, so lets start with that. Here is a diagram showing our setup.
_______________________ _______________________
| Node A | | Node B |
| _______ | ______________ | ______ |
| | eth0 =-=----------| 10 Gb switch |----------=-= eth0 | |
| /------=______| | |______________| | |______=------\ |
| | _______ | ______________ | ______ | |
| | | eth1 =-=----------| 1 Gb switch |----------=-= eth1 | | |
| | /----=______| | |______________| | |______=----\ | |
| | | _______ | ______________ | ______ | | |
| | | | eth2 =-=----------| 1 Gb switch |----------=-= eth2 | | | |
| | | /--=______| | |______________| | |______=--\ | | |
| _|_|_|_ | | _|_|_|_ |
| | knet0 | | | | knet0 | |
| |_______|--\ | | /--|_______| |
| | | | | | | |
| _____|__ ___|____ | | ____|___ __|_____ |
| | App. A | | App. B | | | | App. A | | App. B | |
| |________| |________| | | |________| |________| |
|_______________________| |_______________________|
Here we see two servers, Node A and Node B. Each one has three interfaces; eth0, eth1 and eth2. These three interfaces are then used to create the kronosnet knet0 device. This virtual device can then be used just as you would a normal ethernet device.
The existing ethernet devices can retain their original IP addresses and are not effected or altered by kronosnet. This means that you can drop kronosnet onto an existing configuration and not worry about negatively effecting any of your existing applications. It simply adds another interface, the device, which in turn has it's own IPv4 or IPv6 address.
In this example, knet0 would use eth0 when available as it would be the preferred active0.1. Should it fail, then which ever of the other two interfaces, eth1 or eth2, would be used until the preferred eth0 recovered. All of this happens in a manner transparent to your applications!
Meshing
Any given kronosnet device can use up to eight interfaces. However, you can have a large number of kronosnet devices on a single host. Each kronosnet device can in turn share common interfaces or use their own dedicated interfaces. In this way, you can easily create fully meshed networks.
Lets take the example nodes from the previous sections, add a couple of interfaces, setup a second knet device and move up to four nodes.
____________________________ ____________________________
| Node A | | Node B |
| _______ | ______________ | ______ |
| /----------------| eth0 =-=-------------| 10 Gb switch |-------------=-= eth0 |----------------\ |
| | /------------|______| | /---------|______________|---------\ | |______|------------\ | |
| | | _______ | | ______________ | | ______ | | |
| | | | eth1 =-=---+---------| 1 Gb switch |---------+---=-= eth1 | | | |
| | | /----------|______| | | /-------|______________|-------\ | | |______|----------\ | | |
| | | | _______ | | | ______________ | | | ______ | | | |
| | | | | eth2 =-=---+-+-------| 1 Gb switch |-------+-+---=-= eth2 | | | | |
| | | | /--------|______| | | | /-----|______________|-----\ | | | |______|--------\ | | | |
| | | | | _______ | | | | ______________ | | | | _______ | | | | |
| | | | | | eth3 =-=---+-+-+-----| 1 Gb switch |-----+-+-+---=-= eth3 | | | | | |
| | | | | /----|______| | | | | /---|______________|---\ | | | | |______|----\ | | | | |
| | | | | | _______ | | | | | ______________ | | | | | _______ | | | | | |
| | | | | | | eth4 =-=---+ + +-+---| 1Gb switch |---+-+-+-+---=-= eth4 | | | | | | |
| | | | | | /--|______| | | | | | /-|______________|-\ | | | | | |______|--\ | | | | | |
| | | | | | | | | | | | | | | | | | | | | | | | | |
| | _|_|_|_ | | | | | | | | | | | | | | | | _|_|_|_ | |
| | | knet0 | | | | | | | | | | | | | | | | | | knet0 | | |
| | |_______| | | | | | | | | | | | | | | | | |_______| | |
| | | | | | | | | | | | | | | | | | | | | |
| \---+-----\ | | | | | | | | | | | | | | | | /-----+---/ |
| | | | | | | | | | | | | | | | | | | | | |
| | _|_|_|_ | | | | | | | | | | | | _|_|_|_ | |
| | | knet1 | | | | | | | | | | | | | | knet1 | | |
| | /-|_______|-\ | | | | | | | | | | | | /-|_______|-\ | |
| | | | | | | | | | | | | | | | | | | |
| __|_|___ ____|___ | | | | | | | | | | | | ___|____ ___|_|__ |
| | App. A | | App. B | | | | | | | | | | | | | | App. A | | App. B | |
| |________| |________| | | | | | | | | | | | | |________| |________| |
|____________________________| | | | | | | | | | | |____________________________|
| | | | | | | | | |
____________________________ | | | | | | | | | | ____________________________
| Node C | | | | | | | | | | | | Node D |
| _______ | | | | | | | | | | | | ______ |
| /----------------| eth0 =-=---/ | | | | | | | | \---=-= eth0 |----------------\ |
| | /------------|______| | | | | | | | | | | |______|------------\ | |
| | | _______ | | | | | | | | | | ______ | | |
| | | | eth1 =-=-----/ | | | | | | \-----=-= eth1 | | | |
| | | /----------|______| | | | | | | | | |______|----------\ | | |
| | | | _______ | | | | | | | | ______ | | | |
| | | | | eth2 =-=-------/ | | | | \-------=-= eth2 | | | | |
| | | | /--------|______| | | | | | | |______|--------\ | | | |
| | | | | _______ | | | | | | _______ | | | | |
| | | | | | eth3 =-=---------/ | | \---------=-= eth3 | | | | | |
| | | | | /----|______| | | | | |______|----\ | | | | |
| | | | | | _______ | | | | _______ | | | | | |
| | | | | | | eth4 =-=-----------/ \-----------=-= eth4 | | | | | | |
| | | | | | /--|______| | | |______|--\ | | | | | |
| | | | | | | | | | | | | | | |
| | _|_|_|_ | | | | | | _|_|_|_ | |
| | | knet0 | | | | | | | | knet0 | | |
| | |_______| | | | | | | |_______| | |
| | | | | | | | | | | |
| \---+-----\ | | | | | | /-----+---/ |
| | | | | | | | | | | |
| | _|_|_|_ | | _|_|_|_ | |
| | | knet1 | | | | knet1 | | |
| | /-|_______|-\ | | /-|_______|-\ | |
| | | | | | | | | |
| __|_|___ ____|___ | | ___|____ ___|_|__ |
| | App. A | | App. B | | | | App. A | | App. B | |
| |________| |________| | | |________| |________| |
|____________________________| |____________________________|
More simply:
____________ _____________
| Node A | | Node B |
| _______ | __________ | _______ |
| | knet0 |=----------| Switches |----------==| knet0 | |
| |_______|| /-----|__________|-----\ | |_______| |
| _______ | | __________ | | _______ |
| | knet1 |=----+-----| Switches |-----+----==| knet1 | |
| |_______|| | /---|__________|---\ | | |_______| |
|____________| | | | | |_____________|
____________ | | | | _____________
| Node C | | | | | | Node D |
| _______ | | | | | | _______ |
| | knet0 |=----/ | | \----==| knet0 | |
| |_______|| | | | |_______| |
| _______ | | | | _______ |
| | knet1 |=------/ \------==| knet1 | |
| |_______|| | |_______| |
|____________| |_____________|
In this example, knet0 on each node connects to the same device on the other two nodes. For redundancy, we've added a second kronosnet device, called knet1.
Install
At this time, installation is a very manual affair involving:
- Cloning a git repository
- Switching to a known-good branch
- Compiling and installing
- Setting up pam.d.
- Adding a user and group
- Starting the daemon
- Configuring the interfaces
- Save the config to a file
For The Impatient
This is a pretty dirty little chain of bash commands that will do everything up to starting the daemon. These aren't well documented as most of this will be taken care of in the eventual RPM package.
At the time of this writing, the checkout with hash a7705b6daaf3ec55e17df3db15f2e2c16a0ffa57 is known to work. The code changes daily, and sometimes hourly. I'll update this page as quasi-stable checkouts are found to be stable.
# Meet dependancies
cd ~
yum -y groupinstall "Development Tools"
yum -y install pam-devel
# Setup the user and group.
groupadd -r kronosnetadm
groupmems -g kronosnetadm -a root
useradd kronosnet -b /etc -M -r -s /bin/bash -G kronosnetadm
passwd kronosnet
# Create a pam.d directory entry. We cheat by linking to an existing entry.
ln -s /etc/pam.d/system-auth /etc/pam.d/kronosnet
# Now clone, checkout, compile, make, install and run the daemon
git clone git://github.com/fabbione/kronosnet
cd ~/kronosnet
git checkout a7705b6daaf3ec55e17df3db15f2e2c16a0ffa57
./autogen.sh
./configure
make
make install
kronosnetd
# Finally, login! Note that by default, sessions will close after 60
# seconds of inactivity
telnet localhost 50000
Create a user and group for kronosnet to run as.
### This doesn't add the user to the group for some reason...
groupmems -g kronosnetadm -a kronosnet
Set a password for the new kronosnet.
Configure
The configuration is done in the kronosnet command line which you can access with telnet. Once your setup is how you like it, you can tell kronosnet to save it to a file for loading when the daemon (re)starts later.
Overview
At this time, you must know and and define all nodes in the network, all ethernet interfaces they have and what their IP addresses are. Plans are to make this more adaptable over time, but for now this is an admitted draw-back. For this reason, it is strongly advised that all of the interfaces you plan to use for kronosnet have static IP addresses. Either IPv4 or IPv6 will work fine. In this paper, IPv4 will be used for familiarity's sake.
Nodes
To have a common verbiage, lets define the following two nodes, each with three statically assigned network interfaces.
- an-node03
- eth0 - 192.168.1.73
- eth1 - 192.168.2.73
- eth2 - 192.168.3.73
- knet0 - 192.168.4.73
- an-node04
- eth0 - 192.168.1.74
- eth1 - 192.168.2.74
- eth2 - 192.168.3.74
- knet0 - 192.168.4.74
The kronosnet Command Line
an-node03
configure
interface knet0 0
baseport 50000
mtu 9000
ip 192.168.4.73 24
peer an-node04 1
link 192.168.1.74
exit
link 192.168.2.74
exit
link 192.168.3.74
exit
exit
start
exit
exit
exit
an-node04
configure
interface knet0 1
baseport 50000
mtu 9000
ip 192.168.4.74 24
peer an-node03 0
link 192.168.1.73
exit
link 192.168.2.73
exit
link 192.168.3.73
exit
exit
start
exit
exit
exit
Create The pam.d Access File
ln -s /etc/pam.d/system-auth /etc/pam.d/kronosnet
ls -lah /etc/pam.d/kronosnet
lrwxrwxrwx 1 root root 22 Dec 13 15:27 /etc/pam.d/kronosnet -> /etc/pam.d/system-auth
Run the Daemon
There is currently no init.d script for kronosnetd, so it will need to be started manually.
Must run os 'root'.
kronosnetd
There is no output from that command, and it will daemonized and return you to the shell. You can check that it is running with ps.
Network Setup
In this tutorial, we'll use four nodes, each with three network cards.
- an-node03
- an-node04
- an-node05
- an-node06
To log in for the first time, use telnet to connect to the localhost on TCP port 50000.
telnet localhost 50000
Trying ::1...
Connected to localhost.
Escape character is '^]'.
Welcome to kronosnetd 0.1 (built Nov 28 2010 13:32:00)
login:
At this point, you can login using the kronostnet user name and justatest password. Once logged in successfully, you should see the knet prompt.
login:kronosnet
Password:
Welcome kronosnet (::1) on vty(0)
knet#
To quit at any time, simply enter the telnet escape code. This is usually ^] (<ctrl> + ])
Planned
Below are features that are planned.
Costing
This is planned for 0.1 or 0.2. Basically all links parameters (except the ping/pong timeout) are important/relevant when there is more than one link between 2 given nodes.
They will define the policy on where traffic should flow between A and B.
Links can be active and passive.
2 active links will always send traffic at the same time and they will have the same "cost".
1 active link and 1 passive link: the active link will have a lower "cost" than the passive. Traffic will always prefer the lower cost link. If lower cost link will be unavailable, the next one in list will be used.
This setup can allow gigantic amount of flexibility as you can have:
link1 cost 10 (preferred active)
link2 cost 20 (passive)
link1 cost 10 (preferred active on 10Ge)
link2 cost 20 (1Ge)
link3 cost 20 (1Ge)
if primary fails, use 2 x 1Ge in active/active setup.... etc.
link1 cost 10
link2 cost 10
link3 cost 20
Aggregation
Note that active/active links (or same cost links) will not be aggregated yet (aka you don't get 2Gb/sec but you get 1G + immediate redundancy in case one link fails.
Notes
Stuff here is random
- 8 links per node
- 2^16 (64,000) nodes
- git checkout c2a42967d7dbba7c06a6124760c9881337499986
- conf -> write will save the config and load it when the daemon starts
- /etc/kronosnet/kronosnet.conf
- it will be possible to define it manually, for testing is set to 60 seconds, default will be 600
- Seq # does not have to be sequential, just a number between 0 <= x <= 255
- if interface foo has value 0, no peer can have 0
| Any questions, feedback, advice, complaints or meanderings are welcome. | |||
| Alteeve's Niche! | Enterprise Support: Alteeve Support |
Community Support | |
| © Alteeve's Niche! Inc. 1997-2024 | Anvil! "Intelligent Availability®" Platform | ||
| legal stuff: All info is provided "As-Is". Do not use anything here unless you are willing and able to take responsibility for your own actions. | |||