The OpenFlow (OF) spec defines an OF Channel as such:
The OpenFlow channel is the interface that connects each OpenFlow switch to a controller. Through this interface, the controller configures and manages the switch, receives events from the switch, and sends packets out the switch.
The ONF competition required a communications driver to support OF. The driver was to be usable by either a Switch or a Controller and manage the OF channels to the peering SDN device.
The availability of a driver such as this eases the barrier to entry for developing new controllers or switches that support OF. By encapsulating the wire communications in a shared library, the development of controller or switch can focus primarily on the OF protocol itself.
The submission criteria included:
- Code for driver
- Code for sample controller integrated with driver
- Demo with integrated controller and OF switches (virtual or physical)
- Proven platform independence
- Bindings to two other programming languages
- Preference for code written in C
The CodeChix team put their heads together at recurrent hacking sessions with pizza and a white board. The technical challenges were many.
- Which controller do we use for integration?
- Which programming language for the driver?
- What cloud platform to support a shared test setup and demo?
It took nearly 4 weeks of unearthing open source controllers in every language possible – Java, Python, and C/C++, compiling and setting up test environments and walking the code of each to determine the effort in integrating the driver. Floodlight, ryu, NOX, POX, Trema, and Mul. We finally concluded on Mul for these reasons:
- Mul was written in C. This worked for us as the core competence in the team was in C programming.
- The code was easy to read and therefore lent itself to modification.
- It was being maintained actively by KulCloud based in South Korea and
- The software architect of the open source repo agreed readily to support the integration by answering our design questions, doing code reviews and the likes.
So, four weeks into the project, we had our SDN controller and a vague idea of what the driver was to accomplish.
It took another 4-8 weeks before we worked out the cloud platform for a shared test setup using mininet, and wireshark along with the integrated controller. We set up Ubuntu 12.04 LTS on Amazon’s AWS, installed mininet and wireshark and were good to go. A fairly expensive solution but the best we could find.
The driver had to be a shared library. None of us had built a stand-alone shared library and compiled an application with it. At the end of the project, we learnt how to write Makefiles, manipulate LDFLAGS and CFLAGS for creating shared library, use GNU tools such as ldconfig to install the library and finally compile the custom library with application code.
Hash tables, platform independence, unit test framework – we had no cycles to write all this necessary infra code to support the library. Frantic searching for some open source options led us to GLib-2.0, Gnome’s SDK, not to be confused with glibc, the standard C library. We installed the library, compiled a hello world and proceeded to use it. Glib-2.0 did wonders for the project in these aspects:
- platform independence – with very little to no code change, the library compiled on ARM and x86 alike
- unit test framework – with very well defined API and terrific logging with XML and HTML generated output, the unit test framework was worth its weight in gold.
On the other hand, glib-2.0 was difficult in these areas:
- Hash tables – the largest chunk of unit testing effort and bug fixing was in this area. Never again will we use glib-2.0 for hash tables.
- Event loops – the loops didn’t dynamically accept new events; all events had to be defined before starting the loop processing.
We decided to use github to host our two repos – one for driver and other for integrated controller – on github private accounts. A dev branch was created for each and workflow established. Each contributor would fork a copy of the main repo, clone the fork and merge changes to upstream using pull requests. The process of pull requests allows for collaborative code reviews before merging to upstream. This aspect however is the hardest to manage as it is extremely error prone. Pull requests gave the team many a moment of ‘oh no.. what did I do’. Fortunately git is highly reversible and recoverable.
An existing, working SDN controller was to be integrated with our driver. Given that the functionality in the driver would already be present in any given controller, the primary objective of this task was to break apart the controller code to separate out the functionality that the driver provided. Once this was accomplished, the next step was to call the library API and finally write a whole lot of glue code to make controller and our driver talk to each other. We had to surgically remove and replace a working part with our driver!!
Next time, we’ll share aspects of design of the shared library along with code snippets.