Kernel Summit

Two weeks back was the Linux Kernel Developers Summit. The Kernel Summit is a two day, developer-to-developer, invite-only conference where the kernel hackers sit down and talk about our feelings. It is not, unfortunately, a hack fest. Instead, the big arguments too heated or otherwise involved for the mailing lists are discussed. An attempt is made to reach consensus on the future direction of the kernel. A cabal at its best.

For whatever reason, I failed to blog the conference from the conference. To make reparations, here and now I will go over some of my thoughts.

Monday started pretty slowly. It was fairly clear, right off the bat, that this conference would mirror last year - little contention and lots of agreement. Compared to the summit two and three years ago, the lack of violent disagreement is always surprising. It certainly drives home the point about the state of kernel development. We started off with a processor panel, which was pretty neat. Both Intel and AMD provided us with honest-to-god new information and not a marketing spiel. We moved on to virtual memory, software suspend, kobjects, and then video drivers. For that session, the always charming Keith Packard stopped by to give us an overview of the current world of graphics hardware and the current situation with video drivers. We discussed where to go from there. I told him to be really harsh on us. I do not think he was harsh enough, but he definitely got across the point that the current DRI-fb-X menage a trois is not the way to go.

Following Keith's session, I chaired a session on improving desktop performance from the kernel's point of view. Issues such as boot time, interactivity, desktop-optimized I/O schedulers, and a reduction in the need for polling were discussed. Linus pointed out that he traced some desktop application and it made 30,000 system calls, touching thousands of files, in the initial seconds of start up. Clearly, he noted, the onus lies more so on these applications than the kernel for improvement.

Tuesday found a customer panel and topics on clustered storage, kexec, RAS, networking, asynchronous I/O, multipath I/O, virtualization, security, CKRM, and OSDL. Much of it was yawn material.


Kernel Summit Group Photo

The final session raised some contention. As Slashdot and other fine news reporting agencies have noted, we have decided to delay the release of 2.7. Yes, delay, not cancel. I have never been a huge fan of the kernel development process - especially after working on GNOME - but I am cautiously interested in this new direction for 2.6. Especially since it really is not a new direction. We are, in fact, ready to and capable of forking 2.6 and starting 2.7. But 2.6 is mature, stable, and we are all generally very happy with it. We also do not have any huge stability-crushing baby-killing changes queued up. Instead, we have a lot of incremental changes, new features, and such. We have been able to slip a lot of these into 2.6, in fact -- we merged 4K stacks, huge scheduler behavior changes, and object-based reverse mapping in post 2.6! A solid 10MB of patches a month have found their way into 2.6. Yet stability has not suffered. 2.6 is better off than any previous kernel, and the dynamic duo of Linus and Andrew is really working. Progress is fast, features are merged, yet stability remains. The question we asked, then, is why change that?

The answer, as noted and second guessed and complained about, is that there is no reason to. As long as we can sustain it, we will continue to use Andrew's 2.6-mm tree as a staging ground for new patches. Linus will continue to work on 2.6. 2.6 will remain a stable kernel, yet from time-to-time see more adventurous (but still well-tested) patches. This is the stuff that vendors ship, anyhow. Ultimately, if the size or stability of 2.6-mm ever becomes unwieldy, we can then fork 2.7. In the meantime, 2.6 can remain stable (in both the code sense and the user-space API/ABI sense) yet address the too-long-release-cycle problem.

The only risk is a loss of time on the kernel developer's part. If we realize that this method is unsustainable in the face of stability, we will merely end up branching 2.7 later rather than earlier. I have faith that it will work, however. The combination of Andrew and Linus working together is gold. Andrew's patch scripts let him manage everything on a per-patch basis, and he can easily maintain a tree of patches against 2.6 proper. Linus's use of BitKeeper, on the other hand, allows him to easily pull from the trees of maintainers. Also, Andrew just rocks.

All in all, the kernel summit was a lovely opportunity to get together and see friends I see much too infrequently -- especially Pat, Greg, Zwane, Chris, Rusty, and Andrew. Some good decisions came out of it, but the most important is the very clear notion that the kernel is mature in terms of features and that innovation lies elsewhere. Which is fine with me.