Log in

No account? Create an account

Previous Entry | Next Entry

OLS: Day Three

Virtualization is a big topic in Linux now. Friday's schedule offered no less than seven events on that topic. I'm passingly familiar with User Mode Linux (my web server's on a virtual Debian machine under UML), but this year's star is a virtual machine monitor I'd never heard of called Xen.

I started out the day with a usbmon talk. The idea's simple enough: a human-readable display of USB traffic. The goal is to have software that provides ethereal-like parsing and a tcpdump-like command-line utility, but for USB, plus a binary, versioned API for user mode. zaitcev talked about how he used it to save time debugging and showed us a demo. Since I've used a bus analyzer (for a very different bus type) at work, I feel somewhat comfortable with them; there's something satisfying about seeing every bit going back and forth and understanding just what's going on. Nobody needs to convince me of their value. zaitcev envisions analogous projects for other protocols: scsimon, infinimon, bluemon, firewiremon, wifimon, fibrechannelmon, and so forth. Right on.

Aside from the device driver-writing tutorial, the event at OLS that I was most looking forward to was going to be Jon Trowbridge's Beagle/inotify talk. So I was disappointed to see that the speaker couldn't make it, so the talk was canceled. Dejected, I returned to my hotel room and took a nap.

After I awoke for the second time yesterday, I went to a "non-technical" talk on SeqHoundRWeb.py, a Python-based interface to online bioinformatics data. It started with the shortest-ever lesson on molecular biology so we could get an idea of the kind of data our presenter works with. We learned that the world of genomics includes many large sources of data collected by different methodologies and characterized by different vocabularies. The challenge is integrating them and making them easily available and searchable for researchers. The speaker chose Python largely for the usual reasons: it's a clean object-oriented language, there's good learning documentation, it has broad cross-platform support. He mentioned that it also has good support for other resources like BioPythonbioperl and BioRuby notwithstanding, I suppose.

Next on my agenda was Case Study: Usage of Virtualized GNU/Linux to Support Binary Testing Across Multiple Distributions. The motivation was a need to test system initialization scripts on multiple distributions independently and concurrently, when each test takes a long time to run. Multibooting with grub or lilo wouldn't allow concurrent testing; vmware uses too much memory. The solution in this case was to run User Mode Linux, configured with a Gentoo host operating system with a slightly modified disk I/O "elevator sorting" algorithm. The guest operating systems were Novell Linux Desktop 9, Red Hat Enterprise Linux 3 and 4, and one of the Red Flag products. The result: they saved money and lab/desk space, and they ended up with a quickly re-usable hardware platform. They noted that a virtual Linux setup wouldn't be so useful for hardware abstraction tests, resource sharing, or inter-client communications. Future plans include replacing the host operating system with a hypervisor and addressing resource protection.

Delving deeper into the world of virtualization, I next attended Testing the Xen Hypervisor and Linux Virtual Machine. (In retrospect, without having attended any of the earlier talks on Xen, I was unprepared for one that assumed a degree of familiarity with it.) As part of the Linux Test Project, the researchers used a tool called Xenfc that makes random hypervisor calls. Most of the calls are expected to fail, and the tool checks to see that the correct error codes are returned. They tested Xen's balloon driver, a feature that lets you change the amount of memory that's available to each domain. They tested SMP support and found that SMP in Xen isn't yet stable. And they developed Xentest, an automated testing framework centered around Xen.

For dinner I headed to Elephant & Castle, a chain-restaurant pub, where I had some good Irish stew. I thought it would fun to try a Tom Collins cocktail, but I found I didn't like it much. I tend to assume that classic cocktails are subtle drinks, and this one is vaguely like a Gin and Tonic. What I failed to think seriously about is that fact that it's like a Gin and Tonic except you replace the tonic water with a combination of lemon and lime juices and sugar. Just too sour for my taste.

After dinner, I made it a point to attend the Debian Women: Encouraging Women without Segregation Birds-Of-a-Feather Session. The first thing anyone said about this session beforehand was, "A guy's leading it?" Yes, a guy led it, an enthusiastic Brazilian computer science student who reported that the girls were busy since Debconf 5 had just concluded. He presented the history of the Debian Women project and explained its mentoring program. My impression is that he's someone who generally "gets it," in spite of his habit of referring to women as "girls" while referring to men as men. I'm going to give him the benefit of the doubt and presume that he isn't familiar enough with English to understand the implications of his word choice there. Dana and Erinn were among the 'chix mentioned as being active in the Debian community. Also mentioned were linuxchix, KDE Women, GNOME Women, and—speculatively—Ubuntu Women. Possibly too much time was devoted to laughing at the latest misogynist troll, the response to whom is discussed by Mary. Overall, though, I think it was a worthwhile session.

The final session of the day that I attended was H'Uru - Coding Beyond MYST. The idea is that after a MMOG named Uru was released in 2003 and then canceled, our speaker set out to develop a Linux port of the game, with nothing to work with but a bunch of data files and some hints about the file format by some folks at the Clockwork Orange BBS. Much of the structural data was encoded in a PRP file format, organized like a relational database to store data like game-world geometry and transformations. Other file types contained Python bytecode for more complex interactions, saved game states, and which PRP files make up an age, among other things. Our speaker started by looking at hex dumps of these data files, or what he called "reverse engineering by meditation." He then wrote a utility called prpdump to interpret the data in a slightly more readable format. The Uru gameplay server is apparently a Python interpreter with lots of extension classes, so our speaker used decompyle to understand what the code does.

Uru's graphics were pretty sophisticated; the game used some advanced features of graphics cards, only some of which were supported in OpenGL, if I understand correctly. Its multilayer texturing combined up to four textures in various modes, and each texture possibly was animated, used a different material for each layer, used color and alpha at the vertex, used reflection coordinates, and/or used vertex blending. Our speaker found that rather than using one generic shader for this texturing, it's faster to generate specialized shaders and switch between them. Other performance hacks he discovered were doing static lighting as much as possible and using bounding boxes for occlusion culling.

Unfortunately, he couldn't get a demo running adequately on the laptop he'd brought, so I didn't get to actually see H'Uru. He reported that there's still lots of room for improvement: the shaders are still awfully slow, home-brew collision detection kinda sucks, octrees are "simply bad," sound doesn't work, materials and shading aren't always right… it's a work in progress.



( 1 comment — Leave a comment )
Jul. 30th, 2005 05:27 pm (UTC)
The first thing anyone said about this session beforehand was, "A guy's leading it?"

Xen is pretty wild. In a way, it's very similar to UML, but a lot of the virtual drivers are hefted out of the client OS and into the host interface, and they provide functionality that's closer to real hardware than what UML uses. For example, UML uses a host patch to create a separate address space without the host OS' process tree mapped in, and with a new permission table, but it's all accessed with standard Linux allocation calls. On the flip side, Xen presents the guest OS a virtualized translation lookaside buffer. And while UML hands off what's pretty much a PPP interface for networking, Xen implements ring buffers which look very much like what you'd feed a standard ethernet card. The above is cool, because you can take a guest OS, make very few changes - pretty much eliminating all the register tweaking you do when you boot or hunt for peripherals and just pointing them at Xen's TLB, net buffers, etc - and you're done. It's entirely feasible that you could port an entirely new OS overnight.

Last time I looked at it (this past winter) it wasn't very happy running without X though, which is too bad. Net and disk I/O benchmarks were significantly better than UML's on the same machine, but throwing X into the mix with a colo box is a bit daunting. I'm keeping my machines under UML for now!
( 1 comment — Leave a comment )