The AnandTech Linux XBOX PC Experiment
by Kristopher Kubicki on November 10, 2004 4:00 PM EST- Posted in
- Linux
Distributed Compiling
Distributed compiling was one of the original goals of this project from the beginning. We do not need to throw a real lot of computing power at gcc in order to compile something, but compiling some things (like GCC itself!), we can really benefit by using a lot of different jobs if hard drive and network IO do not slow us down. There happens to be a very excellent program called distcc that acts as a front end to run GCC over several machines at the same time. Since we installed distcc on our original cluster master before cloning it, we only need to jump start the daemon using our cluster_command.sh script.Network IO could really hurt us here. Each slave machine can utilize the entire 100Mbps of network traffic, but our master might be uploading to more than one machine at a time. If we have to upload files to seven other machines on the cluster at once, we would be limited to only 1.7 megabytes (14.3 megabits) per second. It may make a lot more sense for us to run a separate dedicated PC with a gigabit Ethernet card as the master instead - at least for distcc. We will test both cases here to see how network IO affects our build.
Since there are eight machines on the cluster, we want to make sure that there are enough make jobs running to satisfy each processor on the node. Deciding the exact number of jobs under distcc is not an exact science, but fortunately, all of our machines are the same speed, so that will alleviate some headaches. We tried compiling GCC 3.4.2 on our cluster using distcc under 9 and 17 jobs.
# ./cluster_command.sh distccd -daemon # export DISTCC_HOSTS='master slave1 slave2 slave3 slave4 slave5 slave6 slave7' # make - j9 CC=distcc
We didn't really see the performance numbers that we were looking for here. We first anticipated poor performance on the XBOX cluster due to its small amounts of RAM. "In theory", if we can get our cluster to scale to 16 nodes without a huge performance hit, we would see very impressive compile scores. However, running 16 threads on a single user application does not occur that often, even with a large compile like GCC, since many things need to be made in order. A multi-user environment running hundreds of compiles at once would benefit from so many nodes; perhaps a community cross-compiling station. Just in case, the master XBOX cluster is not performing poorly due to the meager 100Mbps network card. We reconfigured the cluster to obey a different host with a gigabit Ethernet card. We ran the same command as above.
It looks like our cluster really didn't get too affected by the network IO after all. If we anticipate running more XBOXes, however, running the cluster from a dedicated master with at least one gigabit Ethernet card would be absolutely necessary.
Facebook
Twitter
RSS
30 Comments
View All Comments
TimPope - Thursday, May 12, 2005 - link
not bad information but i would have liked to see some kind of real world performance using openmosix.. a single x box on its own as a pc is slow but stick 2-4 together using open mosix could make a reasonably good machine and still be pretty cheapHalz - Wednesday, November 17, 2004 - link
The rule followed in the article for the -j option, "number of proccessors + 1", overlooked the logical proccessors of the Xeon's Hyperthreading.. -j should have then been something around 5 instead of 3Halz - Wednesday, November 17, 2004 - link
Simply compiling on the Opteron and Xeon with the same number of threads as the full cluster would have illustraighted a difference.More testing should have gone into finding how many threads was the ideal number for the given platforms.
artifex - Saturday, November 13, 2004 - link
Aikouka, can't you just use one of those "HD Loader" type programs WITHOUT a modchip?I'd be all for modding my PS/2 if I thought I could actually do something useful with it, like stream audio/video from a PC or a ReplayTV or something.
KristopherKubicki - Saturday, November 13, 2004 - link
Halz: what should it have been?Kristopher
Aikouka - Thursday, November 11, 2004 - link
23, yes, you can still do just about anything. I know with the software mod that I use, I've been having problems getting the original MS Dash to load up, but I've gotten around that using other programs for the original dashboard's functionality (dvd etc).You know, you can also replace the HDD with just a software mod, and it's not that hard. So, if you don't want to hardware mod and want more space, you can still put in a bigger HDD. As much as some people don't like the XBOX, in my opinion, it's probably the best console to mod.
24, 2) Modchips also allow hdd loading if you have the PS2 HDD (using HDDLoader.) Also, it lets the warez'ers download and play games on the PS2 that they don't really own.
artifex - Thursday, November 11, 2004 - link
1) what we really need is a usb-based tv tuner that actually works. That would be excellent for adding functionality both to XBoxen as cheap PVRs (though I'd still just use XBMC to stream from my ReplayTV, most of the time), but also would be great for iMacs. I'm sure if someone came up with a decent open architecture design, the community would come up with drivers for both types of systems.2) what are modchips for PS2s useful for, other than playing import games? Especially with the new PS2s having no drive (is there still a header on the new board style to add one back?)
3) did I miss the obligatory dnetc test? You gotta do that, you know.
Booty - Thursday, November 11, 2004 - link
I don't even own an Xbox, but reading this article has me reaching for my wallet...But first, I want to get this straight - I can mod the Xbox and still use XLink, right? I doubt I'd get a Live subscription anyway, but it'd be nice to have that option possible.
Ideally I'd like to throw a bigger hard drive in there and then run XBMC, without losing the normal XBox capabilities.
So if I can do that, I'm goin' to the store this weekend... :)
Halz - Thursday, November 11, 2004 - link
The compile options for the Opteron and Xeon were starving the CPUs; the number of jobs (-j) was no where near optimal.Halz - Thursday, November 11, 2004 - link