Distributed Compiling

Distributed compiling was one of the original goals of this project from the beginning. We do not need to throw a real lot of computing power at gcc in order to compile something, but compiling some things (like GCC itself!), we can really benefit by using a lot of different jobs if hard drive and network IO do not slow us down. There happens to be a very excellent program called distcc that acts as a front end to run GCC over several machines at the same time. Since we installed distcc on our original cluster master before cloning it, we only need to jump start the daemon using our cluster_command.sh script.

Network IO could really hurt us here. Each slave machine can utilize the entire 100Mbps of network traffic, but our master might be uploading to more than one machine at a time. If we have to upload files to seven other machines on the cluster at once, we would be limited to only 1.7 megabytes (14.3 megabits) per second. It may make a lot more sense for us to run a separate dedicated PC with a gigabit Ethernet card as the master instead - at least for distcc. We will test both cases here to see how network IO affects our build.

Since there are eight machines on the cluster, we want to make sure that there are enough make jobs running to satisfy each processor on the node. Deciding the exact number of jobs under distcc is not an exact science, but fortunately, all of our machines are the same speed, so that will alleviate some headaches. We tried compiling GCC 3.4.2 on our cluster using distcc under 9 and 17 jobs.

# ./cluster_command.sh distccd  -daemon
# export DISTCC_HOSTS='master slave1 slave2 slave3 slave4 slave5 slave6 slave7'
# make  - j9 CC=distcc

Kernel 2.6.4 Compile (GCC 3.3.3)

We didn't really see the performance numbers that we were looking for here. We first anticipated poor performance on the XBOX cluster due to its small amounts of RAM. "In theory", if we can get our cluster to scale to 16 nodes without a huge performance hit, we would see very impressive compile scores. However, running 16 threads on a single user application does not occur that often, even with a large compile like GCC, since many things need to be made in order. A multi-user environment running hundreds of compiles at once would benefit from so many nodes; perhaps a community cross-compiling station. Just in case, the master XBOX cluster is not performing poorly due to the meager 100Mbps network card. We reconfigured the cluster to obey a different host with a gigabit Ethernet card. We ran the same command as above.

Kernel 2.6.4 Compile (GCC 3.3.3) (no master)

It looks like our cluster really didn't get too affected by the network IO after all. If we anticipate running more XBOXes, however, running the cluster from a dedicated master with at least one gigabit Ethernet card would be absolutely necessary.

A Beowulf Cluster (Con’t) Distributed Rendering
Comments Locked

30 Comments

View All Comments

  • ukDave - Wednesday, November 10, 2004 - link

    Yes the xbox has a PS1 emulator, only one that i know of. Its called "PCSXbox". Good place for emulator info: http://xport.xbox-scene.com/ , obviosuly a subsection of xbox-scene.

    And before anyone asks, no there are none for the PS2 (nor will there ever be).
  • ukDave - Wednesday, November 10, 2004 - link

    I'm not 100% sure about the SmartXX, as i have the well known Xecuter 2.3b chip which has DIP switches mountable on the outsode of the xbox so that youc an disable it in order to play X-Box live and whatnot.

    From what i've just read over on the forums at xbox-scene, SmartXX is fully X-BoX Live compliant. You simply (as you say Kris) boot back into the MS BIOS. This can be done by turning the xbox on by pressing the EJECT button. To enable the modchip, turn on with the POWER button. (source: http://forums.xbox-scene.com/index.php?showtopic=3...

    SmartXX section of the forums: http://forums.xbox-scene.com/index.php?showforum=6...
  • ksherman - Wednesday, November 10, 2004 - link

    #3, you said there is mulators for n64 and down. Is there an emulator for PSone games? would love to be able to play FF7 again
  • KristopherKubicki - Wednesday, November 10, 2004 - link

    Jeff7181: I think with the SmartXX chip you can actually boot back into the normal MS BIOS and have no problems with X-Link and such. If you install Linux on the last 2 unpartitioned GB of your 10GB drive, your XBOX should basically have no idea that the machine is modded at all.

    LotoBak, ukDave, am I correct on that statement?

    Kristopher
  • ukDave - Wednesday, November 10, 2004 - link

    Oh and Jeff, i believe many mod chips have the facility to simply turn them off with a flick of a switch, so you can continue to use XBOX Live. But as LotoBak says, there is the free X-Link service available too.
  • ukDave - Wednesday, November 10, 2004 - link

    http://www.xbox-scene.com is really the only place to go for XBOX related discussion, tutorials, really anything you need to know that isn't covered in this article.

    I bought a bodged XBOX mod from someone in the UK, got it professionally fixed and re-modded. It now has a 120GB disk for all my games, its connected to my LAN. With the excellent XBMC i can play literally any media file from any PC over the network, be it standard files such as MP3's or AVI's, or images such as .bins/cue's.

    Well worth looking into if you have an XBOX that is currently un-modded.

    Also, /me wants 8 XBOX's :D Although, those 'leccy costs, eek!
  • Jeff7181 - Wednesday, November 10, 2004 - link

    I've been thinking about modding mine to use to play video over the network... but I don't want to lose normal Xbox (and Xbox Live) functionality. Wish that was covered here, but I guess it's not really a "mod your Xbox" article as much as it's a "mod an Xbox to make a cheap PC" article.
  • LotoBak - Wednesday, November 10, 2004 - link

    Nice guys... besides your linking problem

    Knowing a little more about the xbox scene a few things your might consider...

    1) Mods go for as cheap as 10USD retail.
    2) Software exploiting the xbox is always an option which results in a 'free' modchip
    3) http://tutorials.xbox-scene.com to learn about most anything
    4) XBMC dvd menu support is comming!
    5) XBMC irc channel on irc.efnet.info #xbmc
    6) Online gaming is free through Xlink www.teamxlink.com
    7) Emulators for n64 and 'down' are available
    8) This is the cheapest htpc with 480p 720 and 1080i available
    9) xbmc supports many formats
    Audio : wav, aac, ac3, mp3, flac, ra, vorbis
    Video : ogg avi mkv rm, mov containers
    Codec : xivd, divx, mpeg2, mpeg1, quicktime, realvideo and realaudio (limited) and many others. The video engine is based apon mplayer open source project

    Networking :
    Shares :Supports ccx, samba (windows file sharing), and replaytv shares.
    Streaming : Supports a multitude of video and audio streaming formats. webradio is included which has a 'channel' browser for online shoutcast streams. There is also a apple movie trailer browser.

    Scripting :
    XBMC has python scripting language implimented so its open to many possibilities

    XBMC does have some rough edges but it is maturing quickly. If you are interested in contributing to the project check out www.xboxmediacenter.com and the 'XBMC' project on sourceforge

    Feel free to ask questions about xbox stuff here... ill watch it for a while
  • Marlin1975 - Wednesday, November 10, 2004 - link

    http://www.anandtech.com/linux/showdoc.aspx?i=2271...
  • Marlin1975 - Wednesday, November 10, 2004 - link

    Page 6 has HOME instead of page 7 link

Log in

Don't have an account? Sign up now