Distributed Hashing

As far as distributed computing goes, distributed hashing has always been one of the original uses of a distributed network. A novelty to some, a necessity to others, hashing a lot of keys takes a very, very long time depending on the algorithm. Hashing keys does not require as much memory as rendering or compiling, which will probably work well for the XBOX cluster. Below, you can see a quick print-out of how fast OpenSSL works on a single node of our XBOX cluster, in comparison to the Sempron 2200+ machine mentioned earlier.

XBOX

OpenSSL 0.9.7c 30 Sep 2003
built on: Mon Feb 23 18:18:12 GMT 2004
options:bn(64,32) md2(int) rc4(idx,int) des(ptr,risc1,16,long) aes(partial) blowfish(idx) 
compiler: cc
available timing options: USE_TOD HZ=128 [sysconf value]
timing function used: getrusage
The 'numbers' are in 1000s of bytes per second processed.
type              8 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes
md2                557.47k     1546.09k     2091.52k     2292.74k     2362.03k
mdc2                 0.00         0.00         0.00         0.00         0.00 
md4               9312.01k    50638.08k   104728.32k   143168.85k   160380.25k
md5               3405.55k    15094.19k    25995.73k    31601.87k    33746.11k
hmac(md5)         1456.18k     8698.11k    19730.66k    28729.00k    33202.18k
sha1              3989.46k    15676.20k    27201.45k    33287.51k    35621.55k
rmd160            2942.53k    12834.88k    21769.13k    26220.89k    27904.68k
rc4              42422.11k    56691.63k    59720.45k    60751.63k    60867.38k
des cbc           7463.69k     8229.14k     8364.52k     8364.03k     8366.76k
des ede3          2861.27k     2966.78k     2985.30k     2990.08k     2990.08k
idea cbc             0.00         0.00         0.00         0.00         0.00 
rc2 cbc           5796.07k     6218.69k     6275.93k     6291.11k     6313.30k
rc5-32/12 cbc        0.00         0.00         0.00         0.00         0.00 
blowfish cbc     16639.31k    20924.97k    21380.52k    21482.50k    21570.44k
cast cbc         12566.61k    14372.20k    14635.41k    14653.44k    14666.41k
                  sign    verify    sign/s verify/s
rsa  512 bits   0.0035s   0.0003s    286.7   3299.3
rsa 1024 bits   0.0195s   0.0010s     51.2   1049.0
rsa 2048 bits   0.1167s   0.0032s      8.6    314.0
rsa 4096 bits   0.7650s   0.0111s      1.3     90.3
                  sign    verify    sign/s verify/s
dsa  512 bits   0.0032s   0.0039s    313.5    259.5
dsa 1024 bits   0.0097s   0.0120s    103.3     83.4
dsa 2048 bits   0.0317s   0.0389s     31.6     25.7
OpenSSL>

Sempron 2200+

OpenSSL 0.9.7c 30 Sep 2003
built on: Mon Feb 23 18:18:12 GMT 2004
options:bn(64,32) md2(int) rc4(idx,int) des(ptr,risc1,16,long) aes(partial) blowfish(idx) 
compiler: cc
available timing options: USE_TOD HZ=128 [sysconf value]
timing function used: getrusage
The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes
md2               1307.26k     2797.72k     3917.72k     4364.70k     4511.21k
mdc2              3019.04k     3408.81k     3522.52k     3549.56k     3565.32k
md4              11304.80k    39934.94k   117963.41k   229924.63k   318395.57k
md5               8789.64k    29747.05k    81595.00k   144216.44k   185556.36k
hmac(md5)         5081.28k    18230.65k    57049.28k   121655.26k   181687.44k
sha1              8715.38k    26775.46k    63727.77k    97333.12k   115140.36k
rmd160            7386.51k    21200.32k    45464.16k    64026.54k    72683.38k
rc4              88866.45k    94630.03k    96065.66k    96649.60k    97376.13k
des cbc          18098.33k    18586.49k    18717.35k    18809.36k    18772.63k
des ede3          6551.90k     6620.66k     6660.51k     6666.11k     6667.71k
idea cbc             0.00         0.00         0.00         0.00         0.00 
rc2 cbc          15875.50k    16446.24k    16598.92k    16634.42k    16655.25k
rc5-32/12 cbc    73740.31k    88512.81k    92535.12k    94040.06k    94410.97k
blowfish cbc     44137.47k    48594.05k    49798.41k    50179.96k    50288.17k
cast cbc         30171.40k    32328.70k    32877.09k    33049.36k    33011.88k
aes-128 cbc      37067.76k    37903.95k    38377.07k    38499.61k    38531.59k
aes-192 cbc      32505.08k    33104.00k    33376.89k    33556.62k    33580.99k
aes-256 cbc      28702.51k    29251.51k    29532.40k    29604.97k    29624.04k
                  sign    verify    sign/s verify/s
rsa  512 bits   0.0014s   0.0001s    702.7   8190.2
rsa 1024 bits   0.0074s   0.0004s    135.4   2648.0
rsa 2048 bits   0.0452s   0.0013s     22.1    798.2
rsa 4096 bits   0.2956s   0.0042s      3.4    236.2
                  sign    verify    sign/s verify/s
dsa  512 bits   0.0012s   0.0015s    820.1    653.8
dsa 1024 bits   0.0038s   0.0048s    266.1    209.8
dsa 2048 bits   0.0122s   0.0152s     81.8     65.6
OpenSSL>

Our hopes of making this XBOX distributed cluster thing worthwhile experiment are slowly diminishing. Again, we aren't out to break any speed records here, but we would love to see some XBOXes scale well. As another point of reference, our Opteron 150 machine can sign about 1063 RSA 1024 keys per second (in a 64-bit environment); that's approximately 20 times faster than what the XBOX is capable of.

The XBOX is very slow, but having a whole lot of them might scale well. We look to one of our favorite programs, John the Ripper, for more advice. Distrubuted John, or djohn, behaves similarly to make, assigning a different portion of the total keyspace to each machine. Djohn scales extremely linearly, since code cracking times are very long and there are very few places for network latency to interfere with our hashing. Since it takes basically the same amount of computing power to hash the same length key with a different character set, our total password cracking power is equal to N times the password cracking power of one machine, where N is the number of machines in the cluster (assuming the machines are all the same speed). You can see below how various machines performed JTR benchmark tests. JTR was compiled with GCC 3.3.3 with only MMX optimizations.

Distributed John the Ripper 1.6.37 - DES [64/64 BS]

Distributed John the Ripper 1.6.37 - MD5 [32/32]

Distributed John the Ripper 1.6.37 - Blowfish (x32) [32/64]

A star denotes estimated performance. Our eight-way cluster fares pretty well against its workstation competition, but properly manipulating the compile options could tilt the results in any configuration's favor. Also keep in mind that 64-bit compilations of JTR yield up to 20% performance boosts as well on Blowfish, and we get those performance benefits from the x86 platform on which the XBOXes run. Cracking keys shows some immediate promise on our XBOX cluster, but looks to be the only real application.

Not to give the XBOX too much credit here, the puny Sempron performs much better at Blowfish and MD5 hashing. Building an equivalent cluster of Sempron machines would yield more than double the crunching power on Blowfish, and even more on MD5.

Distributed Rendering Final Thoughts
Comments Locked

30 Comments

View All Comments

  • Nepsir - Thursday, November 11, 2004 - link

    I was just wondering if you will do a test with mental ray on the cluster (I think you mentioned something about Mental Ray in the article). It would be really interesting to see the results of that since me and a couple of friends got modded X-Boxes and I have a 3dsMax6 license (Mental Ray is included).
  • gibson042 - Thursday, November 11, 2004 - link

    You might want to look at the "mini-cluster" project (http://www.mini-itx.com/projects/cluster/) when planning/designing/building your EPIA cluster. It's a 12-node (originally 6) cluster developed by Glen Gardner, and appears to be the same size or a little smaller than a stack of an equal number of XBOXes. Of course, the frame is custom built and the hardware is exposed to the world.
  • Aikouka - Thursday, November 11, 2004 - link

    I can say that I love having a modded xbox, because the capabilities are so much greater. As discussed earlier, a software mod can be a great way to do it (although it usually requires a game unless you want to take your HDD out, which is NOT a fun method unless you love swapping IDE cables.)

    It's also good to note that you should be careful when modding these or even make a backup of the hdd on your system, because things can go wrong, and if you like to fiddle with things, then your chances will be a lot higher. I know at one point I was stuck in error 16 and had to pull the HDD out and unlock it using the IDE swap method.

    Are there plans for an article on modding something such as the PS2? I just recently tried a no-solder modchip in the PS2, and it doesn't seem to be going along too well. Maybe it's time for a flip-top.
  • Scarceas - Thursday, November 11, 2004 - link

    "We crimped our own cabling as you can see in the image below."

    I'm still looking for the image, I know its there somewhere!
  • bleugh - Thursday, November 11, 2004 - link

    If anyone wants their xbox motherboards upgraded to 128 megabytes of memory, I can do it

    I'm UK based and use BRAND NEW samsung memory chips

    I can upgrade your existing motherboards, sell you ready upgraded boards, or even just the chips!

    if one of the anandtech team wants their xboxes upgraded i'll happily do it at cost, please contact me to discuss

    Dean
  • LotoBak - Thursday, November 11, 2004 - link

    First, XBox live banning....

    Basically this is how it used to work. When your on live it does a hash of your bios. A mod replaces your bios. If you bios is not the retail ms bios the serial (eeprom) is blacklisted. NOW since halo2 has been released it appears that MS has changed a while bunch of shit on us. Preliminary information indicates that MS is banning by hard drive serial+model numbers, xbox serial, and flagging the gamertag as 'potential pirate'. Thus to unban the xbox both a new hdd and eeprom are needed. Then you must cancel and create a new xbox live account. There are more details about how it all works but bottom line is we dont know how it works now. They can throw new stuff at us anytime now. Rumors are flying that hdd capacities are being scanned as well. For all intents and perposes xbox + modded xbox's do not mix.

    Yes there is a PS1 emu for xbox. That said xbox DVD drives do not read cd-r (or retail ps1 games) Therefor all games must be transfered to the xbox hdd in bin/cue or iso format

    When comparing mods remember, the mod contains no modded bios stuff. They are useless in them selves. You will need to aquire a modded bios for your xbox (online, free). What this means is that every mod can contain the same bios's. Therefor they all contain the same base features (mostly booting unsigned code). Hardware features and support is where you want to base your decision on. If you want cheap + no manufacterer support there are 10$ solutions around for you(enabled/disables with power button). If you want support with a mod with a few extra hardware features (2 bios's, external switch) that'll cost around 30-40. Fancy Shamcy lcd support, 8 bios banks, blah blah blah fancy external thing thoes are up to 75ish. But in the end they all allow you to run the same software on the xbox.

    if you want to learn read some beginner tutorials at http://tutorials.xbox-scene.com
  • Omega215D - Thursday, November 11, 2004 - link

    After seeing Halo 2, it makes me wonder if they really need Pixel Shader 2.0. Its pretty cool that I can play old games on the XBOx through emulators though but i wanna see when the next XBOX kicks off before spending $200.
  • ViRGE - Wednesday, November 10, 2004 - link

    #12, probably not. The Xbox's GPU is a GeForce 4 class GPU - the lack of Pixel Shader 2.0 features can really be limiting.
  • euph - Wednesday, November 10, 2004 - link

    is there any way to use the gpu for the pov-ray rendering?
  • ukDave - Wednesday, November 10, 2004 - link

    Little quote that sums up the SmartXX XBL saga:

    Q: just bought a Smart XX mod and I still don't know if i'll be able to play xbox live so please somebody tell me if i'll be able to play it.
    A: There is no easy way to answer this question. First of all, it all depends on circumstance. Have you upgraded your hard drive? Has your Xbox ever been banned before? And, for newbs, are you thinking about playing on live while using a bios from the chip.

    If you answered yes to any of these questions you've won a lifetime ban from Microsoft!


    Sleep time :) And i don't even use XBL :D

Log in

Don't have an account? Sign up now