Fragwell By Name...

As previously noted, I have been documenting the supercomputer build at [H]ard|Forum (Hardforum, for search simplicity). Well, things are never so simple when I'm deep into them as they may seem on the surface when I start out because I like to go for elegance and longevity in everything I do.

It's damn near winter now and I'm working on the power/data system, not just for the supercomputer, but for the whole Fragwell Laboratory. It's already driving me insane just from playing "Beat The Clock" with Mother Nature.

This is the real deal. NEC 2008 Code-compliant and built to last a lifetime and then some. While there's no way I'll be staying in a trailer park for life, I'm reluctant to stay beyond Age 40, I want the next owner to have something extra that sets this house apart from the others.

For the next decade, this is where I will learn my Kung Fu. I hope I outgrow it before 2020.

Supercomputers - Two of them

I've been documenting this over at the [H]ardForum.

I almost finished Dashcat and had bought my switch when a larger rack of the same brand I used for Dashcat came available. Couldn't pass that up. Dashcat is being expanded to 10 nodes from the original eight and will have UPSes and the master server installed in the rack, in addition to an LCD console and storage drawer. Atop the rack, since it came with an adaptable 5U shelf, I have affixed the ICEBox controller, switch and two Belkin Omniview Matrix 8-port KVMs.

Dashcat only requires eleven ports out of the sixteen total KVM ports available. One will be reserved, on a long cable, for computer repairs so I can use the LCD console as the interface and save space on the workdesk.

As mentioned, the KVMs are Omniview Matrix models. These units allow two consoles with the interface of any two connected machines being displayed at the same time. As I'll only have one console, the second console port is open. What to do? Easy. Run it out to a KVM repeater, through one of the UTP links into the house, into the repeater catcher and to one of the displays on my desk (neither of which I'm using the VGA ports on).

The Omniview Matrix 2x8 was a lucky buy. It supports control through the OSD, hotkey commands, and the front panel. It's interesting that I originally had two of them, sold one for $50, decided to use the remaining one, ended up short by one and bought one on eBay for $25 total.

As for the second supercomputer, it's really an experiment in condensing the computing power of the larger system. Same CPUs, same motherboards, smaller fans, passive channeled airflow. It's not going to be cooled with 100F air. I'm convinced that's impossible for this one. It will, however, be a guinea pig for practical use of the ICE protocol, monitoring CPU temperatures. To apply the same on the original machine, I would need more ICEcards and thermal sensors. I'm still keeping an eye out for them.

Spring Cleaning - Part Two

The move process has turned out to be quite advantageous. When this is done, there will be more than enough space for everything in my shop.

The heavy cabinet that held my toolboxes moved from the South wall to the North wall and now has my working mini-fridge sitting on top of it, placing the fridge at eye-level. I did have to move the shop light controller since the fridge would have been covering it otherwise. I didn't even have to move the Ikea desk because the cabinet and fridge were the same width. That's a win.

With the cabinet gone, I moved my main desk to take its place and rolled the Dashcat rack into place next to the desk. I had enough space left on that wall to move the crappy laptop inventory shelf next to Dashcat, freeing up the East wall for addition of the AC unit and big storage rack from the bedroom.

The AC unit was easy to re-install. I must have done something right last summer when I put it in because there was no trouble at all on that front. I had forgotten the rating of the unit so I checked the specs and found it was a 9000BTU/hr model. I had been expecting a 6500BTU machine so this was a nice bonus and will easily take care of the 6000BTU/hr Dashcat throws out.

The rack from the bedroom is better than the racks I use in my storage bay since the shelves of this one were extruded with no holes in the surface. This means no more small stuff falling through the holes or anti-scuff pads being torn off of merchandise. It also means I have twenty more square feet of storage space.

A small amount of time before work this morning was spent re-positioning my laptop's docking station. It had been sitting on top of the cabinet I already moved. Now, I just need to find a spot for my laser printer.


Still do do before the really hot weather comes along:

Insulate the West end of the shed, including the door (The summer sun heats the wood and the AC can't even fight that entirely)
Remove the wood surrounding the shed foundation (spider haven)

To do at some point soon:

Run the 230V 30A power leg (this is going to take some serious doing)
Install an awning for the shop. Rain sucks. Snow sucks worse.

Spring Cleaning

Winter around here just sucks. Fortunately, I had the good sense to prepare for the inevitable Spring season by gathering gear for yard and house care. While the snowblower still sits under my carport, inoperable, I have plenty of time to work on it so there's no hurry. The other stuff, however, is best done as soon as possible before the hellacious summers are upon us again.

Enter the pressure washer. I bought a decent Karcher pressure washer for $15 last year and was bummed out when I found that the internal water channels of the wand were cracked, preventing pressure from building up. I set the machine aside and picked up a universal wand back in November. I put it all to work today, cleaning windows, yard tools and the driveway and blasting horribly damaged paint from the East shed. The system was powerful enough to tear off the grey layer of the wood and leave the natural colored stuff behind. This was some rather welcome destruction.

The window AC units are going in a bit early to make room for some stuff from the house. It's a welcome change.

I've still got to put the AC unit back in the Fragwell workshop and that will require moving some gear around as the Dashcat supercomputer is occupying the spot where I had the AC unit last year. I do believe it's time to move my desk.

Dashcat Nodes

I started this last night. Now that there's a break in the action, turning the LNXI cases, Verari/RackSaver nodes, Tivo hard disks, "Hershey Bar" RAM and OEM Windows licenses into the heart of the eight-node version of the Dashcat farm. I have two nodes built as part of a trial right now. I'll be working up an assembly line approach for the rest of them. I've got to pick up a few more cases to cannibalize fans from. It has turned out to be the perfect approach since the fans screw right down and hold tightly with the screws that were used to fasten them to the steel wall inside the case.

I wonder what all eight will sound like when powered up?

Configuration:

Dual AMD Opteron 275 2.2GHz dual-core
3.5GB DDR RAM
160GB Seagate hard disk from a DVR.

Through The Fire And Flames

The only thing burned was calories. I've dropped over twenty pounds since my last post. That's been my focus since the start of the year, much like many other Americans. I've still got about thirty pounds to go.

The render farm project changed a bit and I'm sixteen cooling fans, eight rack handles and a managed switch short of having everything I need to build the machine. Other than that, with the warmer weather thawing the ground, I can pull up the pavers I buried the data trunk under and shift the slack to allow connection of the workstations in the house. With that done, I can run a single link until I am able to find a couple of dual-port gigabit cards.

I Live.... Again

I haven't fallen off the map. I've been too busy keeping Plasmafit updated during my training. When I wasn't updating Plasmafit, I was updating Fragwell. I really need to combine the two, actually.

The circuit breaker I spoke of was replaced quickly and dissected for troubleshooting. I don't know (or care) what it did, but there was a 1206 SMD resistor with a scorch mark halfway across the package. That was the cause of the tripping, for sure. The new one has been solid since Day zero.

I found out my recently installed space heater is wired wrong. I would have sparky blackouts in my shop when it moved and caught one in the act one day. The outer portion of the heater case is supposed to be neutral, which is connected to ground via a bunch of other devices in my house. Well. it's not. That heater has the case connected to the hot lead. Someone was playing around inside it. It was sitting on top of the rack for Legacy/Kitten and would touch the case when moved with vibration. No wonder the arc took out a small chunk of the case. I've kept the heater mounted on wood for a while so I didn't have to go tearing it apart. I still have to do it, though.

The Legacy upgrade project went through two sets of upgrades and a name change. It's now being called the Dashcat Personal Supercomputer, or DashcatPSC for short. It's named after my cat, Dash. He was adopted from a family who named him after a character in The Incredibles.

Time to kill some watts

I figured if I was going to run a powerful computer that requires new electrical wiring, I'd have to find out just how much energy I'm using.

So I got a Kill-A-Watt.

And I have learned some things.

The Xeon LNXI compute nodes consume 130W in "waiting on a shout back from the master node with the rest of my BIOS" mode. I don't know what kind of drain this entails, but I do recall that a computer in BIOS mode is often consuming full power to do nothing.

The Opteron LNXI nodes consume 210W in the same mode.

11-26-2009 - Straight Trippin'

Woke up this morning to work in the shop a bit before turkey and found the old GFCI had tripped again with nothing connected to it. It must be replaced. I don't want that happening when I'm in the middle of a render job some fine day, making me lose half my farm.

So long, piece of shit computer.

I've been growing both wary and weary of my workshop computer. The hard disk is old, loud and 40GB small. I don't put anything on it because I'm afraid I'll lose the data when it finally dies. That's about to change.

A while ago, I bought a ThinkPad T60 for my wife from a fellow [H]ard|Forum member. I liked it so much I decided I needed one for myself so when he put more up for grabs, I bit. Mine came with the 9 cell battery, a Kensington lock cable and a mini docking station. Mini docking station? Those are routinely lame. Not with Lenovo.

It's got every port my workshop computer has and then some.

I'm moving my entire operation to the T60.

Better still, one of the LNXI compute nodes was packed with a nice Lenovo keyboard while the other two came with Dell keyboards. I'm typing this post on it right now--hooked to the crap computer.

While the workshop computer's hard disk is noisy and the noise is far from consistent, the biggest annoyance I have with the machine isn't the hard drive. It's the cooling fans. The CPU cooler is a Thermaltake Volcano 6cu that spins at something like 8000RPM and the video card is a BFG GeForce FX5900 Ultra with a squirrel cage blower that sounds like a dustbuster.

I don't expect to be able to hear what's happening outside while working in the shop when the server farm is running. I do, however, expect to hear what's going on when it's not since I have no windows.

While the computer has been a pain in the ass, the core of it may serve a purpose in the future. I'm trying to sell off some computers, but nobody seems to be buying anything they have to do a bit of work on. No matter. I can strip the internals out of all of them, bolt the motherboards together and run them as a small cluster, selling the cases and Windows XP stickers to cover the cost of adding memory. 512MB is too small. I need a gig in each one. I'll worry about all of that after I get the server farm running, however, since my workshop computer's CPU is only capable of completing the Yafaray benchmark in half an hour and each of the ten nodes in this potential cluster is slightly slower still.

More Power!

I installed the Legacy nodes in their new rack last night and today was spent preparing the shop for the new mains leg to be installed.

Rearranging was the hard part. It turns out running the new cord under the house is going to be easy. While I had been planning to tie a string to a rock and skip it under the house to the other side, I found I already have a string in place in the form of the very extension cord supplying the shop right now. I connect a lesser cord to that and pull it under the house to use for pulling both supply cords back for semi-permanent installation.

Much has been done.

The spider problem has been gone for months now. That's no big deal. What is, however, is the fact that Fragwell Labs, while once intended to receive a makeover, is going to be a barn for my server farm during the cold months.

With that in mind, I need more power. While the lab already has a 20A circuit, it has suffered some regrettable limitations due to the way I set the wiring up. When we moved in, the lab was wired with power via a 20A drop from the breaker panel to a piece of Romex and then to a junction box with a GFCI inside. From there, a grounded 12AWG extension cord was plugged into the box, run under the house and out to the lab. That was fine, for a bit, until I was heating the lab with a bunch of idling computers last winter to try to keep it at a temperature where I could work comfortably and ended up blowing the "15A" breaker on my ten year old power strip. This tripped the GFCI so I had to pull the skirting and push the reset button. Not easy with a foot of snow in my way. I have better power strips now, but there's another problem.

Directly inside the shop, the extension cord has a three-way splitter plugged into it. This splitter has a 13A breaker on it and it's very touchy, especially when the ambient temperature is high as is common during Utah summers. Running my lab computer, lighting, portable A/C unit and laser printer is a recipe for a breaker trip. Not fair.

But there's a way around it. This weekend, I run a second 115VAC line off the other side of the phase. While I can't go directly from a single outlet to a power strip and use all 20A since code is such that no single outlet can use more than 80% of the ampacity of a circuit, I can get a tw0-way splitter and use the full ampacity of both lines. While that circuit won't be enough during the summer months where cooling is a necessity, it will be more than enough for winter, even with the server farm and laser printer, assuming I use the printer all that much.

The compute cluster is meant to be powered by a 230VAC 15A circuit when fully loaded. With the 80% rule in effect, that's 12A peak they plan for. I still have 8A left on each rail, which would be enough for the AC unit and my laser printer to coexist.

I have nine compute nodes. Eight work. The cluster power rating looks at ten nodes.

Shake It Don't Break It

I took delivery of a "new" laptop a short time ago and just recently gave it a much-needed hard disk upgrade from 100GB to 500GB. The machine is still a current model in Dell's lineup and it rather powerful. Dual cores and plenty of RAM. Just what I needed.

As I wrote before, the Dual P4 Xeon 3.2GHz system scored a Yafaray test render time of 10.5 minutes with four threads. I knew I'd have to try the same on the new laptop's T7200 Core 2 Duo. The results surprised me. It was a tie.

If you look at clock speeds, it doesn't make sense. But clock speeds don't mean what they used to back in the Pentium 4 days. The Core and Core 2 CPUs architectures are based on the Pentium M, which was based on the Pentium III Tualatin. The Pentium 4 was abandoned after the Prescott core turned out to be a space heater and little else.

I mention the lineages because it predicted the future. I had a Pentium M laptop of 1.3GHz speed that would outpace my 2.4GHz Pentium 4 laptop in everything but floating point math. That's almost double the operations per clock tick right there.

The Core Duo and Core 2 Duo don't support Hyperthreading. That was brought back with the Core i7. The 15 minute time on the 3.2GHz CPUs when run without Hyperthreading explains the tie. The 2GHz Core 2 Duo in my laptop should be equivalent to a pair of 4GHz P4 Xeons when memory bus fighting is taken into consideration since the Dual Xeon rig is dual channel PC2100 DDR while the C2D is Dual Channel PC2-5300.

At 4GHz, the Xeon rig would complete the render in twelve minutes with Hyperthreading disabled. That's within a minute and a half. Something else is at work here and I don't know what it is, but it says a lot about the improvements made with the Core Architecture.

On top of all that, the Dual Xeon rig cranks out heat like mad, nearly maxing a 380W power supply, while the laptop is hard-limited by an AC adapter brick with a 120W output. And I have twice the RAM in the laptop.

I have yet to try any of my cluster computers. I wonder how they will do.

Life, the Universe and Everything

A while back, I hacked together a 32" LCD TV from a wide array of parts. It turned out to be too small. Funny, that, since it's got a widescreen picture the likes of which only the largest old-school tube TVs could approach. I guess it's the HD thing. More detail so you need to be able to see it.

Last Saturday, I got another LCD set. This one had real issues. While it would work correctly with no input signal, any kind of signal would cause it to scramble horribly. I had seen this before. It was power supply related. Replacing the capacitors in the main logic power supply allowed the display to work correctly. Basically, $5 got me a new display. And I grabbed it at the perfect time. The next morning, the ground was wet from fresh rain.

This new display didn't come with a stand of any kind and I knew I wouldn't be able to make one. It was time for a major purchase so I went to Walmart and found a very nice steel and glass LCD stand that would take advantage of the wall mount holes. It only cost me about $170 after taxes so I couldn't complain at all, having ended up with a 42" display for $175.

It doesn't have a tuner so I'll have to buy one. What is does have is a native 1080p image, HDMI input, two component inputs, VGA and the old-school S-Video and composite. It will take a 1920x1200 signal on the VGA input, but it looks noisy. I'll have to try a better cable. I do know that the VGA cable for my XBOX 360 will show a proper display, although it's cut off at the left.

Braaaaaains!

I'm among the undead today after something like four hours sleep last night.

I was cleaning my workshop and then attacked the living room junk collection. I was up until long after 1AM.

I think karma paid me a dividend, though. It would seem there's someone out there who worked for Linux NetworX back in the day. Maybe I can get some advice?

Expect the Unexpected

Tonight I was looking at a motherboard I had intended to use for the home theater PC in the living room and figured out that it would be a good manager node for the supercomputer. It's only packing an Athlon XP, but that's still fast as hell for what it will be doing (shuffling data around and screaming at compute nodes)

In addition, I have ten computers in my storage bay that are packing Athlon XP 2000+ CPUs and could be turned into a reasonable cluster by themselves. They aren't fast, but that matters for dick, really, since they were $5 each and I can stock them with a gig of RAM per node for about $15 each. I don't know how I will be able to set them for Coreboot, though.

Insert Witty Title Here

I'm just dumbfounded. I have a sense of "No way it's that easy".

The supercomputer I bought is going to be quite easy to deal with, it seems. I found it uses LinuxBIOS, now known as coreboot. That explains why it has no signs of hard disks ever being installed in the machine. This is going to save some energy. It still means I have to have the rigs maxed on RAM and Gigabit Ethernet might not be enough.

Cleanup... Like... Everywhere

It's almost 11:30PM and I'm out in my workshop, de-junking and organizing again.

I have a renewed sense of purpose after discovering a few minutes ago that I have a shot at the Linux NetworX IceBox unit that I need for my supercomputer to get it running. I'm so doing this. I have laptops to prepare and sell off. This will happen.

Rendered Useless?

I started working on Blender 3D rendering tests using Yafaray raytrace render tests. I saw that people were scoring around 3 minutes on an Intel i7 920 with the test scene that features a glass block, semi-glossy floor and two Suzanne the Blender Monkey solid models.

I gave it a whirl and: less than 3 seconds

Okay, but it looks like an early 80s pop music video, all burned out and nothing like the intended output. With that fixed, I ran it again: 27 minutes

WTF? How is it that slow? I looked at Windows Task Manager and saw that my Two CPUs and 4 threads were severely underutilized. So I quit for the night and played World of Warcraft with my wife. Suzanne the Monkey could wait.

Still, I couldn't help but wonder if my rack of Dual Xeon 2.4 nodes would still be woefully underpowered due to age at this rate. 27 minutes a frame divided by six nodes is 4.5 minutes and my workstation is using Dual 3.2GHz 553MHz FSB Xeons. Uh oh. Even slower.

But the test I ran is under Windows XP Pro 32-bit and that's insanely slow compared to Linux, based on the other benchmarks I was seeing.

Today I tried again. This time I understood things a bit better. It would seem the exporter is now required and I have to set the parameters from the right-hand menu that comes up when I select the exporter from the render menu. It's set for one thread. That explains a lot. Since I had disabled Hyperthreading to see if that was the issue, I was running with the two CPUs 1:1 style. This time: just under 15 minutes.

That's a lot better. Now to re-enable Hyperthreading and see what that gets me. Should net a 20% decrease in render time. Time to test for that. Reboot!

10.5 minutes under Windows XP Pro when Hyperthreading. That'll do.

Soon I shall see how Linux fares.

Fall Surplus Sale Success

YES!

Standing in line for hours on end, and being able to outrun my competition, paid off. My spoils of war are undergoing restoration now.

Linux NetworX Evolocity Supercomputer

10-node cage, six nodes populated
Dual Intel Xeon 2.4GHz CPUs in each node with 3 or 4GB Dual Channel RAM on a SuperMicro motherboard.

It's the supercomputer I was going to build last year times two and then some, packed in about a tenth of the space and with so much more RAM.

I don't know how many "U" tall the whole cage is, but my plan is to get a rack large enough to put two cages into if I happen to be lucky enough to score a second one at any point.

With Linux supporting PAE on the Xeons, I can go over the 4GB limit of Windows and take advantage of the six RAM slots on each motherboard. Even if I only break up the 512MB modules to fill the empty slots in the 4GB nodes and buy eight more 1GB modules to even the system nodes out, that's still 5GB per node and like $100 invested in RAM modules. The same amount of RAM in a desktop system would cost three times that much.

I just can't run Windows on this system. It's not fair in the least. I do have the licenses for it, though. I'd just be severely limited in terms of memory space.

In addition to the supercomputer, I also picked up a Dell D810 laptop with a busted LCD panel. More likely put out of its misery since it was a 1280x800 panel and not one of the good ones. I can have it fixed for about $80 with a WUXGA 1920x1200 panel. Supposedly, these systems are limited to 2GB RAM, which makes zero sense to me, but whatever. It will likely become my Windows PC while the Dual Xeon 3.2GHz rig with Radeon HD2600 becomes my Linux workstation first and Windows gaming system second.