Overclocking, Beowulf Clusters, Transformers, and Mineral Oil Cooling
Overclocking, cluster computing, and mineral oil cooling are not new ideas, yet they rarely have been combined. Let's look at some examples:
This is the top result on YouTube for the search "world's cheapest supercomputer":
It's an air-cooled GPU cluster in Japan. It had a lot of bang for the buck at the time, but there was no attempt to overclock it or use mineral oil cooling.
This Raspberry Pi cluster by Oracle only needs oil cooling and overclocking to be a real mean machine:
Mineral oil is cheap, non-toxic, non-conducting, and does not evaporate. It also has much better heat transfer properties. It's been used for a long time to cool high voltage transformers. Here is a brief history from: Mineral oils | Transformers Magazine (transformers-magazine.com)
***
Before Elihu Thomson, an electrical engineer working for Westinghouse in the US, patented the use of mineral oil in transformers in 1887, the burgeoning transformer industry had a major problem to solve. As transformers operate, energy losses occur, generating heat. As higher loads are applied, losses increase exponentially, rapidly raising temperatures in a transformer’s core and windings. Without adequate cooling, this heat prematurely ages the transformer, ultimately leading to equipment failure. At the time, the only insulating material used was air, but because these first transformers generated high amounts of losses, they were quickly limited in size by the rapid generation of heat that air failed to properly dissipate. Any attempts at larger devices would fail. Once Elihu Thomson identified oil as a readily available solution, the history of oil as an insulating medium began. Today, several billion liters of mineral oil are used in electrical equipment worldwide.
***
I used mineral oil to cool the overclocked computer I built two years ago.
The white mineral oil jug is visible in the background. The copper BBs and pennies in the oil pan act as heat sinks. In retrospect, I could have gotten even better cooling by lifting up the corners of the pan a few inches to get airflow under it. An even better set-up would have been to put a layer of aluminum foil on the table, set a damp towel on that, and then put the oil pan on top of the towel. That would have gotten me cooling from radiation and evaporation.
NSA has a water-cooled computer complex in Utah. Yes, the best idea their engineers had was to use evaporation cooling in a desert. That's a poor design to say the least.
***
The Utah Data Center (UDC), also known as the Intelligence Community Comprehensive National Cybersecurity Initiative Data Center,[1] is a data storage facility for the United States Intelligence Community that is designed to store data estimated to be on the order of exabytes or larger.[2]
...
The completed facility is expected to require 65 megawatts of electricity, costing about $40 million per year.[6][19] Given its open-evaporation-based cooling system, the facility is expected to use 1.7 million US gal (6,400 m3) of water per day.[24]
***
There's potential to save enormous amounts of power by overclocking. When one CPU can do the work of hundreds or thousands, computers became vastly smaller, cheaper, and more powerful. Computers are about as small and cheap as they're ever going to be, and advances in software are few and far between. Thus, the only avenue of improvement left is heat transfer.
There is a Japanese supercomputer called Tsubame KFC. The KFC part stands for Kepler Fluid Cooling. Like the computer I built, the nodes of the supercomputer are immersed in mineral oil.
Early efforts at liquid-cooled computers had mixed results:
Ahead of overview of TSUBAME-KFC's cooling technology with warm liquid (oil) submersion, we discuss the cooling methodologies; While submersion cooling has been deployed in the past in machines such as the Cray-2, the Florinate coolant utilized was extremely expensive, and moreover evaporated at low temperature of 56 degrees Celsius, and in fact the vapor was collected to be re-condensed, requiring airtight packaging
***
My oil-cooled computer never got above 40 degrees C even during a CPU stress test. Also, one gallon of the mineral oil I used costs about $29.
These guys broke the record for overclocking this year. They used liquid nitrogen as a coolant. Liquid nitrogen is dangerous, expensive, and while it is very cold, its heat transfer properties are poor because it evaporates so quickly. 9 GHz is impressive, but they could have gotten even better performance with mineral oil. I overclocked my oil-cooled computer to 5 GHz. That was the maximum I found. Higher than that, and I'd get a kernel panic when trying to boot up.
I mention all this because rather than spend hundreds of dollars on a computer I don't need to test a concept that's already been proven, I thought it would be better to simply write an article about how to build a machine that combined all these useful ideas.
So here's how it should be done: Make a cluster of the desired size using Intel 13900K chips. Submerge the CPUs in mineral oil. Overclock the chips in 1 GHz implements until 90% of the available power supply is used. A regular US household outlet offers about 1500 W. That's enough power to get to at least 90 GHz. The 9 GHz crew above used 125 W to get there, so 90 GHz would need about 1250 W.
Microsoft did something similar a few years back when then submerged a data center in a waterproof shipping container:
***
Back in 2018, Microsoft sunk an entire data center to the bottom of the Scottish sea, plunging 864 servers and 27.6 petabytes of storage 117 feet deep in the ocean. Today, the company has reported that its latest experiment was a success, revealing findings that show that the idea of an underwater data center is actually a pretty good one.
Good job, Microsoft. Next time, fill the container with mineral oil and then partially submerge it in cold water. Lake Michigan would be best as it has the coldest water in the lower 48. Mount the shipping container vertically in water, like the one on the left in the pic below:
No comments:
Post a Comment