Smart construction and good planning allowed Japan's data centers to escape virtually unharmed from the massive earthquake that rocked the country in March, a Japanese data center executive said Thursday.
Operators there had to grapple with blackouts and shortages of generator fuel and equipment. They have also fought hard to be exempt from nationwide power caps that go into effect Friday. But despite enduring the biggest earthquake in recorded history, none of Japan's data centers were severely damaged or knocked offline by the disaster, the executive said.
"So far there has been no critical damage reported to the Japan Data Center Council," said Atsushi Yamanaka, a general manager with data center operations company IDC Frontier, who gave a talk at the DatacenterDynamics conference in San Francisco about the impact of the quake.
IDC Frontier is a subsidiary of Yahoo Japan. It operates Yahoo's data centers in the country and those of other clients. Yamanaka is also a member of JDCC, an association for operators and suppliers.
Data centers were hit with three major events: the earthquake itself, the tsunami that crippled nuclear power stations along the northeast coast, and power shortages that resulted from the crippled nuclear plants.
Most data centers in Japan exceed the country's already-strict building codes and incurred only minor damage when the earthquake hit, Yamanaka said. Modern data centers in Japan are built on giant "shock absorbers" -- isolators made from metal and rubber on which buildings "float" while the ground beneath shakes from side to side.
Some data centers also have floor-level and rack-level isolators. In addition, all server racks, cooling equipment and other gear is secured firmly to the floor. "I see some U.S. data centers with racks just sitting on the floor, and you don't see that in Japan," he said.
The shock absorbers are most effective at the building level, Yamanaka said, and some of those at the rack-level did not work during the earthquake. Nevertheless, he said, only five server racks were reported critically damaged in all of Japan's data centers.
Another mitigating factor was that 70 percent of Japan's data centers are in the Tokyo region, which escaped relatively lightly. There are no data centers in the northeast where the tsunami hit, in part because that's precisely where tsunamis are expected. Nevertheless, the ground shook quite violently in Tokyo, rocking up to 10 centimeters to the right and left for almost two minutes.
Some companies have altered construction plans and are moving data center projects to the west of the country, which is considered safest, Yamanaka said.
Disaster recovery plans generally went smoothly. Where power was cut off, uninterruptible power supplies and diesel-powered generators kicked in, and companies were quick to order more fuel, Yamanaka said.
Fuel was in short supply, however, as was other equipment such as power cables, which the government directed to the most hard-hit areas. 'You couldn't get a normal power cable with a power strip for one month," he said.
Power shortages remains the biggest problem. The government has reactivated hydro and fossil fuel plants, but starting July 1 all facilities that consume more than 500kWh are ordered to reduce their peak power usage by 15 percent from a year earlier.
Data centers operators have fought the cap with some success. They argued their facilities are critical infrastructure and that many had already slashed their consumption before the earthquake as part of energy-efficiency projects. Also, companies have been "evacuating" servers and storage from Tokyo offices to their data centers because of power shortages in the capital, making it even harder to reduce demand, Yamanaka said.
"We told the government it was physically impossible" to meet the reduction targets, he said. The government eventually backed down and has reduced the target for data centers to between zero and 15 percent, depending on how much they reduced energy use the year before. Fines for noncompliance are high -- the equivalent of US$12,500 for each hour over the limit.
Power problems are likely to worsen this summer when cooling systems work overtime, and could lead to further rolling blackouts.
Despite the preparedness, lessons were learned and companies are updating their disaster recovery plans. Communications was affected more than expected, and while social networking tools were a substitute for phone lines, they were also a place where rumors spread fast, Yamanaka said.
Many customers sent staff to their collocation providers' data centers to check on damage. They often got stranded due to transport failures, and blankets and food were in short supply. Service providers must find ways to communicate better with customers when disasters strike, he said.
Data centers are also trying to find ways to cope with long-term power shortages and the price hikes that result, and how to replenish their fuel supplies more quickly. Service providers may alter contracts so that customers shoulder some of the increased cost for raised energy prices.
JDCC is compiling a more detailed report on the quake and wants to share it with operators overseas, Yamanaka said.
The talk was particularly resonant for data center operators based in areas susceptible to natural disasters. David Smith, a principal with Ecom Engineering in Sacramento, California, said he found the talk "fascinating," and that he had expected data centers to be more affected by the earthquake.
Preparedness is a big part of the solution.
"Because we live in Japan we have regular drills for earthquakes and also fires, and most data centers have self-fire fighting organizations that help each other," Yamanaka said.
"Before kids learn how to read and how to count numbers, we get educated with those earthquake drills."
