This data site was experiencing some very peculiar electrical problems. Main power fuses feeding the combined U.P.S. and chiller loads were randomly blowing at all hours of the day and night.

The site was a major critical data processing centre. The data processing electrical load had gradually increased in size over the years, but unfortunately the supporting U.P.S. systems had not. The U.P.S. system was at 90% capacity with this current electrical load. The U.P.S. battery back-up system was therefore “only good for about 10 minutes” before being completely discharged by this “heavy” electrical load and then the entire data centre load would crash. This site, as electrically designed, had a back-up diesel generator system to feed the combined U.P.S. and chiller circuits, but here is where the plot thickens.

Every time there was an electrical utility power outage, the generator would start up and the transfer switch would automatically transfer to diesel power to feed these critical loads with no problems each time.

So far, everything seems to be operating as specified – right?

The real problem occurred when the utility power was restored.

The transfer switch then operated and attempted to transfer back to normal/utility power from generator power. At that exact same instant in time, these 600-volt rated-400 Amp time delay fuses randomly blew in the switchboard feeding these combined critical loads. Sometimes it was A-phase or B-phase or C-phase fuses or sometimes it was all three phases. At other times, no fuses blew.

These are extremely frustrating random events!

If a fuse blew during transfer, the electrician had a maximum of 10 minutes to replace the fuses and then attempt to re-establish this utility power feed, otherwise the U.P.S. battery system would drain down and the data centre would crash. The electrician had to essentially “babysit” these switchboard fuses 24 hours around the clock until a solution was found – hence the request for electricians who would be able to sleep on the job.

When we arrived at site, there was a bed, TV, fridge, microwave, etc. already set up beside the switchboard for these “sleeping” electricians. This major problem had continued for over six months and no one could figure it out. The 40 dead fuses so far had all been documented and lined up in a failure dated row similar to a row of empty beer bottles. Boxes of new replacement fuses were also close at hand (right beside all the stacked up empty pizza boxes).

The diesel generator group, transfer switch group, U.P.S. and chiller groups had all already been called in and all their systems had checked out okay.

Was it a harmonic resonance problem, or an intermittent out of phase transfer coupled with some type of random diesel frequency instability issue?

It could be anything at this point!

At any time of the day or the night, utility power could be lost and the above sequence of events would tragically unfold again. The electricians would have to quickly respond, replace the fuses and attempt to re-establish utility power each time. Sometimes they would have to continually replace these fuses several times during the same event, until they could get these critical circuits to “hold in”.

To begin the investigation, we installed our customized high-speed monitoring equipment on all the critical feeders. With the now very reluctant, frustrated electricians and data processing personnel standing by, we then created a utility power outage and automatic power restoration sequence. As expected, a fuse blew. While this scenario had been occurring again and again for the last six months at this site, the difference was this time, we instantaneously captured and stored all the voltages, currents, harmonics, phase angles, etc. on these critical electrical feeders at the exact same time as this fuse failure.

After analyzing all this captured data, we decided to go one step further and prepared a detailed electrical co-ordination study for this site to confirm our suspicions.

Part of this site had already been electrically designed for future U.P.S. expansion. The 600/480-volt step-down transformer feeding the existing U.P.S. was therefore much larger in KVA size than what was currently required. Using the transformer nameplate information, including impedance, voltage and KVA values, and plotting this on an electrical time current co-ordination plot revealed the following.

The plotted and predicted transformer inrush time current values exceeded the upstream 400-amp fuse delay clearing times. As a result, the fuses would intermittently blow during transformer re-energization!

This random A/B/C phase fuse blowing factor was also dependent on what part of the voltage waveform electrical power was interrupted and then restored (transformer magnetization B/H Curve). Different fuse curves from different fuse manufacturers were now also examined as a potential solution.

The final agreed upon solutions were to either replace this U.P.S. step-down transformer with a lower KVA transformer or rework the main electrical switchboard and install a higher ampacity fusible disconnect switch/cabling. In this case, the client chose to rework/install this higher ampacity disconnect switch. (approx. time to complete 12 to 14 hours)

Problem solved!

If an up-to-date accurate electrical co-ordination study had first been completed, then all these very embarrassing data processing centre reliability issues and this extremely lengthy/ costly problem could have been completely avoided.

All the beds, appliances, etc. have now been removed from the switchboard room and put in storage. There now is however, a regiment of fully trained “sleep and pizza” electricians ready for the next big electrical babysitting job. Stay tuned.

Patrick J. Lynch, P.Eng. has been president of Power Line Systems Engineering since 1986. He graduated from the University of Waterloo in electrical engineering in 1975 and has successfully directed Power Line’s completion of more than 1,100 complex site disturbance investigations around the world. For more information, visit www.powerlinesystems.ca.

This article was originally published in the December 2019 issue of Electrical Business.