I’m in the process of purchasing a data center UPS at work. Looking at an APC SmartUPS VT currently. I was looking at something larger from Liebert, but the vendor wanted $18k for the install, more than the cost of the hardware itself, and I have a hard time justifying $18k for what should be a couple days worth of work. In the process of all this, has been a lot of attention to power. I’m at a junction right now, I basically don’t have enough power for the UPS I’m looking at, but the UPS is larger than I need, as we plan on building a new data center in the not too distant future. This has lead me to an electrical code question. In the end, I’m probably going to have our electrical contractor do the work over a vendor, because despite not having the confidence of the vendors experience with their own equipment, electrical contractors generally have names, like John, or Bob, and I can chat with them for five minutes respect their work from the conversation. That and they do a good job without charging $4k/hr or whatever the vendor’s project costs come out to be. But yeah. I like small shops. If I can’t find someone with a first name to talk to who can spend ten minutes explaining the engineering of the situation to me, I’m not going to trust their judgement and I’m going to find someone else. Of course, I’m certainly not going to try to do it myself. I’ll worry about vendor inter-operable LACP, they can worry about harmonics. It’s what we both get paid for.
But still, I’ve been communicating with our electrical contractor and a couple vendors all along, but I’m not really satisfied until I understand the mechanics, or perhaps the electrics, of the situation. Tonight I posted my question on an electricians forum. It’s currently up for debate as to if I’m allowed to ask questions there, as they have a policy against answering “how-to” questions to avoid laymen killing themselves, doing illegal electrical work, etc. Hopefully they side with me. As I got thinking about their choice though, I realized how much I think about electricity. Sure, I’ve got all these outlets in the ceiling of my data center, all I really have to do is plug my PDU’s from my racks in and not worry about it, right? I’m IT, that’s facilities. Well, there’s no such thing as facilities in my company, and I previously come from even smaller companies where the concept of departments didn’t even exist, so I might be a semi-rare case here. But I think about electricity a lot. I wonder what the current and peak current of my racks and PDUs are, ensuring I’m not only not overloading a breaker, but evenly balanced across phases. Then when the UPS comes into the picture, I further get to worry about the load on the UPS, run times, etc. All this leads to spending a lot of time figuring out how 120V single phase power relates to 208V three phase power, the difference between KVA and KW for UPS sizing, and why the hell my datacenter was built with NEMA 5-20 plugs instead of something rugged and locking like an L5-30.
Maybe that’s why there are specialized vendors out there getting $18k for an install. But people I work for seem to want me to know whats going on, and more importantly, I don’t sleep at night if I don’t get it anyways. So, other admin folk, how does power affect your daily life (besides windpocalypse 2k6 and the fact that casey lives in the sticks)?
Bryan you cannot feed a larger sized feeder from a smaller sized feeder as you posted elsewhere. You can feed one of the same size like 100 to a 100, but that is very poor design.
I design data centers for Verizon (formely MCI-Worldcom and my best advice is to consult with an certified electrical engineer.
Thanks Dereck. I figured as much. I have two sub-panels. One with a 100A feeder from the main, and another off that with a 100A feeder. I think I mentioned this. Anyways I think the previous occupant likely had a UPS between these panels one for critical data center loads and the other for convenience outlets and the two panels were simply strung together when they left with their UPS.
I’m waiting for my electrician to come back with a quote for pulling new feeders.
I do think you’re missing the point, although perhaps it’s a different mentality for the two industries. Almost all of my experience is self-taught. Either on the job, from a book, or online.
Simply saying that I should consult an engineer is like saying I should not have learned about cars because gasoline is flammable, or I shouldn’t have learned about computers because power supplies are dangerous. I could go on to my experiences with electricity, batteries, etc.
Don’t get me wrong, I certainly understand your position. Electricity is dangerous and more importantly if people muck around with it who aren’t qualified, they create a dangerous situation for other people. However, arguing that knowledge causes dangerous situations is like arguing that guns kill people and it has nothing to do with the person that wields the weapon.
btm:
I happen to be a data center designer and general contractor who only builds DC’s and critical facilities. I would recommend that you visit http://www.datacenteruniversity.com The site has courses you can take on-line that discuss the topics that are currently keeping you awake at night.
Insofar as your selection of UPS, if you need under 16kVA/11.2kW of single phase load I would recommend the APC Symmetra LX unit due to its modular construction and internal N+1 capabilities. If you’re more in the 20/30kVA range, the Smart-UPS VT offers good value for your buck, but the Symmetra VX allows best future expansion and internal redundancy.
Hope this helps!
redebo: I’ve actually been to datacenteruniversity before, and it’s awesome you mention it because it is a killer site. There’s a lot of really nice information there that crosses the bridge between electrician and systems/network engineer that I’m speaking of.
I was totally into the Smart-UPS VT 30KVA. The story goes like this in short. I’ve got a 75KVA transformer on my floor feeding a lighting board and two panels with a 225A main. Off one of these is a 100A breaker that feeds my data center over 4W #2 cu + ground. My data center has a sub panel with a 100A main that has another 100A OCPD/service disconnect that feeds the critical panel. The UPS shall go between the two. The Smart-UPS VT 30KVA calls for a 125A OCPD. I asked APC if I could simply put 100A OCPD’s on both sides and step down to a 100A output fuse and they said no, simply because it’s not the spec, not because they thought there was anything wrong with it.
I had figured this would work until I move to a larger site and get the electrical correctly engineered, but risking the warranty wasn’t really something we want to do. Their 20KVA unit which is 100A OCPD is 6+ weeks ship time through my vendor (lame). I was looking at Tripp-lite’s 30KVA unit which only specs 100A OCPDs, but then I asked my electrician to verify that this would meet code and they said anything over a 15KVA UPS would be to large, failed to provide a reason, and told me they were busy and would talk to me about it next year.
So I guess I’m in search of a new electrical contractor. I explained the situation to building management’s contractor today via email and I’m waiting their response. The funny thing to me out of all of this is how -extremely difficult- it is to reach my satisfaction that this all is safe and meets code. Everyone keeps saying “talk to an electrician” and I’m trying my hardest to… but it’s just not working out for me.
In the interim, I will probably go back and spend some more time @datacenteru.
I sympathize with you, Bryan. It always surprises me to hear about non-telecom datacenters and how they use AC. We have in-house people to do all our DC wiring, and so we only rarely require electricians for AC.
Mentioning the twistlok connectors got me thinking — if you go that far, it seems like you should be wiring your datacenter with 240v. Almost everything you plug in should support it, and some stuff (like the IBM Bladecenter, for one example) requires it. You also get the safety advantage of halved amperage and I think you get less heat output from power supplies too. You could stick in 120v outlets too, for wall warts or any other crap that doesn’t do 240v. You can even get combo NEMA 5-20/6-20 outlets (though I dunno about locking).
Maybe this is commonly done already, I dunno. All we’ve got is 48v and a few 120v outlets on inverter, so for most stuff we don’t even get the luxury of outlets.
btm:
I’m going to give you a full post w/ some recommendations later (my wife is yelling at me to ge into the christmas spirit and stop working) but real quick: I went and looked at the 30kVA APC SmartUPS VT and the reason it requires a 125A breaker is that its max power draw is 83.3A. As you may (or may not!) know, you cannot run more then 80% of a breakers rating through it continuously. If you put a 100A breaker on it, and ran the unit at full load, you’d be pulling more then 80% of the breaker rating! My guess is that APC just bumped it to 125A because that’s the next “easy to find” breaker.
Hope that helps as to why they told you no. Bigger post later w/ some other information for you!
Bryan,
Are you still experiencing problems? If you have given up on the UPS OEMs for help, you may want to consider some of the industry trade organizations; 7x24Exchange, UPtime Institute come to mind.
I found another electrician after my original one kind of left me hanging. He came by and we talked about it a bit and found ourselves on the same page. At this point, to get this project done I think we’re going to upgrade the #2 cable to #1 in the existing conduit, he checked the code, and then upgrade the breakers to 125A. We’ll run the 125A straight into a disconnect, then into the APC 30KVA UPS as originally planned. Nothing appears unsafe as it is, nor would it have been, we’ll just go ahead and satisfy the mystical powers that be.
Bryan,
I am curious, how did you go about selecting the UPS vendor for your application?