Replacement of Level 1 Calorimeter Trigger Power Pan ------------------------------------------------------ Original Rev. 23-July-1991 Current Rev. 9-MAR-2004 The current revision of this file is in: www.pa.msu.edu/hep/d0/ftp/l1/cal_trig/hardware/rack_crate/ cal_trig_power_pan_replacement.txt First verify that there is a problem with the Power Pan. 1. Is anyone else working on part of the L1 Trigger ? Has anyone changed something that effects which pans should be running ? 2. Which Red LED's are ON. Note that the -2V LED's are not as bright as the other 3. 3. Make a voltmeter check of the 4 power supplies in the power pan. For reference if necessary, compare the readings to those of a working neighbor Power Pan. 4. Turn the Power Pan off for 20 sec and try turning it back on. 5. If it still looks bad, turn the Power Pan off and make Ohm meter readings. Compare to the readings from a known working neighbor Power Pan for reference if necessary. Note some of the output capacitors will hold a 10 of 20 mV charge which interferes with Ohm meter readings. Check both from the output terminal to the common terminal of each supply and also from output terminal to output terminal on pairs of supplies. All of this can be done from the front panel. Checking for Probe ---------------------- ----------------------- +5V to Common Short 5V Red to 5V Black -2V to Common Short 2V Red to 2V Black -4.5V to Common Short 4.5V Red to 4.5V Black -5.2V to Common Short 5.2V Red to 5.2V Black +5V to -2V Short 5V Red to 2V Black +5V to -4.5V Short 5V Red to 4.5V Black +5V to -5.2V Short 5V Red to 5.2V Black -2V to -4.5V Short 2V Black to 4.5V Black -2V to -5.2V Short 2V Black to 5.2V Black -4.5V to -5.2V Short 4.5V Black to 5.2V Black 6. Note: It is probably not a good idea to play around too much (e.g. turning the Power Pan On and Off) with a Power Pan which has what appears to be a bad supply. Depending upon what is wrong with the supply it could put out illegal voltages at high currents and thus destroy cards in the crate to which it is connected. To Replace the Power Pan: 1. Turn off all of the L1 Cal Trig Power Pans (racks M103 through M112) using the switch on the front panel of each Power Pan. Then turn off the rest of the L1 system by pressing the Yellow OFF button and not the RED panic stop button (unless there is an emergency). Wait 10 seconds until the blower stops, then remove a lockout key from the power control panel. Details of the procedure to turn the L1 Cal Trig OFF and ON are described in: http://www.pa.msu.edu/hep/d0/ftp/l1/cal_trig/hardware/rack_crate/ cal_trig_power_control_procedures.txt 2. On the power contactor boxes above the back of the racks, remove the AC power cord(s) for the rack you will be working in. The AC power cords come out of the top of the rack and plug into the power contactor boxes. They are labeled near the plugs which go into the contactor box. This step is to make absolutely certain that there is no AC power being supplied to the rack that you are working on. 3. Take the rear door off the rack. NOTE: Steps #4 and #5 are no longer necessary for most of the Power Pans in the Cal Trig because the bolts that connect the large DC power cables to the "Power Distribution Monolith" on top of the power supplies are now put in up-side-down, i.e. the nut is on top and the bolt comes in from underneath. This allows you to remove the nut and then pull the bolt down out of the power cable lugs *without* removing the G10 parts mentioned in steps #4 and #5. 4. Working in the rear of the rack, remove the vertical G10 covers, which cover the card file backplane power bars, of the card file backplane just above the power pan which you are replacing. Remove just the narrow (2" wide) piece of G10 by taking out the two small brass Phillips screws that hold each of these G10 pieces onto the nylon blocks at the top and bottom. 5. Remove the horizontal G10 piece which is just above the back of the Power Pan which you are replacing. This piece of G10 is held in by a screw at each side that goes through the shelf brackets in the rack. 6. Use a clean Chem-wipe or cloth or piece of paper toweling to cover the ERPB card that is directly below the power lugs at the back of the Power Pan that you are replacing. The intent is to protect the ERPB card from any little shards of metal that may come off of the lugs on the DC power cables. 7. Verify the labels on the big DC power cables and relabel if necessary. 8. UN-bolt the DC power cables. Use 2 wrenches and try to balance the torque in the wrenches, so that no net force is put into the power terminals on the Power Pan. This is to prevent damage to the studs on the power supplies themselves. The power terminals and the heavy "Power Distribution Monolith" on top of the power supplies is held on only by the studs on the power supplies. 9. Verify that the AC power cord for the Power Pan is not caught or cable tied down in such a way that it can not move forward. 10. From the front of the rack unscrew the small black G10 strip panel which is just above the power pan that is being removed. This small panel must come off so that the power pan can come out of the rack with its Power Distribution Monolith still attached to it. 11. From the front of the rack, unscrew the 4 front mounting screw on the Power Pan. 12. Now slide the Power Pan about 2/3 of the way out. If it is a lower Tier 1 Power Pan support the front of it on a short piece of 4 by 4. If it an upper Tier 1 Power Pan support it on a cart. Note: it is important that the front of the Power Pan NOT drop down or else the rear power terminals will dig into the bottom card in the card file above the Power Pan. As the pan slide forward it is important not to let the back of it slide side ways or else it can scrap against the radiators and block their air flow. 13. With the Power Pan 2/3 of the way out you can reach back and unscrew the leads of the AC power cord. 14. Now pull the Power Pan the rest of the way out. IT IS HEAVY. It is important to have 2 people lift it. Installing the Replacement Power Pan. 1. The steps are similar the those above with a few extra things to be careful to watch out for. 2. Before installing the new power pan, check for shorts between the individual power supply outputs (in pairs), between all the outputs and the DC common ground return, and between all the outputs and AC (chassis) ground. 3. Before installing the new power pan, record the power pan serial numbers (found on the back of the Power Distribution Monolith) of both the removed pan and the new replacement power pan. 4. When installing the G-10 protector strips, double check that none of the power bus bar to backplane wires are being pinched or stretched. Also verify that none of the power bus bar to backplane foils have been bent out of position (e.g. are shorting to other power bus bars). 5. Double check the AC power cord: Is the proper wire to the proper post? Is the power cord out of the way and not being pinched or located where it is going to vibrate against some sharp metal? 6. Double check the DC power cables. Is each proper cable to the proper terminal? Are the connections tight (try to balance the wrenches and have no net force on the power terminals). 7. Use the Ohm meter from the front of the Power Pan to check the resistances of the supplies to common and from supply to supply as described above. 8. Use the Ohm meter to look at the connector on the line cord going into the rack. Verify that a hot phase line is not connected to the chassis ground. The chassis safety ground is the center round pin. This is a safety check. 9. Verify that the cards immediately above the Power Pan that you have replaced are still fully plugged into their crate and backplane. This is especially important for the CHTCR card that is right above the power supply that has been replaced. Steps for the Initial Power Up of a new Tier 1 Power Pan. 1. In racks M103 through M112; Turn OFF all of the Power Pans. 2. Put the lockout key back into the power control panel. 3. On the Photo-Helic differential pressure gauge, located at the front of rack M113, set the LOW LIMIT set point to 0.40 inches. The LOW LIMIT set point is controlled by the knob on the LEFT side of this gauge and and the value of the LOW LIMIT set point is indicated by the left hand red needle. 4. Press the "Fault Reset" button on the RPSS Panel the Rack Power Safety System Panel at the very bottom of rack M113. 5. Press the Green Start button on either of the Calorimeter Trigger RPSS power control boxes. The fan will start up immediately and 10 seconds later the power contactors will close letting AC power into the racks. 6. Watch the black differential pressure indicator needle on the Photo-Helic at the front of rack M113. After about 30 seconds this black indicator needle should show a differential pressure of between 1.65 and 1.75 inches. After the differential pressure reaches this level, adjust the LOW LIMIT control knob (the left hand control knob which moves the left hand red needle) so that the LOW LIMIT set point is 1.30 inches. 7. If everything looks OK then turn on the newly installed power pan. It is best to have someone in front turn it on while watching for problems and a second person behind the rack watching for problems. 8. Check the voltage readings on the front of the new Tier 1 Power Pan. They should be about: Supply Actual Voltage Output ---------- --------------------------- 5V 5.055 Volts 2V 2.105 Volts 4.5V 4.610 Volts 5.2V 5.230 Volts If the new pan looks like it is running OK then close all the rack doors (front and back) and let it continue to run for about 5 minutes and check the voltages again. 9. If everything continues to look OK, then turn on the rest of the L1 Calorimeter Trigger in the following order. Do not open more than one door at a time when turning on these power supplies. Turn on the Calorimeter Trigger Control Crate Power Pan (top of rack M104). Note it has only 2 LED's Turn on all of the the Tier 1 Power Pans. Turn on the Tier 2 Power Pans (top of racks M106, M110 and M112). Turn on the Tier 3 Power Pan (top of rack M108). 10. Make absolutely certain that all the Cal Trig doors have been closed. 11. It is a good idea to keep a close eye on things for another 5 or 10 minutes. Are all the red power LED's still on? Does anything smell like hot electronics? 12. If everything looks OK then tell the DAQ Expert to have COOR initialize the L1 Framework and the L1 Calorimeter Trigger. Any current special instructions for bring up the L1 Cal Trig will be covered in a recent entry in the log book. Additional Instructions for Replacing a Tier 2 or Tier 3 Power Pan ------------------------------------------------------------------ Replacing a Tier 2 or Tier 3 Power Pan is very similar to replacing a Tier 1 Power Pan with two small differences: 1. There is a DB25 connector at the rear of the Power Pan that needs to be disconnected before the Power Pan is removed. This needs to be tightly reconnected after the new Power Pan is installed. 2. The proper Voltages for a Tier 2 Tier 3 Power Pan are the following +5V --> 5.050 Volts -2V --> 2.040 Volts -4.5V --> 4.555 Volts -5.2V --> 5.230 Volts