Hub-Module Power Distribution Strategy ------------------------------------------- Original Rev. 20-Mar-2015 Current Rev. 19-Oct-2016 This file describes the Power Distribution setup on the Hub Module. The power supplies themselves are covered in a different "as built" file, hub_0_ab_power_system.txt The description of the power distribution includes the 9 main power buses from the DCDC Converters as well as the isolated +12 Volt bus from the ATCA power input system. A principal intent of this file is to describe the various Area Fills that are used on the Hub Module to distribute the 11 power buses. Needless to say - the Hub Module PCB does not have enough power layers to have a separate power layer for each power bus. A careful division of the few available physical power layers into separate area fill sections is needed. In the Mentor design system all of the layers that handle the power distribution are setup as Signal type layers. This is necessary because these Stackup layers need to include signal traces in some areas. Besides the high current area fills the Hub Module also has a significant number of high current discrete traces. Many of these high current traces are involved with connections to the Zone 1 connector and the two ATCA power modules. These high current connections are described near the end of this file. Stackup Layers Available for Power Distribution: ------------------------------------------------ PCB Mentor Stackup Logical Layer Layer Nominal Power Buses Carried on this Layer ------- --------- ------------------------------------------- L9 Signal_5 BULK_3V3 and Diff Signals to Backplane L11 Signal_11 FPGA_CORE, MGT_AVCC, FAN_1V8, ISO_12V, CLK_2V5 L12 Signal_12 FPGA_CORE, MGT_AVTT, FAN_1V8, ISO_12V, SWCH_1V2 L14 Signal_6 BULK_1V8 and Diff Signals to Backplane The normal Hub PCB signal and ground layers are half ounce copper and support the high speed differential signals. L9 and L14 are one ounce because they do need to carry high current but they also need to support quality differential signals. L11 and L12 are one ounce for the high current distribution. Limited Areas where a Given Power Bus Is Needed: ------------------------------------------------ FPGA_CORE only needed under the Hub FPGA and its bypass caps MGT_AVCC only needed under the Hub FPGA and its bypass caps MGT_AVTT only needed under the Hub FPGA and its bypass caps MGT_AVAUX only needed under the Hub FPGA and its bypass caps ISO_12V only needed for the DCDC Converter Inputs and for the MegArray power pins to the ROD BULK_2V5 only needed for the 2 MiniPODs and for the 40.08 MHz and 320.64 MHz Clock Fanout SWCH_1V2 only needed for the Switch Chips and the Phys Chips and their bypass caps and power input filters FAN_1V8 only needed under the GTH Fanout Chips and the DCDC_7 Converter Which Chips Require Which Power Buses: -------------------------------------- Switch Chips require only SWCH_1V2 and BULK_3V3 BULK_1V8 is not needed under the Switch chips and this layer is available for signal routing. Phys Chips require SWCH_1V2, BULK_1V8, and BULK_3V3 Config_Flash requires only BULK_1V8 MiniPODs require both filtered BULK_3V3 and filtered BULK_2V5. Each MiniPOD requires its own cutout and fill on L14 for its filtered 3V3. The fills for both the BULK and filtered 2V5 are on L9 the nominal BULK_1V8 stackup layer. Clocks all Clocks require BULK_3V3, for all Clocks their 3V3 power is filtered and supplied on a separate isolated area fill on L14 which require cutouts in L14 40.08 MHz and 320.64 MHz Clocks require BULK_2V5 and for these clocks their 2V5 power is filtered and supplied on a separate isolated fill on L11. FPGA In total the FPGA needs 6 power buses: FPGA_CORE, MGT_AVCC, MGT_AVTT, MGT_AVAUX, BULK_1V8, and BULK_3V3 is needed for the 8 VCCO pins for Banks 84 and 94. Thus something special is needed under the FPGA because L14 (normally used for BULK_3V3) needs to be used for the 12 pins of MGT_AVAUX in this area. A cutout in the BULK_3V3 fill on L14 is used for the MGT_AVAUX power distribution. BULK_1V8 except for the 3V3 <--> 1V8 translators, the BULK_1V8 bus is only needed North of the Switch Chips BULK_3V3 is needed South of the Switch Chips only for the 25 MHz Ethernet Clock Generator and for the front panel JTAG circuits. ATCA Power Modules note that SynQor wants a special shield plane (basically at VRTN_OUT) under its Isolated DCDC Converter, see page 12 of the manual for the IQ65033QMA10 ATCA power entry module. Discrete Wire Run Connections: Rev. 6-Sept-2016 ------------------------------ There is not enough routing space or copper width available to make a number of power connections on the Hub Module. The proposed solution is to make these connections with discrete wire runs. In some cases these discrete wire runs make up the whole of a given power connection, In other cases these discrete wire runs suppliment a trace run that is too thin to carry the full load current by itself without excessive voltage drop. The following is the list of discrete wire power connections on the Hub Module: Connect the source of IPMV_3V3 WTERM1 IPMC_3V3_SRC power in the ATCA Power Entry WTERM2 IPMC_3V3 Module to most of the loads on this rail that include the IPMC module itself and other components in the area of the IPMC. Nets File: atca_power_entry_nets Connect the source of the WTERM11 CNST_5V0_SRC Constant 5V0 power to the WTERM12 CNST_5V0 loads on the CNST_5V0 rail that are in the upper back side section of the Hub. Nets File: atca_power_entry_nets Supplement the Iso_12V WTerm21, WTerm22, WTerm23 Iso_12V power connection as it runs WTerm25, WTerm26, WTerm27 Iso_12V north through the pinch from the ATCA Iso_12V module up to most of the DCDC Converters. Nets File: atca_isolated_12v_nets Connect the large Hold-Up WTERM31 ENTERED_N48V Capacitors in the NE corner of WTERM32 Hold_Cap_Negative the card to their pins on the ATCA Power Entry Module. Note WTERM41 Hold_Cap_Pos_Pow_enter that these are 100 Volt lines. WTERM42 Hold_Cap_Positive Nets File: atca_power_entry_nets Connect the source of BULK_2V5 WTERM51 BULK_2V5 power to the input of the WTERM52 BULK_2V5_Wire CLK_2V5 power filter. Nets File: dcdc_9_converter_nets Nets File: clock_generation_nets: There is no convenient path from the ATCA Input Module to the hold-up capacitors in the NE corner of the card that is wide enough to carry the full current. This is a 100 Volt bus so it also has safety issues. The run up to these capacitors is done with an AWG #22 wire that has the appropriate safety ratings. This wire run is glued to the PCB with cyanoacrylate glue. There is a fill under the two hold-up capacitors on stackup layer L9. Number 20 or number 22 wire will be used for these discrete wire run. Wire with the appropriate insulation must be used for approval by CERN. Power Through the Pinch: ------------------------ The large number of high-speed differential signals on the Hub Module constrains the locations where other signals can be routed. This results in a vertical mat of traces running West of the Phys Chips. This vertical mat of traces is forced to run under the front panel RJ45 connectors and their magnetics. Mixed in with this vertical mat of signal traces are a number of power connections including: CNST_5V0 Constant +5V from ATCA Power Entry 50 mA or so IPMC_3V3 3V3 supply from ATCA to the IPMC an Amp or two ISO_12V Isolated +12V from the Iso 12V ATCA supply 25 Amps HOLD_CAP_POS_POW_ENTRY from the ATCA Power Entry Module ENTERED_N48V to the Hold Up Caps 100 Volts 6 Amps Is it possible to run any of these nets through the pinch using an Area Fills ? Such a fill will have a shape of about 120 mm long by 29 mm wide in series with a shape about 65 mm long by 22 mm wide. There will be a lot of holes in such a fill because it is running under the front panel RJ45s and their magnetics. Assuming 1/2 oz copper (1.0323 mOhm per square) and no holes such an overall shape will have a resistance of about 7.2 mOhms. I currently think that the two center 1 oz power fill layers Signal 11 and 12 could be used for the ISO_12V. This would have a resistance of about 1.8 mOhms and at 25 Amps will dump about 1.2 Watts. I will also run two vertical #22 wires to help carry the ISO_12V from the ATCA module up to the loads. I've given up carrying any of the rest of these one power signals on the pcb layers. CNST_5V0, IPMC_3V3, and the two HOLD_UP_CAP run are all being made with #22 discrete wires. Power Distribution Revolution/Revelation Rev. 10-Jul-2016 ---------------------------------------- The following are major changes to the Hub's Power Distribution Strategy most of which have not yet been moved into the text above this point or into the other appropriate files: e.g. Hub Power Supply or Hub Routing. 1. Change the BULK_3V3 supply from a 20 Amp module to a 12 Amp module. The current estimate of the BULK_3V3 draw no longer justifies a 20 Amp module. The 12 Amp module is probably a little quieter, has a little better ground pinout, an will probably put a little less noise on the Isolated +12V bus. 2. The current draws on the BULK_1V8 and BULK_3V3 DCDC Converters are about the same. So which bus uses a fill on Signal_5, that can run both over and under the two 400 pin MegArray connectors, and which bus used a fill on Signal_6, that can only run over the top of the MegArray connectors, is not determined by the current draws on these two rails. 3. Whichever supply, BULK_1V8 or BULK_3V3, that is distributed on Signal_6 and thus can only run over the top of the two 400 pin MegArray connectors should have its DCDC Converter located at the top of the stack of 3 converters along the back edge of the card. 4. Between BULK_1V8 and BULK_3V3 which one is on Signal_5 and which one is on Signal_6. Up until now I have had BULK_1V8 on Signal_5 but is that the best setup ? The main differences in the BULK_1V8 and BULK_3V3 fills are: - The BULK_3V3 fill is need under the Switch Chips where as (except for the feed to the JTAG Translators/Buffers) the BULK_1V8 fill can stop at the South edge of the Phys Chips and otherwise is not needed South of the FPGA. - The BULK_3V3 fill is needed under the front edge LEDs and RJ45 connectors where as the BULK_1V8 fill can stop just West of the Phys Chips. My new current guess is that the Bulk_3V3 fill will benefit more from routing both over and under the two 400 pin MegArray connectors than the BULK_1V8 would. Thus my new layout plan is: <----- BULK_3V3 should be on Signal_5 <----- BULK_1V8 should be on Signal_6 <----- Note - this is the opposite from my layout for the past year or more. Everything below here is written assuming this new assignment of power buses to layers for their fills. Recall also that during routing we may need Signal_5 for routing between the FPGA and locations North or East of it and thus loose the space under the two 400 pin MegArray connectors for BULK_3V3 fill. 5. By using a minimized BULK_1V8 fill under the FPGA, then the MGT_AVAUX fill can go on either the BULK_1V8 layer or on the BULK_3V3 layer. In this case a minimized BULK_1V8 fill means: no fill East of the FPGA and that the fill is constrained to just the center section of the FPGA BGA footprint in the North-South direction. Note that this strongly constrains the location of the FPGA's Bulk_1V8 ByPass capacitors and locates none of these bypass capacitors adjacent to Banks: 70, 71, 72. 6. The SWCH_1V2 fill and the individual fills under each Switch Chip for their AVDDL supply can go on the two inner 1 oz layers (Signal_11 and Signal_12) without causing any interference to the FPGA power distribution. Note that Signal_6 which carries the BULK_1V8 fill could also be used for individual AVDDL fills under the Switch Chips. 7. With careful work, the SWCH_1V2 fill under the two Phys Chips can go on one of the two inner 1 oz layers (Signal_11 or Signal_12) without causing serious interference to the FPGA_CORE power distribution. This will take careful layout at the NE corner of the Phys Chips. Obviously the SWCH_1V2 fill layer is the same for both the Phys Chips and for the Switch Chips. 8. With careful work the CLK_2V5 and CLK_3V3 fills under the 48.08 MHz and 320.64 MHz clock generator and distribution chips may fit on the two inner 1 oz layers (Signal_11 and Signal_12) without causing serious interference to the FPGA power distribution. Signal_6 which carries the BULK_1V8 fill and is Not used for the MGT Fanout to Hub FPGA routing is also a candidate for at least part of one of the Clock Fills. 9. Signal_6 is used for MGT_Fanout to ROD routing and thus can not be used for MGT_Fanout to Hub FPGA routing (because the routes are blocked in the MGT Fanout escape). Thus Signal_6 should be used for the MGT_AVAUX fill (which also blocks the MGT Fanout to Hub FPGA routes). 10. With both the BULK_1V8 fill and BULK_3V3 fill under the FPGA minimized (i.e. no fill East of the FPGA and the fill constrained in the North-South direction to only the center area of the FPGA's BGA footprint) then the 4 layers, Signal_2 through Signal_5 can all be used for MGT Fanout to Hub FPGA routing. I.E. we are back to 4 Layers for this MGT Fanout routing. 11. Note - These minimized fills under the FPGA mean that: - Both BULK_1V8 and BULK_3V3 are West of the FPGA only - no fill of these power buses on the East side of the FPGA. - Only the fill on Signal_5 may run South of the MegArray connectors - because in this area on Signal_6 is used by the MGT Fanout to ROD traces. - The FPGA ByPass capacitors for both its BULK_1V8 and its BULK_3V3 must be placed West of the FPGA i.e. not adjacent to some of the FPGA's loads on these rails. 12. We need two individual fills under each of the MiniPODs, one for that MiniPOD's Filtered 2V5 and the other for that MiniPOD's Filtered 3V3. - These fills must extend to wherever we can locate the vias that connect to the inductors for these filtered supplies. The location of these vias is very constrained by the vertical mat of traces running West of the MiniPODs and the signals (both control and high-speed) running on the East side of the MiniPODs. - What layers to use for these files is very constrained because: .In the vicinity of the MiniPODs the fills for ISO_12V on Signal_11 and Signal_12 are already too narrow. .In the vicinity of the MiniPODs the fills for BULK_3V3 and BULK_1V8 (on Signal_5 and Signal_6 are already quite narrow, and may be only 1/2 oz. 13. The fill for the BULK_2V5 connection from the BULK_2V5 Regulator to the MiniPOD and Clock Filter Inputs can go on layer Signal_??? 14. The low current BULK_1V8 and BULK_3V3 runs from their distribution fills down to the JTAG translator/buffer chips located by the front panel J2 Access connector may need to be carried by wide traces on an available Signal routing layers. 15. Some additional Fills and High Current connections are needed in the Hub Module Design: - There is a Shield Fill needed under each ATCA Power Module on the Top Layer - Signal_1. These Shield Fills are connected to Logic Ground via RC networks. - High current connections are needed between the Zone 1 connector, the Fuses, and the ATCA Power Entry Module. The 1 oz layers (Signal_11 and Signal_12) may be used to help carry these high currents with either wide traces or fills. - Make full use of the 1 oz layers (Signal_11 and Signal_12) between ATCA Power modules and at the output of the Isolated +12V module. - The Top and Bottom Surface layers should/must be used at all high current THD and SMD connections and their associated via-arrays. 16. Either Signal_11 or Signal_12 may be available to carry the Shelf Ground up West of the Zone 2 connectors up to the Alignment Pin Receptacles K1 and K2. Making the connection to the Alignment Pin Receptacles in this way would remove the problem of routing the Shelf Ground from the front panel to the Alignment Pin Receptacles (but will make another bump in the high-speed traces). 17. FAN_1V8 fills are placed on both 1 oz layers Signal_11 and Signal_12. 18. Layer Usage for Fills: Signal_5: MGT Fanout to Hub FPGA Routing, BULK_3V3, Signal_11: FPGA_CORE, MGT_AVCC, ISO_12V, FAN_1V8 Individual Switch AVDDL, Individual Phys Chip DVDDL, Signal_12: FPGA_CORE, MGT_AVTT, ISO_12V, FAN_1V8 SWCH_1V2, Signal_6: MGT Fanout to ROD Routing, BULK_1V8, MGT_AVAUX, 19. Still to fit into the above table: BULK_2V5 Distribution: Signal_11 CLK_2V5: Signal_11 or 6 ?? CLK_3V3: Signal_12 or 6 ?? MiniPOD 2V5: Signal_11 ?? MiniPOD 3V3: Signal_6 Ethernet Clock Fill: Signal_11 ?? or 6 ?? BULK_1V8 to the JTAG Translator/Buffers: Signal_11 ?? ISO_12V: May end up on only Signal_12 because of needed fills under MiniPODs High Current Connections to the Zone 1 Connector and to the two ATCA Power Modules: Rev. 19-Oct-2016 -------------------------------------------- High Current Traces from the Zone 1 Connector, through the Fuses, and to the Input of the ATCA Power Entry Module: Layer Usage --------- ------------------------- Signal_1 Zone 1 to F5/6 Signal_2 Zone 1 to F5/6, & F5/6 to ATCA Entry Mod Signal_3 Zone 1 to F5/6, & F5/6 to ATCA Entry Mod Signal_4 Zone 1 to F3/4, & F3/4 to ATCA Entry Mod Signal_5 Zone 1 to F3/4, & F3/4 to ATCA Entry Mod Signal_11 Zone 1 to F4 and F5/6 Signal_12 Zone 1 to F3, & F5 to ATCA Entry Mod Signal_6 Zone 1 to F1/2, & F1/2 to ATCA Entry Mod Signal_7 Zone 1 to F5/6, & F6 to ATCA Entry Mod Signal_8 Zone 1 to F5/6 & F6 to ATCA Entry Mod Signal_9 Zone 1 to F3/4, & F3/4 to ATCA Entry Mod Signal_10 Zone 1 to PreCharge Resistors 0.5 oz 1.0 oz 0.5 oz oz -------------- --------------- -------------- --- Signal Layer 1 2 3 4 5 11 12 6 7 8 9 10 Zone 1 to F1 x 1.0 Zone 1 to F2 x 1.0 Zone 1 to F3 x x x x 3.0 Zone 1 to F4 x x x x 3.0 Zone 1 to F5 x x x x x x 3.5 Zone 1 to F6 x x x x x x 3.5 F1 to Entry Mod x 1.0 F2 to Entry Mod x 1.0 F3 to Entry Mod x x x 2.0 F4 to Entry Mod x x x 2.0 F5 to Entry Mod x x x 2.0 F6 to Entry Mod x x x x 2.0 PreCharge Resistors x 0.5 So what is the resistance and power loss on these traces ? In English units the value that I can remember is: a bar of copper 1 sq inch cross section and 1 ft long has a resistance of 8.30 micro-Ohms at 25 deg C with a coefficient of about + 0.38% per deg C. 1 oz copper is about 0.0014" thick and 2 oz about 0.0028" thick. These Hub high current traces are all 2.0 or 2.5 mm wide. 2.0 mm wide and 2 oz thick gives about 37.6 mOhm per ft or about 0.1234 mOhm per mm. This is about AWG 26 and Fermi Safety would allow me to run this at 3.2 Amps in a bundle of 1 to 3. We expect about a 250 Watt load and the supply sould be about 48 Volts. Thus we expect a total draw of about 5 Amps. The total length of the input and return traces together is about 205 mm or about 25.3 mOhm. Thus we should expect a drop of about 126.5 mVolts and a loss of about 632 mWatts in these traces. The above analysis assumes that all of the power flows on either just the A bus or just the B bus. In normal operation we should have more or less even power flow on both buses. High Current Traces from the ATCA Power Entry Module to the ATCA Isolated +12V Module: Layer Usage --------- ------------------------- Signal_1 Couple Caps Entered_48V_Rtn Signal_2 Entered_48V_Rtn Signal_3 Entered_N48V Signal_4 Entered_N48V Signal_5 Entered_N48V Signal_11 Entered_48V_Rtn Signal_12 Entered_N48V Signal_6 Entered_48V_Rtn Signal_7 Entered_48V_Rtn Signal_8 Hold_Cap_Pos_Pow_Entered Signal_9 Entered_48V_Rtn Signal_10 Couple Caps Entered_48V_Rtn All/most of these traces are 2.00mm. There is basically two runs per layer. There is 3.0 oz to carry the Entered_N48V. There is 3.5 oz to carry the Entered_48V_Rtn. So 3 oz copper and lets call the mostly two times 2.00mm wide traces equivalent to 3.2mm width (see the layout). This gives about 15.7 mOhm per ft or about 51.5 micro-Ohm per mm. This is about AWG 22 and Fermi Safety would allow 6.4 Amps in a bundle of 1 to 3. The total run length (sum of both directions) at this point is about 115 mm. So a total resistance at this point of about 6 mOhm. High Current Traces from the ATCA Iso +12V Module to the Gnd and Power Planes and Discrete Power Wire Vias: Layer Usage --------- ------------------------- Signal_1 GND and ISO_12V connections to Planes and Vias Signal_5 GND and ISO_12V connections to Planes and Vias Signal_6 GND and ISO_12V connections to Planes and Vias Signal_10 GND and ISO_12V connections to Planes and Vias All of these traces are 2.5 mm wide. There are 5 runs per net on each of these 4 layers, i.e. both the module's ISO_12V pin and its power output GND pin make 5 runs on each of these 4 layers, i.e. besides the pin connection itself, there are 20 traces each 2.5 mm wide to carry the current from these pins. The total copper weigth of these 4 layers is 3.0 oz. So this is equal to 5 runs with 3 oz copper. This gives about 4.02 mOhm per ft or about 13.2 micro-Ohm per mm. This is about AWG 16 and Fermi Safety would allow 15 Amps in a bundle of 1 to 3. The ATCA Iso +12V Module can make 25 Amps. If I throw the remaining 6 signal layers (at 0.5 oz each) at this it will cut the resistance in 2 and give us a 30 Amp capacity. I assume that it will be impossible for us to solder this.