Hub-Module Power System ---------------------------- Original Rev. 20-Jan-2015 Current Rev. 2-May-2017 The Hub-Module power system is based on the ATCA architecture. This power system supplies both the components on the Hub-Module itself and provides +12V power to the ROD mezzanine card. An overall view of the Hub Module power supplies is presented in drawing #24. This Hub Module power system document is divided into a number of sections: - ATCA power entry and the isolated +12V supply - Brief description of the Hub's power buses and their loads - Description of the 7 DCDC Converters, summary then details - Description of the 2 Linear power supplies on the Hub - Supervision of the Hub Modules power supplies. - Power requirements information for the various Hub systems ATCA Power Entry and Isolated +12V Supply: ------------------------------------------ Power reaches the Hub Module's zone 1 connector on redundant negative 48 Volt buses. After passing through fuses the bulk -48 Volt power from the backplane enters a SynQor "power entry module". This is either a SynQor IQ65033QMA10 10 Amp 300 Watt module or a SynQor IQ65033QGA12 12 Amp 350 Watt module. In either case these are quarter-brick size modules with the same footprint. The first prototype Hub cards are built with the SynQor IQ65033QMA10SNF-G power entry module. From the power entry module filtered power, monitored by the IPMC, flows to the input of an Isolated +12 Volt converter. This converter makes the Isolated +12 Volt power that operates everything on the Hub Module except for the power supervision circuits and the IPMC mezzanine card. The Isolated +12 Volt power is provided by a SynQor PQ60120QEA25NNS-G 12V 25A 300 Watt quarter-brick module. Higher current modules are available if needed. The Isolated +12 Volt power from this SynQor module is the input power to a number of non-isolated DC/DC "buck" converters that generate the actual power buses used on the Hub Module. This same Isolated +12 Volt power is provided to the ROD mezzanine card on the Hub. The power entry module provides a separate isolated 5V0 supply that is used to operate the power supervision circuits on the Hub Module. This isolated 5V0 supply is available anytime that the Hub receives backplane 48V power. The power entry module also provides separate isolated 3V3 power for the IPMC card. This separate 3V3 power is available anytime that the input module receives backplane 48V power. The IPMC controls when the Isolated +12V module is allowed to operate. When the IPMC and the Shelf Manager have successfully negotiated the power up of the Hub Module then the IPMC assets the enable signal to the Isolated +12V module. The IPMC the monitors the power entry module using its Management I2C bus which runs to both the ATCA power entry module and to the FRU & SDR EEPROM. The power entry and Isolated +12V section of the overall Hub Module design are shown in drawings #06, #07. Drawing #08 shows the various backplane ground connections that are used on the Hub Module. Hub Module Power Bus Table: --------------------------- Power Bus Bus Principal Anticipated Supply Supply NetName Voltage Consumers Load Capacity Name --------- ------- --------------- ----------- -------- ------ FPGA_CORE 0.950 V FPGA INT & BRAM 20.8 Amps 40 Amps DCDC-1 MGT_AVCC 1.000 V FPGA MGT 10.8 Amps 20 Amps DCDC-2 MGT_AVTT 1.200 V FPGA MGT 10.0 Amps 20 Amps DCDC-3 MGT_AVAUX 1.800 V FPGA MGT 0.60 Amps 3 Amps DCDC-4 SWCH_1V2 1.200 V Switch & Phys 4.0 Amps 12 Amps DCDC-5 BULK_1V8 1.800 V FPGA & other 3.0 Amps 12 Amps DCDC-6 FAN_1V8 1.800 V MGT Fanout 13.0 Amps 20 Amps DCDC-7 BULK_3V3 3.300 V Switch & other 4.6 Amps 12 Amps DCDC-8 BULK_2V5 2.500 V MiniPOD & Clk FO 1.3 Amps 3 Amps DCDC-9 The 40 Amp supply is a GE MDT040A0X3-SRPHZ The 20 Amp supplies are GE UDT020A0X3-SRZ The 12 Amp supplies are GE PDT012A0X3-SRZ The 3 Amp linear supplies are LT1764AEQ The low current reference supplies for the FPGA System Monitor are not included in this table. DCDC Converters: ---------------- Both GE (Lineage Power) D-Lynx series and TI (Power Trends) PTH08Txyz series converters were evaluated for use on the Hub Module. The D-Lynx converters were selected based mostly on the PMBus connection offered by their TI TPS40400 control chip. Concerns about using the D-Lynx converters remain in the areas of noise, assembly, and rework. Note that for the 2 low current supplies Linear Technology LDO linear regulators are used both to save space on the PCB and to insure a low noise supply bus. The amperage capacity of each of these DC/DC converters was sized to the load that it must drive. During this selection consideration is also given to minimizing the number of different component types used on the Hub Module. The details of the 7 GE DCDC Converters are shown in drawings: 26A, 26B, 26C. The following table and its notes summarize these designs: Converter Name: DCDC1 DCDC2 DCDC3 --------- --------- --------- Bus NetName: FPGA_CORE MGT_AVCC MGT_AVTT Bus Voltage: 0.950 V 1.000 V 1.200 V Max Bus Amps: 40 A 20 A 20 A Converter Type: MDT040A0X3 UDT020A0X3 UDT020A0X3 * Cin Local: 62 uFd 42 uFd 42 uFd # Cout Local: 220 uFd 580 uFd 580 uFd Cout Remote: 3822 uFd 1013 uFd 1013 uFd R V-Set Theory: 34.29k 30.0k 20.0k R V-Set Real : 34.00k 30.0k 20.0k R Ramp Series: 20.0k 20.0k 20.0k R Ramp Shunt : 34.0k 30.0k 20.0k R Servo: 180 Ohm 180 Ohm 180 Ohm C Servo: 10 nFd 10 nFd 10 nFd R ADRS_0: 10k 15.4k 23.7k R Adrs_1: 84.5k 84.5k 84.5k I2C Adrs: 0o50 0o51 0o52 Anticipated Load in Watts: 19.8 W 10.8 W 12.0 W @ Anticipated Input Current: 1.83 A 1.00 A 1.11 A Design Notes: -- External External Output Output LC Filter LC Filter Converter Name: DCDC5 DCDC6 DCDC7 DCDC8 --------- --------- --------- --------- Bus NetName: SWCH_1V2 BULK_1V8 FAN_1V8 BULK_3V3 Bus Voltage: 1.200 V 1.800 V 1.800 V 3.300 V Max Bus Amps: 12 A 12 A 20 A 12 A Converter Type: PDT012A0X3 PDT012A0X3 UDT020A0X3 PDT012A0X3 * Cin Local: 42 uFd 42 uFd 42 uFd 42 uFd # Cout Local: 88 uFd 88 uFd 580 uFd 88 uFd Cout Remote: 1530 uFd 1510 uFd 1929 uFd 1030 uFd R V-Set Theory: 20.0k 10.0k 10.0k 4.444k R V-Set Real : 20.0k 10.0k 10.0k 4.420k R Ramp Series: -- 40.0k 40.0k 40.0k R Ramp Shunt : -- 20.0k 20.0k 8.87k R Servo: 180 Ohm 180 Ohm - Ohm 200 Ohm C Servo: 10 nFd 15 nFd - nFd 10 nFd R ADRS_0: 36.5k 54.9k 84.5k 130k R Adrs_1: 84.5k 84.5k 84.5k 84.5k I2C Adrs: 0o53 0o54 0o55 0o56 Anticipated Load in Watts: 4.8 W 5.4 W 23.4 W 15.2 W @ Anticipated Input Current: 0.44 A 0.50 A 2.17 A 1.41 A Design Notes: Delayed -- External -- Turn On & Output Self Ramp LC Filter * --> The Cin Local capacitors are all ceramic of 3 different values and are located immediately next to the converter's input terminal. There is an additional Bulk Cin of 2100 uFd consisting of both ceramic and Tantalum capacitors that is shared by all 7 DCDC converters. This Bulk Cin is located within a few cm of the converters and is connected to each converter by a neck of trace typically a one or two cm wide. # --> The Cout Local capacitors are all ceramic of 2 different values and are located immediately next to the converter's output terminal. Note that in the case of converters: DCDC_2, DCDC_3, and DCDC_7 (the 3 DCDC Converters that include an external LC output filter) that the Cout Local includes a 470 uFd Tantalum capacitor in addition to the multiple ceramic capacitors. Cout Remote consists of ceramic and Tantalum capacitors of multiple values distributed around the card. In the case of the FPGA supplies these capacitors are all located near the U1 FPGA. @ --> The anticipated Input Current in Amps for each supply assumes 90% efficiency for that converter. Currently the total anticipated load on the Isolated +12V bus from the Hub itself is 8.46 Amps or about 101.5 Watts. About 91 Watts of this reaches components on the Hub and about 10 Watts is lost as heat in the DCDC converters. DCDC Converter Input Filter Capacitors: --------------------------------------- There is a limit of about 12,000 uFd for the total amount of capacitance that one may have on the output of the Isolated +12V converter without upsetting its stability. Note that this 12,000 uFd is specified with the Isolated +12V supply running at capacity into a resistive load and I do not know how running it at less than full output or running it into a chopping DCDC Converter load affects its stability with the full 12,000 uFd on its output. This 12,000 must be split between the DCDC Converters on the Hub itself and the converters on the ROD mezzanine On the Hub Module itself, the total capacitance presented to the Isolated +12V supply is: 3118 uFd or about 26% of the assumed total available budget. DCDC Linear Supplies: --------------------- Two low current power buses on the Hub Module use linear LDO regulators. One of the linear supplies is MGT_AVAUX and the other is BULK_2V5. Both of these linear supplies are powered from the BULK_3V3 bus. Both supplies use an LC filter between the possibly noisy BULK_3V3 supply and the linear regulator's power input pin. These supplies include a trim pot with a +-5% adjustment range. These LDO regulators are enabled all of the time. Their output ramps up as the BULK_3V3 supply ramps with the regulator output typically trailing the BULK_3V3 supply by about 200 mV. These two supplies do not have PMBus monitoring and control like the 7 DCDC converters do. Details of these low current linear supplies are shown in drawing #27. DCDC Converter Supervision: --------------------------- The IPMC supervises the Hub module power system through its control of the Isolated +12V bus. In addition the IPMC can monitor the ATCA power entry module via its Management I2C bus and it can monitor the 7 DCDC converters via its Sensor I2C bus. The only two supplies that can not be directly monitored in this way by the IPMC are the linear regulators for the MGT_AVAUX and BULK_2V5 power buses. The voltage of these supplies can be read by the FPGA's System Monitor. The System Monitor can internally see the MGT_AVAUX supply and the BULK_2V5 is scaled and connected to an analog input to the System Monitor's ADC. The voltage of all 9 supplies can be accurately measured with a good DVM plugged into header J3 in the NE corner of the Hub circuit board. This step is needed to calibrate the internal monitoring provided by the DCDC Converters. A number of components on the Hub Module (FPGA, Switch chips, and Phys chips) have specific power supply sequencing requirements. These requirements are meet by ramping up together on a volt per volt bases all supplies on the Hub module except for the SWCH_1V2 supply. The SWCH_1V2 converter is not started until the other 8 supplies have ramped up and are stable. The circuits that control the startup of the power supplies are shown in drawings: #28, #30, and #31. The startup process has the following steps: - The process starts when the supervisor sees that the enable signal to the Isolated +12V supply has been asserted and that the Isolated +12V bus from this supply has reached a minimum of 8.5 Volts. - At this point a delay of 500 msec is used to allow the Isolated +12V bus to reach its full value and to stabilise. - Then the enable signal is sent to 6 of the DCDC converters on the Hub Module, all of the DCDC converters except for the SWCH_1V2 supply. - At this point a delay of 120 msec is used to allow these 6 DCDC converters time to complete their internal startup routines. Note that the output of these converters does not begin to ramp during this 120 msec delay because the Ramp Sequence pins on these converters is being held low. - After this 120 msec delay the Ramp Sequence signal to these 6 converters begins to rise. The converter outputs follow the Ramp Sequence signal on a volt per volt bases. The main supplies to the FPGA reach their full output over a 5 to 9 msec period. The BULK_3V3 supply is the last to reach its full output - taking about 3 times longer than a 1 Volt supply. - About 100 msec after these 6 supplies have reached their full output the SWCH_1V2 converter is enabled. The ramp rate of this converter is internally controlled so that it reaches its full output in a few msec. This delayed startup of the SWCH_1V2 supply is required by the Switch chips, not just to prevent cross coupling of the power rails through the CMOS circuits but rather because the Switch does not want its core to start until its I/O has been powered up and is stable. - The two linear regulator supplies MGT_AVAUX and BULK_2V5 receive their input power from the BULK_3V3 supply. They just ramp up about 200 mV behind the BULK_3V3 supply. - Once all 9 supplies are up and stable, as indicated either by their Power Good signals being asserted or as measured for the two linear regulator supplies, then there is a delay of 500 msec for everything to settle down and then the Board_Startup_Reset_B signal is de-asserted. The distribution of the Board_Startup_ Reset_B signal is shown in drawing #32. This completes the power up of the Hub Module. The circuits that supervise this startup process are powered by an isolated 5 Volt output on the ATCA power entry module. This 5 Volt output is active anytime that the card receives backplane 48 Volt power. The shutdown of the Hub Module supplies is initiated when either the enable to the Isolated +12V supply is dropped (the normal shutdown situation) or when the output of the Isolated +12V supply falls below 8.5 Volts. During the shutdown sequence the SWCH_1V2 converter is immediately stopped and the other 6 converters ramp down quickly in an orderly manner for as long as their input power remains available. When the input power falls below threshold then all of the converters are immediately stopped. The shutdown sequence is shown in drawing #29. During shutdown the Board_Startup_ Reset_B signal is asserted as soon as the enable is removed from the Isolated +12V supply and remains asserted for as long as power is available. DCDC Converters with an LC Output Filter: ----------------------------------------- Three of the DCDC Converters on the Hub Module include an LC Output Filter to reduce the switching noise that is present in their output voltage. These 3 converters make power for the MGT Transciver circuits and these circuits specify a peak to peak noise voltage of under 10 mV on their power supply rails. The low current MGT_AVAUX supply meets this requirement by using a linear regulator with a filter inductor before it to remove the high frequency noise. The higher current MGT_AVCC, MGT_AVTT, and FAN_1V8 supplies meet this requirement by using an LC filter after these DCDC Converters. The L for this filter is a high current low resistance 680 nH Wurth inductor. The C for the filter consists of large value tantalum capacitors and ceramic capacitors of multiple values. Most of these capacitors are located near the load point. A potential issue with using an LC filter after these buck DCDC Converters is that their servo control loop may require special compensation for it to be stable. To provide various options for how the feedback network is setup in these 3 converters the Hub Module pcb includes a set of 4 jumpers for each of these supplies. The jumpers that are associated with each of these converters are: DCDC-2 MGT_AVCC JMP1051, JMP1052, JMP1053, JMP1054 DCDC-3 MGT_AVTT JMP1101, JMP1102, JMP1103, JMP1104 DCDC-7 FAN_1V8 JMP1301, JMP1302, JMP1303, JMP1304 Jumpers JMP1051, JMP1052 (and JMP1101, JMP1102 and JMP1301, JMP1302) control whether the the V_SENSE_+ feedback is taken before or after the output filter inductor. JMP1051 selects before the output inductor and JMP1052 selects after the inductor. Jumpers JMP1053, JMP1054 (and JMP1103, JMP1104 and JMP1303, JMP1304) control whether the the RC coupled feedback is taken before or after the output filter inductor. JMP1053 selects before the output inductor and JMP1054 selects after the inductor. This jumpers information is also presented in the "as built jumpers" document. In addition to these feedback source selection jumpers, the layout for each of these 3 converters also includes the SMD pads to mount an 0805 capacitor that directly connects between the converters output and its "Trim" input pin. Power Requirements Information for each Section of the Hub: ----------------------------------------------------------- The following are power supply requirements information about the various components on the Hub Module. This information was used to design the overall Hub Module power supply and to make the overall list of Hub Module power buses shown above. More details about the power requirements of the individual sections can be found in their "as built" documents. Power Supply Requirements for the Hub Module UltraScale XCVU125 FPGA: FPGA Bus Bus Volts Estimated Current Draw --------- ------- -------------------------- VCCINT 0.950 V 20.8 Amps est. +VCCBRAM VCCAUX 1.800 147 mA quiescent VCCAUX_IO 1.800 2 mA quiescent VCCO 1.800 / 3.300 1 mA per bank quiescent VCCADC 1.800 few mA VREFP 1.250 few mA MGT_AVCC 1.000 8.8 Amps est. MGT_AVTT 1.200 12.6 Amps est. MGT_AVAUX 1.800 0.6 Amps est. Note that this FPGA has power supply sequencing requirements. See below for more details about the estimates for FPGA MGT power from the Xilinx tools. Requirements for the Base Interface Switch Power Supplies: A separate 12 Amp converter is used for the Switch's 1.2 Volt requirements. To meet the power supply sequencing requirements of the Switch and Phys chips, this SWCH_1V2 converter is not enabled until after the other DCDC converters on the Hub have ramped up and stable. The total draw on the SWCH_1V2 bus from all 3 switch chips is expected to be 3.5 Amps. The total draw on the BULK_3V3 supply from the 3 Switch chips is expected to be 1.5 Amps. The total heat dissipation from the 3 Switch chips is expected to be 9.2 Watts. Requirements for the GTH Fanout Power Supply: The 74 channel FEX Data Fanout (aka GTH Fanout) has its own private power bus called FAN_1V8. The reasons to have a private converter for this Gb/s speed fanout include the requirement for low noise and the possible need to adjust this supply independent of other loads. The expected load on the FAN_1V8 supply is 13.0 Amps which results in about 23.4 Watts of heat from the 74 fanout chips. Requirements for the Micrel KSZ9031RNX Phys Chip Power Supplies: In the Hub application the KSZ9031RNX requires 3 supply buses: 1.2V supply for: AVDDL the Analog Core DVDDL the Digital Core AVDD_PLL Internal PLL 1.8V supply for: DVDDH the Digital I/Os 3.3V supply for: AVDDH Transceiver Analog Power The total load on each of these power buses from the 2 Phys Chips on the Hub Module is expected to be: 1.2V supply 530 mA from SWCH_1V2 delayed turn on 1.8V supply 130 mA from BULK_1V8 3.3V supply 164 mA from BULK_3V3 Total Power Consumption with 1.8 V digital IO and 3.3 V Transceiver supply and 100% utilization at 1000 Base-T is expected to be about 1.5 Watts for both Phys chips. Requirements for the Clock Generation and Fanout Power Supplies: The 25 MHz Ethernet Clock generator and its fanout are expected to use 55 mA from the BULK_3V3 supply. The 40.08 MHz and 320.64 MHz Clock generators and their fanouts are expected to use 160 mA from the BULK_3V3 supply and 381 mA from the BULK_2V5 supply. All supplies to these clock generators and fanouts are filtered and distributed to the clock components on a separate isolated PCB area fills. A total of about 2.0 Watts is needed for Clock generation and distribution. Requirements for the MiniPOD Power Supplies: The MiniPODs each require: 2.5 V and 3.3 V power. The Transmitter MiniPOD requires: from the 2V5 bus 280 mA typical 365 mA maximum from the 3V3 bus 105 mA typical 185 mA maximum This is a heat load of 1.2 Watts typ 1.6 Watts max. The Receiver MiniPOD requires: from the 2V5 bus 350 mA typical 525 mA maximum from the 3V3 bus 48 mA typical 90 mA maximum This is a heat load of 1.1 Watts typ 1.6 Watts max. The total maximum requirements for both MiniPODs are: Bulk_2V5 bus 890 mA Bulk_3V3 bus 275 mA Heat Load 3.1 Watts Power to both MiniPODs is filtered and distributed on separate isolated PCB area fills. Final best Estaimate on FPGA MGT Power from Xilinx Tools: --------------------------------------------------------- Requirements for the MGT AVCC and AVTT Power Supplies 3-Aug-2016 Ultrascale GTH and GTY channels each draw currents from: 1.0 V AVCC power 1.2 V AVTT power The Xilinx Power Estimator spreadsheet (XPE) was used to evaluate the expected power usage for AVCC and AVTT. The current draw on these supplies depends highly on the line rate and very little on the user data payload. This is quite different from estimating current usage in the main logic area where user data content and most importantly the toggling ratio is the driving factor. For MGT channels the toggling ratio for the highest rate circuitry has to be 50%. XPE thus has all the information it should need to produce an accurate estimate for MGT AVCC and AVTT. The goal was to determine a maximum expected power usage. This requires knowing the maximum line rate considered. The asumptions were - 9.6 Gbps maximum line rate for backplane and all readout channels - 4.8 Gbps maximum line rate for miniPOD channels, i.e. all TTC, combined data and FELIX channels - DFE used for all receiver channels (as it uses more power) - 1.0 V differential swing on Hub FPGA readout output channels - 0.5 V differential swing on miniPOD output channels The MGT channel assignment from 18-Apr-2016 provided the list of GTH and GTY channels the Hub uses only as transmitters and as full transceivers, grouped by usage and by corresponding expected line rates. This list was used to configure the XPE GTH and GTY tabs. 4.8, 6.4 and 9.6 Gbps are backplane speeds still plausible at this time for l1calo. XPE indicates that some current draws are higher at 6.4 than 9.6 Gbps, for GTY AVTT in particular. Two sets of XPE estimates were thus needed, for backplane line rates of 6.4 and 9.6 Gbps. The maximum values are summarized below. The maximum currents from these XPE estimates are: MGT AVCC drawing 8.8 Amps MGT AVTT drawing 12.6 Amps Note: The "EyeScan" feature was NOT enabled in the XPE estimation. Adding this option to all channels increases AVCC only, by ~2 A. The excel file that generated these estimates is 20160714_Hub_full_VU125_XPE_2015p4p1.xlsm in the components/xilinx/PowerEstimator web area. The file hub_0_ultrascale_notes.txt contains more XPE results about this estimate and other MGT power usage studies. Review of the Tantalum Bypass Capacitors: 2-May-2017 ----------------------------------------- The intent of this section (written after the pcb layout was completed) is to review how many Tantalum Bypass capacitors are used on each supply rail and to indicate which Mentor components file holds which of these capacitors. Note that the Tantalum capacitors on the Isolated 12 Volt bus are not covered in this section. This section covers just the supply rails that feed power to the various ICs in the Hub design. Two kinds of Tantalum Bypass capacitors are used on these supply rails: Cap_330_uFd_Tant_V Tantalum Capacitor 330 uFd 6.3 Volt 18 mOhm KO Cap Polymer V Case SMD 1.9 mm tall Kemet Part No. T520V337M006ATE018 Cap_470_uFd_Tant_V Tantalum Capacitor 470 uFd 2.5 Volt 12 mOhm KO Cap Polymer V Case SMD 1.9 mm tall Kemet Part No. T520V477M2R5ATE012 Both of these Tantalum capacitors are the stupid "V" case forced on the design by the ATCA mechanics. DCDC_1 FPGA_CORE 8x 470 uFd from virtex_non_rpcs_comps DCDC_2 MGT_AVCC with External LC Filter 1x 470 uFd pre L from dcdc_2_converter_90_deg_comps 2x 470 uFd post L from virtex_non_rpcs_comps DCDC_3 MGT_AVTT with External LC Filter 1x 470 uFd pre L from dcdc_3_converter_90_deg_comps 2x 470 uFd post L from virtex_non_rpcs_comps DCDC_4 MGT_AVAUX 1x 330 uFd from dcdc_4_converter_non_rpcs_comps 2x 470 uFd from virtex_non_rpcs_comps DCDC_5 SWCH_1V2 3x 470 uFd from distributed_bypass_cap_comps DCDC_6 BULK_1V8 2x 470 uFd from virtex_non_rpcs_comps 1x 470 uFd from distributed_bypass_cap_comps DCDC_7 FAN_1V8 with External LC Filter 1x 470 uFd pre L from dcdc_7_converter_90_deg_comps 4x 470 uFd post L from distributed_bypass_cap_comps DCDC_8 BULK_3V3 4x 330 uFd from distributed_bypass_cap_comps DCDC_9 BULK_2V5 1x 330 uFd from dcdc_9_converter_non_rpcs_comps ------ IPMC_3V3 1x 330 uFd from ipmc_and_associated_comps