Notes on Hub Module SN 02: Latest Rev. 30-June-2017 -------------------------- - Start work on SN 02 at 17:00 1-June-2017 - Check the resistance from the various power buses to Ground looking for shorts: DCDC-1 FPGA_Core about 4 Ohm DCDC-2 MGT_AVCC " 30 Ohm DCDC-3 MGT_AVTT " 180 Ohm DCDC-4 MGT_AVAUX > 1k Ohm DCDC-5 Switch_1V2 about 450 Ohm DCDC-6 BULK_1V8 " 300 Ohm DCDC-7 FAN_1V8 > 1k Ohm DCDC-8 BULK_3V3 > 1k Ohm DCDC-9 BULK_2V5 > 1k Ohm CLK_BULK_2V5 > 1k Ohm CNST_5V0 > 1k Ohm Check Both Ends IPMC_3V3 > 1k Ohm IPMC End IPMC_3V3 about 125 Ohm Power Module End ISO_12V > 1k Ohm Hold_UP + and - > 1k Ohm Check Both Ends - Write Serial Number on the card - Note that the card has a lable on the front surface that says, "First Article" and the back side says "TTM SN 009". - Verify that the ESD Ground connections are OK, i.e. that the RJ45 Shields and the ESD Strips are really connected to the boards Logic Ground through meg Ohm resistors. - File smooth the Top and Bottom Edges. This time it took almost one hour. Filing at an angle from one side and then the other helps to eliminate the file chatter. Air hose off the card. - Clean off some flux residue around J2 and around both of the ATCA Power Modules. - Install the Guide Pin Receptacles and re press in the Zone 1 and 2 connetors and the RJ45s. - Install the 4 backplane MPO connectors. - Install the 4 TSSOP parts on the back side; 3x 24 pin 1x 14 pins. - It is 5.5 hours to this point. - Remove all the SMD parts (resistors and jumpers) that must come off. - Install the required SMD resistors, capacitors, and jumpers. - About 8 hours to here. - Install the post L RC network on the filtered MGT supplies. - Install the discrete power wires. So far it appears that I do not need to connect the big hold-up capacitors in the NE corner for the ATCA Power Entry module to work. So currently there are 6 discrete power wires that need to be installed: 3x Iso_12V, IPMC_3V3, CNST_5V0, Clk_2V5. So far I have not glued down the discrete power wires. - It is about 11 or 12 hours to here, i.e. 3 or 4 hours total for the post L RC networks and discrete power wires. This does not include gluing down the discrete power wires. - Initial power up of SN-02 on 7-June-2017 The card was setup with JMP6 installed so that it does not need a signal from the IPMC to turn on the Iso_12V supply. The card has a wire jumper across R2952 so that non of the Hub rail DCDC Converters on the card should start up. When given 48V power and the ATCA Backplane enable signal the Iso_12V LED only flashed about once every two seconds. Much chasing for the case of this. It is not cause by the un-connected Hold-Up capacitors. It is not cuased by a short or some problem like that on the Iso_12V bus or on the 48V buses. After chasing for a while it's clear that there is no continuous ouput from the ATCA Power Entry module. It appears that the ATCA Power Entry module must not be loaded when the Power Entry module first gets the ATCA backplane enable signal. So remove JMP6 and install a "Hub Power ON" switch across this location instead. If you now turn on the ATCA Backplane Enable, wait one or two seconds, and then enable the Iso_12V supply with the Hub ON switch the Iso_12V rail comes up just fine. The Iso_12V rail is OK so remove the jumper across resistor R2952 so that the Hub's DCDC Converters can turn ON. The Hub DCDC Converters appear to all come up OK. As expected the MGT_AVAUX and BULK_2V5 were both about 100 mV low as their trimmers had been preset for minimum output. All rail voltages look OK and the total draw on the 48 Volt bus appears to be in the 900 mA to 1 Amp range. Note that the Fan_1V8 was drawing its normal 25 Watts or so. The Hub FPGA was just slightly warm to the touch with no heat sink and nothing Configured into it. - Temporary FPGA Heat Sink for SN-02 8-June-2017 The prototype approved for production Hub FPGA heat sink was retrieved from the Machine Shop (along with the 3x types of Brackets and the Fill Block and the MiniPOD HS are almost ready). I milled the 40 mil deep trench for the MTG_AVTT capacitor into the flat surface of the heat sink. It appears that the trench needs to move East from where it is shown in drawing m26. I cut the trench in this HS with an end mill instead of a ball mill. The Fill Block also needs to be trimmed where the ends of the MGT_AVCC capacitors come up against it. The MGT_AVCC capacitors need to be insulated with kapton tape wrt the vias in the pcb, the heat sink surface, and the vertical edge of the Fill Block. - JTAG test of SN-02 8-June-2017 At first the vivado jtag system could not see the Hub's FPGA at all. I traced this to the control line to the U556 nc7sv145 JYAG mux being HI. That must be caused by the un-configured Hub FPGA Select I/O pin pulling Hi and that feeding through R2995 and pulling up on R2998. I need to verify that the correct resistor values have been installed. If that is OK then try moving R2995 from 470 Ohm to 10k Ohm and move R2998 from 10k to 2k Ohm. For today I just jumpered across R2298. Now the vivado JTAG sees the Hub's FPGA - Configure SN-02 via JTAG 8-June-17 Pawel has the JTAG cranked up to 6 MHz. That Configures the Hub FPGA in about one minute. - Additional work on SN-02 on 14-June-2017 Exchange the screws that came with the backplane MPO connectors for a 2-56 5/16" screw with an external tooth lock washer. The supplied screws and washers only get about 3 1/2 or 4 threads into the mounting rods on the MPO connectors. These mounting rods appear to be tapped for about 12 turns of threads. The new 5/16" screws get about 7 1/2 or 8 turns of threads. The ATCA Guide Pin Receptacles are mounted with an M2.5 10 mm screw and flat washer. I need to check what is under this part of the card to see if a lock washer can be used. Rework the resistor values in the circuit that receives the ROD_Power_Good signal, aka ROD Power Control #2. This signal is HI when all ROD power is Good. Change R2998 the pull-down from 10k to 2k Ohm. The ROD_Power_Good signal must be able to supply 1 mA minimum of pull-up current. Change R2295 the series isolation resistor feeding the ROD_Power_Good signal to the 1V8 Select I/O pin on the Hub FPGA from 470 Ohm to 10k or 20k Ohm. This is large enough so that the un-configured state of this Select I/O pin can not overwelm the pull-down resistor. Work on testing the full Hub Clock System. - Both side of the 4 outputs from the 1st 40 MHz Fan Out are running. - Both sides of all 12 40 MHz clocks to the backplane are running. - Both sides of all 9 fanouts of the 320 MHz clock are running. - Made a differential output adaptor for the HP generator so that I can feed a signal to the "Spare Oscillator" location on the Hub pcb. - It looks like the "Safe FPGA" Configuration does not currently include the nets/routing to get the Spare Oscillator signal to the reference output to the 40 MHz PLL. - Additional work on SN-02 on 15-June-2017 Working on testing SN-02 Clock System. The "free run" frequency of the 40 MHz PLL is: 40.0728 MHz. - Additional work on SN-02 on 16-June-2017 Using the differential input adapter and the HP generator set to 250 mVpp output square wave I see the following lock range for the SN-02 PLLs: 40 MHz PLL 40.074 40.084 320 MHz PLL 40.0756 40.0826 Install the Spare Oscillator. Leaving the pin #1 enable open circuited is OK as the data sheet says. I needed to add some pull-downs to ground to get an output. Note this is a PECL output oscillator. I added 309 Ohm pull-downs and that is more than enough. It's making 800 or 900 mV of output swing and I only need 300 mV. The frequency of this oscillator is in the range: 40.07849 to 40.07850 MHz. Start running the Hub FPGA --> Trans MiniPOD --> Rec MiniPOD --> Hub FPGA loop test. Start running it at 4.8 Gbps and then move to 6.4 Gbps. Check PS current: Power Bus: FPGA MGT MGT Swch Bulk Fan Bulk Core AVCC AVTT 1V2 1V8 1V8 3V3 ---- ---- ---- ---- ---- ---- ---- Address: 40 41 42 43 44 45 46 Volts: 0.93 1.00 1.20 1.20 1.80 1.82 3.317 Amps: 1.38 0.06 0.19 1.94 0.69 11.88 - Not Config or Safe Amps: 2.19 1.56 1.06 1.94 0.69 11.88 2.25 4.8 Gbps The 4.8 Gbps test ran for 3:50 with 0 errors. FPGA Temp was about 30 deg C. Amps: 2.38 1.81 1.06 1.94 0.69 11.88 2.25 6.4 Gbps The 6.4 Gbps test ran over Friday night. The FPGA Temp was about 33.5 deg C. The 6.4 Gbps test was stopped Saturday June 17th after about 19 Hrs of running with 0 errors on the 3 links. Next try running at 9.6 Gbps. Amps: 2.62 1.19 2.00 11.94 Amps: 2.50 1.88 1.06 1.94 0.69 11.88 2.25 9.6 Gbps The 9.6 Gbps test ran until Monday morning June 19th with 3 links showing zero errors in 44 hrs of running. The FPGA temperature was 33 to 34 deg C. Next try 11.2 Gbps. Amps: 2.75 2.12 2.00 Amps: 2.69 2.00 1.06 1.94 0.75 11.88 2.25 11.2 Gbps Monday morning June 19th - two of the three links are running OK at 11.2 Gbps but note that the Trans and Rec MiniPODs are only 10 Gbps parts AFBR-811 FN1Z & AFBR- - Hub SN-02 was shipped to Ed at Cambridge on 30-June-2017.