Characterization of Run II Components ----------------------------------------- Original Rev. 15-APR-1997 Most Recent Rev. 23-OCT-1998 This file is the central location for information that characterizes the operation of the various Components in the Run II Trigger Frameworks. This information is typically obtained by making measurements on actual Run II Components. These measurements are made both on individual cards and on setups made of a number of cards and various interconnecting cables. This file is divided up into sections with a separate section for each of the principal cards or setups on which measurements were made. 14-SEP-1998 -------------- Down in the body of this file look for the following sections. Many of these will have their own Revision Date. Foundation Module Characterization Paddle and Pass-Through Card Characterization Register Test Loop Timing Power Supply Current Test 100328 Output Voltage Characterization with 1k Ohm Pull Down XC4013L power usage characterization Final Official Measurement of the P1 Backplane Timing Signal Zo FPGA Timing Test "Xilinx Speed Tests" Voltage drop across Schottky diodes clamping Vcc to Vdd VME Interface Notes Additional P1_TS Notes Foundation Module Characterization: ----------------------------------- Series Terminator Resistors Feeding MSA_IN signals into the Main Array. Currently 33 Ohm looks best for the heavily loaded lines (for example as will be used on the AND_OR Card), but want to keep options open to assemble with 33, 39 47 or 56 Ohm series terminators feeding the Main Array with MAS_IN. Feeding Timing Signals from the BSFS FPGA into the Main Array. 47 Ohm continues to look optimum for this. Feeding Timing Signals from the ECL Receivers to the BSFS FPGA (and the VME Interface FPGA and the 53 MHz driver for Finisar). For the 12 lines that run just to the BSFS FPGA the 47 Ohm 4 Resistor Chip Array packs appear to be best. For the 2 lines that use discrete 1206 chip resistors, as of 18-APR-1997, it is not really clear what value is best. We should be ready to build with any standard value between 33 and 56 Ohms. Paddle and Pass-Through Card Characterization: ---------------------------------------------- Register Test Loop Timing: -------------------------- Using version PLAY 2.6 and running the register test on 4 FM cards (both the Main Array FPGA's and the BSFs FPGA's) for a total of 1560 registers you see the following timings: with a 2 MHz TS_0 signal you get 1.223 million loops per minute with a 50 MHz TS_0 signal you get 2.925 million loops per minute Power Supply Current Test: -------------------------- VCC VDD VTT VEE Conditions Amps Amps Amps Amps ---------------------------------------- ----- ----- ----- ----- TOM and Vertical 3. 0. 1. 1. TOM, Vertical, and 4xFM 13. 2. 10. 9. TOM, Vertical, and 17xFM (@ TS_0 2 MHz) 44. 10. 32. 35. TOM, Vertical, and 17xFM (@ TS_0 50 MHz) 44. 11. 32. 35. 100328 Output Voltage Characterization with 1k Ohm Pull Down ------------------------------------------------------------ A single 100328 was tested with both 470 Ohm and 1k Ohm output pull down resistors to Vee. Vcc was set at +5.00V and Vee was aet at -4.50V 470 Ohm Pull Down 1 k Ohm Pull Down -------------------- ------------------- Output Emitter Output Emitter Voltage Current Voltage Current --------- ------- --------- ------- Voltage High Output - 0.854 V 7.76 ma - 0.759 V 3.74 ma Voltage Low Output - 1.664 V 6.03 ma - 1.576 V 2.92 ma Recall the 100k Nominal Standards: One version is: Vbb is - 1.32 V Logic HI output will be equal or greater than -1.025 V Input voltage must be equal or greater than - 1.165 V to guarantee it will be recognized as a logic HI. Logic LOW output will be equal or less than -1.620 V Input voltage must be equal or less than - 1.475 V to guarantee it will be recognized as a logic LOW. Another version is: Output Logic Hi: - 1.025 V min - 0.880 V max Input Logic Hi: - 1.165 V min - 0.880 V max Output Logic Low: - 1.810 V min - 1.620 V max Input Logic LOW: - 1.810 V min - 1.475 V max Why all this concern about 1k Ohm pull downs ? Because this will be used in driving the Finisar Laser board. The HP G_Link will not have any additional pull downs on its following inputs: DIV0, DIV1, RST*, ED, FF, FLAGSEL, M20SEL, and LOOPEN. On the HP G_Link part all of these inputs are referred to by HP as type "I-ECL" inputs. These I-ECL inputs have internal 16k Ohm pull down resistors and may be floated for logic LOW and tied to GND for logic HI. The I-ECL inputs guarantee that anything above - 1.150 V will be recognized as logic HI and anything less than - 1.500 V will be recognized as a logic LOW. So our 1 K pull down is OK. XC4013L power usage characterization ------------------------------------ The AONMET FPGA has a "Power Burner" component, which is approximately 200 FF's (18% of the FF's in the chip) toggling at some frequency, driving 4 outputs. The frequency is variable, and the outputs can be shut off. Preliminary tests of the power consumption of this component (which is thought to be similar to the power consumption of a typical FPGA in the Framework, which may have more logic but not all running at a high rate) are as follows: Toggle frequency Outputs Power in one chip ---------------- ------- ----------------- 10 MHz on 59 mA (195 mW) 10 MHz off 52 mA (172 mW) 5 MHz on 35 mA (116 mW) 5 MHz off 30 mA ( 99 mW) 2.5 MHz on 20 mA ( 66 mW) 2.5 MHz off 18 mA ( 59 mW) clock off 7 mA ( 23 mW) unconfigured 8 mA ( 26 mW) Doing a very simple calculation, a full crate of AONM's configured this way (assuming for simplicity that the BSF and VME use the same power as the AONMET, which is probably wrong) would consume: 59 mA * 18 FPGA's/board * 20 boards/crate = 21 A (70 W) of 3.3V Thus the planned-for heavy draw of 3.3V does not appear as likely as originally thought, and the dual tied 3.3V supplies are probably overkill. Final Official Measurement of the P1 Backplane Timing Signal Zo 10-NOV-97 --------------------------------------------------------------- --------- We need to know the Zo of the P1 Backplane Timing Signal lines so that we can pick the proper Terminators for the RAY card. Measure this by looking at the signals in both slots #3 and #19, with both 0 cards and 18 cards in the crate, and look at both the normal mode and common mode part of the Timing Signal. The "matched" HP probes and the 350 MHz Tek scoped are used to view Timing Signal #0 running at about 10 MHz. With the scope at 10 nsec per division one can see both edges of this timing signal. A very small 100 Ohm trim pot with "zero length" leads is used as the terminator. Probe both sides of the differential TS#0. Set up the scope for both common mode and normal mode displays. Adjust the pot for best scope display then measure the value of the pot. Repeat. Finally verify the good operation of the scope and probes by connecting both probes to the same side of the signal and make sure that with scope inversion of Ch#2 that the Ch#1 + Ch#2 display cancels. Best Setting in Ohms 18 Cards in the Crate -------------------------------------------------------- Slot #19 Slot #3 ----------------------- ----------------------- Norm Mode Comm Mode Norm Mode Comm Mode --------- --------- --------- --------- 36.0 33.2 34.4 35.0 35.0 35.0 38.2 35.8 34.3 32.7 37.0 33.1 34.5 35.0 34.4 35.8 Average 34.95 33.97 36.0 34.93 Average 34.46 35.46 Average 34.96 0 Cards in the Crate -------------------------------------------------------- Slot #19 Slot #3 ----------------------- ----------------------- Norm Mode Comm Mode Norm Mode Comm Mode --------- --------- --------- --------- 56.1 51.2 49.8 49.1 54.2 49.8 50.6 51.4 51.4 50.4 48.1 48.6 52.2 49.9 51.6 49.7 Average 53.47 50.32 50.02 49.7 Average 51.89 49.86 Average 50.88 So take the following values as the Zo of the backplane: 0 loads --> 51 Ohm Zo 18 loads --> 35 Ohm Zo The Zo in terms of the number of loads will go as: Zo Z'o = --------------- sqrt (1+ Cd/Co) Or in our case: 51 Ohm Z'o = --------------- sqrt (1+ #_of_loads / 16.02 ) So in the operating systems things should look like: L1 FW TRM & AONM Crate 13 Loads 37.89 L1 FW TDM & FOM Crate 14 Loads 37.26 L1 FW TRM & SM Crate 9 Loads 40.81 Expose Group Scaler Crate 16 Loads 36.07 Per Bunch Scaler Crate 10 to 20 Loads 40.02 to 34.01 Gated SM & L2 FW Crate 12 Loads 38.56 Test Framework Top Crate 10 Loads 40.02 Test Framework Middle Crate 8 Loads 41.65 Test Framework Bottom Crate 10 Loads 40.02 So the closest rational value is 39 Ohms |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| Xilinx Speed Tests Original: 19-JUL-1995 ------------------ Tests of Xilinx FPGA timing parameters using NeoCAD TRACE delay calculation software. Simple designs were drawn and implemented in XC4003 and/or XC4025 devices (speed grade -4). The following parameters were tested: 1. Global clock net minimum delay, maximum delay, max skew on a single global clock net, max skew between different global clock nets. I found that the delay was relatively intolerant of loading on a single clock net (heavy loads added about 0.5 ns to the delay). XC4003 XC4025 ------ ------ minimum delay: 6.0 ns 12.6 ns maximum delay: 6.5 ns 15.1 ns max skew - 1 net: xxxxxx 1.0 ns max skew - 2 nets: xxxxxx 2.5 ns 2. Minimum wait necessary between clocking an input IOB and clocking a subsequent CLB (range depending on routing). This is for a low fan-out net. For the next 3 tests, "good" routing places the components near each other so routing is a single length local line, "bad" routing places them in opposing corners so that up to 3 long lines may be required. I did not explore pathological cases like trying to circumnavigate the chip via single length local lines. Note that in high density designs we may end up traversing some fraction of a quadrant on local lines if we run out of long lines. XC4003 XC4025 ------ ------ "good" routing: 11.7 ns 11.6 ns "bad" routing: 15.7 ns 32.0 ns 3. Minimum wait necessary between clocking a CLB and clocking a subsequent CLB (range depending on routing). This is for a low fan-out net. XC4003 XC4025 ------ ------ "good" routing: 7.8 ns 11.5 ns "bad" routing: 12.6 ns 29.1 ns 4. Minimum wait necessary between clocking a CLB and clocking a subsequent output IOB. This is for a low fan-out net. XC4003 XC4025 ------ ------ "good" routing: 10.4 ns 13.7 ns "bad" routing: 14.9 ns 30.0 ns 5. Typical skew on a single "normal" (i.e. longline/local lines) net (maximum is open-ended and good layout/routing must be observed on high-fan-out lines) 5.0 ns for XC4025 Summary: Here I will use numbers regardless of whether they came from XC4003 or XC4025. The goal is to estimate the MAXIMUM acceptable SKEW between two separate input clocks. The minimum delay that we could ever have between two separate clocked actions is 7.8 ns (from test 3 above). Noting (from test 1 above) that two separate clocks can have as much as 2.5 ns skew, we can't have less than 10.3 ns between separate input clocks. Now add in the max skew on a "normal" route (5 ns from test 5). This skew is added to account for either (a) a single output driving multiple inputs, or (b) skew between parallel paths. This gives a minimum inter-clock spacing of 15.3 ns (which is similar to one 53 MHz tick of 18.8 ns). So 53 MHz is fast enough and I also expect (hope) that it's granular enough. OK, but what about skew? i.e. how much "slop" would we want to add to our fastest path? Let's say 20%, or 3 ns between signals (1.5 ns jitter on a single signal). Note that this still allows us to conceivably do things on adjacent ticks, although I doubt that we will. Let's also compare "bad" routing vs. 2 ticks. 2 ticks = 37.66 ns. In the worst case seen in the tests, clocked actions required a minimum 32.0 ns separation. Adding 2.5 ns clock-to-clock skew and 5.0 ns net-to-net skew, we are at 39.5 ns. So even without backplane timing signal skew, the worst test case would not fly in 2 clock cycles, but would instead require 3 clock cycles (56.4 ns). Finally, let's think about the entire path, broken into 2 components: (1) source -> paddlecard -> backplane -> multiple cards skew at source assumed to be negligible, as well as skew through paddlecard. Dan will tell us about skew in backplane between signals, and also between cards. (2) card input -> translator -> "MTG corner" -> distribution to FPGA's -> use on FPGA's Again assume that skew introduced at the card input and through the translators is negligible. How good do we imagine we can make the MTG corner (assuming that there is an MTG corner)? We can probably keep the skew down to a reasonable level (<2 ns) by using registered outputs and carefully watching the routing withing the FPGA. The "array" FPGA's are distributed in an area approximately 10" x 10". Let's say that this costs us another 1 ns in skew from the closest to the furthest FPGA. This is important if we are feeding the same or related timing signals to separated FPGA's. The skews within a single FPGA are described above. -\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/- Voltage Drop Across Schottky Diodes Clamping Vcc to Vdd Rev. 25-JUN-1998 Schottky Doides have been added to the power supply system to clamp the Vcc supply to the Vcc supply. The Vcc supply is thus held at least as positive as one Schottky drop below Vdd. Note that in normal operation Vdd is turned on first and then after it is running the other three supplies (Vcc, Vtt, Vee) come on automatically. If at anytime during normal operation the Vdd supply should fail then the other three supplies are automatically turned off. During normal shutdown (via the DC control switch) all three supplies are normally stop at the same time. If the supplies are stopped with the AC power switch then I expect that Vdd will stay up longest just because it has the least load and all of the energy stored in its supply input section is available for making Vdd. Measurement of the Supply outputs and the Vcc to Vdd Clamp Test 1 ------ Both the 200 Amp Schottky and the 20x 1 Amp Schottkies are used in the Clamp Cards: TOM + VI and 3x AONM and 1x TRM Measurements in Volts Only Vdd Supply Running All 4 Supplies Running ------------------------- ------------------------- Pow Pan TP TOM TP Pow Pan TP TOM TP ---------- ---------- ---------- ---------- Vcc +3.008 +2.997 +5.051 +5.036 Vdd +3.353 +3.347 +3.353 +3.355 Vtt +0.015 +0.011 -2.045 -2.013 Vee +0.110 +0.105 -4.546 -4.521 Test 2 ------ Only the 200 Amp Schottky is used in the Clamp Cards: TOM + VI and 3x AONM and 1x TRM Measurements in Volts Only Vdd Supply Running All 4 Supplies Running ------------------------- ------------------------- Pow Pan TP TOM TP Pow Pan TP TOM TP ---------- ---------- ---------- ---------- Vcc +2.986 +2.974 +5.051 +5.036 Vdd +3.353 +3.348 +3.353 +3.355 Vtt +0.015 +0.011 -2.045 -2.013 Vee +0.110 +0.106 -4.546 -4.521 ------------------------- 367 mV 374 mV Clamp Drop Vcc to Vdd ------------------------- ------------------------- TOM Gnd measures TOM Gnd measures +4.6 mV wrt Pow Pan Gnd -2.2 mV wrt Pow Pan Gnd ------------------------- ------------------------- Test 3 ------ Only the 200 Amp Schottky is used in the Clamp Cards: TOM + VI only Measurements in Volts Only Vdd Supply Running All 4 Supplies Running ------------------------- ------------------------- Pow Pan TP TOM TP Pow Pan TP TOM TP ---------- ---------- ---------- ---------- Vcc +3.037 +3.030 +5.050 +5.037 Vdd +3.353 +3.352 +3.353 +3.352 Vtt +0.000 +0.002 -2.048 -2.040 Vee +0.002 +0.001 -4.549 -4.547 ------------------------- 316 mV 322 mV Clamp Drop Vcc to Vdd ------------------------- ------------------------- TOM Gnd measures TOM Gnd measures +1.2 mV wrt Pow Pan Gnd +1.2 mV wrt Pow Pan Gnd ------------------------- ------------------------- Now just do a linear extrapolation to see what the voltage drop across the Vcc to Vdd Clamp will be with 20 THE-Cards plugged in. This gives: 571 mV measured at the Power Pan 582 mV measured on TOM Well this looks quite big but let's remember that linear extrapolation is not correct because the current through the Schottky diode is actually growing exponentially compared to the voltage drop across the diode. -\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/- VME Interface Notes Rev. 22-OCT-1998 ------------------- There are 3 different arrangements of FPGA types used on THE Cards: "l-to-l": 4013L FPGA used for VME Interface, and 4013L FPGA's used for MSA/BSF also. This is all FM's, and all AONM's except S/N 6 and 7 (the FOM++ cards). "l-to-xl": 4013L FPGA used for VME Interface, and 4028XL FPGA's used for MSA/BSF. This is all TRM's, and all SM's except S/N 64 and 65. "xl-to-xl": 4013XL FPGA used for VME Interface, and 4028XL (or 4013XL) FPGA's used for MSA/BSF. This is all TDM's, 2 AONM's (S/N 6 and 7), and 2 SM's (S/N 64 and 65). In principle, these 3 arrangements can all be serviced by one design. Since the 4013XL FPGA is not bitstream compatible with the 4013L, a separate bitstream is required for the "xl-to-xl" application. In practice, it's more complicated, and there are actually FOUR bitstreams in use in the Framework. They all come from a common schematic, but have slightly different .UCF files. Here I will refer to them by their Version/Revision Numbers as stored on becane. ver4/rev1: 4013L FPGA/17256L PROM. This was the initial VME FPGA design. It has a pinout error--the Config_Error_PAD* pin is in the wrong location, and thus this signal is not visible in the CSR. No new PROM's of this type are being made, but we are not actively removing this PROM from any cards. It may be found on the FM and some TRM's and SM's. It does not reliably work with AONM's (see below) and thus no AONM's should use this part. These PROMs have a horizontal line etched under the part number, near the pin 1 designator. ver4/rev6: 4013L FPGA/17256L PROM. This PROM fixes the Config_Error_PAD* problem above, and appears to work fine in "l-to-xl" applications. It does not work reliably with AONM's (see below) and thus no AONM's should use this part. These PROMs have an "X" etched in the upper right corner. The design is stored on the PC as "vme_4_6.exo" with checksum 00665DAD. ver4/rev12: 4013L FPGA/17256L PROM. The above 2 PROM's have exhibited problems when used on AONM's. Occasionally, MSA FPGA 13 will fail to configure. The solution is to enable the FAST slew mode on the OCB_Write_Strobe_PAD* signal, and add a 33 ohm series termination resistor. This involves moving the OCB_Write_Strobe_PAD* signal from P165 to P174, to simplify adding the resistor. This part is only on the AONM's, which require modification (the series resistor) to work with this design. These PROMs have an "X" etched in the lower left hand corner. The design is stored on the PC as "vme_4_12.exo" with checksum 006771C3. ver4/rev5: 4013XL FPGA/1701L PROM. This PROM is used for all cards with a 4013XL VME FPGA. It also has FAST slew enabled for the OCB_Write_Strobe_PAD* signal, but no series resistor is necessary. These PROMs have an "X" etched in the upper right hand corner (just like the ver4/rev6 PROMs, but they are 1701L devices not 17256L devices). The design is stored on the PC as "vme_4_5.exo" with checksum 01E0C794. The basic problem is that the Write_Strobe* signal should have had a series termination resistor on it from day 1. These workarounds have been tested repeatedly, and with varying VDD voltages, and appear to be sound. -\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/- Additional P1_TS Notes Rev. 30-SEP-1998 ---------------------- 1. The HQ Timing Signals distributed on THE Card now run 36.6 ns (two 53 MHz Accelerator_Clock phases) behind the P1 Timing Signals distributed on the P1 Backplane. This is due to the introduction of a pipeline in the P1-to-HQ Timing Signal processing in the BSF FPGA: ----- ----- P1_TS --- |D Q|----(processing)---|D Q|--- HQ_TS | | | | --|> | --|> | | ----- | ----- | | 53 MHz -+------------------------ The purpose of this pipeline is to minimize skew between the HQ Timing signals (both on a single card and across multiple cards). Without the pipeline, the P1-to-HQ TS delay is NOT well-controlled, so we don't accurately know when a particular FPGA will, for example, latch its MSA Inputs. When programming the Carmen Master Clock, this 2 phase delay must obviously be accounted for, so that the HQ Timing Signals are aligned correctly to the timing environment of the rest of the experiment. This is most crucial at the interfaces of the FW to e.g. the SCL, L1 Subsystems, etc. Also, note that since the 53 MHz clock is now used to latch the other 15 P1 Timing Signals, the phase relationship between the 53 MHz and the other P1 Timing Signals must be controlled. The Carmen Master Clock allows fine control over this parameter, while the Big Ben allows only very crude control. Note also that the various P1 Timing Signals can be skewed with respect to each other by as much as 9 ns in the worst case (data book numbers, not measured numbers), which must be taken into account when "phasing" the 53 MHz clock. 2. The Tick_Clock and TRM_Clock P1 Timing Signals are now active for a single 18.8 ns phase of the Accelerator_Clock, rather than 3 phases as previously. Recall that the BSF has two internal clock "domains." Some flip-flops in the BSF are updated at the BX_Clock (either Tick_ or TRM_Clock) rate, while others are updated at the Accelerator_Clock rate. When passing data between these clock domains, race conditions can exist between the two clock edges. Moreover, the phase relationships between the clocks can vary due to differences in individual FPGA's or to the vagaries of the Xilinx place-and-route tool. Finally, the Xilinx place-and-route tools do NOT take these race conditions into account. This is a very inconvenient (and risky) way to operate. The standard way to resolve this problem is to have just one clock (in this case, the Accelerator_Clock) run ALL flip-flops in the BSF, and use the BX_Clock as the CLOCK ENABLE to the flip-flops which should update at the BX_Clock rate. This eliminates race conditions between the two clock domains, and allows the Xilinx place-and-route software to analyze and guarantee data-passing between the domains. For this to work, the clock enable (BX_Clock) must be valid for only one phase of the master clock (Accelerator_Clock), as it now is.