CMX Clock Generation and Distribution ------------------------------------------ Original Rev. 20-Nov-2012 Current Rev. 30-Apr-2014 This file describes the generation and distribution of the clocks on the CMX circuit board. The main focus of this file is the LHC locked 40.08 MHz and 320.64 MHz clocks for the Logic and GTX Transceiver functions in the 3 FPGAs on the CMX card. In addition to these LHC locked clocks the CMX circuit board includes: GTX Quad reference clocks of 100.000 and 120.000 MHz to the BF and TP FPGAs for G-Link and possibly for S-Link operation of GTX Transceivers, a 20 MHz clock to the Xilinx System ACE, and a 4 MHz clock to the CAN-Bus microprocessor. An overall view of the clocks on the CMX card is shown in the following drawing in the circuit diagrams section of the MSU CMX web site: 26_clocks_overall_view.pdf The generation and distribution of the LHC locked 40.08 MHz and 320.64 MHz clocks is shown in the following 3 drawings: 11a_clock_generation_and_distribution_a.pdf 11b_clock_generation_and_distribution_b.pdf 11c_clock_generation_and_distribution_c.pdf The generation of the 100.000 MHz and 120.000 MHz crystal oscillator based clocks is shown in the drawing: 11d_clock_generation_and_distribution_d.pdf The distribution of reference clocks into the Base Function and Topological Processor GTX Quads is shown in the drawings: 27_gtx_transceivers_base_function.pdf 28_gtx_transceivers_topological.pdf Basic Design of the LHC Locked Clocks: -------------------------------------- - On the CMX circuit board the LHC reference for the LHC locked clocks comes from a TTCDec mezzanine. - The CMX card provides a TTCDec DSKW-1 locked 40.08 MHz Logic clock to all 3 FPGAs. - The CMX card provides a TTCDec DSKW-2 locked 40.08 MHz Logic clock to just the BF and TP FPGAs. - The 320.64 MHz clock that is generated on the CMX card is locked to the TTCDec DSKW-1 output. This 320.64 MHz clock is provided as both a Logic clock and as a GTX Reference clock to both the BF and TP FPGAs. - The CMX card uses a narrow band PLL-VCXOs to generate its clean LHC phase locked 40.08 and 320.64 MHz clocks. - The PECL outputs from the three narrow band PLL-VCXOs are fanned out using low noise PECL clock distribution chips. - Signals from these clock distribution chips are routed as either LVDS level signals to differential Global Clock inputs on the FPGAs and as AC coupled LVPECL level signals to differential GTX Reference clock inputs on the GTX Transceiver Quads. Virtex-6 Clock Input Signal Levels: ----------------------------------- - The signal level requirement for the GTX Transceiver reference clocks is clearly given in the chapter 5 of the GTX User Guide and in the Virtex-6 DC and Switching Data Sheet. The GTX reference clock input is clearly aimed at AC coupled Differential LVPECL signal levels. In the GTH User Guide AC coupled Differential LVPECL reference clocks are specifically recommended. - The Global Clock inputs for the Base Function and Topological Processor logic clocks can be any signal level that is supported by the Virtex-6 Select I/O inputs. It does not appear that the Select I/O inputs will directly accept Differential LVPECL with internal termination. The cleanest solution appears to be to use LVDS_25 signal levels to send the logic clocks to the Virtex-6 global clock inputs (and to the Board Support Spartan 3A FPGA). In all cases internal 100 Ohm internal termination can be used with these LVDS_25 signals. - The 40.08 MHz and 320.64 MHz clock fanout is all done with Differential LVPECL chips and the logic clocks to the Virtex-6 Global Clock inputs may be scaled to LVDS_25 levels with simple resistor networks at the sending end and internally differential terminated at the receiving FPGA end. - The relevant signal specifications are: Virtex-6 LVDS_25 Input: V Input Diff: 100 mV min, 350 mV Typ, 600 mV max V Input CM: 0.30 V min, 1.25 V Typ, 2.20 V max MC100LVEP111 Differential LVPECL Output: Vout HI: 2.155 V min, 2.280 V Typ, 2.405 V max Vout LOW: 1.355 V min, 1.530 V Typ, 1.700 V max Scaling Differential LVPECL to LVDS_25: --------------------------------------- - Scaling both the Direct and Complement LVPECL outputs at the sending end by makes both the differential and common mode voltages in the correct range for reception by the Xilinx LVDS_25 receivers. - The resistor network to scale the Differential LVPECL signals can be 47 Ohm series then 110 Ohm to ground. - At the Virtex input: The expected differential voltage will be 317 mV. The expected common mode voltage will be 1.349 V. - This will result in about 16.9 mA of emitter current to the HI side of the Diff LVPECL driver and 7.6 mA to the LOW side. These emitter currents are right in the normal operating range for these parts. - The internal Xilinx 100 Ohm differential LVDS terminator can be used with these LVDS level clock lines. - Average heat in the 47 Ohm resistor is 8.1 mW Average heat in the 110 Ohm resistor is 16.7 mW So 0603 size resistors may be used in this application. Base Function and TP GTX Transceiver Reference Clock: ------------------------------------------------------ - The GTX transceiver PLLs can multiply up their reference clock by a factor in the range from 4 to 25. - The GTX transceiver PLL output is 1/2 the transceivers line bit rate, e.g. 3.3 GHz PLL output gives 6.6 Gb/s data rate. - The GTX transceiver PLL's reference must be in the range from 62.5 MHz to 650 MHz, 50/50 duty cycle, 200 ps edge speed, AC coupled, 800 mV typical differential amplitude with range of from 210 to 2000 mV differential, 100 Ohm input resistance. Note the somewhat special way that Xilinx defines differential amplitude for the Transceiver Reference Clocks. - The GTX transceivers can operate over the data rate range from 480 Mb/s up to 6.6 Gb/s. What internal PLL ratios do they use ? At the low end: 480 Mb/s --> 240 MHz PLL output 240 MHz divided by 62.5 MHz Ref --> 3.84 ratio At the Hi end: 6.6 Gb/s --> 3.3 GHz PLL output 3.3 GHz divided by 62.5 MHz Ref --> 52.80 ratio 3.3 GHz divided by 650 MHz Ref --> 5.08 ratio - GTX PLL Control Values: The reference clock is divided by "M" before going into the PLL M = 1 or 2 The feedback divider is N1 x N2 where N1 = 4 or 5 where N2 = 2 or 4 or 5 and where N1 x N2 must not equal 4 or 5 There is a final divider "D" between the PLL output and the transceiver where D is either 1 or 2 or 4 Recall that the line rate is 2x the PLL output frequency. GTX User Guide 2v6 pg 117 shows typical reference clocks in the range of about 200 to 325 MHz for line rates of about 4 to 6 Gb/s. We know that for a 6.4 Gb/s line data rate we want/need M = 1 and D = 1. - The actual line rate that we want is 6.4 Gb/s --> 3.2 GHz PLL output which is 80 times the "40 MHz" LHC frequency. 80 is 2 x 2 x 2 x 2 x 5 - Example setups from 40 MHz LHC to 6.4 Gb/s: 40 MHz times 8 external gives 320 MHz reference 320 MHz reference with N1=5 and N2=2 give 3.2 GHz G-Link GTX Transceiver Reference Clock: ---------------------------------------- - Reference Clock for the "Slow Optical" RIO and DAQ Outputs On the CMX card the G-Link for the ROI and DAQ outputs will be implemented with GTX transceivers. This GTX implementation of the G-Link will need a 120.000 MHz non LHC locked reference clock. This is provided by a separate 120.000 MHz LVPECL crystal oscillator and PECL fanout chip. This is CMX Crystal Oscillator #1 reference designator U371. This 120.000 MHz PECL clock signal is AC coupled to the GTX Quad 118 Reference Input on the Base Function FPGA. Board Support FPGA Clock: ------------------------- - The Spartan 3A Board Support FPGA will receive just one clock - the LHC locked 40.08 MHz clock referenced to the TTCDec DSKW-1 output. - The Spartan 3A part can receive this clock as an LVDS signal with internal 100 Ohm termination. - The LVDS input specifications for the Spartan 3A are: V Input Differential: 100 mV min, 350 mV Typ, 600 mV max V Input Common Mode: 0.30 V min, 1.25 V Typ, 2.35 V max - This is really a Xilinx LVDS_25 signal going to an I/O Bank with VCCO of 2.5 Volts. - The same resistor network as described above will be used to connect the LVPECL fanout signal to this Spartan 3A LVDS_25 global clock input. System-ACE Clock: ----------------- - The System-Ace typically runs from a 20 MHz clock. - The 20 MHz clock for the System-ACE needs to be the same as the clock that is running whatever device is connected to the System-ACE Microprocessor port, i.e. its MPU port. - On the CMX card it is the Board Support FPGA that provides the connection to the ACE MPU port. The BSPT FPGA runs from the 40.08 MHz clock and it will provide a 20.04 MHz clock to the System-ACE. In this way the cycles in the MPU port of the System-ACE will be synchronous with activities in the BSPT FPGA. - The clock input to the System-ACE is referenced to its VCCL power bus. On the CMX card the System-ACE VCCL bus will be supplied with 2.5 Volts. Thus the pin #93 clock input to the System-ACE must be a 2.5V CMOS level clock signal. - The System-ACE clock trace will be back terminated at the BSPT FPGA. The source of the ACE clock is pin V14 on the BSPT FPGA. CAN-Bus Microprocessor Clock: ----------------------------- - The clock to the MB90F594 CAN-Bus microprocessor can be either a 4 MHz quartz crystal with associated capacitors connected to the X0 and X1 pins or it can be a clock signal connected to only the X0 pin with pin X1 floating. - Table 3 DC Characteristics page 39 hints that this device has strange input voltage levels, i.e. Vish min of 0.8 x Vcc and Vils max of 0.5 x Vcc. The diagram on page 41 hints that the X0 clock input runs between 0.2 x Vcc and 0.8 x Vcc. Other diagrams e.g. pg 45 indicate TTL type levels of 0.8V and 2.4V. - CMX will provide a real 5V CMOS 4 MHz clock to the X0 pin (pin No. 82) of the MB90F594 CAN-Bus microprocessor and it will float its X1 pin (pin No. 83). This is the setup shown on page 13 of the MB90F594 data sheet. LHC Locked Clock Frequency: --------------------------- - The LHC RF frequency is about: 400.8 MHz - The LHC BX frequency is really: At 450 GeV Proton Injection: 40.0788790 MHz At 7 TeV Proton Physics: 40.0789658 MHz At 450 GeV Ion Injection: 40.0784139 MHz At 7 TeV Ion Physics: 40.0789639 MHz - Round to the nearest 1 Hz: At 450 GeV Proton Injection: 40.078,879 MHz At 7 TeV Proton Physics: 40.078,966 MHz At 450 GeV Ion Injection: 40.078,414 MHz At 7 TeV Ion Physics: 40.078,964 MHz - For all modes the biggest pulls at 40.08 MHz are +- 276 Hz - For all modes the fastest slew rates at 40.08 MHz are 22 Hz/sec The ramp up/down slews are much slower < 1 Hz/sec - The center for Protons is: 40.078,922 MHz +- 43 Hz - The center for Ions is: 40.078,689 MHz +- 275 Hz - The center for both is: 40.078,690 Mhz +- 276 Hz - Order symmetric pull center frequency of: 40.078 700 MHz This will require a pull of +266 Hz - 286 Hz This is a pull of about 7.2 ppm - Order symmetric pull center frequency of: 320.629 600 MHz This is a pull of about 7.2 ppm Components Used: ---------------- - 40.0787 MHz input, 40.0787 MHz output VCXO PLL 3.3V CMOS or AC PECL reference input signal level Differential LVPECL output signal level 50 Hz PLL loop bandwidth Custom 40.0787 MHz PLL Clock 6 x 5 mm 10 pin SMD Connor-Winfield Part No. SFX-524G-CRN1 - 40.0787 MHz input, 320.6296 MHz output VCXO PLL 3.3V CMOS or AC PECL reference input signal level Differential LVPECL output signal level 200 Hz PLL loop bandwidth Custom 320.6296 MHz PLL Clock 6 x 5 mm 10 pin SMD Connor-Winfield Part No. SFX-524G-CRN2 - Clock 1 to 10 Fanout OnSemi MC100LVEP111MNG Differential LVPECL inputs and outputs 3.3 Volt power 5mm x 5mm LLP-32 QFN-32 - Clock 1 to 2 Fanout OnSemi NB6L611MNG Differential LVPECL inputs and outputs 3.3 Volt power 3mm x 3mm 6-VFQFN 16-QFN - 100.000 MHz PECL Crystal Oscillator 7 x 5 mm 6 pin 3.3 Volt SMD Connor-Win Part Number: P143-100.0M - 120.000 MHz LVPECL Crystal Oscillator 7 x 5 mm 6 pin 3.3 Volt SMD Connor-Win Part Number: PGF123-120.0M - 4 MHz 5 Volt CMOS Crystal Oscillator 5.0mm x 3.2mm 4 pin package SMD ECS Inc. Part No. ECS-3961-040-AU-TR Monitoring of the PLL Based Clock Signals: ------------------------------------------ - The narrow band PLL-VCXO components that are used to generate the LHC locked 40.08 and 320.64 MHz clock signals provide an output signal that indicates whether or not they are locked to their reference input. - This "Lock Detect" signal from the three PLLs on the CMX card has been routed so that these signal can be inputs to the BSPT FPGA. I assume that these signals will be bits in a status register which will indicate whether or not the PLLs are locked onto the reference signals from the TTCDec. - Logic can be placed in the BSPT to detect even isolated momentary drop out of a Lock Detect signal. Layout of the Clock Generator Section of the CMX Card: ------------------------------------------------------ - The TTCDec reference clock enters the CMX card at its lower right-hand corner. This input is buffered and sent to the TTCDec input. - The TTCDec mezzanine is located immediately to the West of its reference input buffer chip. The standard pair of SamTec QSH connectors is provided for mounting the TTCDec mezzanine card. - All ground connections to the TTCDec have been connected and locally bypassed 3.3V power is provided to the TTCDec via its QSH connectors. - The CLK_40_DES_2_PLL_2 output and either the CLK_40_DES_1 or the CLK_40_DES_1_PLL_2 output may be selected as the DSKW-2 and DSKW-1 references that are used by the CMX card. - The Clock Generator section of the CMX card is immediately West of the TTCDec mezzanine card. - All components within the CMX Clock Generator run from an isolated filtered 3.3V power plane that services just the Clock Generator section of the CMX card. This power is distributed on a separate "fill" within the CMX pcb.