***************************** * * * D-Zero Run II Upgrade * * Level 1 Trigger Framework * * and * * Level 2 Trigger Framework * * * * Board Support Function * * * * FPGA Description * * * ***************************** Original: 7-MAY-1998 Latest: 26-SEP-2000 Introduction ------------ This document concentrates on the architecture and implementation of the Board Support Function (BSF) FPGA. Some version of the BSF FPGA will be used on all cards in the L1 Framework. The basic design of the BSF will be the same on all cards, but the BSF for some cards (e.g. the Trigger Decision Module) will in addition perform some special functions. The BSF FPGA has the following input and output connections: 1. On-Card Bus 2. P1 Timing Signals 3. Board Global I/O Signals 4. P5 Global I/O Signals 5. High-Quality Timing Signals Like the Main Signal Array FPGAs, the BSF FPGA is accessible and configurable via VME, has a Chip_Status signal which it sends to the VME Interface FPGA, and appears on the JTAG scan chain. The BSF FPGA processes the 16 Timing Signals into the 16 High-Quality Timing Signals and the Capture HSRO Data and Capture Monitor Data signals. In addition, it performs the board-level management of the High-Speed Readout (HSRO) protocol. The BSF also does the processing associated with the 17 Board Global I/O Signals and the 17 P5 Global I/O Signals. On certain cards the BSF will perform more specialized functions as well. Operation--General Comments --------------------------- The BSF FPGA is composed of the following elements: 1. On-Card Bus Interface 2. Timing Signal Processing 3. High-Speed Readout, consisting of A. HSRO Management B. HSRO Header and Trailer C. HSRO Data 4. Board Global I/O 5. P5 I/O 6. Specialized Function Processing 7. LED Output Each element is described below. 1. On-Card Bus Interface This is the standard On-Card Bus Interface from the MSU FPGA Common Block Library. Its operation is described elsewhere and will not be repeated here. The only caveat is that the High-Quality Timing Signal input pins are not used since the High-Quality Timing Signals are produced by the BSF FPGA itself. 2. Timing Signal Processing On the P1 Backplane, 16 differential ECL timing signals (designated P1_TS(15:0)) are bussed to all slots. On each instance of THE Card, these signals are converted to single-ended TTL and routed to the BSF FPGA (additionally, the TTL version of P1_TS(0) is also routed to the VME Interface FPGA). The BSF FPGA processes these signals to produce 16 High Quality Timing Signals (designated HQ_TS(15:0)). These High Quality Timing Signals are divided into 4 groups of 4 signals each (e.g. HQ_TS(3:0), HQ_TS(7:4)). Each group of 4 High Quality Timing Signals is bussed to the 4 FPGAs in a single column of the Main Signal Processing Array (e.g. FPGAs #1-#4). The processing which takes place between the P1 Timing Signals and the High Quality Timing Signals is programmable via TCC, and is limited to: - selecting 1 of 4 P1 Timing Signals to drive each High Quality Timing Signal - forcing each HQ Timing Signal high or low via TCC Note that there are 4 separate sets of 4 HQ Timing Signals. The structure of the BSF requires that the 4 sets of HQ Timing Signals are identical, that is: - HQ_TS(0,4,8,12) are the same - HQ(TS(1,5,9,13) are the same, etc. Specifically, HQ_TS copy one of P1_TS (or are forced H/L) ----- ----- 0, 4, 8, 12 1, 2, 9, 10 1, 5, 9, 13 3, 4, 11, 12 2, 6, 10, 14 5, 6, 13, 14 3, 7, 11, 15 7, 0, 15, 8 The logic used to process the P1 Timing Signal fits into a single CLB. Additionally, the processing of P1_TS into HQ_TS is pipelined into two stages (using the 53 MHz Accelerator_Clock to run the pipeline), and the HQ_TS are latched in the IOB. This minimizes skew between the HQ Timing Signals distributed on the card, at the expense of introducing a fixed 36.6 ns delay between the P1 Timing Signals on the backplane and the HQ Timing Signals on the card. This fixed delay must be compensated for when programming the Carmen Master Clock. Note that the Carmen Master Clock allows the 53 MHz Accelerator Clock to be shifted with respect to the other clock signals. We will take advantage of this feature to compensate for skew between the P1 Timing Signals introduced in the cabling from Carmen, the TOM, and THE Card. Additionally, the Timing Signal Processing block provides timing signals used internally in the BSF FPGA. These internal timing signals are appropriately delayed so that they are synchronous with the HQ Timing Signals provided to the MSA FPGA's. 3. High-Speed Readout Many instances of THE Card use an optical readout path, designated High-Speed Readout, to provide data in the L1 Trigger Framework Data Block. This path uses an optical cable, transporting data with the HP G-link protocol at up to ~850 Mbits/sec, to move data from each card to a VRB card. This transport is described in detail in files in the directory: MSUTRGROOT_II:[HSRO] 3.A High-Speed Readout Management Three P1 Timing Signals, all sourced from the Framework Helper FPGA, are used to manage this readout: 1. Capture_HSRO_Data This signal passes through the BSF FPGA to each of the MSA FPGA's, and is used (in conjunction with a Tick rate clock) to capture data in HSRO Data Capture Registers, for readout. It may also be used internally in the BSF FPGA for the same function, and is additionally used in the BSF FPGA for status/error checking. 2. Capture_Monitor_Data This signal passes through the BSF FPGA to each of the MSA FPGA's, and is used (in conjunction with a Tick rate clock) to capture data in Monitor Data Capture Registers, for readout. It may also be used internally in the BSF FPGA for the same function. 3. Transport_HSRO_Data The BSF FPGA uses this signal to initiate a HSRO cycle, as described below. It is not provided directly to the MSA FPGA's. The HSRO cycle is managed by a state engine (implemented in VHDL) which receives control input from the Helper Function card, and also from the HSRO Daisy Chain on THE Card. The state diagram is as follows: .----------------------------------------. v | Quiescent | | | | Transport_HSRO_Data = 1 | v | Header_Word_0 | | | v | Header_Word_1 | | | | .------------. | v v | HSRO_DCE_In = 1 | MSA_Data ----------' | | | v | BSF_Data_Word_0 | | | v | BSF_Data_Word_1 | | | v | BSF_Data_Word_2 | | | v | BSF_Data_Word_3 | | | v | Trailer_Word_0 | | | v | Trailer_Word_1 | | | v | End_of_Event_State --------------------------' The detailed functionality of each state is well-described in the VHDL source and is not repeated here. The Header and Trailer data provided by the BSF FPGA is described below. A large number of essentially static signals are driven by the BSF FPGA to the HSROCB. They are: - Flag - ED - CAV* - LoopEn - FlagSel - M20Sel - Div(1:0) - Rst* - FF These signals are controlled by a VME-accessible register, see the Programming Interface section for details. 3.B HSRO Header and Trailer Besides managing the HSRO cycle, the BSF also provides Header and Trailer words to the HSROCB. Currently, two words of Header and two words of Trailer are defined: Header Word 0: in D(15:0) 0's in D(19:16) Trailer Word 0: in D(15:0) 0's in D(19:16) Trailer Word 1: Event Number in D(7:0) Status Signals in D(15:8) 0 in D(16) End Event Flag ('1') in D(17,19) 0's in D(18) The Event Number increments with every Capture_HSRO_Data signal from the Helper Functions card. It can be reset either either via the Scaler Reset Helper Function signal or via a TCC write to the BSF Control Status Register (see the Programming Interface section for details). The Status Signals are as follows: Trailer Word 1 Data Bit Signal -------- ------ 8 Data Captured 9 Transporting Data 10 Unexpected Second Capture Without Previous Transport 11 Unexpected Transport Without Capture 12 Unexpected Capture During Transport 13 Unexpected Transport During Transport 14 unallocated 15 unallocated The expected pattern of these bits in the HSRO Trailer Word is: bits 8 and 9 HIGH, all others LOW. These status bits are cleared at the end of every event transfer. A slightly different version of Trailer Word 1 is readable via a Monitor Data read. Note that the status bits available via this Monitor Data read are NOT automatically cleared at the end of every event transfer but only cleared by TCC. See the programming interface section for details. 3.C HSRO Data The BSF provides 4 words of HSRO Data (immediately prior to providing 2 words of Trailer). These 4 words contain the following information: P5_IO(16:0) [16 bits in one word, 1 bit plus expansion room in a second word] BG_IO(16:0) [16 bits in one word, 1 bit plus expansion room in a second word] These 4 words are passed through a normal Beam Crossing History Shift Register, and are also available for Monitor Data readout. This information is available for the Triggered Tick only. 4. Board Global I/O The BSF FPGA has 16 Board Global I/O signals. On all cards except the TDM, these 16 signals simply go to an array of vias near the BSF FPGA for possible future expansion. On the TDM, BG I/O(15:0) are individually routed to each MSA FPGA. BG I/O(0) is routed to MSA FPGA 1, etc. BG I/O(16) on the TDM is simply routed to a via as on all other cards. The processing of these signals is described below under Specialized Function Processing. 5. P5 I/O The P5 connector on THE Card carries 17 differential ECL "global I/O" signals, designated P5_IO(16:0). These signals pass through ECL/TTL transceivers and are routed to the BSF FPGA. These signals are arranged in 5 groups: 4 groups of 4 signals each (P5_IO(3:0), P5_IO(7:4), P5_IO(11:8), and P5_IO(15:12)), and a 5th "group" consisting only of P5_IO(16). This 5th "group" could be used either as a clock/strobe signal or a 17th data signal. The BSF FPGA selects the direction and controls the output enable separately for each group of these signals, via VME-accessible registers. Note that, although these signals are called "I/O" type signals, on-the-fly switching of signal direction is not supported, as the termination/pull-down resistors must be changed when switching between ECL input and ECL output functionality. The processing of these signals is described below under Specialized Function Processing. 6. Specialized Function Processing On certain cards, the BSF may have specialized individual function processing. All of this specialized functionality is a superset of the standard BSF processing. Ideally, a single design could contain all of the specialized functionality for all cards, plus the basic processing described here. 6.A TDM Specialized Functionality Board_Global_IO(15:0) inputs must be ORed together, and driven out via P5_IO(16). This processing currently occurs on all cards all the time, and is not controlled by any special Mode bits. Note that P5_IO(16) must be enabled and set to output mode (see programming interface section below) and the appropriate termination resistors must be installed. 6.B FOM++ Specialized Functionality P5_IO(7:0) inputs are ORed together and driven out to a "spare" FPGA pin (P4, connected to via J26), which is then wired to MSA FPGA #1 and #5 (on P157, labelled RA8 VIA on PCB) P5_IO(8) input driven out to a "spare" FPGA pin (P238, connected to via J27), which is then wired to MSA FPGA #2 and #6 (on P157, labelled RA8 VIA on PCB) P5_IO(9) input driven out to s "spare" FPGA pin (P237, connected to via J28), which is then wired to MSA FPGA #3 and #7 (on P157, labelled RA8 VIA on PCB) P5_IO(10) input driven out to s "spare" FPGA pin (P236, connected to via J29), which is then wired to MSA FPGA #4 and #8 (on P157, labelled RA8 VIA on PCB) Note that this requires the addition of white wires to the FOM++ card. This processing currently occurs on all cards all the time and is not controlled by any special Mode bits. Note that P5_IO(10:0) must be enabled and set to input mode (see programming interface section below) and the appropriate termination resistors must be installed. 6.C Spec Trig Fired to Level 2 TRM Specialized Functionality The Spec Trig Fired to Level 2 TRM's require the L2 Capture Monitor Data signal (rather than the L1 Capture Monitor Data signal which is on P1_TS(10) in M123 Middle where these cards are located). There are no free P1 Timing Signals on that backplane. Therefore the L2 Capture Monitor Data signal enters these cards via P5_IO(0), and must be fed to the normal Capture Monitor Data output of the BSF. Also, this FPGA must receive a signal from the VME FPGA on BG_IO(0) which becomes active (HIGH) when any MSA FPGA FIFO is NOT empty. This involves wiring VME FPGA P187 (via J201) to BSF FPGA P87 (BG_IO(0)). The BSF FPGA will send this signal out via P5_IO(16) (see 6.A, TDM Specialized Functionality: the BSF always ORs BG_IO(15:0) together and sends the result out on P5_IO(16)...note that in this case BG_IO(15:1) are just passively pulled down in the FPGA IOB's and left unconnected). This functionality is controlled by a bit in the Timing Signal Selection Control Register (see programming interface below). Note that P5_IO(0) must be enabled and set to input mode (also see programming interface below) and the appropriate termination resistors must be installed. P5_IO(16) must be enabled and set to output mode, again with the appropriate termiantion resistors. 6.D FM D_Latch Specialized Functionality The FM D_Latch BSF must distribute a Capture Monitor Data signal to the MSA FPGA's, but does not have a dedicated Capture Monitor Data output. Instead, this signal is distributed via HQ_TS(2). The standard HQ_TS(2) logic does not support this, nor can it be modified to do so given the existing timing constraints. In addition, there are two possible sources of HQ_TS(2) in this FPGA. In Rack M122 Bottom, the L2 CMD signal is on P1_TS(10), while in Rack M123 Bottom, the L2 CMD signal comes in on P5_IO(0). Note that this is exactly the functionality described above. The only difference is exactly which pins are used to carry the output signal. This is easily rectified in the User Constraint File, which already must be different between the two designs. Technical note: TWO outputs are used to provide HQ_TS(2) on the FM: one output for FPGA's 1 and 4, the other for FPGA's 13 and 16. Therefore a dummy Capture Monitor Data output is provided on the non-FM BSF, routed to an unused output. This allows both designs to be driven by the same schematic. Again, note that P5_IO(0) must be enabled and set to input mode, and the appropriate termination resistors must be installed. 6.E L2 Answer and L2 Auxiliary Data TRM Specialized Functionality The BSF on these TRM's must perform the FIFO NOT EMPTY processing described above in 6.C, L1 ST Fired to L2 TRM Specialized Functionality. 7. LED Output Each BSF FPGA can drive an LED on its cards front panel. This LED is controlled by a VME register accessible to TCC. Programming Interface --------------------- The Board Support Function FPGA has the following VME-visible registers: Reg Register Addr Access Contents ---- ------ -------- 0 R/W Chip Control Status Register (LSW) 1 R/W Chip Control Status Register (MSW) 2 R/W Enable Scaler Timing Signal Reset 3 R/W Force Scaler Reset 4 R/W P5 Global I/O Dir/OE Control 6:5 R/W Board Global(16:0) Direction Control 8 R/W G-Link Static Control 12 R Board Global I/O Readback LSW 13 R Board Global I/O Readback MSW 14 R P5 I/O Readback LSW 15 R P5 I/O Readback MSW 16 R/W Timing Signal Selection Control 32 R/W HSRO Header Word 0 LSByte TCC written HSRO data 33 R Monitor Copy of HSRO Trailer Word 1 34 R/W HSRO Header Word 1 TCC written HSRO data 35 R/W HSRO Trailer Word 0 TCC written HSRO data 36 R/W Tick History Shift Register Control Reg 40 R Monitor Copy of HSRO Data Word 0 [BG_IO(15:0)] 41 R Monitor Copy of HSRO Data Word 1 [BG_IO(16)] 42 R Monitor Copy of HSRO Data Word 2 [P5_IO(15:0)] 43 R Monitor Copy of HSRO Data Word 3 [P5_IO(16)] The bit allocation in each of these registers is given below. Chip Control Status Register LSW: -------------------------------- Bit Access Contents --- ------ -------- 0 R/W Chip Interrupt Enable ('1': interrupts enabled) 1 R/W Chip Interrupt Request ('1': request interrupt) 2 R Chip Status Signal (Interrupt Request Status) (logical AND of Chip Interrupt Enable and Chip Interrupt Request--this is the logical INVERSE of the Chip_Status*(x) signal from this chip to the VME Interface FPGA) 15:3 --- unallocated Chip Control Status Register MSW: -------------------------------- Bit Access Contents --- ------ -------- 0 R/W BSF LED ('1': turn LED on) 15:1 R/W not allocated Enable Scaler Timing Signal Reset Register: ------------------------------------------ Bit Access Contents --- ------ -------- 0 R/W Enable Timing Signal Reset for Event Number Scaler 1 R/W Enable Timing Signal Reset for HSRO Status signals 15:2 R/W not allocated Force Scaler Reset Register: --------------------------- Bit Access Contents --- ------ -------- 0 R/W Force Reset for Event Number Scaler 1 R/W Force Reset for HSRO Status Signals 15:2 R/W not allocated P5 Global I/O Direction and Enable Control: ------------------------------------------ Bit Access Contents --- ------ -------- 0 R/W P5_IO(3:0) Direction Control ('1': these are OUTPUT signals, i.e. convertor runs TTL -> ECL '0': these are INPUT signals, i.e. convertor runs ECL -> TTL Note: this register controls both the TTL/ECL transceiver chips and the BSF FPGA IOB's connected to those chips. This prevents the possibility of driver conflict on these signals) 1 R/W P5_IO(7:4) Direction Control 2 R/W P5_IO(11:8) Direction Control 3 R/W P5_IO(15:12) Direction Control 4 R/W P5_IO(16) Direction Control 7:5 R/W Unused 8 R/W P5_IO(3:0) Output Enable ('1': Output (as specified by Direction) enabled '0': Both outputs disabled (TTL tri-stated, ECL INV and NINV both cut-off)) 9 R/W P5_IO(7:4) Output Enable 10 R/W P5_IO(11:8) Output Enable 11 R/W P5_IO(15:12) Output Enable 12 R/W P5_IO(16) Output Enable 15:13 R/W Unused Board Global I/O Direction Control Register LSW: ----------------------------------------------- Bit Access Contents --- ------ -------- 15:0 R/W BG I/O 15:0 Direction Control ('1': these are OUTPUT signals, i.e. BSF drives these signals '0': these are INPUT signals, i.e. BSF receives these signals) Board Global I/O Direction Control Register MSW: ----------------------------------------------- Bit Access Contents --- ------ -------- 0 R/W BG I/O 16 Direction Control ('1': this is an OUTPUT signal, i.e. BSF drives this signal '0': this is an INPUT signal, i.e. BSF receives this signal) 15:1 R/W not allocated G-Link Control: -------------- Bit Access Contents --- ------ -------- 0 R/W RST* ('0': Reset G-link transmitter '1': Normal operation) 1 R/W FF ('0': Send FF0 as fill-frame '1': Send FF1 as fill-frame) 2 R/W ED ('0': Disable sending data '1': Enable sending data 3 R/W Flag (if FlagSel is '1', the data on the Flag pin is sent to the G-link receiver) 5:4 R/W Div(1:0): (Selects data transfer rate, i.e. Frame Clock rate. For 20 user data bit mode [M20Sel = '1']: '00': Frame Clock = 29.2 to 62.5 MHz '01': Frame Clock = 14.6 to 37.5 MHz '10': Frame Clock = 7.3 to 18.8 MHz '11': Frame Clock = 6.3 to 9.4 MHz Recall our Frame Clock is 53 MHz) 6 R/W M20Sel ('0': 16 user data bit mode '1': 20 user data bit mode) 7 R/W FlagSel ('0': Flag bit in serial frame alternates 0/1 '1': Flag bit in serial frame = Flag input) 8 R/W LoopEn ('0': Normal operation: serial data emerges on DOUT/DOUT* '1': Loopback: serial data emerges on LOUT/LOUT*) 9 R/W CAV* ('0': Send this frame as Control word '1': Normal operation: send this frame as either Data or Fill word depending on ED and DAV* Note: CAV* takes precedence over DAV*) 15:10 R/W Unused Note: These signals must be programmed by TCC to allow the HSROCB to function. To force the G-link to re-synchronize, and then enable running, TCC should do the following: 1. Reset the G-link chip: write 0x0242 2. Initiate re-synch: write 0x0243 3. Wait ~5 milliseconds to guarantee re-synch 4. Enable normal operation: write 0x0247 Board Global I/O Readback Register LSW -------------------------------------- Bit Access Contents --- ------ -------- 15:0 R Board Global I/O 15:0 Readback Board Global I/O Readback Register MSW -------------------------------------- Bit Access Contents --- ------ -------- 0 R Board Global I/O 16 Readback 15:1 R unallocated P5 I/O Readback Register LSW ---------------------------- Bit Access Contents --- ------ -------- 15:0 R P5 I/O 15:0 Readback P5 I/O Readback Register MSW ---------------------------- Bit Access Contents --- ------ -------- 0 R P5 I/O 16 Readback 15:1 R unallocated Timing Signal Selection Control: ------------------------------- Bit Access Contents --- ------ -------- 1:0 R/W HQ Set 0 P1 TS Selection ('0 0': select P1_TS 1 '0 1': select P1_TS 2 '1 0': select P1_TS 9 '1 1': select P1_TS 10 2 R/W HQ Set 0 Force Bit ('0': use the selected input timing signal '1': force the HQ Timing Signal high or low based on the the state of bit 1 above: bit 1 '0': force output LOW bit 1 '1': force output HIGH) 3 R/W Unallocated 5:4 R/W HQ Set 1 P1 TS Selection ('0 0': select P1_TS 3 '0 1': select P1_TS 4 '1 0': select P1_TS 11 '1 1': select P1_TS 12 6 R/W HQ Set 1 P1 Force Bit (as bit 2 above) 7 R/W Unallocated 9:8 R/W HQ Set 2 P1 TS Selection ('0 0': select P1_TS 5 '0 1': select P1_TS 6 '1 0': select P1_TS 13 '1 1': select P1_TS 14 10 R/W HQ Set 2 P1 Force Bit (as bit 2 above) 11 R/W Unallocated 13:12 R/W HQ Set 3 P1 TS Selection ('0 0': select P1_TS 7 '0 1': select P1_TS 0 '1 0': select P1_TS 15 '1 1': select P1_TS 8 14 R/W HQ Set 3 P1 Force Bit (as bit 2 above) 15 R/W Use P5_IO(0) as Capture Monitor Data ('0': P1_TS(10) is Cap Monit Data '1': P5_IO(0) is Cap Monit Data only program to '1' in L2 STF TRM's and FM D_Latch in M123 Bottom) HSRO Header Word 0 LSByte ------------------------- Bit Access Contents --- ------ -------- 7:0 R/W TCC written data for HSRO Header Word 0 LSByte Fixed data written by TCC and readout in every event. 15:8 R/W Unallocated HSRO Header Word 1 ------------------ Bit Access Contents --- ------ -------- 15:0 R/W TCC written data for HSRO Header Word 1 Fixed data written by TCC and readout in every event. HSRO Trailer Word 0 ------------------- Bit Access Contents --- ------ -------- 15:0 R/W TCC written data for HSRO Trailer Word 0 Fixed data written by TCC and readout in every event. Monitor Data: HSRO Trailer Word 1 --------------------------------- Bit Access Contents --- ------ -------- 7:0 R HSRO Event Number 8 R Data Captured 9 R Transporting Data 10 R Unexpected Second Capture Without Previous Transport 11 R Unexpected Transport Without Capture 12 R Unexpected Capture During Transport 13 R Unexpected Transport During Transport 14 R unallocated (reads back 0) 15 R unallocated (reads back 0) NOTE: The 8 status signal in the MSByte of this word are latched, i.e. once they go high they must be manually cleared by TCC using register 2 or 3. This is different from the version of these bits which are sent to the VRB via HSRO, which are cleared at the end of every transport cycle. Tick History Shift Register Control Reg --------------------------------------- Bit Access Contents --- ------ -------- 2:0 R/W Depth in stage 1 of BXHSR 15:3 R/W unallocated Monitor Copy of HSRO Data Word 0 [BG_IO(15:0)] ---------------------------------------------- Bit Access Contents --- ------ -------- 15:0 R Board Global I/O (15:0) Monitor Copy of HSRO Data Word 1 [BG_IO(16)] ---------------------------------------------- Bit Access Contents --- ------ -------- 0 R Board Global I/O (16) 15:1 R unallocated Monitor Copy of HSRO Data Word 2 [P5_IO(15:0)] ---------------------------------------------- Bit Access Contents --- ------ -------- 15:0 R P5 Global I/O (15:0) Monitor Copy of HSRO Data Word 3 [P5_IO(16)] ---------------------------------------------- Bit Access Contents --- ------ -------- 0 R P5 Global I/O (16) 15:1 R unallocated