THE-CARD Control Path Interface <-> L3 RM Specification ----------------------------------------------------------- Original Rev. 29-JAN-2002 Current Rev. 30-JAN-2002 Introduction ------------ An interface is needed to make available to the L3 RM the signals on the Control Path which connects the L3 RM with the Trigger Framework. The electrical specification of the Control Path is available at: www.pa.msu.edu/hep/d0/ftp/l1/framework/hardware/cabling/ control_path_l3_tofrom_l1.txt The signals on the Control Path are: from the Trigger Framework to L3 RM: a 128 bit mask of triggers that fired at L1 AND were passed by L2 along with the associated 16 bit L3 Transfer Number and a strobe. from L3 RM to the Trigger Framework: 128 individual L1 Specific Trigger Disable lines and one Global Disable all L1 Triggers line. The Trigger Mask, L3 Transfer Number, and Trigger Disable lines are available from the Trigger Framework as differential ECL signals. A THE-Card is capable of sinking 128 ECL inputs and driving 64 ECL outputs. An VME interface to these lines can be provided by the (FPGA) logic on THE-Card. A proposal for this VME interface is described below. A Single Board Computer called the RM provides the interface between VME and the L3 DAQ software. This is the same type of Single Board Computer (SBC) that is used throughout the L3 DAQ System. Known entities/constraints -------------------------- * Interface to ECL is via multiple cards. In addition to its 128 bit Main Array ECL Input and its 64 bit Main Array ECL Outputs, THE-Card has a 17 bit bi-directional ECL connector. Signals from this connector on one of the two THE_Cards will be used for: The Global Disable all L1 Triggers line. Incrementing the FIFO Read Pointer for the input FIFO's on both of THE_Cards. Forcing the FIFO Read and Write Pointers for the input FIFO's on both of THE Cards to reset. * THE-Cards support an A24 D16 slave only VME Interface. This is implemented on a ERNI 5 row P1 connector. ALL VME P1 signals are bussed and supported on this P1 connector. Details about THE_Card's VME interface are in: www.pa.msu.edu/hep/d0/ftp/l1/framework/hardware/the_card/ the_card_vme_interface.txt * The VME Interface on THE-Card was designed for "slow controls" and does not support block mode. * An adaptor card (called TOM) is available for plugging standard 6U P1/P2 VME cards into the backplane. A 6U card plugged into TOM is connected to all the P1 VME signals, is feed power and ground on its normal 3 row P1 and P2 connectors and TOM provides access to all of its P2 signal pins. * A FIFO buffer memory is necessary to store the trigger masks and L3 Transfer Numbers until they are read by the SBC. - QUESTIONS how deep of a buffer can be made? I need to look at this and at the overflow protection. * ECL output line updates can be asynchronous THE-Card ECL outputs that carry the Trigger Disable information form the SBC to the Trigger Framework will update asynchronously. When making the L1 decision for a given beam crossing the Trigger Framework just always obeys the current state of these 129 lines. For a given L1 Specific Trigger to fire both its Individual Disable signal and the Global Disable signal must not be asserted. Control Path Interface General Description ------------------------------------------- * Two THE-Cards are used to implement the Control Path Interface. * These two cards appear in separate pre-defined blocks of VME A24 address space. See the THE-Card VME Interface referenced above for details. * One of these cards will be in slot #4 and will occupy the block of A24 addresses $050000 : $067fffe The second card will be in slot #5 and occupy the block of A24 addresses $068000 : $07ffffe Locating the cards in a different block of address is possible but will require careful thought about TCC software implications. * All VME registers on THE-Card are 16 bits wide. Registers are either Read Only or Read Write (there are never any write only registers). Some bit on some Read Only registers will not convey any information from THE-Card. These bits will always ready zero. Some bits in some Read Write registers will have no effect on the THE-Card but these bits will still read back what is written to them. * Controlling the 64 bits of Main Array Output signal on a THE-Card. These 64 bits of output are used to send the Individual Disables to the Trigger Framework. Because of the function that THE-Card was originally designed for, the printed circuit board traces will only allow these 64 output signals to be controlled 4 at a time. That's right - 16 VME writes are required to setup these 64 bits. These 16 registers are not contiguous in address space. The low order 4 bits in these registers will be used to control the output signals. * Reading the contents of the Input FIFO location currently being pointed to by the FIFO Read Pointer. The 128 bits of FIFO'd input data are read 16 bits at a time. 8 VME reads are required for the SBC to ingest these 128 bits. These registers can be re-read and you will continue to get the same data until you explicitly tell the Control Path Interface to increment its FIFO Read Pointer. These 8 VME registers are not contiguous in address space. * Control Path Interface Management Functions One of the two THE-Cards supplies the Management Functions. These functions are: A R/W register with just one functional bit. This controls the Global Disable of all L1 Triggers signal on the Control Path. A read only register with just one functional bit. This bit is asserted if the FIFO Write Pointer is ahead of the FIFO Read Pointer. This indicates the status of all the input FIFO's on both of THE-Cards. FIFO Write Pointer ahead of the Read Pointer indicates that the FIFO contains at least one entry that the RM has not read. A R/W register with just one functional bit. Writing a "1" and then a "0" causes the FIFO Read Pointer to increment (for all input FIFO's on both THE-Cards) A R/W register with just one functional bit. Writing a "1" and then a "0" resets to zero both the FIFO Read Pointer and the FIFO Write Pointer (for all input FIFO's on both THE-Cards) Two read only registers for reading the current value of the Input FIFO Write Pointer and the Read Pointer. A R/W register that enables or absolutely blocks any interrupt form coming from this THE-Card. A R/W register that enables the generation of an interrupt if the FIFO Write Pointer increments. This bit is automatically cleared when THE-Card begins asserting interrupt. A read only register that contains a bit that indicates whether or not this THE-Card is currently asserting interrupt. Background ---------- FIFO's The only efficient way to implement FIFO's in the FPGA is to make the out of RAM and a Write Address Counter and a Read Address Counter. Using a FIFO implement this way requires a little extra thinking but typically works out OK. It is the only way to get anything more than a very small FIFO on the FPGA that is on the type of THE-Card that I have available for the Control Path Interface. I will get a good estimate of the size of FIFO (depth) that I can fit into this design and let you know what it is. I have not thought at all about what to do if the input FIFO overflows. I very much like your idea of detecting in hardware logic that the FIFO is about to overflow and automatically setting the Global Disable to all L1 Triggers based on that. But note that all the Disables that you have access to on the Control Path are disables to the L1 Trigger. Yes, you instantly stop the L1 Triggers but there could already be up to 16 events in the pipe all of which could receive L2 Accepts and thus need to load their data into this FIFO. So this would require asserting Global Disable when there are still 16 slots available in this FIFO. Not a problem if the FIFO has lots of slots. Clearly this is the cleanest solution, i.e. make the system so that it can not overflow the input FIFO. Probably also a good idea to set a status register bit to indicate if it ever has overflowed. Poll vs Interrupt The intent of the Management Functions described above is to allow the reading of the input FIFO to be accomplished with software that either polls or is interrupt driven. In something like this I typically like to start with a toy program that runs on polling and just displays the data on the screen. The intent is just to demonstrate that the data is there and looks rational. Then move to the real running system that is interrupt driven. Interrupts The intent of the interrupt system it to wake up the SBC when a new event comes into the FIFO (but not to wake it up a second time until the SBC has re-enabled this interrupt source. In the Interrupt Service Routine it would read the data for all of the events that have collected there since the last time it was read. That is between the first event coming in (and the interrupt being set) and the ISR running, additional events may come into the FIFO and once called, the ISR will read all events that it find there. FPGA Configuration At power up the logic needs to be put into the FPGA's on the two THE-Cards that implement the Control Path Interface. Putting logic into the FPGA's is called configuring them and this is done over VME. During the configuration process the only VME cycles that may address these two THE-Cards are those VME cycles that are doing the configuring. During configuration, extraneous VME cycles that address these cards will cause the FPGA configuration process to fail. At power up the Trigger Control Computer (TCC) is used to configure all the FPGA's in the Trigger Framework. Once configured the FPGA's retain their logic until the power goes off. We have had no problem running the Trigger Framework for months without needing to re-configure its FPGA's. The intent is to have the TCC configure the FPGA's on the 2 THE-Cards that implement the Control Path Interface. TCC would connect to the crate that holds the Control Path Interface over a Fermilab Vertical Interconnect. This is the same way that TCC connects to the 8 crates in the Trigger Framework. TCC would not try to configure the FPGA's in the Control Path Interface unless explicitly told to do so. When TCC configures these FPGA's it can optionally load up any registers that are instantiated once the logic is in the FPGA's. So at power up TCC would take the VME bus, configure the FPGA's (requiring perhaps 30 seconds) and then release the bus. TCC would not try to take the bus again unless explicitly told to do so. Control Path Interface Crate Location For the next "N" months the Control Path Interface will be located in the bottom crate in rack M101. This crate is not needed until we have time to work on implementing the "L1 Cal Trig Quadrant Terms". It will be at least 6 months before we work on that part of L1 Cal Trig. At that time I will provide a new crate and power supply to located the Control Path Interface in a different locations. If desired, it could be moved to the 2nd floor L3 Trigger area. That is, the only electrical connections to this crate are: Control Path and Vertical Interconnect and these can both cross from the MCH grounding system to the fixed counting house. So in the long term the Control Path Interface Crate does not have to take space in the MCH.