Magic Bus Transceiver MBT ------------------------------ MSU L2 Group 28-JAN- 1998 Draft 0.31 The purpose of the MBT card is to collect all of the support function that are needed in the Alpha Processor based Level 2 Trigger Crates onto one card. The MBT card has 4 main functional blocks. These are: 1. Receiver for Cypress Hot Link Input connections. This is used in the Global Crate to receive Preprocessor data and in some of the Preprocessors Crates to receive input data from their Level 1 Triggers. There are 7 or 8 channels (to be determined) of Cypress Hot Link input. Data received on these links is buffered and then sent via the M-Bus to the Alpha processor boards. 2. Transmitter for Cypress Hot Link connections. This is used in the Preprocessors to send their data to the Global Crate. There will be 2 channels of Cypress Hot Link output. Data sent out on these links comes from the M-Bus, passes through a FIFO and is then sent out. 3. Provide 128 lines of Parallel Digital I/O. This can be used in both the Preprocessor and Global Crates for realtime scaler and state control. It is also used in the Global Crate to send the L2 Decisions back to the L2 Hardware Framework. 4. Receive the Serial Command Link. This is used in both the Preprocessor and Global crates. The L1 Accept, L2 Decision, and SCL Initialize information is received and made available. The MBT is a single-width 9U 400mm VME-64 for Physics card that has Magic Bus (aka M-Bus) on its 235 pin hard metric P3 connector. Any register that can be written to on the card, may also be read back. Receiver Section: There will be 7 or 8 Hot Link input channels that can all operate in parallel. The decision to have either 7 or 8 channels will be make once it is understood how many input channels the various systems will need. This decision will also take into consideration any addition difficulty in having more than 8 channels. Note that there is effectively one additional input channel which is dedicated to transporting SCL L1 Accept information. The Hot Link receiver chip on each of these 7 or 8 channels feeds its data to a separate FIFO. Each FIFO is big enough to store at least 16 events worth of data from its input link. Note that the actual number of events that a given FIFO can store is determined only by the size of the events. The proposed size for each FIFO is 64 KBytes. This is sized by: 400 tracks/event x 8 Bytes/track x 16 events = 51,200 Bytes. The FIFO's are not hard wired with fixed event boundaries. The FIFO's maintain the boundaries between events by using outband Cypress Hot Link control symbols. The event boundaries need to be maintained so that later on when the MBT is placing an event onto the Magic Bus it will be able to send just the data from a single event to the Alpha Processor cards and then stop. The receiver sections on a number of MBT cards will need to be able to work together to minimize the amount of communication required with the Administrator processor during actual event processing. This control and coordination of the receiver section is performed by a Receiver Control Block on each of the MBT cards. Some of the functions of this control block are outlined below. 1. From M-Bus (or VME?) you can tell it to drain all its event buffers and "initialize". 2. From the M-Bus (or VME?) you can load Test Data into the event FIFO on each of the receiver links. 3. From the M-Bus (or VME?) you can monitor how many events are currently in the FIFO on each of the receiver links. The 5 bit value that indicates the number of events currently held in a given FIFO is also available as a realtime electrical signal on pins of a monitor/test jack. 4. From M-Bus (or VME?) you can tell it the mask of which of its 7 or 8 Link Sources it should find data from and place on the M-Bus as it puts each event onto the M-Bus. 5. From M-Bus (or VME?) you can tell it what low order address bits to use for the Event ID in its cycle of 16 events, and what low order address bits to use as the ID of each Link (i.e. the source ID). This supplies the upper 28 bits of the M-Bus address. The lowest 4 bits will increment in a loop 0 to 15 and are used to indicate the Event ID. For broadcast operation to the Alpha processor boards the upper 24 bits are all zero, bits (7:4) indicate the Link ID (Source ID), and the lowest 4 bits are the 0 to 15 loop of "event ID". A(31:8) fixed at zero for "broadcast" operation to the Alpha's A(7:4) Source ID, aka Link ID A(3:0) Event ID, increments in a loop of 16 with each event 6. From M-Bus you can send it a "go" signal to the Pilot MBT card. The MBT Card then does the following: a. It waits until input event buffers on both the Pilot and any Assistant MBT cards have at least one event worth of data from each of the Cypress Input Links that they have been told to use in point 4 above then, b. It acquires mastership of the M-Bus and places all the data from the lowest number Link that it has been told to use onto the M-Bus. It sets the proper low order address bits to indicate the event number in the cycle of 16 and the Link ID. c. At the end of sending out this events worth of data from this lowest number Link it allows arbitration for M-Bus mastership. d. It either keeps M-Bus mastership or else when it regains it, then it sends out this events worth of data from the next to the lowest number Link that it has been told to use. While sending out this data it sets the proper low order address bits to indicate this Link's ID (the event ID in the cycle of 16 has not changed). e. It continues in order through all the Links that it has been told to use (both the links on the Pilot and on any Assistant MBT cards), sending out their data for this event, and allowing mastership arbitration between the data from each Link until it has sent out this events data from all of the Links that it has been told to use. Then, f. When it has sent out this events worth of data from all of the Links that are in use during this run, it then sends the "end of event" signal to all the M-Bus cards that are receiving this data and it: Removes this event from its input event buffer on each Link. Changes its low order address bits to indicate the next event in its cycle of 16. Hardware signals between MBT cards allow the Administrator to send the "Go" to just the Pilot MBT card. This "go" signal is sent over the M-Bus because that bus is know not to be in use at the time this signal needs to be sent. These MBT to MBT hardware lines then allow the various MBT cards to coordinate sending out the event. This involves waiting until all of the active links on all of the MBT cards contain at least one event before starting to send it out and allowing first one MBT card and then the next to send its data. Between the 8 bit wide Cypress Receiver and the 128 bit wide M-Bus master driver there is a 16 to 1 funnel. It is not practical to have 8 bit wide FIFO and then all of the funnel between the FIFO output and the 128 bit wide M-Bus driver. (for 50 nsec M-Bus write cycles this would require 50nsec / 16 = 3 nsec FIFO parts). It is also not practical to have all of the funnel in front of the FIFO (i.e. 16 parallel chips in the FIFO of each of 8 Receiver channels). So the design will probably end up with part of the funnel in front of the FIFO and the rest of it in front of the M-Bus driver. The FIFO chips will actually need to be 9 bits wide to accommodate the out band Cypress control symbol that is used to indicate event boundaries. An additional feature of the funnel M-Bus driver section is its ability to pad out the final M-Bus transfer of data from each link to an integer number times 16 bytes of data per event. This padding will most likely occur at the output side of the FIFOs. The systems that send data to the Receiver Section do not have to send and integer number times 16 bytes of data per event. 128 Lines of Parallel Digital I/O: This is a simple parallel digital I/O function for sending the answer to the L2 Hardware Framework. The Parallel Digital I/O Section can also be used to get state monitoring control lines into a logic analyzer or scalers. This is a simple register based I/O that transfers data with the M-Bus. During normal event processing operation there should be worst case latency of about 2 usec in getting access to the M-Bus. Because it is easy to implement and is the only simple way to get some control lines into the system, this digital I/O should be bidirectional. On each MBT card this 128 line Parallel Digital I/O will probably be implemented as two 64 bit registers. Using registers of length 64 bits fits the proposed maximum data width of M-Bus programmed I/O transfers that are mastered by an Alpha processor card. In addition to these two 64 bit RW registers there will also be one small control register (probably 8 bit) that is associated with the 128 Line Parallel I/O section. Any write to this control register causes a "strobe" signal to be sent out that implies that the 128 bits contain new valid data. Writing a 1 to a specific bit in this control register forces all 128 I/O lines to zero and sends out a "strobe". This gives the Global Stage Administrator a fast way to send "L2 Reject" back to the L2 Hardware Framework, i.e. what it needs to do 90% of the time. Output Link Section: Each MBT card has two Hot Link output channels. This section contains FIFO's for the data being sent out from the crate and the Cypress Hot Link Transmitter for each channel. Data that is to be sent out is received on the MBT from the M-Bus. Separation between events is maintained both in the FIFO and over the link by Hot Link control symbols. There is no flow control "back channel", i,e, the Cypress Hot-link can always send at full speed, (i.e. the receiver of this data guarantees that it can always accept the data). The FIFO in this output section is just used to hold data in the case that the M-Bus can supply data faster than the Hot Link can send it out. The two output channels are completely separate. Programmable features include, ability to clear all data from both channels, ability to read how many events each FIFO is holding. In addition this 5 bit value that indicates the number of events in each FIFO should be visible in realtime as electrical signals on pins of a test monitor connector. For each Output Channel there are two registers that are used to load data into the FIFO that feeds the output channel. 1. One register is 64 bits wide (to match the Alpha Processor card's M-Bus master). This register is used to send all of the data for the event to the output channel. This register is mapped to a block of M-Bus address (perhaps 64k long) so that if a DMA engine is available for sending the output data it could target this register as a series of addresses. 2. There is a second register at a different address that when written to places the Hot Link control symbol into the FIFO that is used to mark event boundaries. The address of this second register is immediately above the block of address that map to the data register described above. Thus if a DMA engine is used to send a preprocessor's output data to the L2 Global Stage it could be setup to terminate the DMA cycle at this address. Serial Command Link Receiver Section: The MBT will use a standard SCL mezzanine card to receive: the L1 accept information, the L2 Decision information, and the SCL Initialize messages from the SCL data stream. All of the Alpha processors in a crate need to receive the L1 Accept information. It is the L1 Accept information that contains the event's "Geographic Section L1 Event Number" and the L1 Qualifier Mask that indicates whether or not a given event is a "normal event" or a Monitor event or a Mark and Force Pass event. For each event, the Alpha Processors need to receive this L1 Accept information in a way that associates it with the rest of the input data that these processors receive for the event. Thus this L1 Accept information is FIFO'd and then broadcast to the Alpha Processors by circuitry that is identical to that used by the Receiver Section channels. This L1 Accept information is the 8th (or 9th) Receiver Section channel. The only difference is that this 8th (or 9th) Receiver Section channel has its data source right on the MBT card instead of receiving it data over a Cypress Hot Link. Note that between filtering the L1 Accept information out of the SCL data stream and putting it into a Receiver Section FIFO channel, addition bytes need to be added to present the L1 Accept information in the same Software Format as is used by all the other preprocessor channels that feed the L2 Global Stage. The handling of the L2 Decision information is different because only the Administrator Processor needs access to this data. The L2 Decision information is placed into a FIFO so that it may be read (probably over M-Bus) when the Administrator processor has time and can get bus access to read it. The Administrator access the output side of this FIFO by a register that maps to a single fixed address. If the FIFO is empty then when the Administrator reads this register it will read a data value that indicates to it the FIFO's empty condition (i.e. in band control information). The MBT card also receives the SCL Initialize messages from the SCL data stream. This SCL Initialize signal is made visible in a register to probably both the M-Bus and the VME Bus. The SCL Initialize signal can also be enabled to generate a VME interrupt. Thus the Administrator processor can learn of an SCL Initialize Request either by reading a register on the MBT card or by being interrupted by the MBT card. The generation of the SCL Initialize interrupt can be enabled or disabled by a control register. This control register would have the normal global interrupt enable/disable bit and enable/disable bits for all of the possible sources of interrupt. Currently the SCL Initialize is the only interrupt source on the MBT card. The SCL does include a back channel to the SCL Hub End, i.e. to the Trigger Framework. There are 5 pieces of information transported on the SCL back channel: L1 and L2 Busy, L1 and L2 Error, and SCL Initialize Acknowledge. There is no plan to detect L1 Busy in either the L2 Global Stage or the L2 Cal Preprocessor. This is because there is no way for these systems to actually know in realtime how many events are in their various buffers and there is nothing that they can do to stop an overflow once it would begin to occur. However these L2 system can indicate L1 Busy as they are required to during the SCL Initialization process. All of the L2 components will need to be able to detect L2 Busy, and L1 and L2 errors (aka L1 and L2 sync errors). The Administrator can communicate these error or busy or SCL Initialize Acknowledge signals on the SCL back channel by writing to a register that appears on M-Bus. This register just directly connects to the SCL back channel. Continuing Issues: ------------------ 1. Which control registers should appear in VME space and which registers should appear in M-Bus space. Perhaps all of the control registers can easily appear in both. For all of the registers that are only used for setup there could be an on card bus that has a slave to both VME and to M-Bus. All of the event rate registers clearly have to have an M-Bus connection. The only Master bus interface on the MBT is the M-Bus Master interface that services the Receiver Section. 2. How to set the base address of each MBT for both VME and for M-Bus. Many crates will have more than one MBT card so that have be be designed to have a configurable base address. DIP switch, jumpers, FPGA configuration ? 3. Number of Receiver Section Input Channels 7 or 8. 4. Number of MBT cards to build. An estimate is 25 to 30. 5. Cost estimate of the MBT. An estimate is 2.5 to 3.0 $k not counting design labor. 6. Testing. How much can be tested without an ALPHA processor card? Should there be a way to read, via VME, the data that the Receiver Section blows at high speed onto M-Bus? Can we loop channels from the Cypress Output Section to the Receiver Section to make a stand alone test of the Cypress parts? Are different MBT card test features needed to verify the operation of an isolated MBT card as compared to testing/exercising a full level 2 processor crate running close to its real software? 7. Speed and form of Cypress Inputs and Outputs: 160 vs 320 Mb/s, copper vs. fiber. Selection of fiber might reduce the number of input channels per card. 8. An arrangement which might be convenient is an internal data bus for all M-Bus I/O 128 b wide, and a narrower path to control registers and slow I/O such as test data downloading. 9. Digital 128-bit I/O and Cypress Outputs are both accessed by Alpha card programmed M-Bus I/O. Programmed I/O may be provided by 128b or 64b M-Bus cycles. The MBT should allow selection of which mode is in use.