Run II L1 Framework Readout vs. Card Type --------------------------------------------- Original: 19-JUN-1997 Latest: 6-AUG-1999 The goal here is to understand what is read out in all of our cards, and to start making the Run II version of D0 Note 967. We need to describe: - what is read out from each FPGA - what order the FPGA's are read out on each card - what each VRB does with the data from each card - what order the VRB's are read out Definitions ----------- Tick Clock: once per 132 ns clock. The rising edge for Tick "n" is used by the TRM to clock its FIFO to present AOIT (etc.) information for Tick "n" to the AONM. At this same Tick "n" rising edge, the TDM latches AOF (etc.) information for Tick "n-1". Notes for reading the tables below ---------------------------------- "Data Description" and "Bits per FPGA" are self-explanitory. An "X" in the "M" column means that the data is available for MONITOR (TCC access via VME) readout. An "X" in the "H" column means that the data is available for HSRO (Data Block) readout. The "TC" column describes which Tick's data is shifted into the Beam Crossing History Shift Register at the rising edge of Tick Clock for Tick "n" (as defined above). In cases where a single card is used in multiple locations, each with different input or output data types, the tables will only refer to generic "INPUT" or "OUTPUT" data. Additional notes ---------------- There is a strong preference towards 32-bit quantities to facilitate the interpretation of the data block and control the effect of any byte swapping that we may have to face one day or another. This means that a 2- or 4-byte item is preferred over a 3-byte item read from a particular FPGA. In the case where the item being readout is already smaller than 16 bits (e.g. And-Or Term readout with 3 or 4 bits per And-Or Term) the advantage of reading 32 bits per FPGA is less important. We may thus decide to only read one 16-bit word per FPGA from some cards *IF* this would significantly reduce the L1 FW readout size, or help reduce the number of words readout from a particular card. Any card which has all 16 MSA FPGA's read out via HSRO is limited to 32 bits (2 * 16-bit words) of data from EACH MSA FPGA. This limit is self-imposed in order to limit the readout time from each card and prevent one type of card from becoming the bottleneck of L1 FW Readout. The overall readout time of the L1 FW during which the L1FW has to set its Front-End Busy signal will be dictated by the worst case of the card with the most 16 bit words to readout. We know we have to read 32 * 16 bit words from some cards (e.g. the TDM) and we should keep the maximum for any card at about the same length. The Support FPGA readout (which isn't well understood at the moment) will also add 2 * or 4 * 16 bit words to the readout from each card. The Total number of 16-bit words read from each card should always be an even number. TRM --- TRM's are used in the following places: 1. Receive And-Or Input Terms from L1 Trigger Subsystems (4) 2. Receive L1 Busy Signals from Geographic Sections (2) 3. Receive Individual Spec Trig Disables from ??? (4) 4. Receive Global Spec Trig Disables from ??? (1) 5. Buffer L1 Specific Trigger Fired Mask for L2 FW input (2) Recall each TRM FPGA has: 4 INPUTS (AOIT, L1 Busy, etc) 4 OUTPUTS (these are the same info as INPUTS, but post-FIFO) Data Description Bits per FPGA M H TC ---------------- -------------------------- - - -- OUTPUT states 4 states x 4 ticks = 16 X X n-1 4 ticks = Triggered Crossing + Prev + 2 Next OUTPUT scalers 1 4 scalers x 32 bits = 128 X n-1 Synch Error state ? 2 1 state x 1 tick = 1 X X n-1 Synch Error scaler ? 2 1 scaler x 32? bits = 32? X n-1 Synch Error time stamp ? 2 1 value x 32? bits = 32? X n-1 FIFO Write Counter 1 value x 5 bits = 5 X X? ? FIFO Read Counter 1 value x 5 bits = 5 X X? ? Any information used for implementing error detection X ? Notes: 1 OUTPUT scalers used to monitor And-Or Input Term rates 2 Synch Error data not yet fully understood 16 bits of HSRO for AOIT info and FIFO information in second 16 bit word upper with some extra room is the L1 FW->L2 FW TRM a special case? AONM ---- AONM's are used in the following places: 1. Map Physics AOIT to Physics AOF (4) 2. Map Exposure Group AOIT to Exposure Group Enable (2) Recall each AONM FPGA has: 128 INPUTS (e.g. AOIT) 4 OUTPUTS (Physics AOF, EG, etc.) Standard AONM FPGA: Data Description Bits per FPGA M H TC ---------------- -------------------------- - - -- OUTPUT states ? 2 4 states x 4 ticks = 16 X X n-2 4 ticks = Triggered Crossing + Previous Crossing + 2 Next Crossing OUTPUT scalers ? 3 4 scalers x 32 bits = 128 X n-2 Miguel FPGAs in "unused" sites of EG AONM, FEBz AONM, and FOM++ Data Description Bits per FPGA M H TC ---------------- -------------------------- - - -- INPUT states ? 1 128 states x 4 tick = 512 X X n-2 4 ticks = Triggered Crossing + 3 Previous Crossings Notes: 1 Not strictly necessary to record INPUT states, but useful for checking TRM-to-AONM (etc.) cables. Recording additional Previous and Next Crossings provides additional history information about the beam and about the detector around the Triggered Crossing. This information couldn't be captured anywhere else and can be very valuable. Previous Crossing History might provide information about recent activity in the detector that might affect readout accuracy and Next Crossing History might flag events for which the system triggered too early on a low threshold when a high threshold would have been met on the next crossing. 2 Not necessary to record OUTPUT states, but again useful for checking AONM-to-TDM cables. Not particularly expensive in CLB's, and this function is needed in the (very similar) FOM FPGA. Could be Monitor-Only, but note that the FOM dictates that it should be HSRO. 3 Useful for monitoring individual PARTIAL And-Or Fired rates, but the main reason for including this is again to make AONM and FOM equivalent. Note that a few scalers are available elsewhere to monitor a few selected independent arbitrary (as opposed to partial) combinations of And-Or Terms for monitoring. There is only one 16 bit word of HSRO per AONM FPGA Each EG AONM will have - Two standard FPGA to produce the 8 Partial Exposure Group Signals sent to the TDM cards. These FPGA will readout one 16 bit word each - Two standard FPGA programmed to reproduce the 8 Partial Exposure Group Signals sent to the Per-Bunch Scalers for Luminosity calculations. These FPGAs will not readout anything. - Two standard FPGA programmed to reproduce the 8 Partial Exposure Group Signals sent to the Gated (not per-bunch) Scalers. These FPGAs will not readout anything. - A number like 2 or 4 FPGAs programmed with an arbitrary combination of And-Or Terms used in Rate Monitoring only to assist the Trigger Specialist or DAQ operator. These FPGAs will not readout anything. - Four Miguels maintaining FIFOs of 32 Input AOIT's each (total of 128 Input AOIT per EG AONM) for readout of Beam Crossing history information. Each FPGA will read out 8 * 16 bit words. We have arranged *which* extra ticks are readout for the And-Or Input Terms from the TRM and AONM cards so that their history information complement instead of repeat each other. Note that the Triggered Crossing Information is reapeated, and that is to verify proper cabling between the TRM and AONM. TDM --- TDM's are used in the following place: 1. Collect L1 information to produce Spec Trig Fired (8) Since TDM's are a single purpose card, the generic "INPUT" and "OUTPUT" terminology is not used. 1 And-Or Fired (AOIT 127:0) input 1 And-Or Fired (AOIT 255:128) input 8 Exposure Group (AOIT 127:0) inputs 8 Exposure Group (AOIT 255:128) inputs 8 Front-End Busy Disable inputs 2 Individual Spec Trig Disable inputs 4 Correlated Global Disable inputs 4 Uncorrelated Global Disable inputs 1 Specific Trigger Fired output Main Signal Array FPGA: Data Description Bits per FPGA M H TC ---------------- -------------------------- - - -- SpTrg Fired state 1 state x 1 tick = 1 X X n-2 SpTrg Exposed state 1 state x 2 ticks = 2 X X n-2 2 ticks = Triggered Crossing + Previous Crossing SpTrg Phy AOF state 1 state x 2 ticks = 2 X X n-2 2 ticks = Previous Crossing + Next Crossing SpTrg Phy PrtAOF state 2 states x 1 tick = 2 X X n-2 SpTrg ExpGrp Enb state 1 state x 2 ticks = 2 X X n-2 2 ticks = Previous Crossing + Next Crossing SpTrg ExpGrp PrtEnb state 2 states x 1 tick = 2 X X n-2 SpTrg DAQ Enb Decor state 1 state x 3 ticks = 3 X X n-2 3 ticks = Triggered Crossing + Previous Crossing + Next Crossing SpTrg DAQ Enb Corr state 1 state x 1 tick = 1 X X n-2 Itemized Decor Dsb states 9 states x 1 tick = 9 X X n-2 Itemized Corr Dsbl states 5 states x 1 tick = 5 X X n-2 ---- Total number of State bits = 29 bits Sp Trig Fired scaler 1 scaler x 32 bits = 32 X n-2 Sp Trig Phy AOF scaler 1 scaler x 32 bits = 32 X n-2 Sp Trig Exposed scaler 1 scaler x 32 bits = 32 X n-2 or 48 bits? ST DAQ Enable scaler 1 scaler x 32 bits = 32 X n-2 ST DAQ Enable Correlated 1 scaler x 32 bits = 32 X n-2 ST DAQ Enable De-Correlated 1 scaler x 32 bits = 32 X n-2 or 48 bits? ST COOR Disable scaler 1 scaler x 32 bits = 32 X n-2 ST Prescaler Veto scaler 1 scaler x 32 bits = 32 X n-2 ST Auto-Disbl Veto scaler 1 scaler x 32 bits = 32 X n-2 ---- Total number of Scalers = 9 scalers Notes: The Specific Trigger Exposed Scaler can provide an overall number at the end of a run of the live time fraction for each particular specific trigger. This scaler is typically running at close to the beam Crossing rate, and can thus rollover about every 10 minutes. To avoid asking TCC to keep track of an upper 32-bit of counter range, and avoid relying on the fact that TCC is always able to poll these counters, one can increase the size of this counter by 8 or 16 bits. 48 bits corresponds to three 16-bit VME read cycles. The Specific Trigger Enable De-Correlated contribution is used for luminosity computation. The count increment for this scaler is read at regular time interval (~1 mn) to correct the Correlated exposure group contribution. Note that it is NOT Sufficient to read this scaler at the beginning and end of run but that the integrated luminosity measurement must be obtained by adding the measurements over small time intervals. The minimum scaler rollover time of 10 minutes is bigger than the 1 minute interval and seems safe for luminosity calculation, but it still seems safer to also increase the range of this counter as proposed for the overall exposition scaler. The following Scalers are intentionnaly omitted because they are ALREADY monitored elsewhere and/or because, depending on the case, it is more convenient to ADD monitoring of a few independent sources outside of the TDM instead of repeating the scaling over 128 separate Specific Triggers. The disadvantage being that it takes knowledge of the programming of the Specific Triggers to rederive the effect on each Specific Trigger, and it might be hard to interpolate a value for a Specific Trigger when the programming is allowed to change. - Sp Trig Exp Grp scaler This quantity is monitored separately for all 8 Exposure Groups. The Scalers used to monitor this quantity need to repeat the AND-ing of the upper and lower And-Or Term contributions before the quantity can be scaled. - ST FEBz Veto scaler This quantity is monitored separately for all 8 exposure groups. The FOM card used to generate the 8 Front-End Busy Exposure Group Signals will provide these scalers. This is not to be confused with the TRM card receiving the 128 Geographic Section Front-End Busy signals and which can monitor the input Front-End Busy signals before they are combined into 8 Front-End Busy Exposure Group Signals. - ST Indiv Veto scaler (2 scalers) This quantity is monitored separately for all 2 * 128 Specific Trigger Individual Disable Signals. The 4 TRM Card where these external signals are received can provide these scalers. - ST Global Correlated Veto scaler (4 scalers) - ST Global De-Correlated Veto scaler (4 scalers) These quantities are monitored separately for all 4 Correlated and 4 De-Correlated Global Veto Signals. The TRM Card where these external signals are received can provide these scalers. Some of these signals may turn out to be generated internally in the L1FW (e.g. the skip next crossing Correlated Global Disable) and a special treatment might be necessary to monitor these internal Disable Signals. Board Support FPGA: Data Description Bits per FPGA M H TC ---------------- -------------------------- - - -- Mask of Sptrg MFP Flags 1 state * 16 SpTrg X X n-2 OUTPUT states ? 2 4 states x 3 ticks = 12 X X n-2 * Notes: This output is in addition to any standard output from the Board Support FPGA (e.g. Header/Trailer words). FOM --- FOM's are used in the following places: 1. Map ST Fired to L1 Accept per Geo Sect (2) 2. Map/Mux ST Fired to L1 Qual/L3 Trans # (1) 3. Generate Skip Next Tick/L1 Fired Strobe/etc (1) 4. Map L1 FE Busy to Exposure Group FE Busy (1) Recall each FOM FPGA has: 128 INPUTS (e.g. Spec Trig Fired) 4 OUTPUTS (Geo Sect L1 Accept, L1 Qual, etc.) Standard FOM FPGA: Data Description Bits per FPGA M H TC ---------------- -------------------------- - - -- OUTPUT states ? 2 4 states x 4 ticks = 16 X X n-2 * 4 ticks = Triggered Crossing + Previous Crossing + 2 Next Crossings OUTPUT scalers ? 2 4 scalers x 32 bits = 128 X n-2 * PBS --- PBS's are used in the following places: 1. Exposure Group Per-Bunch Scaling (16) 2. Other Item Per-Bunch Scaling (16) Recall each Per-Bunch Scaler FPGA has the following: 32 Common Control Signal inputs Data Description Bits per FPGA M H TC ---------------- -------------------------- - - -- Per Bunch scaler 5 scalers x 32 bits = 160 X n-2 * Com Cnt input states ? 32 states x 1 ? tick = 32 ? X n-2 * Scaler combined input gate 1 state (same for all 5) X Scaler Increment gate 5 states X Per FPGA input states NO, unused ? BeamX in Turn Tick Count 8 bits ? X (diag) * note that it is not clear what "TC" is for the "Other" Per-Bunch Item Scalers. GS -- GS's are used in the following places: 1. Geo Sect BX Number (Tick & Turn) (1) 2. Geo Sect L1 Trigger Number (Tick & Turn) (1) 3. Geo Sect L3 Transfer Number (1) 4. Sequential L1 Trigger Number (1) 5. Exposure Group And-Or Enable (1) 6. ???? (????) Recall each Gated Scaler FPGA has the following: 32 Common Control Signal inputs 6 Per-FPGA Control Signal inputs 32 bits Scaler Channel 0 outputs Data Description Bits per FPGA M H TC ---------------- -------------------------- - - -- Gated Scaler Channel 0 1 scaler x 32 bits = 32 X X * Gated Scaler Channels 1:3 3 scalers x 32 bits = 96 X * Com Cnt Input states ? 32 states x 1 ? tick = 32 ? X * PFPGA Cnt Input states ? 6 states x 1 ? tick = 6 ? X * Combined scaler gate 4 states X * cannot predict in advance what "TC" is for these scalers--it depends on the latency of the input. Don't we need a 64 bit beam crossing number somewhere? probably No: a 32-bit Turn Number only wraps around every ~24 hours while a 32-bit Beam Crossing Number would wrap around every ~10 minutes (at 132ns between Beam X) BSF --- BSF's are used in every card. On some cards, BSF's have "card-specific" functions which may require "card-specific" readout. Other elements in the readout should be common across all cards. What type of things do we want to read from the BSF? All cards: Card ID (8 bits is enough, writable by TCC, in header) Event Num (4-8 bits, increments with L1 Accept, NOT L1 Accept Number but a private scaler in each BSF, should be same across all cards, in header) Word Count (8 bits, only reasonable method is to have TCC write the EXPECTED word count, in header, do we really want this?) HSRO Status (some error and status flags, like "L1 Accept seen before readout complete"...what others? How many? Trailer OK?) Only some cards: TDM outgoing MFP flags? (1 bit per TDM) FOM++ incoming Hardware L1 Qualifiers? (~11 bits on FOM++) ??? *************************************************************************