Data Format from L1 Cal Trig to L2 Cal_pp and to L3 ----------------------------------------------------- Original Rev: 26-MAR-1999 Current Rev: 21-SEPT-2001 The file name of this document will not be changed but its function has been expanded to cover the format of the L1 Calorimeter Trigger readout to both the L2 Cal_pp and the format to L3. Background ---------- For Run II the L1 Cal Trig will use a modified version of the ERPB and DC cards to readout, i.e. the Run I system that feed Trigger Tower data to the L1.5 Cal Trigger. In each rack, Trigger Tower data from the ERPB DC cards will feed into a "Bougie" FPGA which is located at the BSF FPGA Site (17) on a Build A And-Or Network card. In the Bougie FPGA the Trigger Tower data will be scanned to build the EM Mask and the Jet Mask. In addition the L2 Header and L2 Trailer will be added to the data. The output of the Bougie card is G-Link encoded Fiber Optic data that splits with half of the optical signal going to the L2 system and the other half of the optical signal going to a Trigger Mode VRB readout crate for feeding Level 3 DAQ. In the VRB crate, VRB Headers and VBD Headers and Trailers are added to the data that goes to Level 3. These headers and trailers are not in the data that is feed to Level 2. The data on each of the fiber optic links is the readout of a single rack in the L1 Cal Trig system. Overview of Data Available -------------------------- By double-cycling the CTFE EM and HD lookup memories, we can provide two different Et based quantities for each Trigger Tower. One of these quantities must be Total Et (due to the structure of the L1 Cal Trig). The other quantity will be the EM Et. The types of data available from L1 Cal Trig to the L2 Cal_pp are then: Trigger Tower EM Et for each of the 1280 L1 Trigger Towers Trigger Tower Total Et for each of the 1280 L1 Trigger Towers Mask of Trigger Towers with their EM Et's over the L2 EM Et Seed Reference Set Mask of Trigger Towers with their Total Et's over the L2 Total Et Seed Reference Set Note that the L2 Seed Reference Sets defined here are NOT coupled to the L1 Triggering Reference Sets. These are used only for generating the L2 Seed Masks - they play no part in generating the L1 Trigger. The L2 Seed Ref Sets have NO phi dependency, only eta dependency (unlike the L1 Triggering Ref Sets, which can have unique values for each Trigger Tower). There is only one L2 EM Et Seed Ref Set and one L2 Tot Et Seed Ref Set. Detailed Data Format -------------------- The same data is provided to L2 Cal_pp as to the VRB cards. This is G-Link Fiber Optic format with 1 fiber optic link per L1 Cal Trig rack. A rack of the L1 Cal Trig covers all 32 phi's at 4 consecutive eta's The G-links will transfer 20-bits of "user data" in 24 bit G-Link frames. The lower 16 bits of "user data" is the real data that is delivered to Level 2 and readout to Level 3. The upper 4 bits of "user data" is used to mark event boundaries (D16 asserted in the first G-Link frame of the event and D17 asserted in the last G-Link frame of the event). We are running the G-Link in this way because we were requested to do so by others. Besides the information from the L1 Cal Trigger listed above, the data on these links includes the standard Level 2 Header and Trailer words. Some small modifications of these Headers and Trailers was necessary because of the "legacy" nature of the L1 Cal Trig. Where modifications were made we attempted to preserve the "spirit" and function of the original header/trailer entry. The detailed format shown below uses L1 Cal Trig rack M103 as an example, i.e. the rack servicing eta +0 through +0.8, all phi's. In the L1 CAl Trig coordinate system this is Cal Trig Eta Indices +1,..., +4; phi indices 1, 2, ..., 32. Note that the data is defined in terms of 16-bit words. This is how the L1 Cal Trig prepairs the data, how the G-Link transports the data, and how the data is presented to the FIC and the VRB. D15 D8 D7 D0 -------------------------- -------------------------- Header Length (in 4B LW) Number of Objects (<=255) (1) \ (fixed data set =3) | | Header/Object Format (2) Object Length (in 4B LW) (1) | | Stan- Tick Number (4) Data Type (3) | dard | Turn Number MSByte (4) Turn Number LSByte (4) | L2 | Algorithm Minor Version Algorithm Major Version (5) | Header (or processor specific) (5) | | Status Bits (7) Processor Specific Bits (6) / EM Et eta = +1, phi = 1 Tot Et eta = +1, phi = 1 (9) \ | EM Et eta = +2, phi = 1 Tot Et eta = +2, phi = 1 | 128 | Words EM Et eta = +3, phi = 1 Tot Et eta = +3, phi = 1 | of | Trig EM Et eta = +4, phi = 1 Tot Et eta = +4, phi = 1 | Tower | Et EM Et eta = +1, phi = 2 Tot Et eta = +1, phi = 2 | Data . . | . . | EM & . . | Total EM Et eta = +4, phi = 32 Tot Et eta = +4, phi = 32 / EM Et > L2 Seed EM Et Ref for eta = +1, phi = 16:1 (10) \ 8 | word EM Et > L2 Seed EM Et Ref for eta = +1, phi = 32:17 | mask | of EM Et > L2 Seed EM Et Ref for eta = +2, phi = 16:1 | TT . | EM Et . | > Seed . | Ref EM Et > L2 Seed EM Et Ref for eta = +4, phi = 32:17 / Set Tot Et > L2 Seed Tot Et Ref for eta = +1, phi = 16:1 (10) \ 8 | words Tot Et > L2 Seed Tot Et Ref for eta = +1, phi = 32:17 | mask | of Tot Et > L2 Seed Tot Et Ref for eta = +2, phi = 16:1 | TT . | Tot Et . | > Seed . | Ref Tot Et > L2 Seed Tot Et Ref for eta = +4, phi = 32:17 / Set Data Type (3) Tick Number (4) \ | L2 Longitudinal Parity (8) Longitudinal Parity (8) / Trailer The length of this block of data is 152 words (2 byte words) or 19 of the 128 bit Magic Bus transfers. Notes: (1) Number of Objects - Object Lenght These bytes are programmed by TCC and are fixed from event-to-event. We need to define some rational meaning for them. We could at least make Number of Objects x Object Length so it will equal the total "user data" length of 72 32-bit longwords. We will start with Number of Objects = 1 and Object Length = 72 Long Words. (2) Header/Trailer Format top 3 bits, Object Format lower 5 bits. This byte are programmed by TCC and is fixed from event-to-event. We will start with both format numbers set to 1. i.e. hex 21. (3) Data Type is a byte that is programmed by TCC and fixed from event to event. From the L2 Web Documents it appears to label what crate the data comes from with decimal values 98 through 107, i.e. a different value for each one of the L1 Cal Trig racks - fiber optic links. This is good because for our own L1 use we need a "source label" built into each data source. We will use the following map of Data Types: Rack Eta Index Location Coverage Data Type -------- --------- --------- M103 +1:+4 98 M104 -4:-1 99 M105 +5:+8 100 M106 -8:-5 101 M107 +9:+12 102 M108 -12:-9 103 M109 +13:+16 104 M110 -16:-13 105 M111 +17:+20 106 M112 -20:-17 107 (4) The Tick and Turn Numbers are NOT available to the L1 Cal Trig at this time. For now we have replaced these fields with a 24-bit counter which increments in some convenient fashion. This counter should read the same for all 10 G-links, but would NOT be associated with the real L1 Geographic Section Tick and Turn Number. So for now this can only help guarantee that all the L1 Cal Trig data for an event fits together but not that it is the data from the correct event. In the future we plan to get at least a few (e.g. 5) bits of the real Tick number readout in these locations. (5) Algorithm Major/Minor Version or Processor Specific Bits. This word is programmed by TCC and fixed from event-to-event. We will use it to keep track of the FPGA versions in the data path producing this data. We need this to match functionality that we use in the Trig FW readout and because we may go through a number of FPGA versions all producing the same Header/Object formats. (6) Processor Specific Bits For now this byte is programmed by TCC and is fixed from event-to-event. We will start with it programmed to 3. A very likely future change is to make this dynamic event to event with the following meaning: D0 Data Captured D1 Transporting Data D2 Unexpected 2nd Cap w/o Prev Trans D3 Unexpected Transport w/o Capture D4 Unexpected Capture during Transport D5 Unexpected Transport during Transport D6 '0' (reserved for expansion) D7 '0' (reserved for expansion) (7) Status Bits as defined by the L2 Header/Trailer specification. None of these Status Bits have any meaning in this application. They will all be set to zero, i.e. their non-asserted state. This byte is programmed by TCC and is fixed from event-to-event. (8) Longitudinal Parity Bit by Bit XOR For each bit D0:D15 this is an XOR of all the the bits, in that bit position, that have been sent out in this event. (9) EM Et is an 8-bit unsigned integer, 1/4-GeV per count, saturating at 255 (approx. 60 GeV). Total Et is an 8-bit unsigned integer, either 1/4-GeV per count, or 1/2-GeV per count. If 1/2-GeV per count, saturation would be approximately 120 GeV, and "truncation" would be performed as in the L1 Calorimeter Trigger. Since EM Et and Tot Et are unsigned, they have pedestals added to avoid rectifying the electronics noise in the detector. These pedestals are uniform across the detector. The most likely value of the pedestal will be 8 counts, i.e. with zero energy deposited in a Trigger Tower it will read hex 08. This mean that with an energy scaler of 1/4 GeV per count, the saturation point is 61.75 GeV. EM Et and Tot Et also receive symmetric low-energy cuts (i.e. Trigger Towers with Et in some small range [+ or -] around 0 GeV output a count that equals exactly 0 GeV). The size of this cut is yet to be determined, but will be in the neighborhood of 1/2 to 1 GeV. This low-energy cut may not be uniform across the detector, and it may be different for EM Et and Tot Et for a given tower. For negative eta, the readout order would be eta -4, -3, -2, -1. (10) Within a word, the bit ordering is: D0: phi = 1 or 17 . . . D15: phi = 16 or 32 For negative eta, the readout order would be eta -4, -3, -2, -1. Again recall that the L2 Seed Ref Sets only have eta dependency. For a given eta, they are FLAT in phi. General note about the L2 Header and Trailer Information about the L2 Header and Trailer can be found at: http://www.pa.msu.edu/hep/d0/l2/ftp/l2/data_transfer/ l2_header_trailer.pdf http://www.pa.msu.edu/hep/d0/l2/ftp/l2/data_transfer/ data_types_source_ids.pdf Format of the Data Readout to Level 3 ------------------------------------- The data described in detail above is what goes into the VRB's The 10 racks in the L1 Cal Trig are connected to three VBD's in the following way. G-Link Connection VRB Channel Pairs ------------------------------------- VRB 0/1 2/3 4/5 6/7 --- ---- ---- ---- ---- "A" M112 M110 M108 M106 -20:-17 -16:-13 -12:-9 -8:-5 "B" M104 M103 M105 M107 -4:-1 +1:+4 +5:+8 +9:+12 "C" M109 M111 no no +13:+16 +17:+20 conn. conn. VRB "A" is in slot 9 and is the first to be readout by the VBD. VRB "C" is in slot 11 and is the last to be readout by the VBD. The general format of the data as seen in the data block from the L1 Cal Trig is the following: VBD Header 8 Long Words VBD "Word Count Word" i.e. number of longwords read from VRB "A" VRB "A" Header 8 Long Words Data from VRB "A" Channels 0/1 i.e. rack M112 Data from VRB "A" Channels 2/3 i.e. rack M110 Data from VRB "A" Channels 4/5 i.e. rack M108 Data from VRB "A" Channels 6/7 i.e. rack M106 VBD "Word Count Word" i.e. number of longwords read from VRB "B" VRB "B" Header 8 Long Words Data from VRB "B" Channels 0/1 i.e. rack M104 Data from VRB "B" Channels 2/3 i.e. rack M103 Data from VRB "B" Channels 4/5 i.e. rack M105 Data from VRB "B" Channels 6/7 i.e. rack M107 VBD "Word Count Word" i.e. number of longwords read from VRB "C" VRB "C" Header 8 Long Words Data from VRB "C" Channels 0/1 i.e. rack M109 Data from VRB "C" Channels 2/3 i.e. rack M111 VBD Trailer 10 Long Words Notes: When worked with the L1 Cal Trig data in L3 or off line it is probably easiest to understand the structure of the data in terms of 32 bit long words. As seen in L3 or off line, the Data from VRB Channel Pairs will have been scrambled so that for a given channel pair, (for a given G-Link), first all the low order bytes from the 16 bits words are readout packed into 32 bit long words and then all the high bytes from the 16 bit words are readout packed into 32 bit long words. The easiest way to work with the VRB Channel Pair data on L3 or off line is probably to first always unscramble all the VRB Channel Pair data and then to refer to its detailed layout as described above in terms of 16 bit words. It is only the VRB Channel Pair data, i.e. the data that comes into the VRB's on the G-Links, that gets scrambled. All of the headers and trailes and word count works and such are just as explained here in term of 32 bit objects. The VBD header is 8 long words long. Only the first 2 long words are used. The first long word is the total number of long words readout to L3 from this VBD including the VBD Header and Trailer. The most significant 16 bits of the 2nd long word is the L3_Transfer_Number which must be the same for all crates readout for a given event. The least significant 16 bits of the 2nd long word is the Crate ID which is hex 10 for the L1 Cal Trig Crate. VBD Word Count Words In the event data there is a VBD Word Count Word right before the data from each card that the VBD read. The lower 16 bits of a Word Count Word shows the number of longwords that the VBD read from the following card. The VBD Word Count Words are not part of the VBD Header or VBD Trailer. The Word Count Words are put into the data stream by the VBD, i.e. they do not come from the cards readout by the VBD. The VRB Header has the following format: 8 longwords of VRB header. According to the 8/28/00 spec: 31 24 23 16 15 8 7 0 ----------------------------------------------------------------- 0: | Total Byte Count | ----------------------------------------------------------------- 1: | User Info | Slot Number | Event Number | ----------------------------------------------------------------- 2: | Firmware Version Number | bit 1 = Trig Mode | ----------------------------------------------------------------- 3: | Status Word 1 (TBD) | ----------------------------------------------------------------- 4: | Channel 0 Byte Count | Channel 1 Byte Count | ----------------------------------------------------------------- 5: | Channel 2 Byte Count | Channel 3 Byte Count | ----------------------------------------------------------------- 6: | Channel 4 Byte Count | Channel 5 Byte Count | ----------------------------------------------------------------- 7: | Channel 6 Byte Count | Channel 7 Byte Count | ----------------------------------------------------------------- "User Info" is event to event static data that can be programmed by TCC writing to a register on the VRB. The "Event Number" is not related to any of the other things often called event number in the D0 Run II DAQ system, e.g. the L3 Transfer Number, or the L1 Geographic Section Trigger Number. In the G-Link application the pairs of Channel Byte Counts must obviously agree and for the L1 Cal Trig readout they should all be hex 98 = decimal 152. Because of the fixed format and fixed size of the data from the L1 Cal Trig the various byte counts in the VRB header are the same for all events. Checking them is a good "sanity check" on the L1 Cal Trig data block. VBD Trailer This is 10 long words long and does not contain any information that is typically needed when working with event data. The first and second Trailer long words are just a repeat of the first and second Header long words. The next 6 long words are not used. The next to the last long word is the "Token Value". The last long word is the "Negative Checksum".