Run II L1 Framework Monitoring -------------------------------- Original Date: 18-AUG-1995 Most Recent Version: 10-JUN-1997 General Background ------------------ The general idea is that there will be two readout paths in the Run II equipment: the Data_Block path and the Monitoring path. The Data_Block path is the high speed optical path that sends out information with each L1 trigger. The monitoring path provides VME access to information. The equipment will be designed so that ALL information that goes out through the Data_Block path will also be visible from the Monitoring path. This include the ability to have all information that will be accessed via the monitor path, loaded into registers for monitor readout at the same instant and during a 132 nsec period where there is a L1 trigger. Notice that in this system there is no SPY function to capture a Data_Block for examination by TCC's Monitor Pool Filler. In the Run II equipment TCC will access all monitor information in the way that it get the SBSC information in the Run I equipment. Status vs Scalers ----------------- It is not yet clear whether certain signals in the L1 Framework should be monitored by attaching a scaler to them or by just being able to examine by knowing their current status. It is also true that for some signals you want to know both, e.g. for how many BX's has this signal been active and was it active for the BX during which this monitoring information was captured. Scaler readout in the L1 Data_Block ----------------------------------- We should only put into the Data_Block the scalers that actually need to be there. In general we think that this is far fewer scalers than were put into the Run 1 L1 Framework Data_Block. This means that we will have lots of our scaler information only in the Monitor Pool. Thus, periodically, moving this scaler information into the "Luminosity Files" and permanently recording it will be important. Information vs What Beam Crossing It Comes From ----------------------------------------------- Although a "capture_monitoring_information" signal will cause all sections of the L1 Framework to isochronously load their monitoring information into registers for VME readout; not all of this information represents the state of the L1 Framework at the instant this signal becomes active. For example the And-Or Network Input Term States will represent the state of these signals for the Beam Crossing whose L1 trigger decision has just been sent out to the Geographic Sections; where as the Live Crossing Scaler count will indicate the number of live crossings up to this point in time. It is clear that much of the monitoring data needs to be contemporary at the beam crossing level in order for it to be analyzable, e.g. all of the state information must come from the same beam crossing. Generation of the "Load" Monitor Data Signal -------------------------------------------- The idea is to generate one common "Load Monitor Data" signal and to distribute this throughout the L1 Framework. Individual application cards could make delayed versions of this signal, as needed, to guarantee that they do not capture information at a point in time when it is changing, e.g. a scaler card would not want to capture information at the same time as the scaler maybe incrementing. The Trigger Control Computer should be able to control when the Load Monitor Data signal is generated. For example the TCC should be able to asynchronously request that the Framework load monitor information during the next beam crossing or to arm a circuit that will cause the Framework to load monitoring information during the next beam crossing when a L1 trigger fires. In the later case, the circuit that causes the Framework to load monitoring information when a L1 trig fires, may not wait for the news of an L1 firing to loop out and come back on the serial command link. Making this path direct may same the need for history sample buffers. Scaler Types and Widths ----------------------- Data_Path Scaler Card It is thought that 16 scalers will fit onto a Data_Path Scaler Card. Each of these scalers has a "history pipeline". The proper tap on this history pipeline feeds the register that holds the count that will be readout through the optical link and the register that will be readout through the VME path. There are separate load signals for these two registers. Load Monitor Data >-----------------+ | +--v---+ Clk +------+ History Pipeline | VME | VME >---| | +--+--+--+--+--+--+--+--+--+ +-->| Reg |---> >---|Scaler|----->| | | | | | | | | | | +----- + Gates +______+ +--+--+--+--+--+--+--+--+--+ | ^ | | +------+ | | | | Data | Optic Shift >-----+ +-----------+-->| Blk |---> Clk | Reg | Path +--^---+ | Load Data Block >-----------------+ Roll Over vs Scaler Width ------------------------- Width 396 nsec Beam Crossing 132 nsec Beam Crossing -------- ------------------------ ------------------------ 32 1677 sec = 27 min 57 sec 566 sec = 9 min 26 sec 40 119 hr 15 min 40 hr 15 min ------------------------------------------------------------------------------- Draft: Tuesday, June 10, 1997 A. Specific Trigger Fired Mask ------------------------------ Specific Trigger Fired Mask 128 bits = 4 longwords (from Trigger Qualifier/SpTrg Group FOM?) C. Trigger Number(s) -------------------- Level 1 Trigger Accept Number 32 bits = 1 longword This is a sequential number counting the number of L1 Trigger Accepts. Beam Crossing number(s) Turn Count 32 or 64 bits L1 Geographic Section Trigger Number Turn Number 16 bits Bunch Number 8 bits L3 Geographic Section Transfer Number 16 bits D. Level 1 Qualifiers --------------------- Generated by a FOM Same 16 Qualifiers to all Geographic Sections Level 1 Trigger Accept Qualifier State 16 bits for Qualfier #n [n=0:16] Level 1 Trigger Accept Qualifier Scaler 16 x 32 bit scalers for Qualifier #n [n=0:15] E. And-Or Input Terms --------------------- 64 And-Or Terms serviced per TRM Card 4 And-Or Terms serviced per TRM Card FPGA And-Or Term Input State 64 x 16 bit words for Term #n [n=0:255] (4 terms x 4 BX per word) And-Or Term Monitoring Scaler 256 x 32 bit scalers for Term #n [n=0:255] And-Or Term Synchronization Error State One per group of 16 And-Or Term, i.e. 16 Error sources And-Or Term Synchronization Error Scaler And-Or Term last Synchronization Error Time Time at last occurence F. Specific Trigger ------------------- 8 Specific Triggers serviced per TDM Card Specific Trigger States 128 x 32 bit longword for Specific Trigger #n [n=0:128] Specific Trigger Fired State ? Previous Crossing Bit # Current Crossing Bit #0 ? Next Crossing (or would be firing?) Bit # Specific Trigger Exposed State Previous Crossing Bit #2 Current Crossing Bit #3 Next Crossing Bit #4 SpTrg Physics And-Or Fired State Previous Crossing Bit #5 Current Crossing Bit #6 Next Crossing Bit #7 Current Crossing Lower And-Or Terms Bit #8 Current Crossing Upper And-Or Terms Bit #9 SpTrg And-Or Exposure Group Enable State Previous Crossing Bit #10 Current Crossing Bit #11 Next Crossing Bit #12 Current Crossing Lower And-Or E.G.E.S Bit #13 Current Crossing Upper And-Or E.G.E.S Bit #14 SpTrg DAQ Enable State Previous Crossing Bit #15 Current Crossing Bit #16 Next Crossing Bit #17 Specific Trigger Disable Mask Front-End Busy Disable Bit #18 Prescaler Disable Bit #19 COOR Disable Bit #20 Auto-Disable Bit #21 Gated Individual Disable #0 Bit #22 Gated Individual Disable #1 Bit #23 Gated Internal Global Disable #0 Bit #24 Gated Internal Global Disable #1 Bit #25 Gated Internal Global Disable #2 Bit #26 Gated Internal Global Disable #3 Bit #27 Gated External Global Disable #0 Bit #28 Gated External Global Disable #1 Bit #29 Gated External Global Disable #2 Bit #30 Gated External Global Disable #3 Bit #31 ------------- 1 longword Specific Trigger Fired Scaler Count 128 x 32 bit scalers for Specific Trigger #n [n=0:128] Specific Trigger Exposed Scaler Count 128 x 32 bit scalers for Specific Trigger #n [n=0:128] Specific Trigger Physics Andor Fired Scaler 128 x 32 bit scalers for Specific Trigger #n [n=0:128] Specific Trigger Exposure Group Enable Scaler 128 x 32 bit scalers for Specific Trigger #n [n=0:128] Specific Trigger DAQ Enable Scaler 128 x 32 bit scalers for Specific Trigger #n [n=0:128] Specific Trigger COOR disable Scaler 128 x 32 bit scalers for Specific Trigger #n [n=0:128] Specific Trigger Prescaler Veto Scaler 128 x 32 bit scalers for Specific Trigger #n [n=0:128] Specific Trigger Front-End Busy Veto Scaler 128 x 32 bit scalers for Specific Trigger #n [n=0:128] Specific Trigger Auto-Disable Veto Scaler 128 x 32 bit scalers for Specific Trigger #n [n=0:128] Specific Trigger Individual Veto Scaler 2 x 128 x 32 bit scl for Specific Trigger #n [n=0:128] (can be done on TRM) Specific Trigger External Global Veto Scaler 4 x 128 x 32 bit scl for Specific Trigger #n [n=0:128] (can be done on TRM) Specific Trigger Internal Global Veto Scaler 4 x 128 x 32 bit scl for Specific Trigger #n [n=0:128] G. Geographic Section --------------------- 64 Geographic Sections serviced by a FOM Card 4 Geographic Sections serviced per FOM FPGA Geographic Section L1 Accept State 32 x 16 bit words for Geographic Section #n [n=0:128] (4 terms x 4 BX per word) Geographic Section L1 Accept Scaler 128 x 32 bit scalers for Geographic Section #n [n=0:128] 64 Geographic Sections serviced by a TRM Card 4 Geographic Sections serviced per TRM FPGA Geographic Section Front-End Busy State 32 x 16 bit words for Geographic Section #n [n=0:128] (4 terms x 4 BX per word) Geographic Section Front-End Busy Scaler 128 x 32 bit scalers for Geographic Section #n [n=0:128] Geographic Section SCL L1 Error Scaler 128 x 32 bit scalers for Geographic Section #n [n=0:128] Geographic Section Last L1 Error Time Stamp 128 x 32 bit word ? for Geographic Section #n [n=0:128] Geographic Section SCL L2 Error Scaler 128 x 32 bit scalers for Geographic Section #n [n=0:128] Geographic Section Last L2 Error Time Stamp 128 x 32 bit word ? for Geographic Section #n [n=0:128] H. "Foreign Scalers" -------------------- e.g. time since something last happened. some individual scalers, some per bunch scalers? I. Luminosity = per bunch scalers --------------------------------- cf D0 Note 2860 (?) scaler increments between last two snapshots -------------------------------------------- repeat all or most of the scalers above but for the scaler count increment between the current snapshot and the previous snapshot (about 5 second interval) Framework global states ----------------------- running/pause time since last event time since last heartbeat event time since last trigger event delta beam crossings delta time Programming ----------- feedback of programming of all L1 resources Notes: ----- - 32 bit scaler counts up to 4,294,967,303 (unsigned) and will roll over: at 132 ns : after 567 second = 9 minutes 27 seconds at 21. us : after 90,200 seconds = 1,500 minutes = 25 hours - a 40 bit scaler would roll over at 132 ns : after 145,000 second = 2,400 minutes = 40 hours 19 minutes - from the list above, the subset of scalers in danger of rolling over during a run are: And-Or Term Monitoring Scaler 256 x 32 bit scalers Specific Trigger Exposed Scaler Count 128 x 32 bit scalers Specific Trigger Physics Andor Fired Scaler 128 x 32 bit scalers Specific Trigger Exposure Group Enable Scaler 128 x 32 bit scalers Specific Trigger DAQ Enable Scaler 128 x 32 bit scalers Specific Trigger COOR disable Scaler 128 x 32 bit scalers Specific Trigger Prescaler Veto Scaler 128 x 32 bit scalers Specific Trigger Front-End Busy Veto Scaler 128 x 32 bit scalers Specific Trigger Auto-Disable Veto Scaler 128 x 32 bit scalers Specific Trigger Individual Veto Scaler 2 x 128 x 32 bit scl Specific Trigger External Global Veto Scaler 4 x 128 x 32 bit scl Specific Trigger Internal Global Veto Scaler 4 x 128 x 32 bit scl Geographic Section Front-End Busy Scaler 128 x 32 bit scalers Foreign Scalers Luminosity = per bunch scalers - TCC can watch when the 32 bit scalers roll over, and maintain a higher order byte for those scalers. This seems a bit scary and like a lot of responsability on TCC. How hard of a job is it on TCC? can TCC miss when a scaler rolls over? TCC will get access to a snapshot of all scalers about every 5 seconds. When TCC sees that the previous scaler count was "close but below maximum count of the 32 bit scaler" and the current scaler count is "small positive number", TCC can deduce that the scaler just rolled over. If TCC somehow misses a 5 second increment, or if the DAQ is held up and no data block and/or no monitoring information is captured for a while, then TCC can temporary miss the moment when the scaler rolled over, but there is also no monitoring information captured either. TCC can catch up at the next snapshot. For TCC to potentialy miss and lose track, would require that TCC isn't allowed to look at scalers for more than 9-10 minutes while the scaler rolls over twice while TCC only recors once, or just rolls over and catch up with the previous snapshot while TCC thinks the scaler just incremented without rolling over. Requiring TCC to keep track of a higher order scaler byte must thus include guaranteeing that TCC can ALWAYS get access to a scaler snapshot at least once per 1-4 minutes. That can be a regular triggered event with monitoring information, a heartbeat event with monitoring information or a forced monitoring capture with no event. - There are 19 scalers per SpTrg (listed above) that would need to hind a home on the TDM. This seems like a big number, but the TDM is the place of choice. The TDM is where the relevant information is generated, while some quantities could be exported to outside scalers. One notices that the majority of the per-sptrg scalers are monitoring the sources of disable. The compelling reason to have these scalers on the TDM (as opposed to earlier where the sources of disable are received) is that each sptrg can choose to allow/disallow a particular source of disable. In particular, we have (up to) 8 global sources of disable (4 internal and 4 external). Scaling the source itself would only require a total of 8 global scalers somewhere off the TDM cards, as opposed to repeating the scaling 8 x 128 times (once per global disable and per specific trigger). Doing this would limit our ability to monitor how much a given source has disabled a particular sptrg over the last 5 seconds, or since the beginning of the run. It takes some programmed IO from TCC to change the status of the allow/disallow properties of a source of disable for each specific trigger, and is this something we are going to change multiple times every 5 seconds?. If we can argue that the decision to allow/disallow these 2 x 4 global disables is just a decision for the configuration of the hardware in general or of a particular run, then we don't need to implement the 2 x 4 x 128 scalers. If we think we might want to use these resources to implement some software/hardware throttling combination where TCC needs to change the allow/disallow property during the run, then we should implement all or some of the 2 x 0..4 x 128 scalers.