The Level 1.5 Calorimeter Trigger for D-Zero D0 Note 2117 Daniel Edmunds, Steve Gross, and Philippe Laurens April 21, 1994 Department of Physics and Astronomy Michigan State University East Lansing, MI 48824-1116 This D0 note is an introduction to the Level 1.5 Calorimeter Trigger described here in the context of the overall D0 Trigger System. The presentation covers motives and goals along with constraints and design choices, and gives an overview of the architecture and implementation of the system. This document exposes the capabilities and limitations of the Level 1.5 Calorimeter Trigger as a platform, and is a prerequisite for the understanding of more technical details. Page 2 CONTENTS -------- The D-Zero Trigger System The Hardware Trigger Functionality of the Level 1 Trigger Specific Triggers Making Decisions between Beam Crossings Level 1.5 Trigger System Need for a L1.5 CT Level 1 limitations Neighbor sums Software implementation Design Goals Rate and Rejection Processing Time Neighbor range Process on request Why choose a DSP? Main Challenge of the project The DSP C40 The Hydra II Implementation Calorimeter Trigger Tower Energies Parallel Local DSP processing Avoiding special cases Data path L1.5 CT input data L1.5 CT as a Front-End crate Conditions for running the L1.5 CT Frame Code versus Tool Code Multiple Terms Mark and Pass events Crate architecture and control Three DSP boards One 68k CPU board Parallel IO cards and support paddle boards Two multiport memory boards One VBD board Split VME backplane VSB Bus TCC programming Timing performance Answer Time Front-End Busy Future improvements Possible hardware improvements Industry trend Innovative usage Appendices A L1.5 Calorimeter Trigger Cycle Timing Strip B New COOR to TCC message for L1.5 CT C D0 note bibliography Page 3 The D-Zero Trigger System ========================= Triggering at D-Zero can be described as a two step system. The Hardware Trigger (Levels 0, 1 and 1.5) selects events for digitization. The Software Trigger (Level 2) selects a fraction of these events to record for offline analysis. The DSP based L1.5 Calorimeter Trigger presented here is a component of the Hardware Trigger. The goal of the Hardware Trigger is to process every beam crossing (about 300 KHz) and pick the most promising interactions for digitization and full data collection. The Hardware Trigger uses several detector specific data processing systems. These custom devices implement relatively simple algorithms and obtain fast but partial views of the interactions. The Software Trigger receives and processes the events chosen by the Hardware Trigger with an input rate of about 200 Hz (Fig. 1). At this stage, the complete events are available for final selection of what is saved to tape with an output rate of about 2 Hz. The Software Trigger uses a "farm" of 50 general purpose computer platforms that can run arbitrary complex algorithms. The Hardware Trigger ==================== Functionality of the Level 1 Trigger ------------------------------------ The functionality of the Hardware Trigger system can be broken down into three main aspects. - The first task is to generate Triggering Information about a given beam crossing. - The second is to use this information and the state of the other components of the acquisition system to generate a Trigger Decision for each beam crossing. - The third is to direct the collection of the data for each event selected. The heart of the Hardware Trigger is the Level 1 Trigger Framework. It does not generate any information about the beam crossings, but makes the Trigger Decisions and synchronizes the acquisition system (cf. D0 Notes #328 and 705). The Triggering Information is generated by separate Level 1 Trigger Subsystems that provide the input to the Level 1 Trigger Framework. At D-Zero, the Level 1 Muon Trigger, the Level 1 Calorimeter Trigger and the Level 0 Trigger are the major Trigger Subsystems and the major inputs to the Level 1 Trigger Framework. A typical Level 1 Trigger Subsystem analyzes the signals from one of the detector components and makes use of specialized hardware to provide an indication, for each beam crossing, of the activity seen by that detector component. Page 6 Specific Triggers ----------------- The Level 1 Trigger Decision contains 32 separate programmable sets of requirements. Each set of requirements is called a Specific Trigger and can be programmed to represent different physics constraints as measured by the Level 1 Trigger Subsystems. For example some Specific Triggers can be programmed to select events with muon detector tracks, while others ask for electron signal, hadronic jet energy deposit or missing energy. The Specific Trigger granularity is used to control the mix of events going to the Level 2 system and is used by the level 2 system to know what type of event it is and thus what subset of the software filters to apply. A Global Level 1 Trigger Decision is made as the logical OR of all 32 Specific Trigger decisions. Making Decisions between Beam Crossings --------------------------------------- At D-Zero the detector signal itself is not pipelined; the raw signal is not systematically collected and saved over several crossings while awaiting data processing and Trigger Decision. The D-Zero design avoids this pipelining by instead requiring that the Level 1 Trigger Decision be available before the next beam crossing. Thus the signal from an interesting event must be saved and collected before it is overwritten by the next beam crossing. The ability to provide information between beam crossings (3.5 microseconds) depends on detector signal characteristics (charge drift or collection time), cable delays, algorithm complexity, and existence and support of dedicated hardware. All information available between beam crossings is directly exploited in the Level 1 Trigger Framework, which is kept synchronous to the beam crossings. Level 1.5 Trigger System ------------------------ Obtaining additional rejection requires more complex data processing. While additional filtering purifies the stream of events sent to the Software Trigger, it also allows reallocation of some of the precious bandwidth available through the data acquisition system. The ability to examine additional Triggering Information that is only available on a slower time scale (10-200 microseconds) is implemented in the Level 1.5 Trigger System. This late Trigger Information is not available between beam crossings, nor is it synchronous with the beam crossings. The Level 1.5 Trigger System also makes the distinction between generating Trigger Information and forming Trigger Decisions. The Level 1.5 Trigger System is made of separate Level 1.5 Trigger Subsystems that provide the input to the Level 1.5 Trigger Framework. The Level 1.5 Muon Trigger was the only Level 1.5 Trigger Subsystem operating during run Ia, while the Level 1.5 Calorimeter Trigger is being added during run Ib. Page 7 The Level 1.5 Trigger Framework is intimately interfaced to the Level 1 Trigger Framework and can take over and halt the normal Level 1 processing. A Level 1.5 Cycle is only performed when needed to confirm or reject a Level 1 Trigger. After every positive Level 1 Trigger Decision the Level 1 Trigger Framework initiates the data acquisition protocol to start the event collection before the next beam crossing. But when a beam crossing has been identified as needing Level 1.5 Confirmation, control is then transferred to the Level 1.5 Trigger Framework to start a Level 1.5 Cycle. The Level 1.5 Trigger Framework waits until all necessary input from the Level 1.5 Trigger Subsystems is available. This input is asynchronous with the beam crossings, and comes after an unpredictable delay. The Level 1.5 Trigger Framework uses this information to make the Level 1.5 Trigger Decision and returns control to the Level 1 Trigger Framework. The Level 1 Trigger Framework can then complete the data acquisition protocol. When a Level 1.5 Cycle is concluded with the confirmation of the event, the data digitization cycle that was already initiated can continue and the event is fully collected. When the Level 1.5 Cycle leads to the rejection of the event the data collection is interrupted, and all the acquisition Front-Ends must completely unwind (i.e. get themselves ready for the next event). A Level 1.5 Cycle is not performed for every beam crossing, but only when a positive Level 1 Trigger Decision is made that requires a Level 1.5 Decision to decide whether the event needs to be transferred to Level 2. See D0 Note #1638 for details. During a Level 1.5 Cycle, all Level 1 activity ceases. This results in global deadtime during which all beam crossings are totally ignored. The beam delivered during Level 1.5 processing is totally lost. The rate of requests for Level 1.5 Cycles must be carefully managed and most importantly the processing time must be kept as low as possible. Under these circumstances usage at D-Zero of the Level 1.5 Trigger is a balance between increasing event quality and loss of experiment livetime. Need for a L1.5 CT ================== Level 1 limitations ------------------- The Level 1 Calorimeter Trigger provides information from Trigger Towers which are 0.2 in phi by 0.2 in eta (cf. D0 Note #706). Without entering into too many details, this granularity brings the following limitations on both electron and jet triggering. Even though electron showers are typically smaller than Trigger Towers, the energy deposited is frequently shared between two or more Trigger Towers. The threshold that can be safely applied at level 1 while preserving good efficiency must thus be set to a fraction of the desired electron signal. For jets, which span many towers, this problem is much more severe. This excessive lowering of the thresholds leads to poor rejection of background and high trigger rates. Furthermore the ability to trigger on multiple jets or electrons is very poor. Page 8 Neighbor sums ------------- To address the energy sharing problem, the signals from adjacent Trigger Towers need to be taken into account so that the true energy deposit of a candidate object can be computed. Further requirements should also be applied to improve object identification (e.g. electron versus jet). It was shown early on (M.Narain) that simple algorithms that sum energy from neighboring Trigger Towers can provide rejection factors of 5. Such computations could be done with dedicated hardware (possibly even fast enough to be included in the Level 1 Trigger Decision), but would require a very complex and expensive new system that would take many more years to build while producing a platform of limited flexibility. Software implementation ----------------------- A software approach offers many attractive characteristics for a Level 1.5 Calorimeter Trigger Subsystem. The resulting Level 1.5 Calorimeter Trigger platform described here is very general and can assist both in electron and jet triggering. Advantages inherent to the flexibility of the software platform include making incremental modifications or "tuning" of the algorithm possible, or testing of the next version to be released. If need arises, the code can also be switched between normal operation and special runs. Someone could also come up later on with a radically new approach. This platform is also open to more advanced filtering that might not be reasonable to implement in hardware, for example on combinations of objects with angular or mass requirements on di-electron, di-jet, or electron-jet pairs. Design Goals ============ The driving need calling for the addition of a L1.5 CT is to improve the electron triggering performance of the Hardware Trigger. This is the principal goal at this time for the Level 1.5 Calorimeter Trigger. Rate and Rejection ------------------ Relatively simple algorithms applied to the coarse Trigger Tower data were shown (Meena Narain, Sarah Eno) to produce additional rejection factors of 2 to 5 depending on the threshold. Operating input rates are foreseen to be in the 500 to 1,000 Hz range. Processing Time --------------- Existing constraints in the acquisition system already set an upper limit of 250 microseconds on the Level 1.5 processing time. This limit would apply to any Level 1.5 Trigger System and is the maximum time the acquisition can wait while a beam crossing is processed in a Level 1.5 Cycle. For this reason a timeout is built into the Level 1.5 Trigger Framework that automatically accepts events that couldn't be processed within 200 microseconds. Page 9 Another important factor affecting the processing time available to a Level 1.5 Calorimeter Trigger is the incurred deadtime. All beam crossings occuring during a Level 1.5 Cycle are lost. Moreover this deadtime is in addition to (for the most part) the 5-10% deadtime inherent to the operation of the acquisition system during normal data taking. The design goal is for a total budget of 100 microsecond processing time. When applied to a 500 Hz input rate the Level 1.5 Calorimeter Trigger will yield a 5 % deadtime. Neighbor range -------------- The class of electron or jet algorithms envisioned all involve energy sums of the EM and/or Tot (=EM+Had) Trigger Tower signals within plus or minus two Trigger Tower indices of the candidate tower. This is the minimum neighbor range that must be made available to a processing element. Process on request ------------------ The Level 1.5 Calorimeter Trigger does not need to evaluate every beam crossing, but instead must be ready to process a given beam crossing when needed. Why choose a DSP? ================= Main Challenge of the project ----------------------------- The technical challenge in designing a Level 1.5 Calorimeter Trigger System is not in generating the code that ultimately computes the energy sums. The challenge isn't so much in applying the selected algorithm as it is in collecting and making the appropriate data available to a CPU. A wide variety of fairly common processor chips may easily have the necessary speed to execute the required operations, but the data must be supplied to the processor in fast access memory that can be read at full CPU instruction speed (typically on the order of 50 ns access time). The data needed by the Level 1.5 Calorimeter Trigger is spread over 10 racks of Level 1 Calorimeter Trigger electronics. This data must be collected and brought close to (inside) the processor. Another part of this challenge is to accomplish this data transfer without wasting a prohibitive fraction of the time budget before actual processing has even started. The DSP C40 ----------- Various options were available which involved different Processor families, third party computer board products, and the need to build various amounts of additional custom hardware. Without discussing all the options that were examined, this document concentrates on the selected processor and architecture and reviews the important advantages and characteristics. Page 10 The processor we chose is the C40 DSP (Digital Signal Processor) of the TMS320CXX family from Texas Instruments. The C40 is a 20 MHz (i.e 50 ns instruction cycle) 32 bit DSP microprocessor. The C40 also offers 6 communication ports with FIFO buffers and a very flexible DMA engine. Data can be transferred through one of these communication ports from the outside world or between C40 chips at 50 ns per byte. DSPs form a special class of processors that are optimized for fast data manipulation and high throughput. Unlike general purpose processors (Motorola 680X0, Intel 80X86...) they are not particularly suited for supporting an operating system (very limited support for subroutine calling, context switching, priorities, system security, code protection, etc). Conventional applications of DSPs are found in the treatment of audio, video, or tramission signal (compression/decompression, numeric filtering, FFT...). DSP architectures tend to emphasize data access and data transfer using multiple internal busses for instruction and data access. Instruction pipelines are kept short and all instructions are performed in single CPU cycles (i.e. no microcode). Some instructions offer multiple memory access in one CPU cycle (e.g. to get two operands). The different basic blocks of the processor operate independently and special instructions are available to make various combinations of these processing blocks operate in parallel, effectively executing multiple operations per cycle. The Hydra II ------------ We selected a commercial VME processor board. The Hydra II board, manufactured by Ariel, includes four C40 chips, with VME slave and VSB master interfaces (Fig. 3). Most importantly it makes available to the user 4 of the 6 communication ports of each C40, while using the other 2 ports for a ring connection between the C40 chips. Each C40 has some private memory and has access to a shared region that is also visible from the VME bus. The VME region can be used to download executable code and parameters. VME accessible registers can also be used to cause interrupts on any processor on the board. Page 12 Implementation ============== Calorimeter Trigger Tower Energies ---------------------------------- The Calorimeter Trigger Towers are 0.2 in azimuthal (phi) dimension by 0.2 in pseudo-rapidity (eta) yielding a segmentation of 32 towers in phi and 20 towers in both the positive and negative regions of eta, for a total of 2*20*32 = 1280 Trigger Towers. Each Trigger Tower is divided into an EM and a Hadronic section. The Trigger Towers are given a phi index in the range 1..32 and an eta index in the range -20..-1,+1..+20 (Fig. 4) The Level 1 Calorimeter Trigger coverage is spread over 10 racks of Calorimeter Trigger electronics with one rack covering all 32 phi indices for a block of 4 eta indices of either positive or negative value (1..4, 5..8, 9..12, 13..16, 17..20). A hook for future expansion of the Calorimeter Trigger was included in the design of the Calorimeter Trigger Front-End cards (CTFE). At the time, in 1988, the thoughts of a future engine were for a hardware cluster finder, and we chose the 9-bit sum Tot Et (=EM Et+Hadronic Et) signal, after energy lookup. The 1280 Tot Et signals are driven on spare pins at the back of the 320 CTFE cards. The Level 1.5 Calorimeter Trigger can use Tot Et information but getting the EM signal by itself is essential. The 9-bit Tot Et output being hardwired as the sum of the EM PROM output and the Had PROM output means that in order to get EM Et alone the Had Et contribution must be forced to zero. To accomplish this, we were able to use the "double lookup" ability in a different manner than (and beyond) what it was originally intended for (cf. D0 note #706). The second lookup was never used for triggering at D-Zero. The only global Level 1 Calorimeter Trigger energy quantities used for triggering the experiment are the Global Tot Et (Scalar Et) and the Missing Et, both of which are built during the first lookup. The second lookup was thus available for obtaining EM Et alone on the Tot Et port of the CTFE cards. Page 14 Parallel processing ------------------- The L1.5 Calorimeter Trigger processing steps are: 1) collect all Trigger Tower energies from the L1 Calorimeter Trigger, 2) sequentially scan all 1280 Trigger Towers looking for candidates, 3) use an algorithm to confirm some of the selected candidates, 4) combine the results and form a decision based on a global analysis While the final step must regroup and process all information in a single processor, the transferring, scanning and filtering steps can be split across several processors operating in parallel for an increase in performance. Candidate processing is distributed among 11 DSP chips called the Local DSP's, while the final answer is computed by a separate DSP called the Global DSP using the Confirmed Objects found by all the Local DSP's. The natural granularity of one rack (4 eta indices) already gains a factor of ten, which is suitable. This however does not mean that bringing one rack of data to one processor each would be sufficient. Since the confirmation algorithm involves sums that include neighboring towers, additional data for the adjacent Trigger Towers must be provided to each processor. Two racks of data are transferred to each DSP thus providing 8 consecutive eta indices. This enables each DSP to fully process the center 4 etas while always being able to construct sums involving neighbor towers plus or minus two eta or phi indices away from the candidate tower. Summarizing (Fig 5), each Local DSP receives two consecutive eta racks of Trigger Tower data (e.g. rack +1..+4 with +5..+8 or racks -20..-17 with -16..-13) and fully processes the Trigger Towers plus and minus two eta indices around the boundary between the two racks (i.e. +2..+5 or -18..-15 in the above example). This also implies that each rack of data is sent to two DSP chips. Avoiding special cases ---------------------- End effects for the racks covering eta + or - 17..20 are handled by creating fake data for non-existing racks with no energy content. Avoiding the creation of a special case means avoiding special software that would have to be tested separately. Similarly the eta -2,-1,+1,+2 Local DSP does not require any special treatment. The same basic code runs on all Local DSP's. Software constants incorporate all eta dependencies for data analysis, and all processor location dependencies for data transfer. Separate executables are linked for each DSP. Page 16 Data Path --------- The CTFE data is collected from the back of the Level 1 Calorimeter Trigger racks. There are 320 CTFE cards driving four 9 bit quantities each. The prospect of driving a mass of cables from the CTFE cards to a centralized point that would still have to perform the demultiplexing of the EM Et and Tot Et data, plus a remultiplexing to get in the communication port of the C40 Local DSPs was not very appealing. The data collection and serialization is instead distributed to each rack. Demultiplexing and data buffering is also distributed at the CTFE level, with the use of the Energy Readout Paddle Board (ERPB), designed and built at the University of California at Irvine. Each ERPB handles four CTFE cards (Fig 6). An ERPB is plugged in the back of and directly receives the data from one CTFE card. The ERPB provides 3 more sets of connectors to readout 3 more neighboring CTFE cards. Each rack of Level 1 Calorimeter Trigger has thus 8 ERPB cards to handle its 32 CTFE cards. The data serialization to fit the C40 communication port protocol is handled by one Distributor Cap (DC) card in each rack. The Distributor Cap was also designed and built at UC Irvine. This card controls and reads out all the ERPB cards in the rack. The physical distance between the DC cards and the C40 boards of the Level 1.5 Calorimeter Trigger and the single ended CMOS logic levels made it impossible to directly drive the C40 ports. The DC output is carried over long differential ECL lines and received, deskewed, translated and buffered on Cable Receiver Cards (CRC) that are kept physically close to the C40 hardware and can drive the necessary two communication ports per Level 1 rack. L1.5 CT input data ------------------ Each Local DSP gets four byte packed blocks (with four eta values in each 32 bit word). Each Local DSP receives one block of EM Et data and one block of Tot Et data from each of the two racks of Trigger Towers that need to be sent to it. The EM Et data is the raw 8 bit quantity as it appears in the Level 1 Data Block (see D0 Note #967), but with a different zero energy response that is eta dependent (see table below). The energy scale is 0.25 GeV per count. There is no low energy suppression. The Tot Et data is the 9-bit sum of the EM Et and Had Et, truncated down to the least significant 8 bits. It has a zero energy response different from the EM energies and that is also eta dependent (see table below). The energy scale is still 0.25 GeV per count. The Tot Et data includes a low energy cut to suppress the contribution of towers with low energy. The low energy cutoff is the maximum of 0.5 GeV of Transverse Energy and 2.5 sigmas of noise. This cut is bilateral (i.e. also applied to "negative" energies). The noise contribution to the cut overrides the systematic 1/2 GeV cut only at low eta values (eta index less than 4). The minimal 0.5 GeV cut means that any EM or Had transverse energy of one count is reported as zero energy. More information is available in section XIII of D0$LEVEL1:L1SIM.DOC. Page 18 Both EM and Tot Et energy counts given to the Local DSP's properly saturate at 255 counts (i.e. 60 GeV or less depending on the zero energy response). The situation where the 9-bit energy sum overflows the 8-bit maximum is correctly detected and treated to prevent roll over by the ERPB cards before being sent to the Local DSP's. No vertex correction is applied to either EM Et or Tot Et. This assumes that no vertex correction is used in the Level 1 Calorimeter Trigger, as is the case now. Using z-correction in the Level 1 Calorimeter Trigger would affect the performance of, but not prevent using the Level 1.5 Calorimeter Trigger. The Zero energy response of the EM and Tot Et energy counts seen by the Local DSP processors is given in the following table. Eta 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 EM offset 16 16 17 18 19 18 20 20 21 21 21 20 20 22 22 22 22 22 22 22 TOt offset 16 16 18 20 22 22 24 24 26 26 26 26 26 28 28 28 28 28 28 22 L1.5 CT as a Front-End crate ---------------------------- The Level 1.5 Calorimeter Trigger is mostly contained in a VME crate in rack M124 of the first floor of the moving counting house. This rack is situated across the aisle from the Level 1 Calorimeter Trigger racks. There are actually two VME crates installed in rack M124, allowing addition of a second L1.5 Calorimeter Trigger that could work independently on separate algorithms (to run a different set of electron algorithms, or jet algorithms). There is also a small cardfile holding the two CRC cards and an MTG (Master Timing Generator) card. Each crate of Level 1.5 Calorimeter Trigger receives a Trigger Acquisition Synchronization Cable. This gives each crate a separate Start Digitize and Front-End Busy signals. Each crate also receives programming information that allows it to recognize when it needs to process an event. The crate can start processing an event without special intervention from the Level 1 and Level 1.5 Trigger Frameworks. The system must also behave as a regular Front-End and make its information available for readout on the trigger data cable (i.e. data cable #0). In particular this Front-End must follow and obey the full Trigger-Acquisition protocol described in D0 Note #827. Each crate must send some data for every event, including events for which the crate didn't do any processing. For events where the Level 1.5 Calorimeter Trigger did not run its algorithm, it still sends a block of data while flagging it as non relevant. The Level 1.5 Calorimeter Trigger data is included in every event, and can be found in the TRGR bank with crate ID 81. This data is available to the Level 2 Processing nodes. A separate D0 note will describe the Level 1.5 Crate Data Block Format. On the online cluster read the file USER1:[TRGMGR.D0_TEXT.LEVEL_15.CALTRIG]L15CT_DATA_BLOCK_FORMAT.TXT. Page 19 Conditions for running the L1.5 CT ---------------------------------- For decision making purposes, the Level 1.5 Calorimeter Trigger only needs to run for beam crossings where all of the following conditions were met: - At least one Specific Trigger fired - All of the Specific Triggers that fired required level 1.5 Confirmation - At least one of the Specific Triggers that fired needs Level 1.5 Calorimeter Trigger Confirmation. But, in order to make processing in Level 2 easier, the decision was made to design the Level 1.5 Calorimeter Trigger to run any time that at least one of the Specific Triggers that needs Level 1.5 Confirmation fires. In consequence, the Level 1.5 Calorimeter Trigger will sometimes run for events that do not need confirmation. During this time, the system is not immediately available to start processing another event that would need Level 1.5 Calorimeter Trigger confirmation, should there be a request. This design choice was made at the expense of additional Front-End Busy and Level 1.5 Deadtime. Frame Code versus Tool Code --------------------------- The code needed to operate the Level 1.5 Calorimeter Trigger is separated into two classes, Tool Code and Frame Code. The code that implements the algorithm and confirms or rejects objects is called the Tool Code and is kept as isolated and interchangeable as possible. The distinction is also made between Local Tool and Global Tool code. All DSP code, including the Tool Code, is written in assembly language. The Frame Code includes everything that is not directly involved in confirming, or counting objects. In particular for Local DSP's the frame code is responsible for receiving new data, scanning and looking for candidates, calling the Tool Code, sending List of Confirmed Objects or additional data to the Global DSP. The Global DSP Frame Code includes receiving the Local Lists of Confirmed Objects, making the answer available to the 68k (see architecture and control below), and transferring data for readout. The 68k code completely falls in the class of Frame Code, and includes control of the system, readout, and implementation of the mark and pass ability (cf. below). Multiple Terms -------------- The first implementation will only allow a single algorithm to execute. But provision was made in the syntax and structure of the system to allow returning multiple terms to the Level 1.5 Framework. Allowing multiple terms will increase the complexity of the frame code, and increase the processing time, thus the deadtime. A serious study will be needed before deciding whether this option should be pursued. Multiple Tools would be able to coexist in the Local and Global DSP's. The job of the Level 1.5 Calorimeter Trigger is to return Level 1.5 Terms for input to the Level 1.5 Framework. The choice would be available to connect any Term with any of the available Local and Global Tools (i.e. algorithms), and a set of Local and Global Parameters. Several terms may share the same Tool Code, while using different parameters. One Term may only use one Local and one Global Tool. A separate D0 note will describe Tool calling and argument passing. Page 20 Mark and Pass events -------------------- In a manner similar to the Level 2 System, a mark and pass ability was given to the Level 1.5 Calorimeter Trigger. A counter, called the "pass one of N" counter, is associated with each Level 1.5 Calorimeter Trigger Term. Whenever one of the counters reaches its programmed limit, the Level 1.5 Calorimeter Trigger performs its normal task, but also immediately reports all its requirements as met in an effort to have the event readout. The Level 1.5 Calorimeter Trigger flags its data as mark and pass and also provides additional information about the event. The additional data may include rejected candidates, intermediate results, and a complete dump of all the energy inputs it received, and all the thresholds that were applied. These mark and pass events can be selected online for monitoring, and collecting statistics. These events will also be run through a simulator and the output of the simulator and the real data will be compared. This will verify that the simulator and the DSP code are performing the intended algorithm. It will also provide an early detection and warning of hardware problems. These mark and pass events will also be saved and written to a separate stream, to remain accessible without having to rescan the whole data set. These events will constitute a good data sample to study the behavior and performance of the running system. They will also provide a bench test for studying new algorithms. Special attention should be brought to the fact that the Level 1.5 Calorimeter does not have the means to guarantee that an event will be accepted, and that it can only make every effort towards this, but never be certain of the result. It is not the Level 1.5 Calorimeter Trigger that confirms an event; this task is under the sole control of the Level 1.5 Framework. The Level 1.5 Calorimeter Trigger can only generate physics information, for example "yes, I believe there was at least one electron in this event", but it is the Level 1.5 Framework that combines this piece of information with the rest of the triggering information to decide whether this makes the event pass all its requirements. However, while the Level 1.5 Calorimeter Trigger cannot guarantee that a particular event is confirmed, it always learns the outcome. When a Level 1.5 Calorimeter Trigger Term is due for a mark and pass event, but the Level 1.5 Calorimeter Trigger notices that its attempt at passing the event has failed, it will extend its effort to mark and force pass the next event that needs this Level 1.5 Calorimeter Trigger Term evaluated. Crate architecture and control ============================== Three DSP boards ---------------- The 11 Local DSPs are situated on three VME Hydra II boards. One Hydra II board has four C40 DSP processors. The boards are labelled A, B and C. The DSP chips are numbered 1,2,3 and 4 on each board. Each Local DSP uses two of its six Communication ports to connect to its closest on board C40 neighbors. The other four communication ports receive the four blocks of data, EM ET and Tot Et for two racks (Fig 7). Page 22 One of the Hydra II boards also holds the Global DSP (Board B, DSP#2). Each of the other two boards hold a DSP that handles the end of the Trigger Tower eta coverage (A2 and C2) and thus only receives one rack worth of data, using only two ports. One of the other two available ports of these DSP's are used to connect these two Hydra II boards to the Global DSP on the third board. The Global DSP uses two of its six communication ports to connect to its closest on-board Local DSP C40 neighbors. Two other ports connect to the other two Hydra II boards. To reach the Global DSP the Local DSP data must cross between 1 and 3 DSP communication links depending on the Local DSP position. Data is received by the Local DSP's which look for candidates. Each Local DSP applies the algorithm to its candidates to find the Confirmed Objects. All the partial Confirmed Objects Lists are sent to the Global DSP which produces the final answer. One 68k CPU board ----------------- A Level 1.5 Calorimeter Trigger Crate also has one MVME-135 board. This is a 68k VME CPU board with a VSB interface. The two functions performed by this processor are Engine Control and Readout Control. The Engine Control function is to initiate, monitor and reset all DSP's for every event that is processed. The Local DSP's automatically receive fresh data without 68k intervention, and even without explicit DSP involvement, since the DMA transfer was prepared in advance. The Local DSP's only start looking for the arrival of their fresh data after they receive the wake up message from the 68k. This message includes the list of terms that need to be evaluated, and whether this is a mark and pass event. Engine Control also collects the answer from the Global DSP and tells the Global DSP whether to transfer data for readout (Fig 8). Engine Control ends the processing cycle by returning all DSP's to their initialized state, ready for the next event. Engine Control also includes detecting problems, and error recovery. When some processing stage hasn't completed successfully the Engine Control is able to halt and reset the whole system to the state where it is ready for the next event. The Readout Control function interfaces the Level 1.5 Calorimeter Trigger system with the Level 1 and Level 1.5 Frameworks on one side, and the crate readout to the Level 2 on the other side. Page 24 Parallel IO cards and support paddle boards ------------------------------------------- A VME bus cycle takes about 500 nanoseconds, and the 68k has little chance if it relies solely on its software to notice in real time activities that only last a couple of microseconds. Instead, Engine and Readout Control are assisted by four small paddle boards that plug into the back of the crate, right behind some Ironics Parallel IO VME boards. These paddle boards hold some connectors, receiver and driver circuitry, and a small amount of supporting logic. The first hardware support function is to detect Level 1 Trigger Decisions and capture all synchronization signals. Some circuitry detects a Start Digitize signal on the Geographic Section, and captures the TAS (Trigger Acquisition Synchronization) Number. It also automatically asserts the Front-End Busy signal. The Front-End Busy signal can be cleared later on by explicit 68k intervention, once the 68k has had time to notice the activity, and after it finds out if a dummy readout, or full processing is necessary. The second hardware support function is to find out quickly if a processing cycle is necessary, i.e. if any of the Specific Triggers involved in the Level 1 Trigger Decision were programmed with requirements from this Level 1.5 Calorimeter Trigger Crate. The Term Select Paddle board receives the list of which Specific Triggers fired on a dedicated cable and uses a memory lookup to directly list which Terms need to be evaluated. The 68k needs this list to notify all DSP's as soon as possible so that they can be ready to process the Trigger Tower information they are currently receiving, or possibly may have already received. The third hardware support function is to capture, follow and remember the evolution of the TAS protocol while the 68k and DSP's are working on an event. When the 68k finally holds the answer that should be sent to the Level 1.5 Framework, it must find out if the Level 1.5 Framework is still expecting it. Moreover, before reading out data for a given event the 68k must check that the event was not rejected for other reasons while its processing was taking place. This is accomplished with some dedicated hardware watching the TAS signals and remembering the conclusion of the protocol. Page 25 Two multiport memory boards --------------------------- The Level 1.5 Calorimeter Trigger data block is collected in two MVME-214 memory boards. An MVME-214 is a dual port VME/VSB memory card. The bulk of the L1.5 Calorimeter Crate data is written by the Global DSP while the header and some fixed data are managed by the 68k. The Global DSP and the 68k both access the MVME-214 module over the VSB bus. There are two dual port memory modules so that one can always be available for writing the next event via the VSB bus while the other has been handed over for readout of the previous event via the VME Bus. The MVME-214 boards have been modified to include a mode control to allow the 68k to swap which card is being written to and which card is being read. One VBD board ------------- Each Level 1.5 Calorimeter crate has its own VBD card for readout over the Trigger Data Cable #0 (TRGR). The first Crate ID is 81 (in decimal), while 91 has been reserved for the second. The VBD board is controlled by the 68k. The 68k initializes and monitors the VBD. It initiates data transfer into the VBD buffers and directs which sections of 214 memory space needs to be read out. Page 26 Split VME backplane ------------------- The crate backplane is made up of two separate sections (Fig 9) to avoid interference in VME accesses between the VBD reading out the previous event and the 68k trying to control the system and communicating with the DSP boards and parallel IO cards for the current event. The two sections of backplane are linked together by a pair of Vertical Interconnect cards. The 68k can thus control the VBD card using VME addresses mapped through the Vertical Interconnect cards. The Readout Section of the backplane only holds one Vertical Interconnect Slave, the two VME 214 modules, Short IO Memory, and the VBD (with a slot reserved for a data cable repeater if necessary). The Processor Section of the backplane holds one Vertical Interconnect Master linked to the Readout Section of the backplane, and a Vertical Interconnect Slave linked to the Trigger Control Computer (see below). The Processor Section of the backplane also has the three Hydra II DSP boards, the 68k CPU module, and the four Ironics IO modules with their paddle boards. VSB Bus ------- The VSB bus connects the two Dual Port Memory Modules in the Readout Section of VME backplane to the 68k CPU module and the Hydra II board Number B that holds the Global DSP on the Processor Section of the backplane. The VSB bus is used to load data into the MVME-214 Memory Modules for readout by the VBD card. TCC Programming --------------- The Trigger Control Computer (TCC) that already controls and monitors the Level 1 and 1.5 Frameworks and the Level 1 Calorimeter Trigger also loads the executable and configures the Level 1.5 Calorimeter Trigger. The Trigger Control Computer is a VAXStation 4000 Model 60 that uses a pVBA adapter from ZRL to communicate with a small VME backplane. Short cables connect this VME backplane to the CPU enclosure. The small VME backplane only contains the pVBA receiving module and a Vertical Interconnect Master which is able to support the longer link to reach the Level 1.5 Calorimeter Trigger VME Crates. This data path allows TCC to read and write in the VME space of the Level 1.5 Calorimeter Trigger Crates. Using only VME write cycles, TCC is able to download new code in all DSP's and cause interrupts to any DSP or to the 68k. This ability is used to download the executables and send the various parameters to all DSP's and to the 68k. After downloading new code and new parameters, TCC instructs all DSP's and the 68k to examine their parameters. Each DSP and the 68k report back to TCC that they were able to read and check their parameters, and are now ready for the first event. TCC also programs the Term Select Paddle board for the lookup of which Level 1.5 Calorimeter Trigger Term is required by which Specific Trigger. Page 28 The system is programmed by COOR, the Coordination program for the D-Zero experiment. COOR uses text input files to describe all available resources (including the Level 1.5 Calorimeter Trigger), and to describe the desired programming configuration for the whole experiment. COOR translates and transmits to TCC the requirements related to the Level 1 and 1.5 Frameworks, or to the Level 1 and 1.5 Calorimeter Triggers. The specification and details of the messages sent by COOR to TCC are available in D0$LEVEL1:COOR_TCC_MESSAGES.DOC, while the new messages related to the Level 1.5 Calorimeter Trigger are available in appendix B. Timing performance ================== Several aspects must be considered when looking at the timing performance of the Level 1.5 Calorimeter Trigger. The most critical is the response time for sending an answer back to the Level 1.5 Framework. Another critical component is the amount of time that the system keeps its Front-End Busy signal asserted after a Level 1 Trigger Decision that didn't concern it, or after returning a decision to the Level 1.5 Framework. The less critical aspect is the readout time, or more precisely the time until the information is made available to the VBD card for readout. Answer time ----------- The answer delay is what generates global Level 1.5 deadtime. Assuming that the Level 1.5 Calorimeter Trigger is successful at filtering out false electron signals, it will spend most of its time rejecting events. Rejection time is thus the quantity to focus on. The overall sequencing of all the steps needed to operate the system is carefully arranged to place the minimum number of actions in the critical path towards an answer. The answer is returned as quickly as possible, even if further processing is still needed before the system can be ready to process the next event. The time needed for additional processing will be longer for events that are accepted than events that are rejected. But with an input rate below 1000 Hz, 1 millisecond is typically available after sending the answer to the Level 1.5 Framework and before the next solicitation. The 68k also tries to make best use of its time. Some actions can be initiated while checking for their completion is not immediately required (e.g. start the VBD on transferring data) and the 68k can move on to the following step. On the other hand, when the 68k is idle, waiting for the next solicitation or waiting on one of the longer steps of the DSP processing, it can try preparing or executing some steps that are not yet critical, but will be required later (e.g. make sure a buffer is available for readout). Trying to accomplish these steps ahead of time might be successful or might fail, and the 68k keeps track of what work was already done and what it wasn't able to do because of lack of time, or because it was still too early for this step (e.g. resource not available). No experimental results are yet available, but the test measurements and estimates are predicting an average answer time in the 80-100 microsecond range (cf. appendix A). Page 29 Front-End Busy -------------- The time the system keeps the Front-End Busy asserted is also important because it contributes to the acquisition deadtime, and to the overall experiment deadtime. While provision has been made for being able to add event buffering later on, no explicit double buffering will be installed in the first implementation phase. Double buffering is the ability to accept the request for a second event while processing a first event, or in other words, receiving two requests close together before asserting a Front-End Busy signal. By definition, no additional Level 1 Trigger Decision can be issued during a Level 1.5 Cycle, and no double buffering is needed during that time. As soon as an answer is sent back the Level 1.5 Trigger Framework, or if the Level 1.5 Framework decided not to wait for the answer from the Level 1.5 Calorimeter Trigger, a second Level 1 Trigger request can be generated while the Level 1.5 Calorimeter Trigger is still processing a previous event. At the moment, the Level 1.5 Calorimeter Trigger will not allow (by asserting its Front-End Busy signal) a second Level 1 Trigger Decision during all of its DSP processing. This constraint will cause additional Acquisition deadtime, but will greatly simplify the system while it is being brought online. This limitation, and others, can be evaluated and addressed one at a time later on. Page 30 Future improvements =================== Possible hardware improvements ------------------------------ The first implementation of the system was strongly influenced by time constraints, and had to be installed after run Ib had already started in January 94. We pushed the design and froze the specifications in November 1993 and made some compromises and simplifications for the sake of coming online as early as possible (April 1994) with a reliable system. Once the system has been fully debugged, is running and performs its promised triggering service, a number of areas can then be explored. Areas of study will include performance and flexibility. By then, we will be able to measure the actual performance of the system. We will also have data to study and evaluate the relative importance of various aspects. We can judge of the impact of some of the limitations and concentrate our efforts in the most useful areas. The response time is the direct cause of the Level 1.5 acquisition deadtime and may be improved by reviewing some of the design choices. For example we could reconsider the length of the list of Confirmed Objects passed by each Local DSP. This length is fixed and at was generously set at 8 entries which makes it contribute to about 30 % of the response time. It is expected that the first implementation will bring additional deadtime to the acquisition. Some of the deadtime will certainly be hidden by other systems that will be busy at the same time as the level 1.5 calorimeter Trigger. But additional deadtime will occur when the Level 1.5 Calorimeter Trigger keeps its Front-End Busy signal asserted while all other systems were done and had one of their double buffers free to take the next event. Improving the Level 1.5 Calorimeter Trigger acquisition deadtime could include enabling additional buffering in the ERPB cards, at the expense of a greater complexity of the signals to control the ERPB's. Another separate approach could involve additional support logic on the support paddle boards in the back of the Level 1.5 Calorimeter Crate. Industry trend -------------- Unlike most of the custom hardware that has historically been built for hardware triggers in High Energy Physics experiments (and all other components of the D-Zero Hardware Trigger), the Level 1.5 Calorimeter Trigger essentially relies on a commercial processor, and commercial boards. We were able to use standard parts in an application for which they were designed and optimized. This project is clearly following and benefiting from industry trends. DSP applications are a very active sector of the electronics, and computer industry. Besides not having to build as much hardware, we were able and will continue to be able to benefit from the developments and the work done for industrial applications. For example, a new member of the Texas Instrument TSM320 DSP family was recently announced. The C80 has a clock speed double the one of the C40 and has 4 processors embedded on the same chip. Once this part is available in quantity, and third party vendors like Ariel use it in new products, we will be able to upgrade our system with limited effort. Page 31 Innovative Usage ---------------- This Level 1.5 Calorimeter Trigger is now used as an electron filter but offers broad possibilities. The system is a very flexible software platform that is open to a lot of improvements in the field of electron filtering, or new and innovative algorithms. The system can also be duplicated to run several independent crates of Level 1.5 Calorimeter Trigger. Multiple crates can operate in parallel, but independently, and on the same or on different events, while applying different algorithms or requirements. One simple application would be to use a second crate as a jet filter. The Local DSP's get the Tot (EM + Had) Et of complete 5 by 5 Trigger Towers regions (i.e 1.0 in eta by 1.0 in phi) around every Level 1 Trigger Towers, and could locate and properly measure the energy deposit in each jet. Miscellaneous improvements could include rejection of duplicate entries at the Local or Global DSP Tool level to avoid counting electrons or jets multiple times. Fancier algorithms in Global Tools could make angular or mass requirements on pairs of objects (di-electrons, electron-jet, etc). In general terms, one can use the Global Tool to look for any desired correlation between objects found by the Local Tool. More ambitious upgrades could involve importing and correlating information from other detectors (Muon Detector, Central Detector) over more DSP communication ports. Page 32 Appendix A Level 1.5 Calorimeter Trigger Timing Estimates Data Algorithms Control ------ ------------ --------- | | Find Start Digitize | Level 1 Calorimeter 10us | Test SpTrg Fired 15 | Trigger sends | Mask us | Trigger Tower | Tell Local DSP's Go | Energies | Tell Global DSP Go V........................... . | . 25us | Local DSP's . | Find Level 1 . Keep watching the | Candidates . TAS protocol V . for an early 5us | Local DSP's . conclusion each | run 1x2:3x3 . of the L1.5 Cycle. | algorithm . | on each . | candidate . ............................V . Watch | Local DSP's send lists . the Local DSP's 35 | of confirmed objects . and Global DSP us | to Global DSP . perform their task . . but intervene only . . in case of problem | Global DSP . | has all its data . V........................... . | Global DSP . 5 | runs Global . us | Algorithm . | and produces . | L1.5 Cal Trig Terms . V......................... | Read Global DSP 2.5us | results and | transmit to | L1.5 Framework | if necessary Level 1.5 Framework .............V Accept or Reject Event | (e.g. for 2 candidates | Watch TAS protocol Total Time ~ 90 us) 1-4us | to find out if ************************ | event needs | transfer ....................................................V | | 150 | Global DSP 4us | Write Status us | copies L1.5 for | in VME 214 | results to transfer | Setup VBD | Dual Port only | Start VBD | Memory (only V | if readout | | is required) 2us | Toggle Double Buffer | | Clear Front-End Busy | V | . Wait for next | . L1.5 Cycle and | . perform housekeeping V V tasks Page 33 Appendix B New COOR to TCC message for the Level 1.5 Calorimeter Trigger -------------------------------------------------------------------------------- This is an extension to D0$LEVEL1:COOR_TCC_MESSAGES.DOC (after LIBTEST LEVEL1) Updated 13-JAN-1994 13. LEVEL 1.5 CALORIMETER TRIGGER PROGRAMMING a) Definitions b) L1.5 Cal Trig Term Reference Sets L15CTERM REFSET c) L1.5 Cal Trig Term Local DSP Tool Definition L15CTERM LOC_DSP d) L1.5 Cal Trig Term Global DSP Tool Definition L15CTERM GLOB_DSP e) L1.5 Cal Trig Term Frame Code Parameters L15CTERM FRAMECOD f) Specific Trigger versus L1.5 Cal Trig Term L15CTERM ST_VS_TM g) Start L1.5 Cal Trig System L15CTSYS START h) Load L1.5 Cal Trig executables L15CTSYS LOADCODE -------------------------------------------------------------------------------- 13. LEVEL 1.5 CALORIMETER TRIGGER PROGRAMMING a) Definitions The messages necessary to program the Level 1.5 Calorimeter Trigger are devided in two classes, reflected by two separate message types: L1.5 CT Term requirement and programming (L15CTERM messages) L1.5 CT system and program control (L15CTSYS messages) The L1.5 Cal Trig Term contributes to the overall Level 1 and 1.5 Triggering System in the form of Level 1.5 Trigger Terms. Each Crate of L1.5 Calorimeter Trigger can provide up to 8 Level 1.5 Terms to the Level 1.5 Trigger Framework. The requirements programmed for each L1.5 CT Term can be classified in four categories, reflected as four separate message subtypes. Each one is further described in the operation summary below. Reference set for selection of Trigger Tower Candidates Local DSP Tool and parameter selection Global DSP Tool and parameter selection Frame Code services The syntax and definition are made here to allow the full flexibility of 8 independent L1.5 CT Term per L1.5 CT Crate, each having independent Reference sets, Local and Global DSP Tools (i.e. algorithms) and parameters. In contrast, it is important to note that the initial implementation of the L1.5 CT system will only allow a single L1.5 CT Term with a fixed Local and Global Algorithm, with variable parameters. In each message, a particular Level 1.5 CT Term is specified by its L1.5 CT Crate Number, and its relative Term number in the crate. The first crate that will be implemented will be crate number 0. A second crate might be added to the system in the future, and would become crate number 1. The particular term in the crate is identified by its relative number in the crate with a number from 0 to 7. Note that this L1.5 CT number is most likely NOT the same as the Level 1.5 Framework Term number that will receive the L1.5 CT Answer from this L1.5 CT Term. COOR will need a table to describe the correspondence between L1.5 CT Crate and Term Numbers and which Level 1.5 Framework Term number they are connected to. Page 34 Understanding the details of the messages described below requires some understanding of the operation and the structure of the Level 1.5 Calorimeter Trigger. One way to summarise the work of the Level 1.5 Calorimeter is as a Level 1.5 Trigger Subsystem that can confirm (or reject) one or more Candidate Trigger Towers as a valid electron signal. Note that the same L1.5 Cal Trig (or a second crate) can also be used to work on confirming or rejecting jet candidates. The system structure also allows more complex requirements to be made on the list of confirmed objects. The system has been designed with enough flexibility to allow these different options without hardware modifications. When a beam crossing requires a Level 1.5 computation, the energy of all Trigger Towers is sent to the Level 1.5 CT. The Transverse EM Energy and the Tot (=EM+HAD) Transverse Energy of each Trigger Tower are sent to the L1.5 CT. The first task of the L1.5 CT is to identify the list of Trigger Towers that are candidates for each L1.5 CT Term. There is one independent such list for each of the L1.5 CT Terms. The Candidate Trigger Towers are identified by applying a Reference Set to the energy of each Trigger Tower. This can be done to the EM or TOT enery depending on the type of object sought. The energies in each Reference Set can be set uniformly over the whole detector, but can also be arbitrarily specified differently for any subset of Trigger Towers. Note also that these L1.5 CalTrig Reference Sets are independent from the L1 CalTrig Reference Sets, even though a logical programming for a L1.5 CT Ref Set will duplicate the L1 CT Ref Set that was used to select the event sent to the L1.5 CT. The second task of the L1.5 CT is to submit the candidates for each L1.5 CT Term to the Tool algorithm and Parameters chosen for the L1.5 CT Term. This part of the L1.5 CT computation is called the Local Algorithm and is executed by the Local DSP's. Each of the 11 Local DSP's only receives enough Trigger Tower Energy information to process the Trigger Towers that it is responsible for. The algorithm is designed to look in the energy deposited in the candidate tower and its neighbors to recognize the type of object sought (electrons or jets) and accept or reject the candidate as a legitimate signal. The Candidate Trigger Towers that satisfy a L1.5 CT Term Local DSP Tool are then called Confirmed Objects. The third task of the L1.5 CT is to collect all the Confirmed Objects and form a final answer for each L1.5 CT Term. This part of the L1.5 CT computation is called the Global Algorithm and is executed by the Global DSP. The Global DSP is the only processor that collects information covering the whole detector. It only gets the list of confirmed objects from the Local DSP's, and does not have access to the Trigger Tower Energies. The trivial case for a Global Algorithm is to simply verify that at least one of the candidates has survived to become a Confirmed Object. The last task of the L1.5 CT in this summarized view is performed by the Frame Code and allows overriding a L1.5 CT Term Answer for a variable fraction of the events. This mechanism forces a positive answer for one of every n events, and thus provides a monitoring sample of events (that would have otherwise been rejected) to be passed to the acquisition system. Note that the L1.5 CT cannot really force an event to be accepted. It is the Level 1.5 Framework that makes the final decision. This decision might include a contribution from other Level 1.5 Trigger Subsystems (e.g. L1.5 muon Trigger) that could reject the event. When a pass_one_of_n counter rolls over for any of the L1.5 CT Terms, all L1.5 CT Term Answers are forced high for that event in order to increase the chances of passing the event. Page 35 b) L1.5 Cal Trig Term Reference Sets 01234567 L15CTERM REFSET CRATE(0) TERM(2) TYPE(EM) SIGN_ETA(POS NEG) MAGN_ETA(1:20) PHI(1:32) THRESH(10000) This message specifies the Reference Set (or a portion of it) that needs to be associated with a particular L1.5 CT Term. The L1.5 CT Term is identified by its crate ID specified as an integer (0 or 1) inside the pair of parenthesis following the keyword "CRATE". The L1.5 CT Term is further identified by the relative term number within the crate specified as an integer (from 0 to 7) inside the pair of parenthesis following the keyword "TERM". The reference Set must be further described by its type "EM" or "TOT" specified inside the pair of parenthesis following the keyword "TYPE". This type is used to select whether the reference set energies must be applied to the EM Et or Tot Et Trigger Tower Energies. The set of Trigger Towers for which this message programs the reference energy is specified using the same syntax as for the Level 1 Trigger Tower Reference Set messagess (cf. paragraph 9.f for details). It might take more than one such message to describe a non uniform reference set. When several such messages are needed to describe the full Reference Set for one L1.5 CT Term, the reference set type must agree in all the messages. The energy value to be programmed is specified inside the pair of parenthesis following the keyword "THRESH". It is specified as an integer in units of MeV of z-corrected Transverse Energies. The Trigger Control Computer will perform the translation to the appropriate quantified value that will be inclusive of the specified value. A given message can only refer to one and only one L1.5 CT Term. The keywords CRATE, TERM and TYPE, and their values must all be present, and in this order, in each message. They must appear first, and before the Trigger Tower Range and Threshold value. More than one Trigger Tower Range and corresponding Threshold can be specified in a single message (like for the Level 1 Trigger Tower Reference Set messages), in which case the CRATE, TERM and TYPE are not repeated. This message only causes TCC to translate and re-format the specified information. TCC also loads it in dual port VME memories accessible to the L1.5 CT Processors. The acknowledgment message returned by TCC to COOR for this message only testifies to the success of understanding the request, and to the success of writing the dual-ported VME memory in the L1.5 CT Crate. In particular, the L1.5 CT is still idle at this time, and does not try accessing or using this data until the START message defined in 13.g below. c) L1.5 Cal Trig Term Local DSP Tool Definition 01234567 L15CTERM LOC_DSP CRATE(0) TERM(2) USE_TOOL(657) WITH_PARAMS(1 2.00) This message specifies which Local DSP Tool algorithm needs to be associated with a particular L1.5 CT Term and which set of parameters need to be used with the Local algorithm for this L1.5 CT Term. The L1.5 CT Term is identified by its crate ID specified as an integer (0 or 1) inside the pair of parenthesis following the keyword "CRATE". The L1.5 CT Term is further identified by the relative term number within the crate specified as an integer (from 0 to 7) inside the pair of parenthesis following the keyword "TERM". Page 36 The Tool is identified by its unique Tool ID specified as an integer inside the pair of parenthesis following the keyword "USE_TOOL". The list of parameters needed by the Tool to operate is specified inside the pair of parenthesis following the keyword "WITH_PARAMS". The same Tool ID may be used for more than one L1.5 CT Term. There is only one Tool allowed per L1.5 CT Term. Parameter definitions are Tool dependent. Different Tools might have a different number and type of parameters. The exact number and type of parameters that are defined for a tool must all be given a value. There is no provision for default values. Successive parameters are separated by one or more blank character (space). The only 2 types of parameter allowed are integer and real numbers. TCC will perform the binary translation to integer or real format appropriate to the DSP architecture, but TCC has no knowledge of which parameter should be integer or real numbers. Thus a real number must always include a decimal point to denote its type, even if the real number value needs no decimal digits (i.e. "2.0" is preferred, but "2." is ok, while "2" is not appropriate for a parameter of type real). Integer or real numbers may take negative values. There may or may not be one or more blank characters separating the negative sign ("-") from the aboslute value. Real numbers cannot use the exponential form and must all be expressed using only the characters "0" through "9", "." and "-". A given message can only refer to one and only one L1.5 CT Term. The keywords CRATE, TERM, and USE_TOOL, and their values must all be present, and in this order, in each message. They must appear first, and before the list of parameters. This message only causes TCC to translate and re-format the specified information. TCC also loads it in dual port VME memories accessible to the L1.5 CT Processors. The acknowledgment message returned by TCC to COOR for this message only testifies to the success of understanding the request, and to the success of writing the dual-ported VME memory in the L1.5 CT Crate. In particular, the L1.5 CT is still idle at this time, and does not try accessing or using this data until the START message defined in 13.g below. d) L1.5 Cal Trig Term Global DSP Tool Definition 01234567 L15CTERM GLOB_DSP CRATE(0) TERM(2) USE_TOOL(645) WITH_PARAMS(5 6.00) This message specifies which Global DSP Tool algorithm needs to be associated with a particular L1.5 CT Term and which set of parameters need to be used with the Global algorithm for this L1.5 CT Term. The message syntax and parameter constraints are exactly the same as for the Local DSP Tool Definition (cf. paragraph 13.c) e) L1.5 Cal Trig Term Frame Code Parameters 01234567 L15CTERM FRAMECOD CRATE(0) TERM(2) PASS_ONE_OF(300) This message specifies the Frame Code requirements that need to be associated with a particular L1.5 CT Term. The L1.5 CT Term is identified by its crate ID specified as an integer (0 or 1) inside the pair of parenthesis following the keyword "CRATE". The L1.5 CT Term is further identified by the relative term number within the crate specified as an integer (from 0 to 7) inside the pair of parenthesis following the keyword "TERM". Page 37 The mark and pass ratio is specified as an integer (greater than or equal to 1) inside the pair of parenthesis following the keyword "PASS_ONE_OF". The L1.5 CT will keep updating a counter for each event that required the computation of this L1.5 CT Term. There is one separate counter per L1.5 CT Term. When any of the pass_one_of_n counters reaches its defined value, all L1.5 CT Term Answers are forced high for that event (in order to maximize the chances of passing the event). A given message can only refer to one and only one L1.5 CT Term. The keywords CRATE, and TERM, and their values must all be present, and in this order, in each message. They must appear first, and before the pass one of n requirement. This message only causes TCC to translate and re-format the specified information. TCC also loads it in dual port VME memories accessible to the L1.5 CT Processors. The acknowledgment message returned by TCC to COOR for this message only testifies to the success of understanding the request, and to the success of writing the dual-ported VME memory in the L1.5 CT Crate. In particular, the L1.5 CT is still idle at this time, and does not try accessing or using this data until the START message defined in 13.g below. f) Specific Trigger versus L1.5 Cal Trig Term 01234567 L15CTERM ST_VS_TM CRATE(0) TERM(2) SPTRG(1 2 5 14) This message specifies the set of Specific Triggers that require the computation of a particular L1.5 CT Term. The L1.5 CT Crate will use this information to determine which (if any) of its L1.5 CT Terms need computation when a particular set of L1.5 Specific Trigger fired. The L1.5 CT Term is identified by its crate ID specified as an integer (0 or 1) inside the pair of parenthesis following the keyword "CRATE". The L1.5 CT Term is further identified by the relative term number within the crate specified as an integer (from 0 to 7) inside the pair of parenthesis following the keyword "TERM". When more than one L1.5 CT Terms are defined, the L1.5 CT Crate will combine all the corresponding lists to find out if any of its L1.5 CT Terms are needed, and decide which one(s) need evaluation. A given Specific Trigger may appear in lists for more than one L1.5 CT Term. The list of Specific Triggers is specified inside the pair of parenthesis following the keyword "SPTRG". One or more Specific Trigger may be specified, each separated by one or more blank character (space). All unspecified Specific Triggers cannot cause the specified L1.5 CT Term to be evaluated. This command does not accept negative numbers. A given message can only refer to one and only one L1.5 CT Term. The keywords, and their values, must all be present in each message. The keywords CRATE, and TERM, and their values must all be present, and in this order, in each message. They must appear first, and before the list of Specific Triggers. This message only causes TCC to translate and re-format the specified information. TCC also loads it in dual port VME memories accessible to the L1.5 CT Processors. The acknowledgment message returned by TCC to COOR for this message only testifies to the success of understanding the request, and to the success of writing the dual-ported VME memory in the L1.5 CT Crate. In particular, the L1.5 CT is still idle at this time, and does not try accessing or using this data until the START message defined in 13.g below. Page 38 g) Start L1.5 Cal Trig System 01234567 L15CTSYS START CRATE(0) The programming of the L1.5 CT system cannot be modified once the system is running. The proper sequence of events is for COOR to load the L1.5 CT executable (cf. message below). Then COOR can define all the L1.5 CT Terms with their reference sets, tools and parameters. TCC prepares the data and requirements passed by COOR and loads the DSP and 68k memory. During this time the L1.5 CT is waiting, and the L1 framework should be PAUSEd. COOR must use this message to ask the L1.5 CT to read and check all the parameters loaded by TCC and report on the success of this initialization. The actions taken by the L1.5 CT at this time are defined in detail in other documents, but this phase, initiated by this START message from COOR, includes verifying the existence of the requested Tools, and making the Tools verify that the parameters passed for each L1.5 CT Term are consistent with each corresponding Tool. If this phase was successful, the L1.5 CT Crate is now ready for the first event. Whenever the programming needs to be modified, COOR needs to start by reloadingthe code (cf. message below), then define all the L1.5 CT Terms again, and restart the crate. This is still true when one L1.5 CT Term was defined and the crate was started, and COOR needs to add the definitions for another Term. This message only affects one L1.5 CT Crate at a time specified as an integer (0 or 1) inside the pair of parenthesis following the keyword "CRATE". A given message can only refer to one L1.5 CT Crate. h) Load L1.5 Cal Trig executables 01234567 L15CTSYS LOADCODE CRATE(0) host_directory_name This message asks TCC to reload the executables of all Local DSP's, Global DSP and 68k Engine Control code. The files follow some fixed naming convention to be defined later (but internal to TCC and L1.5 CT system). COOR only needs to specify the directory where TCC will find the various files. The directory must be a valid directory name, and reside on the D0 cluster. Logical names may be used, as long as they are defined system-wide on the D0 cluster. This message only affects one L1.5 CT Crate at a time specified as an integer (0 or 1) inside the pair of parenthesis following the keyword "CRATE". A given message can only refer to one L1.5 CT Crate. Page 39 Appendix C BIBLIOGRAPHY Here is a list of the documents mentioned in this D0 Note, and other documents related to the Level 1.5 Calorimeter Trigger. D0 Note 328 D0 Trigger Framework, M. Abolins, D. Edmunds, J. Linnemann, February 24, 1986. D0 Note 705 The Level One Framework: D0 Note 328 Revised, M. Abolins, D. Edmunds, P. Laurens, J. Linnemann, B. Pi, May 26, 1988. D0 Note 706 The Level One Calorimeter Trigger for D0, M. Abolins, D. Edmunds, P. Laurens, J. Linnemann, B. Pi, October 25, 1988 D0 Note 967 First Level Trigger Data Block Description, P. Laurens, Rev A. May 4, 1990, Rev B. June 25, 1992. D0 Note 1638 The Level 1.5 Trigger Framework for D0, D. Edmunds, S. Gross, P. Laurens, April 12, 1992. D0$LEVEL1 (use $ LIBTEST LEVEL1 for latest version) COOR_TCC_MESSAGES.DOC Description of the Communication Standard between COOR and the Trigger Control Software, B.Gibbard, D.Edmunds, P.Laurens. USER1:[TRGMGR.D0_TEXT.LEVEL_15.CALTRIG] on the D0 online cluster L15CT_DATA_BLOCK_FORMAT.TXT Description of the Level 1.5 Calorimeter Trigger Crate readout format. L15CT_TOOL_CALLING.TXT Description of the L1.5 CT Tool calling convention