Topology of TCC access to VME Space for communication to L1 Framework and L1 Calorimeter Trigger ----------------------------------- Updated: 28-Jun-01 =================================== ======================== H L1 Trigger H H L2 Trigger _ H H Control Computer H H Control Computer |B| H H _ L1 TCC H H L2 TCC |i| H H |B| d0tcc1.fnal.gov H H d0tcc2.fnal.gov |t| H H |i| in M125 H H in M125 |3| H H |t| H H Windows NT | | H H |3| H H Intel CPU .+. H H | | H H PCI-Bus | H H |6| H ====================|=== H |1| Windows NT H | H |7| Intel CPU H +----------------|--+ H .+. PCI-Bus H |PCI Expansion | | H | H |Bit3 Model SS32 | | ===|=============================== | _ in _ .+. | | | |B| M125 |B| |B| | | | |i| |i| |i| | | | |t| |t| |t| | +--|-------------------------------------+ | |3|......|3| |3| | | | VME Communication Crate in M124 | | | | up | | | | | | .+. _ _ _ _ _ | | |6| to |6| | | | | |B| |I| |V| |V| |V| |V| | | |1| 7x |1| | | | | |i| |r| |e| |e| |e| |e| | | |8| 618s |8| | | | | |t| |o| |r| |r| |r| |r| | | .+. .+. ._. | | |3| |n| |t| |t| |t| |t| | +--|--------|-------+ | | | |i| | | | | | | | | | |Optical | Tot 6x | |6| |c| |M| |M| |M| |M| | |Links .__ L2 | |1| |s| |a| |a| |a| |a| | ._______ Crates | |7| | | |s| |s| |s| |s| | | | | | |I| |t| |t| |t| |t| | +-|--------+ | | | |O| |#| |#| |#| |#| | |.+. L2 | | | | | | |0| |1| |2| |3| | ||B| Crate | | ._. .+. .+. .+. .+. .+. | ||i| | | | | | | | 6U VME32| ||t| e.g. | +-----------|------|---|---|---|---------+ ||3| L2GBL,| | | | | | || | L2CAL,| | | | | | ||6| etc | ___________. | | | | ||1| | | | | | | ||8| | | | | | | |._.9U-VIPA| | | | | .__________________ +----------+ | | | | | | | | | | | _____________. | ._____________ .___________ | | | | | | | | | | | | #=============# | #=============# | #=============# | #=============# | | H M122 H | H M123 H | H M124 H | H M101 H | | H L1 & L2 FW H | H L1 FW H | H L1 FW H | H L1 CT H | | H H | H H | H H | H H | | H +---------+ H | H +---------+ H | H +---------+ H | H +---------+ H | | H |._.L1 FW | H | H |._.L1 FW | H | H |._. SCL | H | H |._.L1 CT | H | \____|V|Crate | H \____|V|Crate | H \____|V| Hub | H | ___|V| Read-| H | | H ||e|(For- | H | H ||e|(TRM | H | H ||e| End | H | H ||e| Out | H | | H ||r| eign | H | H ||r| & | H | H ||r| | H \_H ||r| Crate| H | | H ||t| Per | H | H ||t| AONM)| H | H ||t| | H | H ||t| | H | | H || | Bunch| H | H || | | H | H || | | H | H || | | H | | H ||S| Scal-| H | H ||S| | H | H ||S| | H | H ||S| | H | | H ||l| ers) | H | H ||l| | H | H ||l| | H | H ||l| | H | | H ||v| | H | H ||v| | H | H ||v| | H | H ||v| | H | | H ||#|Custom| H | H ||#|Custom| H | H ||#| VIPA| H | H ||#| VIPA| H | | H ||0|9U VME| H | H ||0|9U VME| H | H ||0|9U VME| H | H ||0|9U VME| H | | H |._.A24D16| H | H |._.A24D16| H | H |._. | H | H |._. | H | | H +---------+ H | H +---------+ H | H +---------+ H | H +---------+ H | | H +---------+ H | H +---------+ H | H +---------+ H | H +---------+ H | | H |._.L1 FW | H | H |._.L1 FW | H | H |._.L1 FW | H | H |._.L1 CT | H | \____|V|Crate | H \____|V|Crate | H \____|V| Read-| H \____|V|Crate | H | | H ||e|(Expo | H | H ||e|(TDM | H H ||e| Out | H | H ||e|Bougie| H | | H ||r| Group| H | H ||r| & | H H ||r|Crate | H | H ||r|& CTRO| H | | H ||t| Per | H | H ||t| FOM) | H H ||t| | H | H ||t| | H | | H || | Bunch| H | H || | | H H || | | H | H || | | H | | H ||S| Scal-| H | H ||S| | H H ||S| | H | H ||S| | H | | H ||l| ers) | H | H ||l| | H H ||l| | H | H ||l| | H | | H ||v| | H | H ||v| | H H ||v| | H | H ||v| | H | | H ||#|Custom| H | H ||#|Custom| H H ||#| VIPA| H | H ||#|Custom| H | | H ||1|9U VME| H | H ||1|9U VME| H H ||1|9U VME| H | H ||1|9U VME| H | | H |._.A24D16| H | H |._.A24D16| H H |._. | H | H |._.A24D16| H | | H +---------+ H | H +---------+ H H +---------+ H | H +---------+ H | | H +---------+ H | H +---------+ H H H | H +---------+ H | | H |._.L2 FW | H | H |._.L1 FW | H H H | H |._.L1 | H | \____|V|Crate | H \____|V|Crate | H H H \____|V|Spare | H | H ||e| | H H ||e|(Disa-| H H +---------+ H H ||e|Crate | H | H ||r| | H H ||r| ble | H H | VME | H H ||r| | H | H ||t| | H H ||t| TRM | H H | Communi-| H H ||t| | H | H || | | H H || | & | H H | cation | H H || | | H | H ||S| | H H ||S| Scal-| H H | Crate | H H ||S| | H | H ||l| | H H ||l| ers) | H H | is | H H ||l| | H | H ||v| | H H ||v| | H H | actually| H H ||v| | H | H ||#|Custom| H H ||#|Custom| H H | located | H H ||#|Custom| H | H ||2|9U VME| H H ||2|9U VME| H H | Here | H H ||2|9U VME| H | H |._.A24D16| H H |._.A24D16| H H +---------+ H H |._.A24D16| H | H +---------+ H H +---------+ H H H H +---------+ H | #=============# #=============# #=============# #=============# | | | | #=============# #=============# #=============# #=============# | H M103 H H M104 H H M105 H H M112 H | H L1 CT H H L1 CT H H L1 CT H H L1 CT H | H H H H H H o o o o o H H | H +---------+ H H H H H H H | H |L1 CT | H H H H H H H | H |Bus Distr| H H H H H H H | H |Crate | H H | H |._______ | H H \____|Comint || H H H ||-------|| H H H || BBB |_____to M103..M106 middle backplane H H ||-------|_____to M103..M106 lower backplane H H || (Tot |_____to M107..M110 middle backplane H H || 7x |_____to M107..M110 lower backplane H H || BBB's)|_____to M111, M112 mid & low backpl H H ||-------|_____to M105, M109, M111, M107 upper backplane H H || BBB |_____to M103 upper backplane (i.e. itself), for MTG H H |.-------.| H H H +---------+ H H H +---------+ H H H |L1 CT | H H H |Eta +1:+4| H H H |Phi 1:16| H H H +---------+ H H H H H H H H +---------+ H H H H H H H H |L1 CT | H H H H H o o o o o H H H |Eta +1:+4| H H H H H H H H |Phi 17:32| H H H H H H H H +---------+ H H H H H H H #=============# #=============# #=============# #=============# -------------------------------------------------------------------------------- Access to the Level 1 Trigger Framework Crates (and L1 Calorimeter Trigger Readout Crates) ============================================== Reminder: --------- L1 Trigger Framework Crate Address Space Remember that inside each Framework crate, VME data access is A24/D16 only. The 24-bit VME address space is subdivided into Card, Chip, and Register Addresses as follows: (cf. the_card_vme_interface.txt) A A A A A A A A A A A A A A A A A A A A A A A A 2 2 2 2 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 ----------------------------------------------- | Card | Chip | Register 0 0 0| Address | Addres | Address Framework Rack Each Framework rack has (up to) 3 of our custom 9Ux400mm crates (L1FW or L2FW, Scalers) or 2 VIPA crates (VRB, SCL). The Run II L1 Calorimeter Trigger also uses one crate of the same type as the Run II L1 Framework, and one VIPA crate for VRB, as included in this description. Vertical Interconnect --------------------- Each crate has a Vertical Interconnect Slave Card in Slot 1 acting as the Crate Controller and providing remote access to a window of VME access space into the crate. There is no VME Master (i.e. capable of initiating VME cycles) in our crates other than the Vertical Slave. The Vertical Interconnect Slave is connected to a Vertical Interconnect Master located in our dedicated VME Communication Crate. VME Communication Crate ----------------------- The VME Communication Crate contains a number of Vertical Interconnect Master VME cards. Each remote VME crate has a Vertical Interconnect VME Slave card and is attached to one of the V.I. Master card by a custom serial cable. Each V.I. Master connects to up to 4 V.I. Slave cards. Given that each of our racks will hold (up to) three 9U VME crates, we can allocate one V.I. Master per rack with one spare V.I.Slave Channel per V.I. Master Card. Trigger Control Computer ------------------------ The VME Communication Crate is connected to the L1 Trigger Control Computer (L1 TCC) via a Bit 3 Model 617 PCI to VME Bus Adapter consisting of two modules and an 8 or 25 foot copper cable. One module plugs into the PCI bus inside L1 TCC, the other is a VME module inside our VME communication crate. The bit3 Model 617 VME module also acts as VME crate controller for our communication crate. User software inside the TCC can use the Bit3 device Driver to map several windows of its vitual address space to access windows of VME address space in the Communication Crate. L1 TCC Access to VME ----------------- When L1 TCC accesses a 16 bit word at an address mapped through the Model 617, a A32/D16 VME cycle is generated in the Communication Crate. If the VME address falls within the address space of one of the Vertical Interconnect Master cards, and within one of its V.I. Slave channels, a A24/D16 VME cycle is generated in the attached remote crate via its V.I. Slave. D16 is the only data size used in our custom trigger cards. But D8 and D32 data transfer size are also possible. A24 is the only access mode used in our custom trigger cards. But A16 access mode is also supported by the Vertical Interconnects in our trigger crates. Rack/Crate Address ------------------ Each V.I. Master card has a switch selectable base address. The switch selects the upper 6 of the 32 bit address lines (A26-A31). The next lower 2 Bits (A24-A25) select which of 4 V.I. Slave is accessed via this V.I Master. We can allocate 2 of the upper 6 V.I. Master Address Bits as a Rack address We also require the next most significant bit to be set in order to stay away from the lower addresses which are currently unused but are reserved for possible use of a CPU module in the VME Communication Crate. 5 2 1 5 2 1 5 2 1 1 5 2 6 3 1 1 5 2 6 3 1 1 5 2 6 3 1 4 2 1 2 6 8 4 2 6 8 4 2 1 2 6 8 4 2 6 8 4 2 1 2 6 8 4 2 6 8 4 2 G G G M M M M M M M M M M k k k k k k k k k k B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B -------+-------|-------+-------|-------+-------|-------+------- A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A 3 3 2 2 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 --------------------------------------------------------------- |Rck|Crt| | Card | Chip | Register 0 0 0 1|Num|Num|0 0 0| Address | Addres | Address \-/ \-/ | | | \--> Crate Number i.e. Vertical Interconnect Slave Address | 0 = Top Crate | 1 = Middle Crate | 2 = Bottom Crate | 3 = reserved | \------> Rack Number i.e. Vertical Interconnect Master Address 0 = Rack M122 (Per Bunch Scalers + L2FW) 1 = Rack M123 (L1FW + L2 Monit Scalers) 2 = Rack M124 (SCL + L1FW Readout) 3 = Rack M101 (L1CT Readout) Allocating 2 out of the possible 6 bits of V.I. Master Address space for a Rack Number provides for up to 4 Framework Racks while we currently only envision needing 3 racks to implement the L1 FW, L2 FW, Scalers, Serial Command Link Hub-End, and Fast Readout, plus 1 Rack for Calorimeter Trigger Readout. Hex Vertical Vertical Prefix Master # Slave # Crate Location and Function 0x10 0 0 Upper Crate of Rack M122 Foreign Per-Bunch Scalers 0x11 0 1 Middle Crate of Rack M122 Exposure Group Per-Bunch Scalers 0x12 0 2 Lower Crate of Rack M122 L2 Framework, Helper Functions 0x13 0 3 Reserved 0x14 1 0 Upper Crate of Rack M123 L1FW AOIT TRM's, And-Or, L1 Busy L1FW Global Disable TRM 0x15 1 1 Middle Crate of Rack M123 L1FW Trigger Decision L1FW Framework Output 0x16 1 2 Lower Crate of Rack M123 L1FW Individual SpTrg Disable L1FW Scalers, Tick&Turn Scaler 0x17 1 3 Reserved 0x18 2 0 Upper VIPA Crate of Rack M124 SCL Hub End 0x19 2 1 Lower VIPA Crate of Rack M124 VRB Readout of L1FW 0x1a 2 2 Reserved 0x1b 2 3 Reserved 0x1c 3 0 Upper Crate of Rack M101 VRB Readout of L1CT 0x1d 3 1 Middle Crate of Rack M101 L1CT Bougies and CTRO 0x1e 3 2 Lower Crate of Rack M101 L1CT Spare Rack 0x1f 3 3 Reserved -------------------------------------------------------------------------------- Access to the Level 1 Calorimeter Trigger Crates ================================================ The L1 Calorimeter Trigger is also controlled by L1 TCC. We use an Ironics Parallel IO VME Card in the VME Communication Crate, with 12 bytes of VME address space at base address 0xf020. The Ironics Card implements six 8-bit Parallel Ports which can be used either as Input or Output. The Ironics Card is used to drive a Run I Comint Card stripped of most of its Run I functionality, except for driving the L1 CT differential ECL Control Bus (CBus). The Cbus is the programming path to the Calorimeter Trigger. Only one CBUS is used on the Comint Card (CBus #2 is unused) to drive all 7 Bus Buffer Boards (BBB) in the CBus Distribution Crate which in turn drive all 25 Crates of the L1 Calorimeter Trigger. cf. http://www.pa.msu.edu/hep/d0/ftp/run1/l1/framework/cards/comint.txt The series terminated TTL signals from the Ironics IO Ports are connected via twist and flat cables (with every other signal grounded) to the Comint Card of the L1 CT CBus Distribution Crate which is the top crate of M103. A paddle board in the back of the Ironics card provides the series termination and the mapping of Ironics Ports to Comint Card Cables. cf. http://www.pa.msu.edu/hep/d0/ftp/l1/framework/hardware/pb_ironics_cbus When writing out to an Ironics Port, the complement of the desired output data needs to be written to the corresponding Port Data register, as the data will be inverted before it is driven off the card. There is no such inverting when reading in port data. Table of Ironics IO Port assignment VME Ironics Address Comint CBus Port Offset Card Signals 1 0x00 Port A 7:0 Mother Board Address 1:8 Ironics->Comint 2 0x02 Port B 15:8 Card Address 1:6 Strobe Direction Ironics->Comint 3 0x04 Port B 7:0 Function Address 1:8 Ironics->Comint 4 0x06 Port C 7:0 Write Data 1:8 Ironics->Comint 5 0x08 Port D 7:0 Read Data 1:8 Comint->Ironics 6 0x0a Port E 7:0 Unused -------------------------------------------------------------------------------- Access to the Level 2 Trigger Processor Crates ============================================== cf. www.pa.msu.edu/hep/d0/ftp/tcc/vme_access/tcc_to_l2_trigger_communication.txt