The following are notes from a meeting with Myron Campbell of U of M on August 11, 1997. I. Alpha-in-VME status ---------------------- 1. The drawings for the prototype Alpha-in-VME board are nearly done. A prototype board is expected to arrive at U of M in October. 2. The Alpha will have a floppy, disk, keyboard, mouse, and printer port. 3. The code for the flash PAL is in C, and thus can be altered. The intention is to alter it so that it can be booted on command [by setting a register? by VME interrupt?] rather than only on power-cycling. 4. The PC164 tech manual EC-QPFYA-TE is available on the web, as is the PC164 Motherboard Digital Unix manual, at www.digital.com/semiconductor/alpha/dsc-pc164mb.html 5. The prototype will include the PCI/VME interface (the Universe chip) and the PCI/MBus interface for MBus write cycles to Alpha memory. The latter exists in prototype and is under test; it was redesigned in Xylinx. It should be possible to run without wait states, at full PCI speed. 6. The prototype will not include either the FRED port lines, or the ability to execute MB write cycles from the alpha. This will be prototyped as a PCI card. 7. MSU now has two copies of the current drawings for the alpha card, and a copy of the transparencies from the CDF L2 design review. 8. It was agreed to add functionality to the prototype before construction of generation of an interrupt in the case of a) EV_DONE on MBUS and b) DMA FIFO empty. [Question: should this be the on-board FIFO_EMPTY for a particular processor, or the MBus FIFO_EMPTY which is the AND of all processors?] 9. Since the Alpha Board will have the ability to become a MBus master, it needs arbitration logic, probably on the present prototype, since that logic can't be easily contained on a PCI card. 10. MSU would appreciate some written documentation from a programming point of view on the prototype Alpha board, eg how to go about setting up the PCI/MB bridge, where in address space the relevant registers are, etc. 11. The PCI/MBus Interface has a separate Alpha memory address counter for each of the 256 MBus broadcast addresses; each counter must be reset by the Alpha to the base memory address before a new transfer to that MBus broadcast address. Transfers may be interleaved among MBus broadcast addresses, but this is discouraged because it generates extra PCI memory cycles. All alphas must accept data from all MB broadcast addresses. II. Prototype status -------------------- 1. The Universe VME/PCI interface is fully tested. 2. The testing of the PCI/MB DMA is nearly complete. 3. In both cases, two cards were made but only one populated. The testing was at the level of functional correctness, not at the level of performance. If MSU requests, the other cards could be populated. The other suggested mode of operation was to have MSU people visit U of M and perform throughput tests there, as this is much less demanding in setting up test rigs. 4. Within CDF, many of the preprocessors are to be done at U of M. The tracking may be split among Pisa and UCLA, or go solely to UCLA. V. Magic Bus ------------ 1. The MBus specification is essentially frozen. MBus backplane drawings are complete and ready to send out for manufacture. MSU is to send quickly a request for how many are desired. MSU has a 3-page document, but the document does not match the MBus lines on the alpha board drawings; for example the MOD_DONE lines do not appear on MSU's documentation. MSU would like an updated document reflecting the present design. MSU would like 6 MBus backplanes made for D0 (two 2-crate setups for a final system and a test rig, plus 2 spares). Below are some clarifications/understandings of the MBus spec discussed at the meeting. 2. MBus "Write" or "Read" cycles are defined from the point of view of the bus master. 3. Whenever a source relinquishes mastership, a full arbitration ensues, allowing sources "upstream" on the bus to obtain mastership even though the previous master was "downstream". Sources contributing to an event can contribute in any order. They are allowed to relinquish and then re-request mastership in a single event, though this is discouraged from a performance point of view because it results in additional PCI address cycles. 4. There is no interaction of "EV_LOADED" and arbitration/mastership issues. In particular, if an Alpha card is "upstream", it can become master and initiate MBus cycles after an "EV_LOADED" has been issued but before "START_LOAD" has been issued. 5. Source location on the MBus is primarily an order-of-arbitration issue. There is no fixed relation between location and the MBus address that a source may choose to put on MBus. In particular, a MBus source card at a particular slot may choose to send data corresponding to several Broadcast Addresses. Cards with multiple data sources may allow arbitration between sources; MSU intends to do so. IV. Additional Functionality ----------------------------- 1. Additional functionality on the Alpha board will appear first as a PCI board prototype; the present Alpha board will not be held up. Jerry Huang will start work soon on the PCI prototype. A target time for a design review, pending schedule of those involved, is the week of September 8. Another possibility is the week of September 15. 2. The Fred timing port will be moved to PCI directly instead of hanging of ISA to avoid long latency associated with ISA address decoding. It is proposed to have either 64 bits of bidirectional I/O lines, or 32 bits of each direction. Either solution is felt to be adequate for D0 purposes. A latency of less than 64 ns for setting or reading of a line is highly desirable, as these lines may be used for scaling at 132ns period of internal states, and measuring latency of occurrences which should last less than 500 ns. 3. The other critical piece of functionality is the ability of the Alpha to communicate with specific MBus addresses (not just broadcast mode). This was viewed by U of M as a facility to support testing, but is a vital part of the normal functioning of the processor from MSU's intended application. The proposed implementation is to make this path naturally 64 bits wide instead of the full 128 bits of MBus. However, an auxiliary register will be provided, which will be sent along in the data transfer, so that the register could be either passively sent along to fill out the 128 bits, or actively filled with information, as the application requires. The requirements from the MSU point of view are a. Arbitration: the Alpha card now can be a MB master, so must arbitrate. b. Mapping: the window of addresses mapped to MBus address space on the particular card is settable. The window is [xxxxx bits] of address space. c. Alpha must be able to initiate a MBus write cycle to an arbitrary MBus address. Given the auxiliary register mechanism, the write may effectively be 64 bits or 128 bits. d. Alphas must be able to respond to a MBus write cycle to a non-broadcast address within that card's window of addresses by writing to Alpha Memory via PCI. The implementation may choose to only respond by taking 64 bits of data during the cycle, ignoring the other 64 bits on MBus, [or better, capturing the additional 64 bits an auxiliary register, overwriting if multiple reads occur...] e. Performance: From MSU's point of view, the performance requirements are as follows: i) bus arbitration takes less than 10 ns per occupied MBus slot. i) be able to send 256 bits (32B) of information to a MBus address (not on an Alpha card) in 0.5 micro sec after bus mastership has been granted. (10 KHz; "fail" reply to L2 framework via MBT card) ii) be able to send 100B messages to a MBus address (not on an Alpha card) in less than 1.5 micro sec after mastership. (20 KHz; L2 Cal preprocessor shipping data to L2Global via MBT card) iii) be able to send (or read) 16 B of information between two Alphas less than .5 micro sec after bus mastership. (20-40 KHz; Admin and Worker handshaking on decision on failed event) These requirements range from 20-70MB/s of data transfer. If the transfers ran at full speed PCI non-block transfer speed for for short transfers (8B address cycle, 8B data cycle for each item), corresponding to half-width transfers on MBus at full speed, that would give a safety factor of 2, since MBus at full speed is 160MB/s running at half-width. It would be desirable to have the additional functionality for Alpha boards: f. Alpha can initiate a MBus read cycle. 64 bits of information would flow via PCI to Alpha memory, the other 64 bits being ignored or captured in an auxiliary buffer g. Alpha can respond to a MBus read cycle pointed at its window of addresses by presenting the 64 bits of data at the mapped memory location in addition to whatever information was presented in an auxiliary 64 bit register. Finally, there is a question of PCI bus contention. If the PCI/MBus interface is busy transferring an event, how does this impact the latency of a request for an Alpha to set/get a Fred port line? How does it interact with with a request to initiate or respond to a non-broadcast MBus read or write cycle? MSU would probably prefer that it be possible to arrange priorities so that Fred > MBus write > Mbus non-broadcast read > MBus broadcast write, or at least to not suffer the full latency of reading an input broadcast event while trying to do I/O to Fred or other MBus devices from the Alpha. V. MB Sources ------------- 1. U of M plans for an input card for MBus sources at this point lean towards Cypress Hotlinks in fiber form. Channel Link is the other contender, but appears to have considerable noise susceptibility. HP g-Link was not recommended for new designs, also on noise-susceptibility grounds, by HP. The g-link protocol only has sign-reversal on a successive-byte basis, so there is nothing to forbid getting 10 0's in a row on the serial link. The new versions will have the standard 8bits in 10 bit frame encoding (as Cypress has) which force balance of on/off bits within a 10 bit frame, giving better noise immunity. Also the new chipsets will have direct TTL interface.