CMX Review: Richard's comments > Design entry: While I don't completely agree with the > sentiments given on page-7 about avoiding schematic entry > for this type of design, I'm interested to see how this > goes. Do you have a scheme for running any > design-rule-checks on the final net-list being submitted > to layout, such as detecting single-pin nets (usually due > to typos)? I first got addicted to this type of design entry when laying out backplanes. That's an easy case to picture where the "intellectual content" of the design is more clearly expressed in a table than in a schematic. Other examples of why I like this text based design entry are: - What's the point in drawing hundreds of bypass capacitors ? One can just list these parts and their connections. - What's the point in drawing a box and adding 431 ground symbols to it for the ground pins on a Virtex-6 part ? This is more clearly done in a text table where you can just pull the pin number content directly out of the Xilinx documentation. For this example see the file: http://www.pa.msu.edu/hep/atlas/l1calo/cmx/hardware/net_lists/Base_Fpga_Power/bf_fpga_ff1759_ground_n2p.txt - Lots of contemporary schematics do not show much (any) of the actual connectivity - they just show boxes with named nets that go to some other page. The interesting bit, the connectivity, is not explicitly shown. It's left to the reader of the schematic to build up an understanding of the connectivity in their head. With text based design entry one at least has space and a format where enough comments can be added to explain the circuit. - In this text based entry, as in most schematic capture programs, we can write hierarchical designs, e.g. if there are 16 channels doing the same thing you only describe the circuit once and then give the rules for incrementing some numeric fields in the reference designators and net names. - Searching for net names, reference designators, and pin numbers is easier and more robust in one text file than it is hunting around through many pages of pdf file. The design rule checks start by putting the overall net list in alpha-numeric order sorted by the number of connections a given net makes. From the snap shot of the CMX design that is presented in this review you can see this sorted net list on the web at: http://www.pa.msu.edu/hep/atlas/l1calo/cmx/hardware/net_lists/Build_CMX_0_Net_List/cmx_0_overall_net_list.txt.nls From here you can make all of the tests of the type, I the designer know that there should be 16 nets with a net name like Data_Bus_Line_xy and that each one should connect to 5 components. So I can test: Are all 16 of these nets showing up in the overall design and do they all visit 5 components ? We have all of the normal net list checks, e.g. a given pin on a given device can belong to only one net, reference designators must be unique... To continue this design entry heresy we have embraced single point nets. For example we enter all unused pins as single point nets with fixed format unique net names like No_Conn_U53_Pin_12. In this way for a typo to slip into the design we must have two counter balancing mistakes. I also draw representative schematics of all important parts of the design. These schematics are hopefully drawn in way to help humans understand what the circuit is doing. They are not drawn in a way to just mechanically extract a net list. These drawings are in the CMX Review document and on the web at: http://www.pa.msu.edu/hep/atlas/l1calo/cmx/hardware/drawings/circuit_diagrams/ Try text based design entry on something like a backplane and you may get addicted to it. It's much like moving FPGA logic design entry from schematic to VHDL text. > CAN bus monitoring: Do you intend passing any of the > monitored current values to the CAN/DCS system, such as > that of the FPGA core supply current? CAN-Bus monitoring is one of the points that I do not understand. I flagged this as a point that needs to be cleared up in my list of topics for the CMX Review. Your guidance on how we should handle this is strongly welcome. Current status: - With only 8 direct analog inputs to the CAN-Bus micro- processor I have used all 8 of them to monitor power bus Voltages on the CMX. This covers all of the basic supply voltages on the CMX. - To pickup Current monitoring via CAN-Bus we could add an external analog mux (that is controlled by the CAN-Bus micro-processor ?) but how much software work is that to re-write the micro-processor code and is the path that carries this monitoring information up to higher levels ready for more channels ? - How important is the accuracy of this stuff ? Should I use a precision reference for the micro-processor's ADC ? I think that I see some cards with just bulk 5V used as the ADC reference. - What are the on card connections from VME to the CAN-Bus micro-processor for ? What connections are required or wanted ? Should there be none so that no matter what the state of CMX - its CAN-Bus monitoring will always be working because its an independent path that will allow us to see the power supply Voltages and some Temperatures. - As currently designed, we can read out the various power supply Currents via VME from the System Monitor ADC in the Virtex-6 parts. This current monitoring is important because we may not have a good understanding of what the FPGA core supply currents will be. > Test-points: Are there any oscilloscope test-points for > say supply voltages and currents, and VME-- strobes, > writes and buffer enables for debugging use? Also any > test-points on clocks and a few select data lines for > measuring latency? These are clearly important features. The CMX design currently has: - We have the 40 pin connector J13 to allow monitoring of all power supply Voltages and Currents. Each sample pin has an adjacent GND pin to help provide some oscilloscope characterization of these supplies. Another purpose of J13 is to allow us to plug in one cable that goes to a rotor-switch and then to a good DVM so that we can easily and accurately measure the 9 Voltages and 7 power supply Currents that are visible on J13. This gives us a way to confirm the accuracy of the CAN-Bus and Virtex System Monitor readouts. - The 40.08 and the 320.64 MHz clocks that are distributed on the CMX card have explicit monitor points. These monitor points make use of otherwise unused channels in the 2 clock fanout chips. Except for trace routing delay these clock monitor points are in time with the clocks that are delivered to the FPGAs. - For digital testing J14 provides 10 signals from each of the 3 FPGAs on the CMX and includes multiple ground pins. This connector is not setup for direct plug into a multi-channel logic analyzer pod because we also want to be able to use J14 for emergency connections between FPGAs if necessary. - We have not included any test points on the 400 backplane signals because we want these lines to be as clean as possible. Thus, if it does not work it will be hard to see anything to find out why. The "wisdom" of this choice can be discussed. - We currently do not have scope probe-tip type coax jacks to allow careful observation of noise on the GTX analog power planes. Instead we have been extra careful about the filtering of the GTX power. The "wisdom" of this choice can be discussed. > Backplane 160 Mbps signals: Could you specify, or > reference to, the functional requirements between the > incoming backplane clock and data signals. ( ie are clock > and data transitions aligned, and on which clock edge > begins the data frame? ) I believe that making these 400 backplane inputs work 100% reliably with 6 nsec signals is the risky part of this project. Philippe and I are not directly involved with the Base Function firmware so the logic to ingest these 400 data lines using the clocks from the 16 processor cards is not something we have focused on. Our only focus has been the fidelity of these traces and routing these signals to the correct pins on the FPGA. It's my understanding that these signals are received in 3 steps: - The individual data lines from a given processor are delayed in the FPGA I/O Block so that they line up correctly with the clock signal from that processor. - The 4 transfers from a given processor that make up one beam crossing's worth of information are assembled into one long word of 96 bits. - When, for a given beam crossing, all of the processors have delivered their 96 bits of data and this has been assembled into 16 words of 96 bits each then all 16 of these words are synchronously transferred into the logic in the Base Function FPGA that processes this data at a 40.08 MHz rate. If there is not a document that defines a common timing scheme that the sources and receiver ends of this data path are working towards then we should talk about this during the CMX Review. Documentation on this may go back to the demonstrator card for the 160 MHz backplane signalling. Additional Comments to Richard's Questions: ------------------------------------------- From Richard in reply to CAN-Bus Monitoring > I have some answers to your CAN questions, but others > know the CAN uC better than me: > > Accuracy: I'd recommend using a precision reference for > the ADC. Using the bulk 5V for this was a mistake on the CPM. > On our later CAM module I added a 4V reference which seems > to work ok: > > http://epweb2.ph.bham.ac.uk/user/staley/CAM/CAM_CAN_schematics.pdf > > as an example. > > Current: There is some code in the CAN uC project for > controlling a multiplexer, if its decided that currents need > to be monitored by DCS. The CPM has a 4:1 analogue mux to > sample three 'current' voltages, but that still needs one > ADC input pin you don't have spare. Anyway, yes it's good > to have the current values available via VME. > > VME-CAN Link: This 8-bit(?) bus was there for checking the > state of the CAN uC via VME, but I'm not sure we ever used > it. Likewise for the VME_Reset to the CAN uC, an option > never used. The set-up became that CAN is run independent > of VME. I see you've noticed the non standard I/O levels > of this device. We had to use 'HCT devices, or similar, > for translating to a full 5V swing for the uC inputs. > TTL levels just didnt work. From Richard in reply to backplane 160 Mbps signal timing > As for the backplane timing, I'll prepare a slide for > next week's joint meeting should we want to discuss > this further. From Wojtek in reply to 160 MHz backplane signals > The scheme is briefly documented in the CMX firmware spec > document in the review docs directory -it certainly deserves > a separate short document that we ought to provide. I have > been assuming that edges of 80 MHz clock are aligned to the > center of the data word and not to the data transition > time. The iodelay circuits are implemented in the firmware > to adjust the relative alignment of the signals if they are > found to be somewhat off from this ideal (Sam points out > that these may not be needed). Also as mentioned in the > firmware document if the timing requirements are adverse > e.g. the data window is constrained to be narrow or jitter > large it seems beneficial to 'push' the clock wrt the center > of data by a small amount (0.5 - 1 ns). This is also > mentioned in the fw spec document. > > As far as framing the 4 words and discovering which sequence > forms an event the source modules need to provide a > 'training' pattern (this needs to happen only once after > start of the run/powercycle) as we discussed in the > hw/firmware meetings (and as documented in the firmware > writeup). This pattern needs to be impossible in physics > mode and we have to agree when it is sent in the course if > the run. > > I hope this adds some info that was missing... let me know > if anything is unclear. From Sam in reply to 160 MHz backplane signals > This is interesting to know. Sending the 80 MHz data clock > with the edges in the center of the 160 Mbit/s data words > means the CPM and JEM would need to produce it with a 160 > MHz internal clock. I had otherwise been under the > impression that the clock edges would aligned with the data > word edges, and that standard DDR input timing would be > sufficient. > > What has been foreseen for the JEM and CPM? From Uli in reply to 160 MHz backplane signals > ... so far 80MHz DDR has been assumed... > regarding the relative timing of the 400 input lines of > the CMX, Bruno has measured the trace length variation > for the 25 lines each to the mergers on the JEMs: > > SUM merger lines: 180mm max. , 171mm min > JET merger lines: 338mm max. , 288mm min