HAWC Time Stamp Generation ------------------------------ Original: 10-NOV-2009 Current: 13-NOV-2009 This note is a draft specification describing how an accurate time stamp will be included in the data recorded for each HAWC event. Goals: ------ A goal of this design is to use channels in a CAEN V1190 TDC module to introduce the time stamp into the data stream. This is the same TDC module that is used to record the HAWC PMT signals. The motivation for this is to keep things as simple as possible by not introducing an addition component into HAWC's high event rate DAQ system. The TDC channels that are used for the time stamp will operate and be readout in the same way as the PMT TDC channels. The time stamp will provide a resolution of ?? usec with an absolute accuracy of ?? usec rms. Assumptions: ------------ For this design I will assume that we are using the VX1190A version CAEN TDC modules, that we have Trigger Matching readout, and that the TDC are setup in the following way: -- For now I have pulled out all of the details from this section -- -- There is a ton of stuff here that needs to be understood about -- -- how we will uses these complicated TDCs. -- Implementation Details: ----------------------- The overall event time stamp consists of 3 components. In steps of increasing resolution these components are: 1. The DAQ computer handling a given event adds to that event's data a time stamp based on that computer's system clock time. All of the DAQ computers are synchronized via NTP time service to a local NTP time server. NTP time service is a standard Internet protocol that has been in use for many years and is available for basically all computer operating systems. A local NTP time server is an inexpensive device. For the HAWC application it makes sense to use an NTP time server that uses GPS as its time base. The advantage of using a local NTP time server is that in this way HAWC's operation will not depend on long network lines back to some other service. GPS based NTP time servers are inexpensive enough that it's very practical to have a redundant server running all of the time. Doing this is a common practice. The absolute accuracy of the computer's NTP controlled system clock will be in the mill second range. The latency between an event trigger and the DAQ computer adding its time stamp to that event's data plus the uncertainty of the computer's system clock will together be less than one second when the DAQ system is running normally. 2. The next finer resolution component of the time stamp comes from information carried by 28 TDC channels. These 28 TDC channels each carry one bit of time stamp information. These bits represent time values of: 10s of usec 4 BCD coded bits 100s of usec 4 BCD coded bits 1s of msec 4 BCD coded bits 10s of msec 4 BCD coded bits 100s of msec 4 BCD coded bits 1s of sec 4 BCD coded bits 10s of sec 4 BCD coded bits The timing information carried by these 28 bits has 10 usec resolution and the value remains unique for 1 minute. There should be no trouble matching up this information with the computer clock information described in point #1 above. These time stamp bits are sent to the TDC once every 50 usec. They are sent at the beginning of each period when the absolute clock time is either **.****00 sec or **.****50 sec. The 28 bits are encoded in the following simple way. - At the beginning of each 50 usec period all 28 bits make a synchronous positive edge in their TDC channel. - If the time stamp value represented by a given bit at the beginning of the 50 usec period is a "zero" then that bit makes a negative edge in its TDC channel 1 usec after its positive transition. - If the time stamp value represented by a given bit at the beginning of the 50 usec period is a "one" then that bit makes a negative edge in its TDC channel 2 usec after its positive transition. Thus all 28 bits show a pulse in their TDC channel once every 50 usec. The leading positive going edge of this pulse is synchronous with the beginning of each "integral" 50 usec absolute clock time period. The pulse lasts 1 usec for zeros and 2 usec for ones. Note that some encoding scheme of this type must be used because the TDCs only remember data for 104 usec. That is, you can not just send the 28 raw binary bit lines to the TDCs because most of the time most of the lines will not make a transition during the maximum time window of the V1190 TDCs. Yes, one could use fewer bits (perhaps 20) if the time value was represented in straight binary instead of binary coded decimal. However the precision time keeping world seems to prefer thinking in decimal fractions of a second rather than in straight binary so that's the way that I have proposed building this system. Clearly it could be built and would work just fine either way. The generation of these 28 bits of time stamp information is controlled by a GPS clock with a parallel data output. There are a number of companies that make such devices. There are even GPS clocks available that provide both parallel output data and an NTP time server all in one device. Note that a small straight forward piece of custom built hardware is needed between the GPS clock's parallel output and the TDC inputs. This custom hardware just converts the clock's parallel output signal levels into a once every 50 usec pulses that indicates the level of a given bit by the pulse width on a line sent to the TDC that represents that bit. This "box" would be fully synchronous logic operated by a 10 MHz clock signal from the GPS receiver. 3. The finest resolution time stamp information comes from the TDC data itself for these 28 channels. Each of these channels will provide a delta time measurement from the most recent rising edge of its input signal until the arrival of the Trigger signal. All 28 of these TDC channels should give the same value for this delta time. This delta time is always less than 50 usec so it fits comfortably within the time window of the V1190 TDC. We thus have an accurate time stamp for the arrival time of the Trigger signal. The TDCs can measure all of the PMT signal edges with respect to the arrival of the Trigger signal. Box between the GPS Clock Parallel Output and the TDC Input ----------------------------------------------------------- The basic function of this box as described in point #2 above is straight forward and easy to implement. An interesting escalation of this box would be to also take a precision clock signal from the GPS receiver (e.g. 10 MHz) and use it to generate a precision 40 MHz clock for the TDCs. This box would fan out the 40 MHz clock with a separate feed to each TDC module. The point of this is to allow the whole TDC system to run synchronously. Without such a setup each TDC module will run from its own on-board 40 MHz oscillator and thus all TDC modules will have a slightly different frequency (aka measurement time base) and all modules will be constantly drifting in phase wrt each other. An interesting escalation of this box would be to also have it synchronously fan out the Trigger signal to each TDC module. The alternative is to use a LEMO cable daisy chain across the front of the TDC modules to distribute the Trigger signal. A significant problem with the LEMO daisy chain Trigger distribution is that each TDC module will receive the Trigger at a slightly different time and that will make it harder to line up the PMT edge to Trigger signal delta times across TDC modules. An interesting escalation of this box would be to also have it synchronously fan out the CRST and CLR signals to all of the TDC modules. The CRST signal resets the "Bunch Counter" in the TDC which is the fundamental 25 nsec "coarse time counter" in the TDC. The point of this is to provide the same value in the coarse time counter of all TDCs, i.e. have a synchronous TDC system. Sourcing the CLR signal from this box results in all TDC control signals being clock synchronous and coming from the same place. The CLR signal erases data from the TDC's output buffer and performs a "global reset" on the TDC module.