D-Zero Hall Log Book for 2003 ------------------------------- The most recent entries are near the beginning of this file. This file begins in January 2003. Earlier D-Zero Hall Log Books are on the web either in the directory with this file or else at: http://www.pa.msu.edu:80/hep/d0/ftp/run1/l1/inventory_logs. DATE: At: Fermi TOPICS: DATE: At: Fermi TOPICS: ------------------------------------------------------------------------------ ------------------------------------------------------------------------------ ------------------------------------------------------------------------------ DATE: 30,31-DEC-2003 At: Fermi Topics: Cal Trig power supply work, work with Victor Martinez, Cal Trig timing work, +16,8 EM- open cable, Install the #2 Quad Term Cable, -14,13 CTFE PROM Trouble, Loading Gains and Ped's from TRICS, Count Comparator Registers Cold Start Instructions are at the end. Replace the failed Tier 1 power supply in Lower T1 M104. This failed last week on Tuesday night late. It's -4.5 Volt brick stopped working. Pull PDM-08 which consisted of: +5.0 V MSU SN # 28 7-FEB-1992 -2.0 V MSU SN # 5 27-SEPT-1988 old model 2500A-2 -4.5 V MSU SN # 19 9-NOV-1988 old model 2500A-2 <-- dead -5.2 V MSU SN # 24 9-NOV-1988 old model 2500A-2 The Pioneer Magnetics model 2500A-2 supplies are the ones that are no longer supported and we have not made an internal rebuild proceedure for this type, i.e. they are junk. Install PDM-09 which consists of: MSU SN# 25 is its -2.0 Volt brick MSU SN# 59 is its -5.2 Volt brick MSU SN# 27 is its -4.5 Volt brick MSU SN# 22 is its +5.0 Volt brick Work with Victor Martinez while doing the Power Pan swap. He did most/all of the work and has copies of (and URL's to) the documentation for changing the Power Pan and for Cal Trig power control proceedures. Install hose clamps on the water manifold under the M104 Lower Tier 1 Power Pan while it is pulled out. There was a clamp on one end of one of the manifolds - I put on 3 more clamps so that they all are taken care of now. Rebuild of PDM-08 +5.0 V MSU SN # 28 7-FEB-1992 -2.0 V MSU SN # 54 21-JAN-1992 MSU 6 Cap rebuild -4.5 V MSU SN # 84 7-JAN-1992 MSU 6 Cap rebuild -5.2 V MSU SN # 85 7-JAN-1992 MSU 6 Cap rebuild PDM-08 was run without load (but all these brick had recently been load tested). Then PDM-08 was labeled as tested ready to use and put in the Spare Power Pan cabinet. The remaining Spare Bricks at Fermi are: 2x 5V 600A Powertec MSU #19 and #25 3x 5V 200A Pioneer MSU #67, #72, #95 all MSU 6 Cap 3x 5V 200A Pioneer Refurbished MSU #97 and #98 and SN #416254 1x 2V 325A Pioneer MSU #79 MSU Cap See Log Book entry from 23:25-Nov-2003 for Fermi spares list. We now have zero spare 2.0 V 250 A bricks at Fermi. At MSU ?? Tier 2 M105 T&SS Delay Cable Install the Timing delay cable for M105 Tier 2. The easiest place to install this cable was in the back of the Control Crate in the top of M103. See the Log Book entry from 11,12-DEC-03 for details about this cable. This cable is 7 sections long of standard Twist and Flat. Installing this cable sould make the |eta| 1:8 Tier 2 have the same timing as the |eta| 9:16 Tier 2. Tier 3 still needs a delay cable that is 3 sections long. Checked signals in M105 Tier 2 wrt M109 Tier 2 and I see that now with this delay cable, M105 Tier 2 has moved so it is 3.5 nsec behind M109 Tier 2. This should be just about perfect. +16,8 EM- We know that +16,8 EM- is an open circuit. This was checked with the "target resistor pack". Now check +16,8 EM- with the Philips/Fluke RLC meter. EM+ EM- HD+ HD- Spare Cable 358 nFd 3.18 nFd 358 nFd 308 nFd 2.6 nFd Based on this I assume that the "open" is in the BLS end connector. Install the Quad Term #2 cable. It required only 9 sections of standard twist-flat cable. This is for And-Or Terms 16:27 from the Quad Term CTOM card in M101 Mid Slot #14. -14,13 CTFE Trouble After powering things back up it appeared that we had a problem with the Cal Trig. The first indication of trouble was from loading up the Cal Trig with \Scratch\L1CT_Setup_Version_1.msg (the file that loads all 8 Ref Sets with rational values and loads all Count Comparators. The resulting And-Or rates did not look right for normal ZB running. The And-Or Terms from the Total Et Ref Set Count Comparators that require only 1 or more TT's > Ref Set Threshold were running at 7.586 MHz for all 4 of the Tot Et Ref Sets. The TT_ADC_Mon display looked fine, i.e. the analog or ADC part of -14,13 had not blown up. With the Ref Sets still loaded up the next test was to use the TRICS Cal Trig Test menu and use the "List TT's over Ref Sets" option. This showed that -14,13 was over all 4 of its Total Et Ref Sets but was just fine with its EM Ref Sets. That fits exactly what the And-Or rate display was showing. Cal Trig Tests "List TT's over Ref Sets" had immediately shown which TT was in trouble. Look at the High Speed readout data and it showed that -14,13 was reading out $ff for both Tot_Et and EM_Et. The thing between TT_ADC_Mon and HSRO data is the PROM's. Use the TRICS CTFE PROM check option (remember that currently you need to check the Energy PROM's on page 7 and the Momentum PROM's on page 2) and everything in EM land was OK. In HD land -14,13 always failed - typically reading a constant 255. -13,13 HD -15,13 HD and -16,13 HD typically would pass OK (just showing an offset of 255). So this test clearly pointed at the -14,13 HD PROM. Pull the CTFE card and the -14,13 HD PROM had its Vcc pin not in the socket but just kind of touching the side of the socket contact. This PROM tested OK so I just straightened its bent pin and put it back in the CTFE card. Put everything back together and loaded up the Gain and Ped DAC's from TRICS using \D0_Config\Load_L1Cal_GainsPeds.mcf (which you execute from the Master Command File menu). That is slick. Everything now looks OK. During the investigation of this the one test that I made that I do not understand is the HD Tree Browser (recall the files are in \Scratch\ and you run them from VME_Access CIO). I could not see the 255 extra counts on the HD Energy Tree and so far I do not know why I could not see them. HSRO Readout from M112 I saw one event where it looks like it may have slipped a channel at phi 13. Review of the location of the Count Comparator reference registers in the Tier 3 Counter Tree CAT2 cards. Tier 3 is MBA = 159 Tier 3 CAT2 for Tot_Et Ref Set #3 is Card Address 15 Tier 3 CAT2 for Tot_Et Ref Set #2 is Card Address 13 Tier 3 CAT2 for Tot_Et Ref Set #1 is Card Address 11 Tier 3 CAT2 for Tot_Et Ref Set #0 is Card Address 9 Tier 3 CAT2 for EM_Et Ref Set #3 is Card Address 7 Tier 3 CAT2 for EM_Et Ref Set #2 is Card Address 5 Tier 3 CAT2 for EM_Et Ref Set #1 is Card Address 3 Tier 3 CAT2 for EM_Et Ref Set #0 is Card Address 1 On a CAT2 Card Comparator #1 Reference Register Bits 1:6 is Function Address 19 Comparator #2 Reference Register Bits 1:6 is Function Address 22 Comparator #3 Reference Register Bits 1:6 is Function Address 25 Comparator #4 Reference Register Bits 1:6 is Function Address 28 It was a long fight to get the Spark running this week. I need to bring the other CPU card here and try that. Cold start instructions Rev. 31-DEC-2003 We are running: TRICS V10.4.K See the full instructions at: www.pa.msu.edu/hep/d0/ftp/l1/framework/hardware/rack_crate/ master_clock_instructions.txt www.pa.msu.edu/hep/d0/ftp/l1/framework/hardware/rack_crate/ framework_power_control_procedures.txt www.pa.msu.edu/hep/d0/ftp/l1/cal_trig/hardware/rack_crate/ cal_trig_power_control_procedures.txt Overview: Only after Master Clock is running and loaded, then turn on TFW and then the L1 Cal Trig (one power supply at a time) then: Configure FPGA's. Use the normal TRICS "Master Command File" menu for this. It configures all FPGA's except for the M101 Routing Master. Get the Routing Master FPGA's Configured and Running 1) After the Routing Master is powered up, wait for its SBC to boot (less than 1 minute) 2) Login to the SBC with 'ssh d0runsu@d0sbc001b' The password is same as the online d0run account. 3) On the SBC run 'reset_all.sh stop' which will stop the relevant readout processes. 4) Use TCC's Configure FPGA's menue to execute the dcf that configures the Routing Master FPGA's. This file is D0_Config\M101_Routing_Master_All.dcf Verify that the "ECL Output Enabled" box is checked before executing this dcf file. If you see an error on the first attemp to configure these FPGA's try it again. It must say zero errors. 5) On the SBC run 'reset_all.sh start' Use the TRICS "System Control Status" menu to: Set the L1 Cal Trig Eta Coverage to 1:20 Check the "TTI File Loads DAC's" box Do a Full Initialize of TFW and L1 Cal Trig Use the TRICS "Master Command File" menu "Run a Master Command File" option to select and then execute the file \D0_Config\Load_L1Cal_GainsPeds.mcf This loads the current Gains and Pedestal files into the L1 Cal Trig. Use the TRICS "System Control Status" menu to: Set the L1 Cal Trig Eta Coverage to 1:16 Do a Full Initialize of the TFW and L1 Cal Trig verify that it is a clean init of all parts. For reference only, the current Gains Ped's files are: D0_Config\Gains_17_20_1_32_rev_a.tti # TT eta's 17:20 D0_Config\Gains_1_16_1_32_rev_5.tti # TT eta's 1:16 D0_Log\Find_DAC_V5_0_F_20031212_Edited.tti;2 # TT eta 17:20 D0_Log\Find_DAC_V5_0_F_20031212.tti;1 # TT eta 1:16 ------------------------------------------------------------------------------ DATE: 18:20-DEC-2003 At: Fermi Topics: Cal Trig Timing in M101 Mid crate, Exclude 2 Channels, Cal Trig VRBC SCLR Sync Loss, Trigger Tower work No change in the Cold Start since last week. TRICS was restarted Friday night. The full length of the "Summer 2003 Shutdown" will effectively have been from about 7-SEPT to about 17-DEC. Check Timing of M101 Middle CTOM Clock Signals Measure the M101 Mid P1_TS_6 CTOM Input Clk which comes from CMC TL 16 Measure the M101 Mid P1_TS_12 CTOM Output Clk which comes from CMC TL 17 against Upper Tier 1 Energy CAT2 Clk CTMBD Backplane T&SS "P" which comes from CMC TL 7 M101 Mid P1_TS_6 is 85 nsec in front of Upper Tier 1 CTMBD Backplane T&SS "P" M101 Mid P1_TS_12 is 217 nsec in front of Upper Tier 1 CTMBD Backplane T&SS "P" If all the cable and fanout delays were exactly the same then I would expect M101 Mid P1_TS_6 to be 18.8 nsec in front of Tier 1 CTMBD Backplane T&SS "P" (because CMC TL 16 is currently generated one RF bucket earlier in the CMC than CMC TL 7). So the cable and fanout delay to the Tier 1 crates is about 65 nsec longer than the cable and fanout delay to M101 Mid. This is about what you would expect. There are 20 sections of twist and flat and a BBB card in the path to the Tier 1 crate. The intent is to Run Tier 3 about 12 nsec in front of Upper Tier 1. It is currently running about 20 nsec in front of Tier 1. So after the intent is implemented, I would expect the T&SS delay to Tier 3 to be about 53 nec longer than the T&SS delay to M101 Mid backplane P1_TS_x. Recall that once the P1_TS reaches a THE-Card in M101 Mid then the signal will be delayed between 1 and 2 ticks of RF by the BSF FPGA, i.e. it will be delayed between 18.8 nsec and 37.6 nsec. Call the delay in the BSF FPGA's on the M101 Mid CTOM's 30 nsec. It is about 2 1/2 sections of cable from the Tier 3 outputs to the Cal Trig And-Or Term patch panel where as it is about 15 sections of cable from the M101 Mid CTOM cards to the Cal Trig And-Or Term patch panel. So it takes the outputs from the M101 Mid CTOM's about 35 nsec longer to reach the And-Or Term Patch Panel than it does the outpus from the Tier 3 CAT cards. On the other hand from the time a Tier 3 CAT card gets its input clock until its outputs are stable is about 35 nsec (see: ww.pa.msu.edu/hep/d0/ftp/run1/l1/caltrig/cards/ cat2_rev_b_description.txt and fmln_rev_a_description.txt). Where as the delay from the CTOM receiving its output clock until its outputs are available is 5 nsec (or less). So the Tier 3 signals are about 30 nsec behind Summary: M101 Mid CTOM's delayed 30 nsec in BSF and 35 nsec on cable relative to the Tier 3 signals. Tier 3 has 30 nsec of logic propigation delay relative to the M101 Mid CTOM signals. So the M101 Mid CTOM needs to receive its P1_TS_x about 35 nsec in front of the Tier 3. But the delay from CMC to M101 Mid P1_TS_x is 53 nsec less than the delay from CMC to Tier 3. So the T&SS should be sent from the CMC to M101 Mid about 18 nsec later than it is sent to Tier 3. Made the 7 section long T&SS delay cable for M105 Tier 2 but I have not installed it yet. Did not make the 3 section long cable for Tier 3. Excluded Channels Edit \D0_Config\Excluded_Trigger_Towers.msg and add -6,9 HD and -6,24 HD to the list. These are currently the two worst channels for noise and we can not even hold a good pedestal on -6,9 HD. Instead of chancing a waste of beam time we decided just to get these 2 channels out of the way for now. The only other channel that was already Excluded is +6,23 EM. L1 Cal Trig VRBC SCL Receiver Sync Loss The SCL Receiver on the L1 Cal Trig VRBC lost sync twice this week and sat there with its amber/yellow LED out. I was here both time and kicked it with the Init_Post_Auxi_L1CT.vio and it started up just fine. As a stop gap measure lets change the "TRICS Console Etiquette" to include always keeping the Init_Post_Auxi_L1CT.vio loaded in the VIO menu. That makes it a bunch easier to tell the shifter what to do. They do not have to find the correct file. As a stop gap measure I have put a sign in the crate showing where to look for the amber/yellow LED. L1 Cal Trig And-Or Term Rates After an Initialize the Cal Trig And-Or Term rates are at 7.5 MHz. During a store the programmed And-Or Terms are whatever the rate is. After the store the Terms that were programmed and used during the store show 0 Hz rate. Why ? The terms that were not programmed / used continue to show 7.5 MHz. Work on TT's This is a summary of the Trigger Tower work that was done this week. I will move this information to our TT Repair Log Book next week. Started the week with 19 channels to examine/repair. These were the clearest problems or the problems that Bob most wanted fixed. Dean has almost certainly fixed 6 of the problem channels There are 5 additional channels that we now know exactly what is wrong with them (cable problems and one CTFE PROM problem). I will take care of these when we have a full day that I can have the L1 Cal Trig turned off. For the next couple of weeks I have Excluded the 2 noisy channels -6,9 HD and -6, 24 HD from taking part in the L1 Cal Decision. -6,9 HD has had to be excluded before. It currently has enough noise that we can not hold a good pedestal on it. The source of noise on these 2 channels is not understood. The new "class" of noise, line locked noise, first seen as wide pedestals in -5,3 HD -5,5 HD -5,7 HD is not understood. An idea for the next test is to: verify that we can see the noise and then test-swap the BLS's that service -5,3 and -5,6 and see if the noise moves. Crossed Cables -------------- -8,23 and -8,24 EM/HD were crossed at the Cal Trig end. --> Dean fix this on Wednesday the 17-DEC-03. Dead TT Signals --------------- +14,29 HD Looks dead - there is no signal on either side of the differential cable. The cable looks OK. I can "see" the output capacitors on the Driver hybrid. The EM side of this TT looks OK. --> During access Dean replaced the Driver Hybrid. This TT now looks fine on the scope. +19,3 HD Looks dead - there is no signal on either side of the differential cable. I can not see the Driver hybird output capacitor on either side of the HD signal. EM signal looks fine. I expect that the problem is either a damaged or missing HD Driver hybird. --> During access Dean installed a Driver Hybrid. This TT now looks fine on the scope. +19,7 HD Looks dead - there is no signal on either side of the differential cable. I can not see the Driver hybird output capacitor on either side of the HD signal. EM signal looks fine. I expect that the problem is either a damaged or missing HD Driver hybird. --> During access Dean installed a Driver Hybrid. This TT now looks fine on the scope. +19,30 HD The signals on the cable look fine. CTFE EM and HD gains are correct. Pedestal width from Find_DAC looks normal. I expect that it is a CTFE Energy Lookup PROM problem. -19,29 HD Looks dead - there is no signal on either side of the differential cable. The cable looks OK. I can "see" the output capacitors on the Driver hybrid. The EM side of this TT looks OK. -19,30 HD There is a short in the HD- side of the cable at the MCH-1 end of the cable. Moving the cable so that the short goes away then I can see signals on both sides of the cable. The EM side of the TT looks OK. Channels with about 1/2 of the Expected Signal ---------------------------------------------- -10,2 EM CTFE EM and HD gains are correct. No cable shorts. I can "see" the output capacitors on the Driver hybrid. The wave form on the EM- signal looks small and wrong. --> During access Dean replaced the Driver Hybrid. On the scope the EM- signal looks OK but it is only 3/4 the amplitude of the EM+ signal. If this channel still looks bad in the Examine then I will test the cable using the target resistor pack at the Platform end. -13,18 EM There is zero EM+ signal. The cable is not shorted. I can not "see" a normal output capacitor on the + side of the EM Driver hybrid. The EM- signal looks fine. I expect that it is either a damaged EM Driver Hybird or an open cable. +3,6 HD CTFE EM and HD gains are correct. There is a signal on both sides of the cable. The cable is not shorted or open. --> Between one and 2 weeks ago this BLS card was replaced. --> We need a new run to see if that fixed the problem. +13,23 EM CTFE EM and HD gains are correct. There is a EM + cable short at MCH-1 end. +14,11 EM CTFE EM and HD gains are correct. There is a EM + cable short at MCH-1 end. +16,8 EM The EM- coax has an open center conductor. Do not yet know where the open is. Noisy TT Signals ---------------- -6,9 HD 40 mVpp of sin wave on each side of the cable. A good differential signal. 100 nsec period (not 132 nsec). There is also big readout noise on this channel that looks like SMT readout noise. But this 100 nsec stuff is there even when the DAQ is stopped. The EM side of this TT looks OK. This 100 nsec stuff is not line locked. In the past we have had to exclude -6,9 HD because it had too much noise. Right now we can not hold a good pedestal for -6,9 HD. -6,24 HD 40 mVpp of sin wave on each side of the cable. A good differential signal. 145 nsec period. EM side of this TT is OK. This channel also has a lot of SMT readout noise pickup. But this 145 nsec stuff is there even when the DAQ is stopped. This 100 nsec stuff is not line locked. -5,3 HD and -5,5 HD and -5,7 HD all have a very similar problem. 30 to 40 mVpp of sin wave on each side of the cable. It is a good differential signal. 400 nsec period but this is not locked to RF Clock. -5,3 HD is the cleanest example of the problem. The EM side of these 3 TT's is OK. The adjacent phi's look OK. This 400 nsec stuff comes in line locked bursts. Each burst lasts about 4 msec. There is a burst each 1/2 cycle of the line frequency. The EM side of these TT's has no hint of this line locked stuff. But there are other HD channels in the area that do show it. They include: PHI Eta -4 Eta -5 Eta -6 Eta +5 --- -------- -------- -------- -------- 1 0 33 mV -- 0 0 2 0 jv 0 0 3 30 mV -- 54 mV -- 0 0 4 0 jv 0 0 5 28 mV -- 35 mV -- 0 0 6 0 0 0 0 7 0 33 mV -- 0 0 8 0 0 0 0 9 0 20 mV -- 0 0 10 0 jv jv 0 11 0 20 mV ww 22 mV ww 0 12 27 mV -- 20 mV ww 26 mV ww 0 13 26 mV -- 20 mV ww 22 mV ww 0 14 jv jv jv 0 15 jv jv jv 0 16 jv 0 jv 0 17 0 22 mv -- jv 0 18 0 jv jv 0 19 0 25 mv ww 22 mV ww 0 20 25 mV -- 24 mv ww 22 mV ww 0 21 24 mV -- 29 mv ww 19 mV ww 0 22 jv jv 17 mV ww 0 23 jv 29 mv -- jv 0 24 0 jv jv 0 25 0 22 mv -- jv 0 26 0 0 jv 0 27 0 20 mv -- jv 23 mV -- 28 0 jv jv 0 29 0 19 mv -- jv 23 mV -- 30 0 jv 0 0 31 0 jv jv 22 mV -- 32 0 jv 0 0 -- --> the center of the noise trace is flat (there are line locked bursts of noise) ww --> the center of the noise trace is a line locked saw tooth the actual amplitude of the noise remains more or less constant jv --> just visible (i.e. you can see some component of the scope display is line locked.) The "normal" noise at eta -5 and -6 HD is about 15 mV. Eta -4 HD is about 18 mV. On Friday, having spend more time looking at these, I would no longer put a lot of meaning into there being a difference between jv and 0. On Friday I did go back and look again at a number of the noise channels that were originally surveyed on Thursday and they look the same. ------------------------------------------------------------------------------ DATE: 11,12-DEC-2003 At: Fermi Topics: New TRICS, Check L2_UnBiased_Sample, Check Expose Group AONM, Quad Term CTOM files, Find DAC Runs, L1 Cal Trig Crate Timing Cold Start: same as for week of 11:15-Nov-2003 except: You no longer need to execute \D0_Config\Init_MissingEt.cio by hand it is now part of \D0_Config\Init_Post_Auxi_L1CT.cio The pedestal files are: D0_Log\Find_DAC_V5_0_F_20031212_Edited.tti;2 # TT eta 17:20 D0_Log\Find_DAC_V5_0_F_20031212.tti;1 # TT eta 1:16 TRICS L2 UnBiased Sample Start running Trics V10.4 Rev K (24-NOV-2003). Then with Scott Snyder test that programming of the TDM MFP_Counters for generating the hardware input for L1_Qualifier #3 i.e. the L2_UnBiased_Sample qualifier is all working OK. Edit D0_Config\Init_Post_Auxi_L1FW.rio so that it now sets up FPGA sites 1 and 5 on the FOM++ card so that L1_Qualifier #3 comes from its hardware input source. I have put everything into Init_Post_Auxi_L1FW.rio to enable L1_Qual #3 to use its hardware input, but I commented out the actual rio Write commands - until we get the go ahead from L2 to do this. Nothing that I did should make a functional change in Init_Post_Auxi_L1FW.rio. Note also that Init_Post_Auxi_L1FW.rio still contains the rio Write to enable the FOM++'s P5 transceiver outputs. That should no longer be necessary now that we are running TRICS V10.4.K DCF DCI Verified that \DCF\EG_AONM_Miguel.dci calls \L1_AONM\L1_AONM_32_3.exo Now, does L1_AONM_32_3.exo have registers for P-Terms ?? I did not think so. Made \DCF\Quad_Term_CTOM_Miguel.dci It calls for the 4036XLA BSF and in sites 1:8 it calls \CTOM\CTOM_2_1.exo and in sites 9:16 it calls Miguel_5_1.exo Made \D0_Config\M101_Quad_Term.dcf For M101 Middle Crate Slot 14 it calls Quad_Term_CTOM_Miguel.dci Tested M101_Quad_Term.dcf and it Configured all 17 FPGA's on the card with zero errors and does not look like it disturbed anything else. Find_DAC runs Friday morning: Find_DAC runs Clean run on eta's 1:16. In the eta 17:20 run there were 3 TT's with too much noise: EM +20,24 EM +20,26 and HD +20,14. This makes the message "Error Tag" in the .tti file. Need to scan the .hst file for both "fail" and "warn". L1 Cal Trig Crate Timing Checked the Tier 1 to Tier 1 crate timing by comparing the CTMBD T&SS "P" Lemo Monitor signals on the Lower Tier 1 crates in M105, M107, M109, and M111 with each other. They are all the same within 1 nsec or so. Check the Upper Tier 1 crates wrt the Lower Tier 1 crates and see that the Lower crate is just a slightly over 9 nsec behind the upper crate. There is 3 sections of twist-flat cable from the Upper Trier 1 crate down to the Lower Tier 1 crate so the expected skew was for the Lower crate to be 8 nsec behind the Upper crate, i.e. very close to the measured 9nsec. Checked the timing of the Tier 2 and 3 crates wrt the Tier 1 crates. In the Tier 1 crates the Tier 1 Energy Adder CAT2 Clock is driven by CTMBD Backplane T&SS signal "P". This is CMC TL_7. In the Tier 2 crates the Tier 2 Counter Tree CAT2 Clock is driven by CTMBD Backplane T&SS signal "A,B,C,D,E,F". This is CMC TL_11. In the Tier 3 crate the Tier 3 Momentum CAT3 Clock (this is the same signal as Tier 2 Counter Tree CAT2 Clock) is driven by CTMBD Backplane T&SS signal "N". This is CMC TL_11. Currently CMC TL_11 is generated 18.8 nsec in front of CMC TL_7. Both are 37 nsec long pulses. M105 Tier 2 T&SS "A:F" is 51 nsec in front of M105 Upper Tier 1 T&SS "P" M105 Tier 2 T&SS "A:F" is 60 nsec in front of M105 Lower Tier 1 T&SS "P" M107 Tier 3 T&SS "N" is 39 nsec in front of M107 Upper Tier 1 T&SS "P" M107 Tier 3 T&SS "N" is 48 nsec in front of M107 Lower Tier 1 T&SS "P" M109 Tier 2 T&SS "A:F" is 31 nsec in front of M109 Upper Tier 1 T&SS "P" M109 Tier 2 T&SS "A:F" is 40 nsec in front of M109 Lower Tier 1 T&SS "P" M111 Tier 2 T&SS "A:F" is 23 nsec in front of M111 Upper Tier 1 T&SS "P" M111 Tier 2 T&SS "A:F" is 33 nsec in front of M111 Lower Tier 1 T&SS "P" You would need to add 7 sections of twist-flat to M105 to make it the same as M109. You would need to add 3 sections of twist-flat to M107 to make it the same as M109. ------------------------------------------------------------------------------ DATE: 4,5-DEC-2003 At: Fermi Topics: Spares to Fermi, L1_Busy Problem, Noisy Excluded TT, Find_DAC runs, Check Power Supplies Cold Start is the same as for week of 11:15-Nov-2003 except: You no longer need to execute \D0_Config\Init_MissingEt.cio by hand it is now part of \D0_Config\Init_Post_Auxi_L1CT.cio The pedestal files are: D0_Log\Find_DAC_V5_0_F_20031205.tti;1 # TT eta 1:16 D0_Log\Find_DAC_V5_0_F_20031205.tti;2 # TT eta 17:20 Brought 2 TDM cards to Fermi for the Spares Cabinet. These are TDM SN #13 and TDM SN #14. They do not have HSROCB's installed. We now have at least one of all the TFW card types in the Spares Cabinet. Thursday morning I connected some cables inside the TFW so that we could monitor on a scope what was going on with the L1_Busy problem that appeared at the end of the shutdown. Although that problem was fixed late this afternoon (L2 software loop problem) I went ahead and made some scope measurements during the current store just as a record of normal operation. On the scope you can see: Yellow top trace These are the L1_Acpt's. Blue middle trace These are the L2_Decisions coming from the L2 Global Stage to the TFW for distribution. Pink bottom trace This is a flag showing when there are 12 or more outstanding L1_Acpt's i.e. 12 or more L1_Acpt's have been issued for which their L2_Decisions have not yet been issued. L1_Acpt is just a copy of the L1_Strobe signal from the FanOut Box in M123. The signal to indicate receiving a L2_Decision from L2 Global is a spare copy of the L2_Answer_TRM_Write_Clock which comes from connector #8 pins 11&12 on the M122 distribution box. Run the scope on peak detect mode. That appears to give a clean view of the 132 nsec pulses even with the scope on 10 msec/div. The scope pictures are on the web at: www.pa.msu.edu/hep/d0/ftp/l1/framework/pictures/ I captured scope pictures during 2 different types of operation. L2 is Collection Monitoring Data: There are 4 pictures in this series: L2_Mon_1.tif ... L2_Mon_4.tif The scope is triggering on there being 12 or more outstanding L1_Acpt's. The horizontal scale is 10 msec/div i.e. a total of 100 msec is displayed on the scope. You can see that L2 Global stops sending out L2_Decisions. L2 Global can not make L2_Decisions because various parts of the L2 System are busy collecting monitor data. For a time L1_Acpt's continue to be issued. The pink trace comes up when there are 12 outstanding L1_Acpt's. The systems that use VRB's for their readout go L1_Busy when there are 14 outstanding L1_Acpt's. Next the L2 System finishes its monitoring tasks and then rapidly issues a batch of L2_Decisions to catch back up. The total time spent doing the L2 monitoring tasks appears to be about 30 msec. The amount of lost beam appears to be about 20 msec. In some minutes of looking I did not see the 12 or more outstanding L1_Acpt's flag become asserted except when the L2 System was collecting monitoring data. I.E. except for the pause during monitor data collection, L2 was always keeping up with the L1 rate. Normal L2 operation: There are 4 pictures in this series: L2_Norm_1.tif ... L2_Norm_4.tif The scope is triggered on just a random L2_Decision coming out of L2 Global. pictures 1 and 2 are at 1 msec/div pictures 3 and 4 are at 400 usec per div. The L2 System is so fast that at the 1.2 kHz L1 rate that existed when these pictures were taken it is easy to correlate with the eye which L2_Decision goes with which L1_Acpt. In pictures 3 and 4 I set the vertical cursors on the scope to measure the L1_Acpt to L2_Decision latency which you can read as 280 usec and 304 usec in pictures 3 and 4. The other thing that I was able to measure was how fast the L2_Decisions come out of the L2 Global once the monitoring data collection tasks are finished. As expected 14 L2_Decisions come out right away. It appears that these 14 L2_Decisions typically come out of L2 Global in about 1.6 msec. That is about 115 usec between L2_Decisions coming out of L2 Global. Check +6,23 EM on the scope. This is the one and only Channel that we have excluded since the end of the shutdown. It looks happy on the scope. I will leave it Excluded for this trip. Friday make some Find_DAC runs. Tell TRICS to ignore L1 Cal Trig. Use VME Access to verify that the current Gains files are loaded. Then make a Find_DAC run: EM & HD eta +- 1:16 phi 1:32 keep 2 target 8 write both files. It made the following file and found a good pedestal for all channels. D0_Log\Find_DAC_V5_0_F_20031205.tti;1 # TT eta 1:16 Make a Find_DAC run: EM & HD eta +- 17:20 phi 1:32 keep 2 target 8 write both files. This run found pedestals for all the channels. It did report one problem, "DAC to ADC Ratio Problem for -18,9 HD". This looks to be just a little way outside of the normal limit. D0_Log\Find_DAC_V5_0_F_20031205.tti;2 # TT eta 17:20 You can see the report of this "DAC to ADC Ratio Problem" only in the .hst file and VME_Access log files. This problem is not reported in the .tti file (which is only meant for loading into the hardware). The official way to check the Find_DAC run is to scan the .hst file for the word "warning". - everything written to the console also goes to the HST file. You don't need to go back to the VME_Access logfile. - all problems judged non-critical are reported with "Warning: ..." This is the string you should look for. - all problems judged serious are reported as "Failed: ..." or "Aborting: ..." but will always also generate an "Error_Tag:" in the .TTI file so that you are forced to notice them when you try to load the TTI result file. Start scanning through all the power supply plot files M103B & M103C all look OK <= 50 mVpp M104B & M104C all look OK <= 50 mVpp both have very clean -4.5 & -5.2 M105B & M105C on B the -2V is 75 mVpp on C all looks OK <= 50 mVpp M106B & M106C all look OK <= 50 mVpp M107B & M107C all look OK <= 50 mVpp M108B & M108C on B all look OK <= 50 mVpp on C the -2V is 75 mVpp M109B & M109C on B the -2V is 75 mVpp on C all looks OK <= 50 mVpp M110B & M110C on B the -2V is 75 mVpp on C should look at -2 and -4.5 M111B & M111C on B -2V is 75 mVpp and -4.5V > 100 mVpp on C -2V is 75 mVpp M112B & M112C on B and C the -2V is 75 mVpp The worst one is M111B -4.5 V which shows > 100 mVpp on StripTool but shows only about 20 mV on the Fluke (which is not too bad compared to the seriously sick supplies earlier in Run II). So I need to learn how to relate StripTool numbers to Fluke numbers. I need to survey all the supplies above that looked 75 mV or worse. M106A all look OK <= 50 mVpp -2V is worst it may be slightly over 50 mVpp -4.5V is clean all sould be adjusted -2V almost hits -1.9 V on its peaks M108A all look OK <= 50 mVpp -2V is worst it may be right at 50 mVpp -4.5V is clean all voltage adjustments are OK M110A all look OK <= 50 mVpp -2V is worst it is slightly over 50 mVpp -4.5V is clean all sould be adjusted -2V hits -1.9 V on its peaks M112A all look OK <= 50 mVpp -2V is worst it may be right at 50 mVpp -4.5V is clean all voltage adjustments are OK for now. ------------------------------------------------------------------------------ DATE: 3-DEC-2003 At: MSU Topics: Working with the Power Supply Volage Display for L1 Cal Trig and TFW. In /mnt/group/d0l1/rackmon/ aka /home/d0l1/rackmon/ there are now copies of Philippe's files for generating the formatted display of L1 Cal Trig and TFW power supply voltages. These files are: l1cal_most.runme, l1cal_rest.runme, l1fw_most.runme, l1fw_rest.runme After a setup d0online you can run one of these files just by: prompt: filename & These files all generate a display with correct rack numbers and power supply identification (and for L1 Cal Trig the eta range information). The update rate is 1 Hz and these displays do correctly show the noisy power supplies. The background color changes when a power supply is turned off (or failed). Recall the L1 Cal Trig Tier 1 Power Supply work that was planned for the shutdown but was not completed: Pull M109 Lower Tier 1 and replace its -4.5 and -5.2 Volt bricks. Pull M112 Upper Tier 1 and replace its -4.5 Volt brick. Pull M109 Upper Tier 1 and Capacitor Band-Aid all of its bricks. All of the planned Tier 2-3 Power Supply work was completed. I needed to verify that the noisy power supplies actually do show up in the StripTool display. I had a big question about this because I do not know what is actually being plotted. The plot updates once per second but does it plot the average of the 13 or 15 epics reads of that paramater or does it just plot the value from one ADC conversion out in the Shea box ? The are StripTool parameters to control, "Data Sample Interval" and Graph Redraw Interval" but I'm not certain what they mean and so far there is no documentation that helps. The delault is: Data Sample Interval = 1 second Graph Redraw Interval = 1 second Noisy supplies vs quiet supplies do look different with these default settings. The difference is easy to spot. Changing to a Data Sample Interval of 0.5 sec may make the difference even more striking. I tried a large number of combinations and 1 sec / 1 sec is OK with 0.5 sec / 1 sec being optimal. To make this test I watch Philippe's numeric table display to verify that noise that one can see in it also shows up on the StripTool plots. In the numeric table display M111B (i.e. upper Tier 1) -4.5V shows 113 mV pp in 30 seconds of noise In the numeric table display M112B (i.e. upper Tier 1) -4.5V shows 24 mV pp in 30 seconds of noise That is about how they look in StripTool. Thus so far I do not think that there is any averaging of all the ADC data (epics samples) going on in StripTool. The plot files in the ~/rackmon/ directory are now setup for a SampleInterval of 0.5 seconds. The way to use them is: prompt: StripTool plot_filename & ------------------------------------------------------------------------------ DATE: 2-Dec-2003 At: MSU Topics: Restart Trics D0TCC1 was using ~77% of its pagefile. Trics and Trigmon were restarted and the pagefile usage dropped to 14.5% ------------------------------------------------------------------------------ DATE: 25:26-Nov-2003 At: MSU Topics: Rack Monitor Voltage Display Geoff added the account 'laurens' to the 'onl_ctl' group to be allowed to update epics control files. Philippe cloned the l1cal template file /online/ioc/templates/lvl1.dbt (lvl1 stands for low voltage level 1, but this is really l1cal; there is also a lvl1c.dbt for the comint/bbb communication crate). Two new files were created lvl1fw.dbt and lvl1fwc.dbt which can instantiate 4x different voltage fields +5.0V, +3.3V, -2.0V, -4.5V for lvl1fw.dbt, this is for the typical FW supply +5.0V, +3.3V, -2.0V, -5.2V for lvl1fwc.dbt, for the SCL hub-end and FW readout The file /online/ioc/node/d0olctl16/d0olctl16.dbg was updated to add devices L1CAL_LV_M101AB L1FW_LV_M101C L1FW_LV_M122ABC L1FW_LV_M123ABC L1FW_LV_M124ABC The ending letter(s) are a reminder of which crates are being monitored (A=top,B=mid,C=bot). Some crates share a power supply (M101AB) while other crate power supplies share a voltage monitoring point, e.g. M122ABC. The file rack_voltage_monitoring_points.txt in http://www.pa.msu.edu/hep/d0/ftp/l1/cal_trig/hardware/rack_crate/ was used to determine which input of which Shae box is used. This file was also updated to show the correct settings of L1FW-style power pans with respect to the -4.5V vs -5.2V supplies. d0olctl16.dbg is the "generator" file which calls the "template" files. There were two things to update: (1) add calls to the lvl1fw/lvl1fwc.dbt files and (2) update the call to the rm16.dbt template to list the new device names. Then we need to create the "database" file setup d0online setup onl_ioc cd /online/ioc/node/d0olctl16 gmake d0olctl16.db Geoff did the following steps to load d0olctl16.db into the vxworks epics node, but we should be able do it ourselves next time. Edit /online/ioc/node/d0olctl16/startup to load this manually updated piece of the database instead of the official one. # Load the EPICS database #dbLoadRecords("epics.db") dbLoadRecords("d0olctl16.db") To reboot d0olctl16 connect to the terminal server port with the console to d0olctl16, type "version" to verify this is the right node, type to reboot the node, wait, then check there are no errors from loading the new database, then exit. > telnet t-d0-mch3 2041 [savage@d0ol30 ca]$ telnet t-d0-mch3 2041 Trying 131.225.230.65... Connected to t-d0-mch3.fnal.gov (131.225.230.65). Escape character is '^]'. **** You type here. **** -> version VxWorks (for Motorola MVME2300 - MPC 603r) version 5.3.1. Kernel: WIND version 2.5. Made on Sep 17 1999, 17:44:22. Boot line: dc(0,0)d0olboot:/online/ioc/ppc/mv2300/d0olctl16/vxWorks e=131.225.230.53:fffffe00 h=131.225.231.199 u=vxworks tn=d0olctl16 s=/online/ioc/ppc/mv2300/d0olctl16/startup value = 178 = 0xb2 -> ^x Now the processor reboots Once rebooted check that there are no error messages from loading d0olctl16.db. To exit: -> ^] telnet> quit Connection closed. Geoff will make this change official by inserting data into the database which only controls group experts are permitted to do. My understanding is that these updates will eventually get released and this will overwrite the /online/ioc/ area (e.g. onl_templates, but I couldn't guess where the .dbg files are stored). Now lets look at the new variables by creating another gui to display these new voltages. There now is one common display for all of Cal Trig (Tier #1, 2, 3 and communication) and one for all of L1FW (mostly L1FW, but also L1Cal readout, and L3 Routing Master). These two displays are started using a ".runme" configuration file and share the same display engine python script (now renamed to "l1volt.py"). setup d0online cd /home/laurens/rackmon l1cal.runme & # this is for the old values, all L1CAL l1fw.runme & # this is for the new values, mostly L1FW Philippe asks Geoff about plotting the quantities. Geoff looks into it. This seems to require that l1volt.py use CaFloatGui.py instead of CaGui.py, but this file is not in the onl_apps product (as in $ONL_APPS_DIR/src/ca), so there now is a (maybe temporary) copy of this file in the rackmon directory. The second issue is that the plotting feature doesn't like non-existing records on a given line. This may be fixed in the future, but for a workaround we can easily split our 2 displays into 4 separate displays. So we make 4 (maybe temporary) additional files. l1cal_most.runme l1cal_rest.runme l1fw_most.runme l1fw_rest.runme To generate a new plot window, just right-click on the device name on the lefthand side, and wait a few seconds for a window to pop-up. All 4 voltages are displayed together and clicking on a voltage name on the righthand side of the plot will display its scale on the y-axis. There are buttons on the bottom of the window to adjust the scale and position of the x-axis. ------------------------------------------------------------------------------ DATE: 23:25-Nov-2003 At: Fermi Topics: L1 Cal Trig, Find_DAC, TRICS start, Well Cooked HSRO_CB, Printing at D-Zero, Enable HW L1 Qual setup and rio File, edit Init_Post_Auxi_L1CT.cio, edit Init_Post_Auxi_L1FW.rio, Geo Sect 0x13 Status Cable, SCL Init and the Cal Trig Readout, Work setting things up for CTOM & Quad Terms, Spares at Fermi. Cold Start is the same as last week (11:15-Nov-2003) except: You no longer need to execute \D0_Config\Init_MissingEt.cio by hand it is now part of \D0_Config\Init_Post_Auxi_L1CT.cio The pedestal file for eta 1:16 is \D0_Log\Find_DAC_V5_0_F_20031123.tti;1 I have updated the full cold start file and taped it near the TCC console. L1 Cal Trig was "on accident" emergency powered down about 1 AM Sunday morning. Daniel and Philippe get it running again and it all looks fine at Sunday 9 AM. The log book claims that at 6 AM x10 "broke" and was power cycled to fix it. I see no indication that M101 Mid FPGA's have been Configured so I doube that x10 was power cycled. I need to make a quard to go over the Stop Buttons. Sunday morning I pulled L1 Cal Trig out of the global ZB to make another pedestal run on it. Ignore Cal Trig, Load Gain rev 5 for eta 1:16 Run Find_DAC over eta 1:16 +- EM/HD all phi Keep every 2 target 8 write both files. The file name is: D0_Log\Find_DAC_V5_0_F_20031123.tti;1 # TT eta 1:16 This file looks all OK except for +6,23 EM had too much noise. I will run with this new 1123 pedestal file. Re-load all gains and pedestals. Later Sunday morning re-start TRICS. The page file usage was at about 75% after re-start it was just below or at 20%. On Monday about mid day it jumped from about 24% to about 37%. Tuesday morning it is still at 37% Well Cooked HSRO_CB About 16:15 Sunday afternoon the TFW safety system shut it down and the DAQ Shifter came and got me. In the TFW safety System, Global Permit was gone. The RMI showed a smoke trip. It smelled smokey but there was no visible smoke. L1 Cal Trig was still running - the VESDA had not stopped it yet. Hard to tell which TFW rack was in trouble. The smoke detector in M101 had a red LED illuminated and M101 smelled. It was quickly clear that the probem was in the M101 Mid or Top crates - so I stopped working on the problem itself and got: TFW, SCL Hub-End and Routing Master running again so that ZB could start back up. Then started looking for the problem in M101 Mid or Top crate. The problem was in M101 Middle Slot #10. This is the Spark for reading out M104. The HSRO_CB on this card was cooked. Pull AONM Build A SN #30 with HSRO_CB SN #65. It was setup with Card_Address $1C and species $60 Install AONM Build A SN #08 with HSRO_CB SN #51. AONM Build A SN #08 had previously been used for a time in the Routing Master. AONM Build A SN #08 had a tag on it that said, "AONM SN #08 Build A with 4036XLA BSF Was Routing Master L3 Disable 64:127 but MSA_Output_00 non-inv is stuck at -2V. 15-OCT-2002". After replacing the Slot #10 Spark the L1 Cal Trig was put back into ZB and its data looks OK. The cooked HSRO_CB SN #65 has a seriously burned spot in back of the connector near the center of the connector. I do not understand this as there are no power pins in that area of the connector. Is there a bypass capacitor in that area ?? AONM Build A SN #30 is probably OK for non HSRO use. Its connector for mounting the HSRO_CB is a little damaged but may actually be OK. Before the cooked HSRO_CB was pulled off the Vee to Gnd resistance was about 0.4 or 0.5 Ohms and the other supplies looked fine. After the cooked HSRO_CB was pulled of the Vee to Gnd resistance returned to normal (i.e. something hi). Before using AONM Build A SN #30 again I need to very carefully check its HSRO_CB connector to make certain that there are no power supply pin problems. Received 3x SCL Receivers from Miroslav that the L2 group has some questions about. I swaped them with Ted. I got 4 from Ted and kept one in our official SCL Spares box. That box now has 1 standard spare and one spare with delayed 7 MHz Clock for Mikes Sequencer Controlers. As per the 28 Oct 2003 note from Greg Cisko that I got a copy of from Philippe, on D0SUNMSU1 I fixed the /etc/printers.conf to reflect the new printer queue names. I kept just the DAB 1 5 6 queues in the file and dumped the 10**9 other fermi stuff. lp -d dab1_hp8150 now works again. The other queues that should work are: dab1_color, dab1_color_trans, dab1_hp8150, and dab1_hp8150_d. Enable Hardware L1 Qualifiers In \Scratch there is now a file called HW_L1_Qual_Env.rio It sets the FOM++ BSF so that P5(15:0) are enabled inputs to the FOM++. It sets FOM++ Main Array FPGA sites 1,2,5,6 to use their Hardware source for the L1_Qualifier. All of the notes about the cabling and setting up of the Hardware L1 Qualifiers are in either: www.pa.msu.edu/hep/d0/ftp/l1/framework/hardware/aonm/ fompp_initialization.txt or l1_qualifier_signal_path.txt or special_functions_fom.txt Also see the 6:7-Nov-2003 log book entry. Work with the Hardware L1_Qualifiers: Double checked that the Collect_Status L1_Qualifier_7 is working OK. As expected this Qualifier comes up before the L1_Acpt and remains set until about 17 or 18 BX_Clocks after the L1_Acpt is issued. This is a copy of TFW Capture_Monitor_Data_Armed. It should come up when TCC arms Cap_Mon_Data and by design of the L1_FW_Helper FPGA it will remain asserted for 16 or so ticks after the L1_Fired_Strobe. Play with the L2UnBiasedSample L1_Qualifier_3. I see two issues: The value that is left sitting in the MFP control register pair for L1 Trig's that have never been told anything about what MFP value they should use is: Reg Adrs 72 = $ffff Reg Adrs 73 = $00ff Using the TRICS COOR menu item L2_UnBiased_Sample to setup the MFP Counter for a given L1 Spec Trig does not seem to work. Using manual rio does seem to work. Philippe traced both problems to my insisting on changing to bit assignment of the "force load" bit in the MFP control register. Edit the Init_Post_Auxi_L1FW.rio file This now enables the P5(15:0) input signals to the FOM++ and sets the FOM++ Main Array FPGA sites 2 and 6 to use the hardware source for L1_Qualifier #7. Thus L1_Qualifier #7 will now by default implement the Collect_Status function. FOM++ Main Array sites 1 and 5 were not touched, i.e. L1_Qualifier #3 will remain for now "standard" type i.e. FOM input. Init_Post_Auxi_L1FW.rio now does 12 rio writes. Verified that COOR Initialization of the system was still working OK after this edit. Edit Init_Post_Auxi_L1CT.cio The full contents of the file \D0_Config\Init_MissingEt.cio comments and modification notes and all has been added to the end of the file \D0_Config\Init_Post_Auxi_L1CT.cio So from now on all the set of Momentum Tier 2 and Tier 3 will automatically happen with each Initialization of the L1 Cal Trig. At Cold Start we no longer need to execute by hand the \D0_Config\Init_MissingEt.cio file. I haved edited the cold start instructions text file. Geoff Savage stopped by. I told him about how nice the new display was and about how would would use the right click strip chart when it is available. He said that we will get it automatically when it is ready, i.e. Philippe should not need to change his files for the readout displays. GS $13 SCL Status Cable Geographic Section 0x13 never showed any L1_Busy. This is STT. They had broken their SCL Status about about 2 or 3 feet from their end. I spliced the cable and they are running OK again. SCL_Initialize and Cal Trig Readout One of the things that was supposed to get done during the shutdown was new firmware in the Cal Trig VRBC. That did not get done and in fact Ted is still working a the final uniform known good version of it. There for we still have the issue of once a month or so having the Cal Trig readout's SCL Receiver loose sync and the VRBC not telling it to resync. Is it easy to add kicking the Cal Trig's VRBC to the list of things that TCC does during an SCL_Init ?? When in the cycle should be do it ?? Work an setting things up for CTOM and Quadrant Terms In \EXO\ make a directory CTOM. It's strange that depending on what program you use to display the directory structure, sometimes the new directory appears as CTOM and sometimes it appears as Ctom. Anyway, into \EXO\CTOM\ I ftp'd ctom_2_1.exo from desmo. Now for the dcf and dci files. The Quadrant Term card M101 Middle Slot 14 is an interesting case because it is an AONM printed circuit board that will use both CTOM and Miguel and perhaps even SHED FPGA's in its Main Array. The arrangement that has been used so far in most locations appears to be to have a dci file per FPGA design and in that dci file to specify both the Ver/Rev of the FPGA design and the Chip_Mast where that FPGA design should be loaded. Having one dci files per FPGA design is good because it means that there is only one file to edit when a given FPGA Ver/Rev is changed. The disadvantage in the way that we have used it so far is that it effectively locks a given FPGA design to being loaded into a given set of FPGA sites. This is OK for 99% of the TFW but was akward so far in at least 5 places: Global Disable TRM in M123 Top Slot 20. For that card the dcf file calls a special dci file, i.e. \dcf\Glb_Disable_TRM_Shed.dci This dci file specifies: L1_TRM_1_1.exo and Shed_1_1.exo FOM++ in M123 Mid Slot 16 For that card the dcf file calls a special dci file, i.e. \dcf\fomppl_miguel.dci This dci file specifies: fomppl_3_1.exo and fomppl_4_4.exo and Miguel_5_1.exo EG_AONM in M123 Top slots 5 and 12 For these 2 cards the dcf file calls a special dci file, i.e. \dfc\eg_aonm_miguel.dci This dci file specifies: L1_AONM_32_3.exo and Miguel_5_1.exo L1Bz_FOM in M123 Top slot 19 For that card the dcf file calls a special dci file, i.e. \dcf\L1Bz_FOML_Miguel.dci This dci file specifies: L1_FOML_1_2.exo and Miguel_5_1.exo L1_Await_L2 Scaler in M123 Bot slot ?? For that card the dcf file calls a special dci file, i.e. \dcf\L1AL2_GS.dci This dci file specifies: GS_7_1.exo and L1AL2_4_2.exo What version of Miguel to use is specified in at least 3 places: \dcf\fomppl_miguel.dci \dfc\eg_aonm_miguel.dci \dcf\L1Bz_FOML_Miguel.dci What version of L1_TRM to use is specified in at least 2 places: \dcf\Glb_Disable_TRM_Shed.dci \ Do NOT \dcf\L1_TRM.dci / Agree ?? What version of L1_FOM to use is specified in at least 2 places: \dcf\L1Bz_FOML_Miguel.dci \dcf\L1_FOML.dci What version of Gated_Scaler to use is specified in at least 2 places: \dcf\L1AL2_GS.dci \dcf\GS.dci What version of Shed to use is specified in at least 2 places: \dcf\Glb_Disable_TRM_Shed.dci \dcf\Shed.dci <--- Never Used ? What version of L1_AONM to use is specified in at least 2 places: \dfc\eg_aonm_miguel.dci \ As expected these \dcf\L1_AONM.dci / do not agree / because P-Terms. So the handling of special cards is already a pretty complicated business. In the end no dcf or dci was made for the CTOM. Spare Cards at Fermi In the Spares Cabinet: 2x COMINT 5x CAT3 5x CAT2 7x BBB 2x Large_Tile 3x CHTCR 8x CT_MBD 1x MTG 1x FMLN ?x HSRO_CB 1x FM 1x AONM Build A with 4036XLA BSF SN#30 no HSRO 3x SM 1x AONM Build B 1x FOM Build B 1x TRM 1x TOM 1x Master Clock Selector-FanOut 1x SCL FanOut 1x SCL Hub Controller 2x SCL Status Concentrators 1x VRB 1x VTM In the Power Supply Cabinet: 2x Tier 1 Power Pans 2x Tier 2-3 Power Pans ?x eta 1:4 Term_Attn_Boards ?x eta 5:20 Term_Attn_Boards In the Transporter Box out on the Side Walk: 15x CTFE Spare Bricks: 2x 5V 600A Powertec MSU #19 and #25 5x 5V 200A Pioneer MSU #67, #72, #84, #85, #95 all MSU 6 Cap 3x 5V 200A Pioneer Refurbished MSU #97 and #98 and SN #416254 1x 2V 250A Pioneer MSU #54 MSU 6 Cap 1x 2V 325A Pioneer MSU #79 MSU Cap Issues with the spares: zero TDM zero good CTFE ------------------------------------------------------------------------------ DATE: 17-Nov-2003 At: MSU Topics: Remote tests of L1 Cal Trig Check EM and HD PROM on page 7 over |eta| 1:16 and all phi. No errors. Walk through the 10 racks using Philippe's /scratch/ .cio files to paint a pattern in each rack and then verified that the correct data appeared in the readout. All racks look OK including M112 which had its 4th down from the top ERPB changed last week. Used the Tree Browser i.e. .cio files run from VME Access to look at the Momentum Tree. All looks OK. Looking at Tier 2 inputs a common thing to see was a jump of 62 or 63 or 64 from the expected input of 256. This in fact makes sense because you are reading "non captured" data on the fly 6 bits at a time. After the various tests, re-loaded Gains and Ped's, Initialized L1 Cal Trig, loaded to /scratch/test_setup*.msg file to load Ref Sets and Count Comparators, verified that AOT rates looked rational, and left L1 Cal Trig in the ZB run. ------------------------------------------------------------------------------ DATE: 11:15-Nov-2003 At: Fermi Topics: Shutdown week 10, Work on L1 Cal Cold Start is at the end. Quadrant Terms Finished all the cabling for the Quadrant Term stuff. There are 0 (aka zero) spare Front_PB's at Fermi. There are one and two of all the other PB's. Install in Slot #14 of M101 Middle Build B FOM card Serial Number #09B. Set its Address Switch to $28. Set its Spicies Switch to $D2. FOM SN# 09B has one "white wire" on it. This FOM has passed all of the tests including single chance test. This FOM has an HSRO Transmitter on it. Connect more Timing & Control Signals from the Carmen Master Clock Sequencer #2 to the P1_TS_ in M101 Middle. Cable the following added signals: Can Be Master Clock M101 Middle Distributed Used by Sequencer #2 P1 Backplane Enters Crate to Main Array CT_FOM Time Line Timing Signal on TOM_PB on HQ_TS_ as ------------ ------------- ------------- ------------- ------- CMC_TL_16 P1_TS_6 J3 pins 11&12 HQ_TS_2 In_Clk CMC_TL_17 P1_TS_12 J4 pins 15&16 HQ_TS_1 Out_Clk Edit the Sequencer #2 clock file to start generating CT_FOM Input_Clk and Output_Clk on Time Lines 16 and 17. The new Sequencer #2 clock file is clk_sequencer_2_12NOV03.txt. Clock was re-loaded after power outage on 14-NOV. Set M110 ADC_Clk to 1,4 ON the bottom 3/4 of M110 was ADC_Clk 2,7 ON Set M111 ADC_Clk to 2,7 ON the bottom 3/4 of M111 was ADC_Clk 2,5 ON Set M112 ADC_Clk to 1,4 ON the bottom 3/4 of M112 was ADC_Clk 2,5 ON Check all of M110, M111, M112 with a scope to verify their ADC_Clk signals. All were OK on the first pass except for M111 Phi 28 which needed to have its keys clicked a 2nd time. See log book entry form 9:13-JUNE-2003 for a description of the original problem setting these switches. Pull the 4th ERPB down in rack M112. The problem there is either the 3rd or the 4th ERPB and I think it is the 4th ERPB. Pull ERPB SN #28 and install new from the spares box ERPB SN #85. There was some difficulty getting the new one to plug in to the backplane correctly (but that was not the cause of the readout problem). See log book entries for: 9:13-JUNE-2003 and 8:11-JULY-2003. Checking Philippe's new L1 Cal Trig Power Supply Voltage Monitor Displays M104_A OK M110_B M110_C OK M112_B M112_C OK M112_A OK M110_B M110_C OK M103_B M103_C OK M104_B M104_C OK M105_B M105_C OK M106_B M106_C OK M106_A OK M107_B M107_C OK M108_B M108_C OK M109_B M109_C OK M110_A OK M108_A OK All displays are connected to the correct sources. All power supplies look more or less OK for now. L1 Cal Trig starts up OK. The gains files to use are: D0_Config/Gains_17_20_1_32_rev_a.tti # TT eta's 17:20 D0_Config/Gains_1_16_1_32_rev_5.tti # TT eta's 1:16 For eta 1:16 this is the new gains file with the new better gains to match the Cal Precision Readout. Tried running with no TT's Excluded. Watch the And-Or Term rates with the my "cal trig test set" of Ref Set Values and Count Thresholds loaded from /Scratch/. Major problem - rates in 100's of KHz. Use the test software to see who is over Ref Set #0. Only +6,23 EM shows up. So put +6,23 EM back on the excluded list. The Missing Et Terms have been moved to AOT's 28, 29, 30. 31 OK. You must remember to execute the file \D0_Config\Init_MissingEt.cio at startup. I forgot that the water to M106 was not turned on. I ran for about 1 hour with it not turned ON. The thermometers looked HI and the air coming out of the front door did not feel right when I was looking at Tier #3. Try running PROM checks: EM PROM's look OK over |eta| 1:16 full phi tested on page 0 HD PROM's (tested on page 0) blow up at all locations with a message something like HD Tier 1 CAT2 Input 7 instead of 8 ==> expect PROM(7) -> 8 buts seems 7 HD Tier 1 CAT2 Input 9 instead of 8 ==> expect PROM(9) -> 8 buts seems 9 Px PROM's checked on page #2. All look OK except for +13,27 +14,27 +15,27 +16,27 so what ever is wrong it is probably a CTFE or CAT2 input problem. The problem has something to do with the bit of value 16. See 25:27-JUNE-2003 and 9:13-JUNE-2003 and 8:11-JULY-2003. This appears to be a known problem that comes and goes but is likely only a readout problem. Py PROM's checked on page #2. All looked OK. CHTCR PROM's tested for all 4 EM Ref Sets and all 4 Tot Ref Sets and they all look OK. Find_DAC Run Init full eta 1:20, set eta to 1:16, Ignore Cal Trig, Load Gain rev 5 Run Find_DAC over eta 1:16 +- EM/HD all phi Keep every 2 target 8 write both files. The file name is: D0_Log/Find_DAC_V5_0_F_20031114.tti;1 # TT eta 1:16 This file looks all OK except for +6,23 EM had too much noise. When we try to include 0x10 in the ZB run it hangs the Taker at download time. Possible idea about what is going on is that COOR was recently changed so that the Calorimeter Precision ReadoutCrates are now by defaulted downloaded at trigger download time and that COOR also things that it needs to download something to the L1 Calorimeter Trigger readout crate. Hints working on d0mino & online: If while trying to ssh from d0mino to an online machine if you get: It is also possible that the host key has just been changed. Please contact your system administrator. Add correct host key in /home/edmunds/.ssh/known_hosts Just get ride of the current /home/edmunds/.ssh/known_hosts file on d0mino and let ssh make a new one. If while trying to start xemacs on d0mino or online you either get no xemacs or a strange xemacs do a setup xemacs. I can no longer print from d0sunmsu1. I do not know who recently changed what. lp -d dab1_hp8000 used to work just fine. I put copies of the Cal Trig Voltage Monitor display stuff in the d0l1 directory, i.e. /mnt/group/d0l1/rackmon It runs fine from their. Note that a bunch of home directories were moved with the cluster boot on Friday. Cold start instructions Rev. 15-NOV-2003 We are running: TRICS V10.4.J and VME_Access V5.0.F See the full instructions at: www.pa.msu.edu/hep/d0/ftp/l1/framework/hardware/rack_crate/ master_clock_instructions.txt www.pa.msu.edu/hep/d0/ftp/l1/framework/hardware/rack_crate/ framework_power_control_procedures.txt www.pa.msu.edu/hep/d0/ftp/l1/cal_trig/hardware/rack_crate/ cal_trig_power_control_procedures.txt Overview: After Master Clock is running and loaded, turn on TFW and Cal Trig (one power supply at a time) then: Configure FPGA's The normal Master Command file for this does everything except for the M101 Routing Master Get the Routing Master Configured and Running 1) After the Routing Master is powered up, wait for its SBC to boot (less than 1 minute) 2) Login to the SBC with 'ssh d0runsu@d0sbc001b' The password is same as the online d0run account. 3) On the SBC run 'reset_all.sh stop' which will stop the relevant readout processes. 4) Use TCC's Configure FPGA's menue to execute the dcf that configures the Routing Master FPGA's. This file is D0_Config\M101_Routing_Master_All.dcf Verify that the "ECL Output Enabled" box is checked before executing this dcf file. If you see an error on the first attemp to configure these FPGA's try it again. I must say zero errors. 5) On the SBC run 'reset_all.sh start' Tell TRICS that the Cal Trig eta coverage is 1:20 Init TFW and Cal Trig Tell TRICS to completely ignore the Cal Trig By hand using VME Access load the Cal Trig Gains files. (made sure that the "No Write to DAC's" box is not checked) Load then in the order indicated below. By hand using VME Access load the Cal Trig Pedestals files. (made sure that the "No Write to DAC's" box is not checked) Load then in the order indicated below. Tell TRICS that it has control of the Cal Trig again and set the Cal Trig eta coverage to 1:16. If you are going to be using the L1 Cal Trig Missing Et trigger then you need to run \D0_Config\Init_MissingEt.cio from the CBus I/O menu. You access the CBus I/O menu from the main TRICS menu. Init the TFW and Cal Trig verify that it is a clean init of all parts The current Gains files are: D0_Config/Gains_17_20_1_32_rev_a.tti # TT eta's 17:20 D0_Config/Gains_1_16_1_32_rev_5.tti # TT eta's 1:16 The current Pedestal files are: D0_Log/Find_DAC_V0_0_D_20030627_Edited.tti;2 # TT eta 17:20 D0_Log/Find_DAC_V5_0_F_20031114.tti;1 # TT eta 1:16 ------------------------------------------------------------------------------ DATE: 6:7-Nov-2003 At: Fermi Topics: Shutdown week 9, Work on L1 Cal, Isolate the SCL Status Cables, Hardware L1 Qualifiers, L1 Cal Trig And-Or Terms. Mike Utes has finished the SCL Status Signal isolator card. He installed it at the widow on the 2nd floor. This takes care of isolating the 20 pair cable that carries SCL Status signals from the 3rd Floor Electronics area down to the SCL Hub-End. He made 3 cards. The setup at the 2nd floor window uses the power supplies that were originally for the Run I ECL isolators. He also still has the ECL isolators if they are needed. On the FOM++ which is SN# 13B I installed 120 Ohm Terminator Resistors at R146, R147, R148, R149, R150, R151. Pin #1 is at the top up side of the card. These are CTS 750-63-R120. No resistors were installed at R152, R153. At R154, R155 I installed a 56 Ohm Pull Down Resistor with its pin #1 at the top up. These are CTS 750-61-R56 Ohm. On the 8 TDM cards I installed at R154, R155 a 56 Ohm Pull Down Resistor with its pin #1 at the top up. These are CTS 750-61-R56 Ohm. No other resistors were installed on the TDM's. On the FOM++ I connected the hardware input L1 Qualifier to only FOM++ Main Array FPGA's #1 - #5 and #2 - #6. There are no jumper wires to FPGA's #3-#7 or #4-#8. The Capture Monitor Data Armed signal to make the Collect Status L1 Qualifier comes from pins #15-#16 of connector #15 on the timing signal fanout box in the back of M122. The value loaded into the P5 Direction - Enable register for the FOM++ needs to be fixed. I think that the only other sotware change it to tell the mux in FPGA's 1,5,2,6 to use the Hardware input L1 Qualifier. So far things were only tested with the Collect Status Qualifier. That appears to work just fine. Part of turning on - testing the Hardware L1 Qualifiers was to move to running TDM_30_3.exo So the dci file for that has been edited to point to _30_3. The dci file has comments that show how to back up if necessary. Ran ZB for some hours with things more or less OK. The only issue is that the DAQ Shifter said that once in a while $1f was missing from the event. Then things hung. The symptoms were: After a SCL_Init the system would run for order of 10 to 300 events then $1F would go 100% L1_Busy and 100% L2_Busy. This hang *always* happened with L1_Await_L2 showing 7 outstanding events. The other strange thing was that using the VME_Access HSRO_Test event dump we were always seeing the same event from the SBC. Asking the DAQ Shifter to reset the SBC in $1f fixed the problem. Currently I do not *think* that any of this is a TFW problem. Cleaned up the patch panel for the L1 Cal Trig And-Or Output Terms. It now has feed of GAP and STROBE to all 4 And-Or Term cables and all the cables are installed and labeled. The patch panel is setup so that the twist and flat cables come off the back (the side that is hard to get to) and the output of the L1 Cal Trig plugs into the front (the side that is ease to get to). The layout is: And-Or Terms as seen 128:143 144:159 0:15 16:31 from the front of the panel The generation of the copies of GAP and STROBE is detailed in the 8:10-Oct-2003 log book entry. I have put pin #9 to pin #19 jumpers into MTG channels 17:24. This is for the most direct path through the MTG. So to make up of the inversion in the output stage of the MTG (which itself makes up for the inversion in the 16R8 PAL output) the patch cable that brings the Carmen Generated GAP and STROBE signals into the front of the MTG has an inversion of these signals. ------------------------------------------------------------------------------ DATE: 29:31-Oct-2003 At: Fermi Topics: Isolation Tests, install SCL to GS0x76=STT6 L2 UBS handling at L2, L3, etc Shutdown Trigger Framework, etc. Use this oportunity to involve one more DAQ shifter, Oleksiy Atramentov, who was interested in learning. Find some Andor Cables at the top of M123 somewhat stirred up and some dangling in front of the door. Change the Subnet Mask with Stu Fuess for d0tcc1 and d0tcc2. The online network has now spread from just 131.225.231.x to 131.225.230.x and the subnet mask needs to be set to 255.255.254.0. All vxworks nodes were moved to 230, online host nodes, tcc, etc, stayed in 231. This will make access control list management easier. Remind Stu about needing to hold down the shift key while trying to logout from the trigger account in order to get the chance to login as administrator (otherwise the auto-login feature logs you right back in). Also disable the "Messenger" service as there is an exploitable buffer overflow problem and we do not need this feature. Note that d0tcc1 already had it off, while d0tcc2 needed this changed. Disconnect FPD Andor Terms (Note: Daniel Mendoza tells me later that he has disconnected those at the far end too). Disconnect the status cable for 0x4F, the coax cables at the back of the SCL patch pannel for Geo Sect 0x08,09,0A,0B,0C,0D, & 78,79, and also the two cables to the L1Cal Test Area as a precaution. The coax for 0x4F is disconnected at the Hub End Fanout Card. Explore the issue of adding one more GeoSect for a new STT test crate in M201. This rack currently has nothing but a PC+monitor and a logic analyzer. Jodi Wittlin said last week that they need this "in the next 2 weeks". Laying on the top-back of M124 are 5 unused SCL coax cables. Four are labelled Spare-1,2,3,5 to M224 (44ft). The last one is Spare-3 to M324 (64 ft). These cables go to the second floor through the hole in the subfloor between the two aisles near M224, which is about as close to M201. They currently come through the floor tile inside M224, come up the panduit cable guide on the right side of M224, and dangle. It should be possible to reroute one of these cables to M201, once the target is clear. Locate box of SCL cabling hardware. Install one more coax feedthrough on patch panel for GS#0x76. Install one more SCL patch cable between patch panel and channel 0x76. Print some labels. Wait for a quiet time in the evening when the lights were returned to re-route and rename the cables formerly labelled SCL081 spare-1 to M224. Both the coax and status cables are now going under the floor into the base of M201 and up the right side cable guide. They are now labelled "M201-0 SCL STT6 GS#76". Everything is now connected on our side. Find a big puddle around the MCH1 air conditioner and running behind M124..M120. The explanation is that some water drain line had to be emptied for the isolation tests and was disconnected at a higher point in the building which flushed the content inside the Liebert in MCH1. Some water was vacuumed off, while some must have gone into the subfloor and wicked under our racks. The Master Clock's Epics CPU had to have its network address changed but this was not done before the power was turned off. Fritz wouldn't wait for power to be restored, and pulled the 9U card out. No lemo cables were disconnected to accomplish this feat, but many cables had to be unwound from the panduit channel. I witnessed the removal of the CPU card, not its return. Michael Begel has some questions about the L2 unbiased sample mechanism. He wants to make sure he can continue to normalize all the SpTrgs. We know the L2 Global will have to lie about the mask of L2 SpTrg Fired. The question is what do they do, and how will Michael be able to correct the L2_Accept scalers that TCC sends to the luminosity server. The next question is what will L3 use for a L2 SpTrg Fired mask to decide which filters to run. The issue is that it should not be able to pass events that didn't truely pass L2. We had a chat about what UBS means for the L1FW and we went to see Reinhard and Adam Yurkewicz. Adam is taking over L2 Global code. There is a document that tries to define this kind of thing but it is not totally explicit about this. Adam agreed to add a paragraph. My own summary is that, for events flagged as UBS, the L2 Global will return the L1 SpTrg Fired Mask unchanged to the L2FW, but that L3 Scriptrunner will still get and use the real L2 SpTrg Fired Decision Mask to run the L3 filters. The mask used by L3 could thus possibly be blank. This mask will also be the L2 SpTrg fired mask written into the Event Header. The UBS event is guaranteed to go to the monitoring stream (if it exists) because L3 checks the UBS L1Qualifier independently of any filtering. The UBS event will also go to the event stream if a L3 SpTrg Script ends up passing the event. There is only one event stream, and one monitoring stream (which is typically only defined for physics runs). With this information, Michael believes he can correct the L1/L2FW counters to provide normalization. He gets to look at every events, but only at the event header (not L1 or L2 the data chuncks). He can spot all events having the UBS flag set, then correct our scalers by decrementing the L2_Accept scalers by one for all SpTrg in the L1 SpTrg Fired Mask that were not also in the true L2 SpTrg Fired Mask, both masks being in the event header. One other question came up regarding the difference between ForcedWrite and a UBS ratio of 1. ForcedWrite is a standard L1_Qualifier and thus a property that can be given to any (one or more) L1SpTrg(s). It means that any time this (one or more) L1 SpTrig(s) fires, the ForcedWrite Qualifier will be set. We went back to the L2 TDR, where it is said that the effect of ForcedWrite vs UBS is identical within L2 but that L3 was expected to send the event to the event stream for ForcedWrite as opposed to the monitoring stream. That was the original plan. Michael says this is not currently possible: L3 will only send events to the event stream if they pass at least one L3 script (which certainly could be made to always pass for that L2 SpTrg), but the L2 SpTrg Script must have accepted the event for this L3 Script to run, which is not guaranteed by ForcedWrite alone. There seems to be strong reasons behind this L3 rule. I cannot remember or come up with a compelling reason for having a ForcedWrite mechanism in addition to and separate from a UBS=1. Independently and after all this, Michael reads the current L2 UBS COOR documentation which is derived from our COOR->TCC syntax document. The doc says that the The Level 2 Unbiased Sample feature cannot be disabled. Michael argues that this will cause problems. One argument is that there is NOT always a monitoring stream defined, but Michael agrees this should be addressed by having L3 quietly dump these monit-only events if the monitoring stream is not defined. Another argument involved forced_reject. We know that the L2FW will also dump SpTrg that were forced_reject, but if the UBS flag was set by that forced_reject SpTrg, other SpTrg may end up being to L3. It is not clear whether there is a reall lasting issue here, but things would be easier to describe and manage if the UBS feature could be turned off. I don't remember why the "cannot be disabled" statement was made, probably because there is no explicit enabling control. I am fairly sure that TCC can effectively disable this feature by simply leaving bit #31 high which should continuously reload some programmed non-zero value. Grounding tests: the shorts under the end muon iron were dealt with. Some shorts under the platform were removed, but there are still at least two shorts under at least two pillars supporting the platform. In all these cases only 20mill G10 was used instead of something like a 1/4". Overall many shorts have been removed, but these last two are at about the worst possible place for a ground loop to induce noise into the calorimeter. Picking up the detector and changing the shims is risky butpossible, but it would take several weeks to re-survey the detector position. Ground tests switched on Friday from being DC based (which had reached 100 A) to 60Hz AC with a pickup coil and a scope. I think it was established that no current goes to the MCH and there are still at least two shorts under the detector. I believe it was one of the four posts from both south footings. Testing stopped around 12:15, power returned over the next 30 mn. All SCL coax and status cables and the FPD andor cables were reconnected. However Jim F. and Stu F. had picked just that time to start upgrading the OS on d0ol07 and the clock could not be loaded. This was going to take hours, so Philippe left to be home on time for trick or treating. The daqexp (Miroslav Kopal, the new L2 hardware expert) had a crash course on restarting the clock and the framework, including running without L1CT. Instructions were to try restarting the system and call Philippe's Cell phone for assistance from the road, with the option of turning back or driving back later. It turned out that the Master Clock started up ok, but the framework had a power problem. There was a drip detector error preventing power up (cf. the water puddle). I need to talk to Miroslav to get full details, but that was eventually solved. Later that evening, once all network reconfiguration issues were solved, still no runs could go through. The symptoms were reported as first L1FW L1Busy with 14 L1 Awaiting L2, but after resetting L2 Global, it was L3 at 100% L3 Disable. This was guessed to be a RM configuration problem. Reconfiguring the FPGAs did the trick. Looking at TCC's logfiles, it turned out that the RM FPGAs were first loaded correctly (at second try: 1 error, then 0 error), and the L1FW was initialized successfully, but the RM FPGAs were somehow loaded yet again with 2 errors, and no second try. That explains it. ------------------------------------------------------------------------------ DATE: 25-Oct-2003 At: MSU Topics: Cooling Water Outage cancelled There was a scheduled cooling water outage for Saturday 7am-3pm. On Friday Philippe had sent notes to the night (Philip Perea) and afternoon (Brigitte Vachon) DAQ shifters to request that they read the documentation and shut the trigger framework off during that time. When the word came out Friday at 2:30pm that the outage was not affecting DZero, another note was sent. The night shifter didn't read the last note until after he had shutdown the trigger framework. The day shifter (Norm Buchanan) had pretty much managed to recover on his own. Apparently the Accelerator Timing rack was off, and Norm had managed to restart the Master Clock in Free Running mode. He was just calling because of the errors at the end of the FPGA configuration sequence. Philippe asks Norm to check on the number of errors (21 errors = ok), explains how to tell Trics to Ignore L1CalTrig, and call back as needed. Norm continues with the power up procedure on his own with no further help. ------------------------------------------------------------------------------ DATE: 18-Oct-2003 At: MSU Topics: Scheduled Power Outage Night shifter (Miroslav Kopal) shuts down the Trigger Framework. Afternoon shifter (Robert Harrington) restores the Trigger Framework, and just calls to notify Philippe when he is done. He reports not noticing the 21 FPGA configuration failures. Philippe points out that he must have turned on both power pans in M101, and asks him to turn the L1CT readout crates off again. Later on Robert calls again, as they cannot power up the rack for L2 Global. There seems to be a water leak fault (and smoke at times?). Philippe helps him locate the water leak sensor, but it doesn't seem wet. The plan is to run the fans from wall power for a while to try and dry any humidity or condensation. It was later reported that the RMI in that crate was replaced but that the most likely cause was that a cable was not plugged in all the way. ------------------------------------------------------------------------------ DATE: 12:16-Oct-2003 At: Fermi Topics: Power outage, Isolation tests Rack Monitor Voltage Display (Geoff) L1Cal Pulser Run Analysis (Reiner) Sunday afternoon: Took a walk on the prairie trail and in the woods. Very pleasant. I couldn't print via D0server3 using NT Fermi domain, but laurens' pwd had expired. Tried again later, but d0server3 was probably already down in preparation of the power outage. This may also have something to do with the move to the newer version of the NT file system and I have to remember what registry variable has to be changed. Figured out how to print from the d0olc command line: flpr -q "flpr" is a fermi unix product and stands for FermiLPR. It sends requests to a centralized print server, which I think was "fnprt.fnal.gov". Note that there is no space between "-q" and the queue name (!). For example dab1_hp8000 is the HP printer near the control room. Could not figure out how to print from the command line of an online linux node. Monday's power outage is short, but is followed by the isolation test and network plus host computers down time will push framework service recovery until late afternoon. This is similar to next Saturday's long 8 hour down time. Check the DAQ shifter schedule and the same two DAQ experts will be on shift Monday and Saturday for the early morning and late afternoon shifts. Send them both a note saying that Philippe will be on site on Monday, but they will have to do it by themselves next Saturday and could use this time to practice. Monday: Power down the trigger framework and master clock with Miroslav Kopal. he will do this for us on Sat morning. We also powered down the Routing Master. The control room binder in which the printed version of the power control procedure are kept is labelled "DAQ Shifter's Guide". Dan's old pager number was handwritten on the framework power document. This is now updated with Dan's new pager number, and Philippe's cell phone number was added. Unplug FPD andor term and SCL links according to Dan's list. One exception: Dean's test area has to be 0x4F instead of 0x4E. There is no 0x4E cable, and 0x4F matches the cables that Dan showed Philippe before the power outage. Marvin found lots of paths for ground shorts, including pieces of steel leaning around, and catwalk chains not isolated. The sprinkler system was supposed to be "dry" (=no pond water in the pipes until needed) but isn't. There is a second FPD cable going to the NIM crate above the Master Clock. There are also 5 coax cables coming in the same path that we traced through MCH1, but these are currently dangling unconnected in front of M114. There was a metal plate for a shelf through the 2nd floor hole in the wall that used to pass the L1 cables to/from the L2 trigger from Run I. The isolation test will be continued on Tuesday. Dean wants to know if the Turn number is reset during SCL_Initialize. The muon system divides the beam crossing clock by 4 and Dean sees this noise in the calorimeter preamps (->also trigger pickoff). All the muon crates reset their divide by 4 counter during SCL initialize. Robert Zitoun wants to sort the events by turn number modulo 4 and see what kind of differences are seen. I verified in the TTS FPGA description that the Tick and Turn are reset during initialize, as this is how the L1FW starts the two separate TTS time zone scalers to be the desired number of beam crossings apart. This is also spelled out in the SCL specification. I also pointed them at the reference to the l1tfw_util::GetTickAndTurnNumbers method to get the 32 bit version of this scaler out of the TFW data, if needed, since the 16 bit turn number rolls over in less than 2 seconds. Tuesday: Michael Begel came down worried about the master clock. Even though the accelerator clock rack is close to MCH1 where the master clock is for RunII (wasn't it on the 2nd floor in Run I) the clock cables go under the floor to the oposite corner of the control room, up to the 2nd floor, out the window (for the MCH2 in its pulled back position), up and accross to the 3rd floor level of the MCH (out in mid-air in the assembly hall), then gets inside MCH3 (not sure where), and then down to M100. Michael says that's 750ft. The 5 cables were disconnected at the 2nd floor window. Georg Steinbrueck is inquiring about getting one more Geographic Section for STT in M201. The allocation table shows 2 spare inputs between STT (0x70-75) and D.Mendoza Test Stand (0x78). Quick look in the MCH, and cannot spot additional cables routed. There is no light in the MCH, and I will check again tonight. There are a number of SCL coax cables marked spare from M224 laying on top of M124. Went to the the video projection of "Fermilab Stories", available online http://vmsstreamer1.fnal.gov/VMS_Site_02/Edu/FermilabStories/index.htm Went to the Lederman Science Center. Great place to take kids and friends with hands on display of particle physics. There is a gift shop with lots of shirts, posters, and gizmos. http://www-ed.fnal.gov/ed_lsc.html The fields around the east side smell very flowery today, and the people at the center think that it may come from a candle factory nearby. Geoff Savage gives Philippe a private tutorial on using his new "Channel Access GUI" interface. This is a generic tool to track EPICS variables. This is a more flexible tool than Fritz's generic display. The real work was done by Geoff, and Philippe has very little to do to leverage Geoff's work and display our variables. Geoff gives Philippe a tour of the L2 power supply voltage GUI as an example to adapt to make a L1Cal display. This is using his generic "caGui" display tool. Robert Harrington is the after4pm DAQ shifter and started restarting the clock pretty much by himself, by reading the instructions carefully. The only bump on the way was that the 5 coax cables had not been re- connected until after the downloading of the clock. Robert H. is able to restart the Framework and R.M. with minimal assistance, and we went over the fact there will be 21 errors during FPGA configuration and that he needs to tell Trics to ignore L1CT. Philippe installs the MS03-39 patch (RPC hole) on d0tcc1, and the Windows 2000 Service Pack 4 plus the patch on d0tcc2. L3 needed to be restarted, as it had been started during the MCH power outage and had woken up without the R.M and other pieces. Trigger Framework only run started with no problem afterwards. The Shell station at the corner of Kirk road and Route 38 (going to Tia Maria) also accepts the MSU gas card at the pump. Reconnect the FPD andor terms, SCL coaxes, and SCL status for 0x4F. Wednesday: Long discussion with Jovan to get an overview and a detailed look at how the TAB system is controlled, what data is transferred, what programmable parameters are left to control, how the current software is put together. etc. Fermilab waveform generator tested to send data to the ADF. This is a digital wafeform generator, sending a train of only positive voltages. It was "discovered" that when the amplitude is increased, the pedestal seen by the ADF decreases, because of the AC coupling. Implement a new L1Cal power supply display following Geoff's model. setup d0online cd /home/laurens cd rackmon l1cal.runme & <-- for all the Tier #1, 2, 3 crate supplies l1calc.runme & <-- for the L1 Cal Control Crate supplies All the hard work was provided by Geoff and the additional stuff is minimal. The plotting feature doesn't currently work, but will eventually be fixed. I think Geoff said one had to right click on the name to get the plots. Thursday: Run IIb L1cal meeting (short). Jovan mentions that he might get to the point where he could test sending data out to the VRB/L2. Jovan leaves next week Tuesday or Wednesday. Philippe sits down with Reiner for a refresher on the d0 cvs code environment, the software framework infrastructure to analyze events, and an overview of his L1Cal Pulser project, plus a quick tutorial on how to get data out of SAM. Check around M224, and there are a number of SCL coax and status cables dangling in front of it. This rack only has network equipment at the moment. Depending on the original and future purpose of these cables, this might be a way to get one more GS to STT. Plug TT(-6,21) back in, and put our scope away. Send note to Reinhard and Miroslav to inquire about a possible test of the TAB sending data to L2CAL. Reinhard is not on site until Fiday or Monday, Jovan will try to contact them. ------------------------------------------------------------------------------ DATE: 11:12-Oct-2003 At: Fermi Topics: Run IIb trigger workshop. Schedule was a frequent topic during the workshop. The original schedule was tied to Silicon installation during summer '06. Waiting too long (especially if the luminosity doesn't improve) implies having comparable amount of data before and after the upgrade. This makes for a more complicated analysis, and less attractive prospects. The projected acclerator luminosity schedule (which is not trusted in quantitative aspects) shows a systematic interuption every summer around the end of fiscal year with a slow luminosity start up after that. This much is probably close to reality. This is the time windows that DZero should use to install the upgrade(s), and then do physics commission while luminosity ramps back up. It is clear that we cannot be ready by summer '04. We should decide to have the new schedule show we are ready for installation by summer '05. Columbia made 13 TAB PCB's and have found no problem (so far) with the board they assembled. They are thus in a position to already own all the production boards. The GAB card was (1) de-scoped by pulling out the SCL and VME interface functionality and (2) simplified by reducing it to a single FPGA. This makes for a simple layout, and they feel they are or can be ahead of the original schedule. The other components of the upgrade are in good shape. L1CTT is redesigning daughter cards (with more logic to look at single fibers instead of pairs, and lower thresholds) which will be tested in current system in summer '04. L1CalTrack is a clone of L1Muon with upgraded pieces and is not a bottle neck. Layer 0 is also aiming for Summer '05. Producing/testing the ADF cards is thus the throttling factor. According to the material Dan and Philippe gave to Maris, MSU is also on track to test bench test pre-production cards in late spring '04, do another integration test around summer '04, bench test production cards in fall-winter '04, and start technical commissioning on the sidewalk spring '05 for installation readiness by summer '05. One idea: there might be a double advantage in producing the ~5x Rev2 of the ADF prototype/pre-production cards in the US instead of France, as this may gain time and allow using the same company that would eventually make the ~80 more cards. This decision need not be made immediately. There was a Control/Monitoring sesssion and Philippe was asked to present a L1Cal TCC control architecture diagram, and discuss the tasks involved in control and monitoring. Bob Hirosky (U of Virginia) is considering taking ownership of the ADF FPGA Firmware, pending engineering support decisions. Todd Adams (Florida State) wants to help, so he inherits the "examine" responsibility. He understands this involves unpacking of the new data, and Bob Kehoe's code. It wasn't specified if or how much FSU will help actual debugging. Dennis Shpakov with Darien (Northeastern) signed up for deriving the filter coefficients from the raw digitized sampling data. UIC's name is associated with commisioning/infrastructure. I don't know what this really means. One aspect was made clear. Mark Adams (UIC) is signing up to help study the BLS cable issue and produce the 80 pigtail BLS cable harnesses but MSU would lead and/or sign off after deciding on the final location for the ADF crates. Somebody (I think it was Mark Adams too) is signing up to make the ADF SCL fanout 6U card (no VME bus). Saclay made the one channel prototype running from the evaluation kit, but will not make the 4 (or 6) channel final board. One question was whether Saclay should start the paperwork to send the ADF card back to Saclay right away (but Denis will be busy with his single channel ADF card, and LVDS tester, and will not need the ADF card right away). Dan could have it for a couple of weeks before the card goes back to France. Misc notes: a full test of the TAB Jet algorithm on real data would require 9x9 trigger towers. The current 1x splitter board, with the 3x more boards being built by Saclay will support a reduced test over 4x4, and nobody was asking for more BLS signals. Will we need to buy FPGA evaluation kit(s) e.g. for the SCL fanout, or the LVDS link test? The "Cuisine of India" restaurant part of the Ogden Mall at 1255 E Ogden in Naperville has good food, and good service. ------------------------------------------------------------------------------ DATE: 8:10-Oct-2003 At: Fermi Topics: Shutdown week 5, Work on L1 Cal, Run IIB tests with Columbia nd Saclay, Prep for Isolation Test, L1 Cal Trig GAP and Strobe It required about 10 power sycles of the SPARKstation 10 to get it running this week. It had been perfect for a year or so but it is now getting painful at startup again. Power Supply Work Bring back to Fermi rebuilt 2.0V 325A bricks MSU SN# 34 and 83. Use these to put MM-1 and MM-4 back together with their original parts. Put MM-1 back in M106 - it runs the Eta 1:8 Tier 2 in M105 Put MM-4 back in M108 - it runs the Tier 3 in M107. Run IIB Cal Trig Test Ed had the Saclay computer setup and running so Denis and Emmanuelle could start work when they arrived on Wednesday. John Anderson supplied a new HP 1650B logic analyzer with a full complete set of probes. Columbia people arrived Thursday morning and needed a 6V and a 48V supply. This was taken care of with parts from Tom Reagan's area. The SCL cables for Label on Cable Geographic Patch Cord to the Run IIB Section Label Cal Trig Area ---------- ---------- ---------------- 7A Spare #1 Cal Trig Test #1 7B Spare #2 Cal Trig Test #2 Platform Isolation Test Things the need to be unplugged before the Platform Isolation test on Monday. The intent is to unplug all the cables that run to equipment that is not on the isolated Platform-MCH ground system. And-Or Term cable from Forward Proton Detector. Unplug this cable at the AOT patch panel AOT's 112:127 Unplug i.e. at the patch panel unscrew the SMA connector on the LMR-200 SCL cable for Geographic Sections 0x 08 ! Third floor FCH test stand 0x 09 ! Third floor FCH test stand 0x 0A ! Level 2 test stand on FCH-2 0x 0B ! Level 2 test stand on FCH-2 0x 0C ! Level 2 test stand on FCH-2 0x 0D ! Level 2 test stand on FCH-2 0x 78 ! Third floor FCH test stand 0x 79 ! Third floor FCH test stand On the front of the SCL Hub-End unplug the MCX connector on the LMR-200 cable for Geosection 0x 4E. [Correction: This must be a typo, and this is G.S. 0x4F] On the back of the SCL Hub-End carefully unplug from the Status Concentrator cards the flat Status Cables for Geographic Sections 0x 78 and 4E. Just 78 and 4E [ditto: this must be 4F] Setup of the Gap and Strobe Signal Fanout for the L1 Cal Trig See also last weeks Log Book entry for details about the initial work on this topic. Details about the signals supplied by the MTG in M103: Tier 1 BBB TSS Pair (Cable MBD MBD Maps this to Driven Bus TSS Backplane Tier 1 Backplane Buse by Pair) Input Bus Function -------- ------- ----- --------- ------------------------ MTG FA_0 19 4 C Read A/B MTG FA_1 21 6 A Write A/B MTG FA_2 25 10 J Energy MS Bit Adrs MTG FA_3 26 11 D Energy LS Bit Adrs MTG FA_4 28 13 L Momentum MD Bit Adrs MTG_FA_5 29 14 K Momentum LS Bit Adrs MTG FA_6 23 8 Not used - static drive MTG FA_7 30 15 S Not used - static drive Tier 2,3,4 Tier 2,3,4 CBus FeedSignal Buffer Source Pair Circuit Function --------- --------- ------- ------------- MTG FA_11 10 CA-2 static level MTG FA_12 12 CA-4 static level MTG FA_13 14 CA-6 static level MTG FA_14 16 FA-2 static level MTG FA_15 17 FA-3 static level Special MTG FA_31 once per 5 second pulse Recall that the MTG Channels are numberede 1:32 while the control registers for the channels are FA_0 through FA_31. In the table above the MTG outputs are refered to by their FA numbers. There are two copies of GAP and STROBE generated by the #2 Sequencer for the L1 Cal Trig. For now the 2 copies of GAP and STROBE are identical. These 4 signals are generated by the following Sequencer #2 Time Lines: TL # 9 Strobe copy 1 TL #12 Strobe copy 2 TL #10 Gap copy 1 TL #13 Gap copy 2 We will continure to use these 2 copies of both GAP and STROBE just in case at some point in the future we need to make 2 flavors of GAP and STROBE. The fanout path is the following: TL # 9 Strobe feeds MTG Bit Inputs 21 and 22 TL #10 GAP feeds MTG Bit Inputs 17 and 18 TL #12 Strobe feeds MTG Bit Inputs 23 and 24 TL #13 GAP feeds MTG Bit Inputs 19 and 20 The MTG Bit Inputs are routed directly to the MTG outputs. These we now have 4 copies of GAP and 4 copies of STROBE. They are: MTG Output 17 is GAP #1 which comes from TL #10 MTG Output 18 is GAP #2 which comes from TL #10 MTG Output 19 is GAP #3 which comes from TL #13 MTG Output 20 is GAP #4 which comes from TL #13 MTG Output 21 is STROBE #1 which comes from TL # 9 MTG Output 22 is STROBE #2 which comes from TL # 9 MTG Output 23 is STROBE #3 which comes from TL #12 MTG Output 24 is STROBE #4 which comes from TL #12 MTG Card Output Polarity Recall that the output driver on the MTG card is "inverting", that is the Direct output pin on the driver is tied to the even pin number on the output connector and the Complement output pin of the driver is tied to the odd pin number of the output connector. This is all rational when the "L" type PAL is used in the MTG Channel. If you skip the PAL and just directly jumper the Bit Input to the Output then you need to invert either the differential input pair or output pair. To make a jumper header to use in the output section of the MTG channel (instead of the PAL), use a 20 pin header and jumper ExtBit Input pin #9 to BitOut pin #19. ------------------------------------------------------------------------------ DATE: 29-SEPT:3-Oct-2003 At: Fermi Topics: Shutdown week 4, Work on L1 Cal Trig: Water Leaks, Power Supplies, Readout, And-Or Term Strobe and Gap signals. Cold Start: We are running TRICS 10.4.J and VME_Access 5.0.F Just tell TRICS to completely ignore L1 Cal Trig Water Leaks The M106-M107 leak is the bottom radiator. It is not the normal end cap of the header pipe leak but rather the leak is from the side of the header pipe. It is in the radiators header pipe on the M107 side of the radiator and was spraying right against the G10 shield on the M107 side. I believe that all of the leaking water was contained. It took about 5 hours of work to get the 4 radiators out. I gave all 4 of the radiators to Pete Simon to work on. He will patch the leak and solder over all the header pipe ends. Only the upper 2 radiators had the special fancy clamps on them that tried to stop leaks in the ends of the header tubes. It took a little over 8 hours to re-install these 4 radiators. Next time that I need to do this it would be easier wtih the small ratchet set and a 5/16 socket. There is no water at all right now from the M110-M111 leak. I assume it has stopped because the electronics is off and it is cold. The biggest puddle that I saw was about the size of a quarter. Power Supplies To get the M106-M107 radiators out I pulled Power Pan MM-1 from the top of M106. This pan had to come out anyway for repair of its -2V brick. MM-1 consists of: +5.0 Volt 50 Amp MSU SN# 88 15-SEP-93 -2.0 Volt 325 Amp MSU SN# 34 25-APR-91 -4.5 Volt 600 Amp MSU SN# 10 18-DEC-90 -5.2 Volt 10 Amp Linear Pull MM-4 from M108 - it runs the Tier 3 in M107. MM-4 consists of: +5.0 Volt 50 Amp MSU SN# 92 15-SEP-93 -2.0 Volt 325 Amp MSU SN# 83 29-MAY-91 -4.5 Volt 600 Amp MSU SN# 11 11-JUN-91 -5.2 Volt 10 Amp Linear Return the two -2.0V bricks MSU SN# 34 and MSU SN# 83 to MSU for rebuild. Put MM-2 and MM-3 back together with their original bricks (see log book from last week). Use external Capacitor Band-Aids on both the +5.0V and the -2.0V supplies, i.e. on both Pioneer Magnetics supplies. Put MM-2 back into rack M110 and put MM-3 back into rack M112. These are the racks from which these Power Pans were pulled a week ago. Cal Trig Readout work Cook enough Configuration and Transmit-Sequencer 22V10 PAL's for the Sept 2003 Distributor Caps that still need to be installed in the running system (6) and for spare DC cards (2) plus (1) spare of each for the spare PAL storage box in the cabinet at D-Zero. See the 17:20-SEPT-2003 log book entry for all the cooking details. Install the 6 remaining Sept 2003 version Distributor Caps with their new PAL's in the following order: M107 DC SN# 12 M108 DC SN# 13 M109 DC SN# 1 M110 DC SN# 2 M111 DC SN# 3 M112 DC SN# 4 Strobe and Gap for Cal Trig And-Or Terms. The L1 Cal Trig is currently using 2 blocks of 16 And-Or Terms. The Strobe and Gap signals for this come from 4 Time Lines on Sequencer #2, two identical copies of each signal. We now need to send Cal Trig And-Or Terms to a 3rd block of 16 - and thus need a 3rd copy of Strobe and Gap. In the full plan the Strobe and Gap signals sould come from the CTRO card. It is not practical to make a 3rd copy of each of these signals in Sequencer #2. So for now the cleanest thing to do is to use spare channels on the MTG in M103 to fanout Strobe and Gap. We should make 4 copies of each. This will require 8 MTG channels. We can go back to making just one copy of each in the Sequencer #2. The MTG in M103 is currently used to make: the static Lookup Page Select Address Lines, the TCC controlled Read A/B Write A/B signals for the 29525's a bunch of static ECL level signals that are sent to un-used sections of the 10116 receivers on the Bus Buffer Boards and the Cal Trig Mother Board Drivers to make the bias circuit in these receiver chips happy, to generate a once per 5 second pulse that is used as a stand in to cause the generation of the Collect Data L1 Qualifier. The current setup of the MTG in M103 is: Tier 1 BBB TSS Pair (Cable MBD MBD Maps this to Driven Bus TSS Backplane Tier 1 Backplane Buse by Pair) Input Bus Function -------- ------- ----- --------- ------------------------ MTG FA_0 19 4 C Read A/B MTG FA_1 21 6 A Write A/B MTG FA_2 25 10 J Energy MS Bit Adrs MTG FA_3 26 11 D Energy LS Bit Adrs MTG FA_4 28 13 L Momentum MD Bit Adrs MTG_FA_5 29 14 K Momentum LS Bit Adrs MTG FA_6 23 8 Not used - static drive MTG FA_7 30 15 S Not used - static drive Tier 2,3,4 Signal CBus Source Pair Circuit Function --------- ------ ------- ------------- MTG FA_? 10 CA-2 static level MTG FA_? 12 CA-4 static level MTG FA_? 14 CA-6 static level MTG FA_? 16 FA-2 static level MTG FA_? 17 FA-3 static level Special MTG FA_31 once per 5 second pulse See: www.pa.msu.edu/hep/d0/ftp/l1/cal_trig/timing/ cal_trig_timing_generation_ii.txt for details. The next step is to figure out which 8 channels in this MTG to use to fanout the Strobe and Gap signals for the L1 Cal Trig And-Or Terms. ------------------------------------------------------------------------------ DATE: 17:20-SEPT-2003 At: Fermi Topics: Shutdown week 2, Work on L1 Cal Trig Readout, Install more L2 Scaler Cables, High rate running, Power Supple work. COLD START Use the System Control Status Menu to tell TRICS to total ignore the L1 Cal Trigger. Cal Trig Readout Tests Try programming the 21-JULY-2003 version of the Configuration GAL using the DC_CONFG.JDC file. This results in the expected error at Vector #36 problem. Watching the screen it may make this error only at High Vcc. Does is process the rest of the Vectors after #36 ?? Switch to using DC_CONFG.JED and cook 5 parts that way without the Vectors. CheckSum = 8685. Program 5 of the 21-JULY-2003 version Transmit Sequencer GAL using the DC_TRANS.JDC file. These parts cook OK. CheckSum = 585F. I broke a pin on a 7C291A U26 PROM so I cooked some new ones. This is from the file U26NW = U26.NOW. The CheckSum = 0003F8d6. Using the modified Sept 2003 DC with its new Trans Sequencer and Configure PAL's it appears that the same standard contents in U26 that we have been running for the past 2 years is just fine - which is what I think should happen. This is the U26NW = U26.NOW file. Install the Sept 2003 version DC cards with their new PAL's in the following order: M103 DC SN# 8 M104 DC SN# 9 M105 DC SN# 10 M106 DC SN# 11 Watch the RED LED on the back of the Distributor Cap as the DC is told by \D0_Config\Init_Post_Auxi_L1CT.rio to Configure the ERPB LCA's and this LED does flash. Look at some normal readout and at Philippe's arrays of a different number "Excluded into" each CTFE channel in a rack. Both look OK. Major edit to \D0_Config\Init_Post_Auxi_L1CT.rio to completely change the section about how the Cal_Trig_Readout_Helper TCC Manual Control signals are used to wake up the DC and Configure the LCA's in the ERPB's. The old existing version of \D0_Config\Init_Post_Auxi_L1CT.rio was copied \D0_Config\Obsolete\Init_Post_Auxi_L1CT.rio_saved_on_19SEPT03 Spark used for L1 Cal Trig Readout for the past >1 year has been Spark_8_2.exo Now with the Sept 2003 version DC and its new PAL's we run Spark_10_3.exo The Bougie.dci file has been edited to point to Spark_10_3.exo. How fast should the new readout operate per L1_Acpt cycle ? 25 MHz divided by 2 is 12.5 MHz which is a 80 nsec period. 128 TT's per rack x 80 nsec = 10.24 usec. In the CT_Readout_Helper there is a 14 tick delay before the DC_Transmit_Trigger is asserted. This is 14 x 132 nsec tick = 1.848 usec. There may be up to another 500 nsec spent waiting around before actually starting to move data. 10.24 + 1.85 + 0.5 = 12.59 usec. Level 2 Scalers Plugged in 8 more L2 Scaler Cables. These 2 Gated Scaler Paddle Cards are both in Slot 11 of M123 bottom crate. Slot 10 upper (P2) is still open. We still have 2 more Gated Scaler Paddle Cards here at Fermi. High rate running ---> Time to Process L1_Acpt TFW L1 Cal Trig ---------------------------- L1 Acpt Rate % L1_Busy % L1_Busy TFW L1 Cal Trig ------------ --------- ----------- --------- ----------- 4,277 Hz 3.2 % 7.3 % 7.5 usec 17.1 usec 17,038 12.3 28.3 7.2 16.6 19.056 14.1 31.9 7.4 16.7 This is before making any changes to the L1 Cal Trig readout. Power Supply Work >From M112 pull the Tier 2-3 Power Pan that supplies the Tier 2 Crate in rack M111. This pan pulled from here is Tier 2-3 MM- MM- consists of: +5.0 Volt 50 Amp MSU SN# 93 15-SEP-93 -2.0 Volt 325 Amp MSU SN# 82 25-APR-91 -4.5 Volt 600 Amp MSU SN# 29 7-FEB-92 -5.2 Volt 10 Amp Linear >From M110 pull the Tier 2-3 Power Pan that supplies the Tier 2 Crate in rack M109. This pan pulled from here is Tier 2-3 MM-2 MM-2 consists of: +5.0 Volt 50 Amp MSU SN# 91 15-SEP-93 -2.0 Volt 325 Amp MSU SN# 48 15-MAR-91 -4.5 Volt 600 Amp MSU SN# 16 11-JUN-91 -5.2 Volt 10 Amp Linear Pulled from these Power Pans the -2 V bricks and take back to MSU for rebuild -2.0 Volt 325 Amp MSU SN# 82 25-APR-91 and -2.0 Volt 325 Amp MSU SN# 48 15-MAR-91 ------------------------------------------------------------------------------ DATE: 15-SEPT-2003 At: MSU Topics: New L1 Cal Trig Gains Calibration The following table compares the gains that we ran in the L1 Cal Trig prior to the shutdown that began on 8-SEPT-2003 with the gains that we propose to run after the shutdown. These proposed new gains are from Rahmi Unalan analysis. See his email note from 14-AUG-2003. EM -- Proposed Proposed to to Theoretical Currently Proposed Current Theoretical Required Operating new Percent Percent TT Eta Gain Gain Gain Change Change ------ ----------- --------- ------- -------- ----------- 1 1.54 1.28 1.27 - 1 % - 18 % 2 1.48 1.23 1.22 - 1 % - 18 % 3 1.37 1.14 1.13 - 1 % - 18 % 4 1.24 1.03 1.02 - 1 % - 18 % 5 1.08 0.90 0.89 - 1 % - 18 % 6 0.93 0.77 0.76 - 1 % - 18 % 7 0.76 0.42 0.58 + 38 % - 24 % 8 0.66 0.55 0.50 - 9 % - 24 % 9 1.53 1.27 1.37 + 8 % - 10 % 10 1.26 1.05 1.13 + 8 % - 10 % 11 1.05 0.87 0.94 + 8 % - 10 % 12 0.87 0.72 0.77 + 7 % - 11 % 13 0.70 0.70 0.62 - 11 % - 11 % 14 0.57 0.57 0.50 - 12 % - 12 % 15 1.49 1.49 1.49 0 % 0 % 16 1.22 1.22 1.22 0 % 0 % HD -- Proposed Proposed to to Theoretical Currently Proposed Current Theoretical Required Operating new Percent Percent TT Eta Gain Gain Gain Change Change ------ ----------- --------- ------- -------- ----------- 1 1.54 1.54 1.75 + 14 % + 14 % 2 1.48 1.48 1.68 + 14 % + 14 % 3 1.37 1.37 1.44 + 5 % - 5 % 4 1.24 1.24 1.31 + 6 % - 6 % 5 1.79 1.73 1.99 + 15 % + 11 % 6 1.54 1.49 1.94 + 30 % + 26 % 7 1.26 0.82 0.82 0 % - 35 % 8 1.10 1.06 0.95 - 10 % - 14 % 9 1.53 1.53 1.53 0 % 0 % 10 1.26 1.26 1.26 0 % 0 % 11 1.05 1.05 0.95 - 10 % - 10 % 12 0.87 0.87 0.78 - 10 % - 10 % 13 0.70 0.70 0.63 - 10 % - 10 % 14 0.57 0.57 0.51 - 11 % - 11 % 15 1.49 1.49 1.60 + 7 % + 7 % 16 1.22 1.22 1.31 + 7 % + 7 % Note: Incorrect Hadronic BLS resistors were used for TT eta's 1:8. Because of the incorrect HD BLS resistors, in the eta range 1:8 the Term-Attn-Brd's only receive 60% of the of the planned signal level. For TT eta's 1:4 this was compensated for by using special Term-Attn-Brd's that corrected for the BLS HD signal levels. In the range TT eta 1:4 if everything is ideal then the proposed and theoretical gains should match. For TT eta's 5:8 the incorrect HD BLS resistors are compensated for by running the Term-Attn-Brd at a higher HD gain than what you would need to based just on geometry. To clearly show this the "Theoretical HD Gain" listed for TT eta's 5:8 in the table above includes the boost in gain that is required to make up for the incorrect HD BLS resistors in this eta range. For TT eta's 5:8 the listed "Theoretical HD Gain" is 1.66x what you would expect based on geometry alone. Because the "Theoretical HD Gain" is listed this way, in the range TT eta 5:8 if everything is ideal then the proposed and theoretical gains should match. For reference recall where the boundaries are: TT eta's 1:6 are from the CC BLS cards TT eta's 7:20 are from the EC BLS cards. The BLS to L1 Cal Trig calibration ranges are TT eta's 1:8 9:14 15:20 In the EM Calorimeter it is TT eta 7 that almost does not exist. In the HD Calorimeter it is TT eta's 5 and 6 that only partially exist. ------------------------------------------------------------------------------ DATE: 8:12-SEPT-2003 At: Fermi Topics: First week of Shutdown, Water Leaks, MCH-1 Breaker, Master Clock, Noise Study Cal Trig Cold Start: Set coverage to eta 1:8. We are running only the first 4 racks and Tier 3. You may want to set the ignore CT errors and keep moving forward. Use files: \Scratch\Gains_1_8_1_32_rev_5.tti \D0_Log\Find_DAC_V5_0_F_20030912.tti The file \D0_Config\Init_Post_Auxi_L1CT.vio has been modified so that the VRB's only readout data for the first 4 racks. See 26:30-AUG-2003 for the rest of cold start (or pointer to it) Water Leaks Leak between M110 and M111. I have seen the green LED for this channel on the 16 channel Drip detector both OFF and ON. With the Cal Trig turned off for some hours there is a small amount of water on this drip detector strip - perhaps 10 drops near the connector end of the strip. So far I can not see any water running down the mud-flaps. The other water leak that needs to be investigated & fixed during this shutdown is between M106 and M107. M106:M107 is still leaking. The breaker for all of MCH-1 was replaced Monday morning as part of the power outage. I do not know whether or not the shut trip circuit was tested to verify that nothing in it is flakey and causes random trips. The power outage for Friday 12-SEPT-03 has been cancled. The next power outage that I know of is Monday 29-SEPT-03. Master Clock Could not reload the Master Clock after the power outage on Monday the 8th. The Clock GUI could talk to and control the various PCC and Sequencer control status registers just fine - but it could not put data into the "next adrs", "static", or "dynamic" buffers. When you asked the GUI to load a buffer you saw no VME activity LED on the module in question. You did see the VME activity LED when you just talked to a CSR. Fritz tracked it to being a hex vs decimal problem with the data base template file that was introduced early in June when he added the alarms to the Master Clock. We did tests all of this 25-27 June but we did not power cycle the Master Clock as part of the tests. The buffers were loaded, the test did VME IO to lala land so the buffers were never touched. EPICS is setup do it does not care that the VME cycles timed out. Fritz said there there is a problem in the EPICS document. Noise RF/4 Fell off about as expected as the central iron opened. Most channels fell to constant values by the time it was open 15" or so. Boris has chokes for all A Layer PTD's. An issue is well they saturate when the torid is ON which has about 200-400 Gauss in the area where the torrids will be. Another big question is that many times when Muon PTD power cycles the RF/4 noise levels that we see on some channels come back to different stable values. It is not clear that the change in RF/4 levels have to do with whether or not the PDT's are all in snyc or not. The cores may be helping. Noise 10 MHz After the power outage - during the controlled turn ON the 10 MHz levels were low. On the Tek portable scope we could see the 10 MHz line come and go as the SMT Sequencers were turned ON/OFF. We could even see that the 10 MHz came from specific SMT Sequencers. The next day the 10 MHz line was about 4 times higher even with all SMT Sequencers turned OFF. So far no understanding of what is going on. In this state the 10 MHz line goes away when the PreAmp power is tuned OFF. Welder Noise It appears 100% uniform in phi. We monitored it in North South East and West. With iron closed it is about 50% bigger in North than it is in South. In North it looks saturated on one side of its swing. When East iron opened the South Welder noise fell by a factor of about 3 on both its East and West sides and North did not change. When West iron opened that was not much change. With both open North is about 3.5 or 4 times bigger than South. The current thought about this entry point is the 10**9 Instrumentation cables that go into the cryostat and clearly are not shielded like the HV is. ------------------------------------------------------------------------------ DATE: 4-SEPT-2003 At: MSU & Fermi Topics: Missing Et Trigger Run Ran the Missing Et L1 Cal Trig in the beam. This was on 4-SEPT-03 at about 1:20 in the afternoon Chicago time. The luminosity was about 14E30 at the time. We used 4 Missing Et triggers with thresholds of 10, 15, 20, and 25 GeV. The run lasted about 10 minutes. We did not see any DAQ or readout crate problems during this 10 minute run. During the run the total L1_Acpt rate was about 30 Hz. All of these events were passed at L2 and at L3. This was a recorded run, Run Number 180,888 Spec Trig's 0 and 1 were used for the zero bias triggers to get the standard luminosity normalization data. The Missing Et Trigger setup was: Before Run During Run Spec MEt MEt And-Or Net Spec Trig Trig Threshold Comparator Fired Rate PreScale Fired Rate ---- --------- ---------- ---------- -------- ---------- 2 10 GeV 0 2739 Hz 270 9.2 Hz 3 15 GeV 1 496 48 8.3 4 20 GeV 2 40 4 8.4 5 25 GeV 3 5.7 1 4.9 The "Before Run And-Or Network Fired Rate" was with the DAQ system quiescent (i.e. no SMT readout noise). The numbers are the average of 3 TrgMon sweeps of 5 seconds each. The "During Run Spec Trig Fired Rate" is the average of 3 TrgMon sweeps of 1 minute each. This was using Momentum Look Up Page #2. Coverage was TT eta's 1:16 We do need to fix the COOR Resource file so that it points to And-Or Terms 152:155 for Missing Et Comparators 0:3. The following snapshots were extracted from 5 successive web trigmon summaries. (The time shown is local MSU time, one hour ahead of FNAL) Level 1 Trigger Framework Specific Triggers 04-Sep-2003 14:17:47 Integrat Period L1/L2FW = 56.9 s / 56.9 s L1 Accept = 29.92 Hz/ 1278546 Operational:Yes Current:Yes Triggered:Yes L1 FW Paused= 0.2 % /NowRunning L2 Bypassed:No Outstanding L1 Accept: 0 L2 Accept = 29.92 Hz/ 540193 Last FPGA Configure = ----------------- L2 Accept/Reject= 100 % / 0 % Last FW Initialize = 04-Sep-2003 13:06 L2 FW Stalled by L2 Busy = 0.0 % Last SCL Initialize = 5 mn 6 s Luminosity Block Num = 0x 0028 50d2 Last LBN Increment = 3 s Tick / Turn = 140/ 205906563 Allocated SpTrg: 6 ExpGrp: 1 GeoSect: 70 Spec| L1 | L2 Accept |And-Or|Prescl|Total|ExpGp|ExpGp| L3 | COOR|Exp Trig|Accept| | Fired| Ratio|Expos| Live|L1 Bz|Disab|Disab|Grp ---#|----Hz|----Hz|----%|-------|----Hz|------|----%|----%|----%|----%|----%|--# 0| 0.53| 0.53|100 | 1001|506.6k|770000| 0.0| 0 | 0.1| 0 | 0 | 0 1| 0.54| 0.54|100 | 719|1.717M|3.400M| 0.0| 0 | 0.1| 0 | 0 | 0 2| 8.68| 8.72|100 | 2710|3069.4| 270| 0.1| 0 | 0.1| 0 | 0 | 0 3| 8.33| 8.37|100 | 2546|769.20| 48| 0.5| 0 | 0.1| 0 | 0 | 0 4| 8.17| 8.18|100 | 4311|250.29| 4| 5.6| 0 | 0.1| 0 | 0 | 0 5| 5.36| 5.39|100 | 1862|149.99| 1| 22.6| 0 | 0.1| 0 | 0 | 0 Level 1 Trigger Framework Specific Triggers 04-Sep-2003 14:18:49 Integrat Period L1/L2FW = 66.9 s / 66.9 s L1 Accept = 31.06 Hz/ 1280625 Operational:Yes Current:Yes Triggered:Yes 2| 9.49| 9.46|100 | 3343|3065.3| 270| 0.1| 0 | 0.1| 0 | 0 | 0 3| 8.32| 8.28|100 | 3100|775.94| 48| 0.5| 0 | 0.1| 0 | 0 | 0 4| 8.11| 8.11|100 | 4854|256.25| 4| 5.6| 0 | 0.1| 0 | 0 | 0 5| 5.35| 5.30|100 | 2217|155.11| 1| 22.6| 0 | 0.1| 0 | 0 | 0 Level 1 Trigger Framework Specific Triggers 04-Sep-2003 14:19:52 Integrat Period L1/L2FW = 61.8 s / 61.7 s L1 Accept = 30.64 Hz/ 1282519 Operational:Yes Current:Yes Triggered:Yes 2| 9.64| 9.67|100 | 3940|3095.8| 270| 0.1| 0 | 0.1| 0 | 0 | 0 3| 8.45| 8.44|100 | 3621|789.30| 48| 0.5| 0 | 0.1| 0 | 0 | 0 4| 8.40| 8.40|100 | 5373|254.07| 4| 5.6| 0 | 0.1| 0 | 0 | 0 5| 4.51| 4.52|100 | 2496|151.56| 1| 22.6| 0 | 0.1| 0 | 0 | 0 Level 1 Trigger Framework Specific Triggers 04-Sep-2003 14:20:54 Integrat Period L1/L2FW = 61.8 s / 61.9 s L1 Accept = 30.31 Hz/ 1284395 Operational:Yes Current:Yes Triggered:Yes 2| 9.34| 9.31|100 | 4517|3119.6| 270| 0.1| 0 | 0.1| 0 | 0 | 0 3| 8.11| 8.15|100 | 4126|794.97| 48| 0.5| 0 | 0.1| 0 | 0 | 0 4| 8.56| 8.55|100 | 5903|252.66| 4| 5.6| 0 | 0.1| 0 | 0 | 0 5| 4.78| 4.78|100 | 2792|151.60| 1| 22.6| 0 | 0.1| 0 | 0 | 0 Level 1 Trigger Framework Specific Triggers 04-Sep-2003 14:22:01 Integrat Period L1/L2FW = 67.1 s / ---- s L1 Accept = 31.96 Hz/ 1286542 Operational:Yes Current:Yes Triggered:Yes 2| 9.88|------|100 | 5181|3133.0| 270| 0.1| 0 | 0.1| 0 | 0 | 0 3| 8.56|------|100 | 4698|807.96| 48| 0.5| 0 | 0.1| 0 | 0 | 0 4| 9.23|------|100 | 6523|263.81| 4| 5.6| 0 | 0.1| 0 | 0 | 0 5| 4.93|------|100 | 3124|158.63| 1| 22.6| 0 | 0.1| 0 | 0 | 0 ------------------------------------------------------------------------------ DATE: 26:30-AUG-2003 At: Fermi Topics: Work on TT's, Power Outages, TCC1 restart, Master Clock error display, Pseudo Term test, Missing Et Runs, Survey for Noise Measurement, Pbar Injection Marker Cold Start - the only change to the cold start instructions in 28:31-JULY-2003 is that we are now running TRICS 10.4.I The dates of the power outages during the shutdown are: Sept 8, 29 Oct 13, 18, 21, 25 Check some TT's +6,21 EM Check on the scope. I can see nice noise pulses. Pulses vary between 100 mV and very big. Make a single TT Ref Set run. Collect ?? events in Run # 180,686. I'm not certain how many events were collected. The DAQ Shifter had to stop the run to start doing other things. I think we got about 5 events. I do not know what they looked like on the event display monitor, i.e. did Precision Cal Readout see anything. -9,22 HD This channel has been Excluded since 6-DEC-2002. Checked on a scope. See once per N minutes massive pulse that sometimes saturates and often has an after pulse. Make a single TT Ref Set run. Collect 21 events in Run # 180,685. Precision Readout is clearly involved. There are 10's of GeV in 20% of the Calorimeter channels. There is a ridge of stuff at the most negative eta, a ridge in the center, and a ridge at the most positive eta. Mostly blue and some red. When this channel sparks something is effecting all 3 cryostats. -8,9 HD Checked on a scope. See 150 mV pulses associated with readout. Stop the L1_Acpt's and the noise pulses stop. -6,25 EM Checked on a scope. See a 100 mV to 150 mV noise pulse about every 20 usec. +8,27 HD Checked on a scope. See continuous 200 mVpp moise bursts. +9,21 EM +9,21 EM has been excluded since 8-JAN-2003 Checked on a scope. See once per 5 minutes massive pulse or series of pulses. Make a "single TT Ref Set" run. Collect 12 events in Run # 180,684. Precision Readout is clearly involved. There are 10's of GeV in 20% of the Calorimeter channels. There is a ridge of stuff at the most negative eta, a ridge in the center, and a ridge at the most positive eta. Mostly blue and some red. When this channel sparks something is effecting all 3 cryostats. As part of this "single TT Ref Set runs" make a .msg fine in \Scratch called Setup_L1CT_Single_TT_Run.msg The easiest way to make these runs is to edit this file and the execute it. That is much master than making of each COOR message. To "Un-Exclude" the TT that you want to look at you typically need to write $ff into FA=81. A big question is how do you know that the noisy Cal Cell has not been supressed in the Precision Cal Readout ?? TCC1 restart Wednesday morning after the store was over D0TCC1 had a problem. TRICS stopped but this was probably caused by some other process using up all of the page file. The TRICS menu window was gone. The TRICS Log window was still displayed but it was not moving once every 5 seconds with new monitoring data. There was a window that said Exception unknown software exception 0x000000fd occured in application at location 0x005b60a7 click OK Terminate and then another window Exception unknown software exception 0x000000fd occured in application at location 0x77e8bc20 click OK Terminate and then TRICS Log window goes away. This may have all been caused by trying to figure out why the screen saver had stopped working for the past 2 or 3 weeks, or by running FTP, or by a vnc connection that got into trouble. D0TCC1 was booted and TRICS auto-started. Philippe added performance monitor to the tasks that wake up with auto-start. Once everything was running again it looks like the system and application programs start up using about 15% of the page file. Watch things that evening and yes, screen blank is back to working normally so yes, something was stuck. Master Clock Error Display Wednesday after the store the Beams Division "booted" the Low Level RF system. This caused the Master Clock to throw the normal Missing Sync, Sync Timing, and Sequencer Hold errors. What showed up on the Significant Event Display (at the Captains station anyway) were errors named TRG_CPCC_M000/SMIS and TRG_CPCC_M000/STIM. I checked with Fritz and the M000 is not a type of M100. He is going to send me instructions about how to write text to put behind the Guidance button for the different errors that Master Clock can display. The place to look for information about how to write Guidance files is: www-d0online.fnal.gov/www/groups/ctl/html/ses_guidance.html Test of Pseudo Terms Recall that what is actually implemented in the Pseudo Term L1 AONM FPGA is defined in: www.pa.msu.edu/hep/d0/ftp/l1/framework/andor_terms/ pseudo_term_implementation.txt Start test 10:50 on Thursday. Stop the SCL via the push botton on the Single Chance Test Menu. Edit the \DCF\l1_aonm.dci file to change from \l1_aonm\l1_aonm_32_3.exo to \l1_aonm\pt2_aonm_1_1.exo Stop Monitoring requests via the control menu. Configure all of M123. 685 configure 0 errors. Exit TRICS 10.4.H Start TRICS 10.4.I Full System Initialize looks OK and the SCL comes back ON. Coor downloads a standard TFW_Only run and it runs OK. Recall that pt2_aonm passes AOT 251 (Tick Select #0) as AOT 351 i.e. the highest number P-Term in the Upper AONM. Add AOT 351 to Spec Trig #0 i.e. make it -247 255 351 AO-Fired drops to 47712 HZ AOT 351 does not show up in the TrgMon Spec Trig #0 display. Adjust the PreScale to bring it up to 2 Hz Add AOT 251 to Spec Trig #0 i.e. make it -247 251 255 351 Everything stays the same Change it to -247 252 255 351 i.e. AOT for Tick Select #1 and P-Term of Tick Select #0 Now adjust Tick Select #1 wrt Tick Select #0 Set Tick Sel #1 is 1> Tick Sel #0 see the And-Or Fired rate = 0.00 Set Tick Sel #1 = Tick Sel #0 see the And-Or Fired rate = 47712 Set Tick Sel #1 is 1< Tick Sel #0 see the And-Or Fired rate = 0.00 Note that the TrgMon Main Spec Trig display showed And-Or Fired = 0.00 but the TrgMon display for Spec Trig #0 showed a And-Or Fired increment of 0. I watched this for at least 5 sweeps of TrgMon. Ask COOR to release this trigger. Full Initialize at TCC to clean up after this playing around. It looks OK. Ask COOR to re-download TFW_Only and it runs fine again. Give the system to Elizabeth and DAQ Shifter. They download a real Trigger File that includes the P-Term that is an OR of 3 Hardware AOT's. In the rates display the ORing looks OK. End of test 11:56 Missing Et Runs People are ready to make some runs with the Missing Et trigger. A question is what Symmetric Low Energy Cut to use. Currently the file D0_Config\ Init_Post_Auxi_L1CT.cio is set to select Momentum Lookup Memory Page #2 for the first lookup. Using Page #2 has a rational set of Symmetric Low Energy Cuts to use for these first beam tests. If we want to change which Momentum Lookup Page is used, we have .cio files with obvious names to do this in the \Scratch directory. There is no point in trying Page #0. Note that Init_Post_Auxi_L1CT.cio has been set to select Page #2 for the first lookup since 10-JULY-2003. For Momentum Lookup we have the MSBit Low for the first lookup and Hi for the second lookup. The Missing Et trigger is based on the first lookup. Recall that since a power up we also need to execute D0_Config\ Init_Missing_Et.cio by hand before the Missing Et trigger can be used. For Reference: Momentum Lookup Page Number Symmetric Energy Cuts Implement by this Lookup Page ----------- ----------------------------------------------------- 0 Zero Symmetric Energy Cut at all eta. 1 1.0 GeV Et Symm Energy Cut for TT eta 1:16 2 1.5 GeV Et Symm Energy Cut for TT eta 1:12 2.0 GeV Et Symm Energy Cut for TT eta 13:16 3 2.0 GeV Et Symm Energy Cut for TT eta 1:12 3.0 GeV Et Symm Energy Cut for TT eta 13:16 Survey to get ready for Noise Measurement This was looking with the scope plugged into the Lemo output on the CTFE i.e. not the normal look and the Inverting side of the differential signal. In all cases this is looking at the HD side of the TT. TT ----- ------------------ +1,2 1800 uV clean +1,19 1100 uV clean -4,26 800 uV clean +6,8 700 uV <-- 10 MHz +8,16 1400 uV clean -8,15 500 uV -6,19 400 uV <-- 10 MHz +12,13 700 uV clean -9,8 1400 uV clean <-- 10 MHz Dates of some of the previous studies are: 29-MAY-03 24-APR-03 10-APR-03 Pbar Injection Marker Pass to Gaston Gutierrez the name Alvan Harms as some one that he can talk with about getting a signal that indicates when the Pbars are injected into the Tev. The 279 module in slot 17 does not appear to be used. Do not touch the 279 module in slot 20 because it make the Sync marker for the Master Clock. ------------------------------------------------------------------------------ DATE: 12:15-AUG-2003 At: Fermi Topics: Work on TT's, SCL Status GS 25, SCL Receivers back, Cal Noise work, new pager The connector at the G.S. end of SCL Status Cable for G.S. 25 had been put on wrong. The normal problem. Replace the connector and Reinhard checked it and it is now OK. I got 2 SCL Receivers back from Don. He has about 4 more spares that he is using to test the new VRBC cards. I gave one of the two that I got back to STT people. They now have all that they need. I put the one spare in the labled storage cabinet. Talked with Marvin and Dean about Cal noise pickup from the welder. The double shielded transformer will only cost $1300. Marvin has talked with Ted about making an optical SCL and it sounds like that is going to happen and thus take the SCL links completely off the list of possible problems. Trigger Tower Work -14,2 HD This was a center to center short on the CTFE. The problem was a flaky short on the underside of the black plastic part of the socket for the Terminator Pack. Watched the short on the Ohm meter go away as I scraped the underside of the socket. Then air hosed it well. -6,22 EM Last trip this had been a 40mV to 50mV pulse every 2 usec or so. Dean cut the resistor for (eta,phi,layer)=-11,44,7 (crate,adc,bls,tower,depth) =2,10,3,1,6. Dean also says that this is a "high noise level" channel in the Cal Precision Readout and thus has been suppressed in the readout (thus when we triggered on just this TT we did not see anything in the event display). Current theory is that this is a Pre-Amp problem. Now this week with the cut resistor -6,22 EM signal looks OK. With no beam in the machine all you see is a 40mV or so (each side of the line) normal waveform bump once every minute or so. -6,28 EM Last trip this had been a 800 mV (each side of the line) "spark" that took 1100 nsec to rise. We looked at this with a single TT Ref Set and collected events with clear evidence of sparking at (eta,phi,layer)=-12,56,1 (crate,adc,bls,tower,depth)=9,2,7,3,0. Dean cut that resistor. Now this week with a cut resistor -6,28 EM lookes OK. You still can see a signal of about 25 mV on each side of the cable that takes about 1100 nsec to rise. So we think the Calorimeter is still sparking and cutting the resistor reduced the effect of this by a factor of 30 or so in the Trigger Pickoff signal. +6,23 EM Last trip this had been a 100 mV to 150 mV (on each side of the cable) pulse that happened every 5 or 6 usec. Dean cut the resistor at (eta,phi,layer)= +11,46,7 (crate,adc,bls,tower,depth)= 3,1,3,0,6. Dean also says that this is a "high noise level" channel in the Cal Precision Readout and thus has been suppressed in the readout (thus when we triggered on just this TT we did not see a very big effect in the event display). Current theory is that this is a Pre-Amp problem. Now this week with the cut resistor +6,23 EM signal looks OK. With lots of watching between stores I have only seen things that look like real energy deposit waveform of 1 or 2 GeV. +8,28 EM Last trip did see one big fast rise time spark looking pulse. Dean looked at data from a ZB run and thinks that he knows what Calorimeter cell it is. Dean cut the resistor for (eta,phi,layer)=+16,55,7 or (crate,adc,bls, tower,depth)=6,6,1,3,6. Now this week we see Tuesday night right at the beginning of the store, -3,17 BLS card blows up. This killed the precision readout from that BLS card and killed the TT pickoff signal. I could clear see the problem with -3,17 on TCC's Cal Trig display of TT over Ref Set "even once". Exclude -3,17 EM and HD both on the fly and in the Excluded TT List file. Looking at -3,17 on the scope it is 200 mV pp each side of the line EM with a 1.4 usec period. HD is the same but 50 mV pp. Wednesday afternoon Dean fixed this BLS. Now it looks OK on the scope. Stop Excluding it Wednesday morning during store. Cal Jet rates start to creep up. CalMuon shifter can see +2,29 HD in the L1 Cal Trig Examine display. Change runs and Exclude +2,29 HD. That takes care of the problem. On the scope you can clearly see that +2,29 HD is making bursts of noise of about 100 mV on each side on the line. Record 4,702 events of this noise in Run # 180,051 with a 4 GeV Tot Et Ref Set focused just at +2,29 HD During this run you could see this noise in the Precision Readout event display. +2,29 HD is now in the Excluded TT list file. Thursday morning. The current worst TT is -8,9 HD Received a new pager from Dmitri. The phone number for the new pager is: 517-232-1037 To use it: Dial the number. You will hear a beep or perhaps three quick beeps (but no verbal prompt). Enter the phone number at which you want to be called back, and then type the pound (#) sign. Then hang up. There was enough TT stuff going on this week that I wanted to send you a note to make certain that we have stayed coordinated. New Friends ----------- -3,17 This is the TT that had trouble starting Tuesday night during the ramp right at the beginning of the store. It was Excluded for the store Tuesday night and Wednesday morning. You fixed it on access Wednesday afternoon and it is no longer Excluded from the Cal Trig. While it was bad what we could see on the scope from MCH was a sine wave with a period of about 1.4 usec and 200 mV pp for EM and 50 mV pp for HAD. You found a SCA card with shorted power bus. +2,29 This HAD signal because noisy Wednesday morning during the store. It became enough of a problem that we Excluded it that morning (and then trigger rates settled back down and muon quit having readout problems). On the scope it looks like bursts of noise of about 100 mV just on HAD. Both sides of the cable have this noise and it looks differential. It is not symmetric noise like SMT pickup. Run # 180,051 has about 4702 events taken with a 4 GeV Total Et Ref Set looking at just +2,29. We did see energy from these events in the Precision Cal Readout. For now +2,29 HD will be Excluded from the Cal Trig. Old Friends ready to rejoin the trigger --------------------------------------- -6,22 EM This had been a 40mV to 50mV pulse every 2 usec You think this is a Pre-Amp problem. For now you cut a resistor on the BLS. Now this TT signal looks fine. I will stop Excluding this TT from the Cal Trig as soon as things settle down. -6,28 EM This had been a 800 mV "spark" that took 1100 nsec to rise. These "sparks" had happened once every few minutes. You cut a resistor. I can still see the spark but now it is only 25 mV. So I think that the Cal is still "sparking" and that you got the correct resistor. I will stop Excluding this TT from the Cal Trig as soon as things settle down. +6,23 EM This had been 100 mV to 150 mV pulse that happened every 5 or 6 usec. You think this is a Pre-Amp problem. For now you cut a resistor on the BLS. Now this TT signal looks fine. I will stop Excluding this TT from the Cal Trig as soon as things settle down. Old Friends still in need of help --------------------------------- +8,28 EM We have seen a large fast rise-time "spark" on the scope for this TT. Typically this is very low rate. From ZB data you had a hint about what cell is the cause and you cut a resistor. Looking at this channel with a scope on Thursday I could see a pulse of 75 to 150 mV every couple of minutes. It looks more or less like a normal energy pulse but comes to a point at the top. Thursday night make a single run with a 3 GeV EM Ref Set on just +8,28 EM. In 2 hours collect just 1 event (which on the scope looks like at least 15 GeV). This is Run # 180,124 For now I will continue to Exclude +8,28 EM. -4,7 We have seen both -4,7 HD and -4,7 EM look funny on the scope. You were able to see something in ZB data for -4,7 HD and you have cut a HD resistor. Thursday on the scope -4,7 HD still had some big pulses at the once every few minutes rate. -4,7 EM on the scope had perhaps a few un-explain small normal energy shape pulses. Make Run # 180,098 with a 4.5 GeV Total Et Ref Set on just -4,7. Collected 14 events. We did see energy in the Cal Precision Readout Event Display - mostly blue but at least 1 event with some red. For now -4,7 EM & HD will remain excluded. +2,25 EM This is an Excluded channel and this week I could see low rate but large amplitude "sparks" on the scope. I made a run with a 3 or 4 GeV EM Ref Set focused just on +2,25 EM. This is Run # 180,076 It should have 36 events. I saw the first event and the last event of this run on the Event Display. Precision Cal Readout is involved. There is a stripe of red for most all CC eta's at phi = 25 in these events. For now +2,25 EM will remain excluded. -4,25 HD Thursday on the scope -4,25 HD had some very big strange looking pulses at the once every few minutes rate. Make Run # 180,104 with a 4.0 GeV Total Et Ref Set on just -4,25. Collected 25 events. We did see energy in the Cal Precision Readout Event Display. For now -4,25 HD will remain excluded. Still need more investigation ----------------------------- +6,21 EM Checked this channel on Thursday with the scope. I can see nice noise pulses. Pulses vary between 100 mV to very big. Need to make a single TT Ref Set run on +6,21. For now +6,21 EM will remain Excluded. +9,21 EM Checked on a scope and saw no problem. I will check it again next trip and if it is still OK I will try putting it back into the trigger. -9,22 HD Checked on a scope and saw no problem. I will check it again next trip and if it is still OK I will try putting it back into the trigger. Summary of Runs --------------- Number Run Number Looking at of Events ---------- ------------ --------- 180,051 +2,29 HD 4702 180,076 +2,25 EM 36 180,098 -4,7 EM & HD 14 180,104 -4,25 HD 25 180,124 +8,28 EM 1 (but it should be a good one) ------------------------------------------------------------------------------ DATE: 28:31-JULY-2003 At: Fermi Topics: Trips of the L1 Cal Trig - Fan Belt, Survey of the Trees, First test of Run IIB Cal Trig, Trigger Tower work, Ring-of_Fire Source, new Tree Dump command files. Cold Start For the full version see 20:23-MAY-2003 current setup is We are running: TRICS V10.4.H and VME_Access V5.0.F The current Gains files are: D0_Config/Gains_17_20_1_32_rev_a.tti # TT eta's 17:20 D0_Config/Gains_1_16_1_32_rev_4.tti # TT eta's 1:16 The current Pedestal files are: D0_Log/Find_DAC_V0_0_D_20030627_Edited.tti;2 # TT eta 17:20 D0_Log/Find_DAC_V5_0_E_20030728.tti;1 # TT eta 1:16 If you are going to be using the L1 Cal Trig Missing Et trigger then you need to run \D0_Config\Init_MissingEt.cio from the CBus I/O menu. L1 Cal Trig Power Trips The problem that caused the 5 trips of the L1 Cal Trig appears to have been the air blower. I never heard/saw it but it was reported that the blower would begin to squeek and slow down and the differential pressure would drop and the magnehelic would finally trip it off. This was reported to me by the day shift on Monday. Monday early day shift they replaced the drive belt. The old belt did not look glazed and it did not smell of hot belt. Pete said that it was loose. Very strange. At 21:00 after end of store we turned off L1 Cal Trig and Pete and Jim looked at the blower some more. They also checked the contactor box for it. All looks OK. We stopped and started the blower 3 times and all appeared well. Late Monday night after the L1 Cal Trig had been running for 1 1/4 hours, make a run of Find_DAC. EM,HD +- 1:16 1:32 2 8 x x. The file that it makes is: D0_Log/Find_DAC_V5_0_E_20030728.tti;1 There are no errors in it and it looks OK when loaded. Up date cold start to use this file. Need to remember to run the misset.cio file need to add it to cold start. Need to make a best effort to setup all the CAT2 cards. Look at some more Missing Et rates. Luminosity = 0 L1 Acpt rate = 0 Hz Page 2 10 GeV 1296 Hz 1207 1109 1259 20 GeV 9 Hz 8 5 4 30 GeV 4 Hz 4 1 3 40 GeV 2 Hz 1 1 1 Page 3 10 GeV 14 Hz 14 17 22 11 17 20 GeV 7 Hz 6 7 8 6 7 30 GeV 4 Hz 3 6 3 2 3 40 GeV 3 Hz 2 4 2 0 1 Survey of Adder and Counter Trees Tier 1 Crates All the Tier 1 crates except for the 2 highest eta racks M111 and M112 have all 4 CAT2 cards installed and plugged in. In M111 and M112 Tier 1 only the Px and Py CAT2 cards are installed and plugged in. The EM and HD CAT2 cards in M111 and M112 Tier 1 are not installed. All the Tier 1 CAT2 cards use their Un-Corrected output. All the Tier 1 crates have Large Tile cables plugged into their Px and Py CAT2 card Comparator outputs. The Tier 2 crate for eta 1:8 in M105 has the following setup: 4x Missing Et Adder Tree CAT2 cards installed and plugged in. The +Px and +Py CAT2 cards use their Un-Corrected output. The -Px and -Py CAT2 cards use an inverter cable and their Corrected output. 4x EM and HD Adder Tree CAT2 cards installed but not plugged in. All 4 of the Energy Adder Tree CAT2 cards use their Un-Corrected output. 16x Counter Tree CAT2 Cards installed and plugged in. All 16 Counter Tree CAT2 cards use their Un-Corrected output except for slot 14 CA = 26 TOT Et REF. SET 2 FOR |eta| 1 to 4 which is using its Corrected output (because that particular CAT2 card has something broken with its Un-Corrected output drivers). 2x Large Tile cards Orange front panel installed but not plugged in. 1x CTMBD The Tier 2 crate for eta 9:16 in M109 has the following setup: 4x Missing Et Adder Tree CAT2 cards installed and plugged in. The +Px and +Py CAT2 cards use their Un-Corrected output. The -Px and -Py CAT2 cards use an inverter cable and their Corrected output. 4x EM and HD Adder Tree CAT2 cards installed but not plugged in. All 4 of the Energy Adder Tree CAT2 cards use their Un-Corrected output. 16x Counter Tree CAT2 Cards installed and plugged in. All 16 Counter Tree CAT2 cards use their Un-Corrected output. 2x Large Tile cards Orange front panel installed but not plugged in. 1x CTMBD The Tier 2 crate for eta 17:20 in M111 has the following setup: 8x open slots 8x Counter Tree CAT2 Cards installed and plugged in. The Tier 2 to Tier 3 cables are not plugged into these 8 cards. These Counter Tree cables will plug into the Un-Corrected output when this eta coverage is used in the Counter Trees. 1x open slot 1x Large Tile card Orange front panel installed and plugged in. 8x open slots 1x CTMBD The Tier 3 crate in M107 has the following setup: 2x Missing Et Px and Py CAT3 cards are installed and plugged in. These CAT3 cards use their Corrected output. 3x open slot 3x HD Adder Tree CAT3 cards installed but not plugged in. There are no cables attacted to these CAT3 outputs. 1x open slot 2x EM Adder Tree CAT3 cards installed but not plugged in. There are no cables attacted to these CAT3 outputs. 1x open slot 8x Counter Tree CAT2 cards installed and plugged in. The Comparator outputs from these 8 CAT2 cards are used. 1x open slot 1x FMLN 1x open slot 1x Large Tile card Orange front panel installed but not plugged in. There are no cables attacted to this Large Tile card's output. 1x CAT3 card installed but not plugged in. There are no cables attacted to this CAT3's output. 1x open slot 1x CTMBD The files run_ii_mXXX_card_addresses.txt were updated in www.pa.msu.edu/hep/d0/ftp/l1/cal_trig/hardware/rack_crate/ to show which cards are not installed or not plugged in. Note that the file for m107 was already made to show the LTCC card as NOT plugged in in anticipation of its desired and future status. Run IIB Tests on the Sidewalk The first of the Run IIB L1 Cal Trig got started on the sidewalk this week. This was tests of the Columbia card holds the SCL Receiver. They wached - tested SCL Initialize and L1_Acpt to their Geographic Section. No test of L2_Acpt. Trigger Tower Work Aws had some more TT's to look at. +13,2 EM looks flat in the pedestal plot. EM BLS signal looks OK can see SCA switching noise HD BLS signal looks OK can see SMT readout noise Hand Pulser Test EM = 652 mV HD = 648 mv looks OK. -14,2 HD looks flat in pedestal plot. EM BLS signal looks OK can see SCA switching noise HD BLS signal looks OK can see SMT readout noise Hand Pulser Test EM = 552 mV HD = 0 mv looks bad. Note that the Term-Attn-Brd for -14,2 was replaced once already - see 8:11-JULY-2003 log book entry. Dig into this and finally figure out that it is a center conductor to center conductor short on the CTFE. Trigger Signal Collision Hall work done Monday night 28-JULY-2003 Aws & Joe. Eta Phi --- --- -11 1 HAD driver was in backward. New driver was installed. +16 8 EM minus cable is open circuit. +13 2 BLS board checked and looked ok, cables looks ok. +17 25 BLS board checked and looked ok, cables looks ok. +17 28 BLS board checked and looked ok, cables looks ok. Note that the correct sockets in the BLS test stand were not used for the eta +17 LM cards. I'm not certain exactly what was tested. Wednesday afternoon work on this some more check again what we see on the +17,25 EM cable. There is almost no signal on the EM- cable. What signal is there has nothing to do with the more normal looking signal on the EM+ cable. Dean looks at it and guesses broken trace. This is the bottom half of a "L" card. Wednesday night - get access +17,25 EM this is the bottom half of a "L" card so we need to move sockets and cables in the BLS Test Stand. Now we see the bad signal. Replace the funny looking EM driver and now the signal looks OK. +17,28 HD this is the bottom half of a "L" card so we need to move sockets and cables in the BLS Test Stand to test this card. We could see the HD problem in the Test Stand. Replace the HD driver and the problem goes away. +20,1 EM This is the top half of an "M" card - so need to move cables and slots to test it. We could see nothing wrong with it. Good signal on EM+, EM-, HD+, HD-. +5,23 This EM is an Excluded channel because it is a sparker in the EM section. Dean cuts the resistor for Tower 3 Depth 6. Thursday continue BLS signal debug work It is not 100% certain that cutting resistor for Tower 3 Depth 6 has fixed the sparks in the BLS signal for TT +5,23 EM. We may have captured one more spark event from that TT on the scope. -6,22 EM is an Excluded channel. Look at it on a scope and you see pulses of about 40 mV to 50 mV amplitude every 2 usec or so. We set up a Ref Set just on this TT and take some events. So far it appears that we do not see energy in the Precision Readout. So this may be a problem with a summer or driver. Will check in the Test Stand. -6,28 EM is an Excluded channel. At low rate (once every few minutes) we see something that looks like a big signal with very long rise time on the scope. It rises for 1100 nsec probably until the Pre-Amp saturates. Each side of the signal rises about 800 mV in that 1100 ns. We put a Ref Set on it and collect 18 events in 1 hour 10 minutes. We do see energy in the Precision Cal Readout for these events. +6,23 EM is an Excluded channel. We see a 150 mV to 200 mV pulse on each side of the line about once every 5 or 6 usec. Note that +6,23 had a cut resistor in Run I. We use a Ref Set to trigger on these and we do see a small effect in the Precision Readout, i.e. all events have a small amount of signal at this eta phi in the Precision Readout. +6,21 EM is an Excluded channel. Once every few minutes we see a pulse of 100 mV to 150 mV on each side of the line. It takes about 700 nsec to rise and goes to a pointed shape, i.e. it does not look like a real energy deposit. A run with this TT in the Ref Set was taken along with +8,28 shown next. +8,28 EM is an Excluded channel. On the scope we did see one big fast risetime pulse. A Ref Set was put on just +6,21 and +8,28 and the first few events in the file should be interesting for the diagnosis of the +6,21 and +8,28 problems. Then they turned on the beam and we picked up a bunch more event which may not contain the problem. The log book reference for the L and M card layout is 5:7-FEB-2003. Ring-of-Fire Tests. Tuesday evening shift Tom Diehl became suspicious that welding in the weld shop was causing ring-of-fire events. This was during beam Physics running. Wednesday afternoon after the store was lost we made an explicit test of welder noise pickup. No problem - we can see it. Triggered on 2 TT >= 4.0 GeV EM in just eta -7. This gives a pure sample of ring of fire events. There appeared to be just as much energy in the TT eta 8 ring. The worst of the noise appears to happen when the arc is struck and not during the continuous part of the arc drawing the bead. Welding in the weld shop makes more noise pickup than welding in the fridge area - but we do see noise from fridge are welding. Typical waveform of the noise pickup as seen on the BLS Trigger pickoff cables is a burst of sin wave that lasts order of 10 usec and is order of 600 nsec period and order of 150-200 mV on each side of the cable. Dean's ring-of-fire log entry At the start of the shift, studies were being done to confirm a source of noise seen intermittantly for the last year in the calorimeter readout and only associated with some work being done by welders yesterday evening. We did confirm that every time the welder started a new weld, a spike in the L1 trigger rate, and an associated spike in the events to the host was seen. These events were seen to have a ring of EM towers light up. Either Ieta=14, 15, -14 or -15 or any combination of these eta rings, for all 64 phis. The size of the signal was about 3-5 GeV per tower, and it was previously seen to involve the EM4 layer only. While similar, this is different from the run 1 "ring of fire" which was limited to Ieta = 14 or -14, EM layer 3. In addition many hadronic towers in both EC cryostats had significant energy, typically 5-10 GeV, and typically on the bottom phi region. Now that we know this can cause noise, we need to understand how the noise get inside north/south cryostats and misses (or is atleast much smaller in) the CC cryostat. Then one of the noise sources that create large missing Et events will be eliminated. I looked at the welder - it is a 60 Amp circuit on the 480 V delta using only 2 of the phases. That makes sense from the point of view of AC welding. The symbol drawing on the back clearly shows SCR's so I guess this thing can make high frequency noise. A big question is does its noise get into Cal via the GND system or via the AC power distribution system. CAT Tree Dump command files. These are CIO command using the recently added keywords to read CAT2 and CAT3 (and FMLN) input registers and format the numbers to the screen. These files also include the recently added keywords to "seek" the output of CAT2 and CAT3 cards, but these commands are ALL commented out in order to prevent possible confusion. "Seeking" a CAT card output involves re-programming its comparator #0 to perform a binary search of the output value. The comparator is returned to its original value, but this feature could cause abnormal temporary triggering if the resource was already in use. This feature could prove useful in certain occasions to compare the output of a CAT card to the input of the next card, but is typically not a crucial information. Additionally only static states can be "seeked" in this manner. If we need to use this command, the files will simply need to be edited to remove the comment flag. As they are now, these files are safe to execute even with beam in the machine. These files may be executed from VME_Access V5.0 Rev F. The next revision of Trics will be able to handle these files, but Trics V10.4-H does not know the new read_CAT2/CAT3_operand keywords. The syntax description file syntax_rules_cbus_io.cio in www.pa.msu.edu/hep/d0/ftp/tcc/trics_ii/ has been updated The following files were copied to in \Trics\Scratch\ Dump_Tree_Energy_EM_Tier1.cio Dump_Tree_Energy_HD_Tier1.cio Dump_Tree_MissingEt_Px_Tier1.cio Dump_Tree_MissingEt_Py_Tier1.cio These 4 files above dump the input to all Tier#1 CAT2 cards, one for each CAT2 type EM, HD, Px, Py. The cards appear in phi, then rack order for eta(1:20) except EM and HD for which only eta (1:16) is available. Dump_Tree_MissingEt_Tier23.cio This file dumps the input to all Tier#2 CAT2 Px and Py cards for eta (1:16) and both Tier#3 Px and Py CAT3 cards and the FMLN Card. Dump_Tree_RefSet_EM_Tier23.cio Dump_Tree_RefSet_Tot_Tier23.cio These 2 files dump the input to all Tier#2 CAT2 cards for eta(1:20) and Tier #3 CAT2 card. There is one file for EM Et, and one file for Tot Et Ref Set Counts. Each file dumps all Reference Sets #0, #1, #2, #3 in order. The file Dump_MissingEt_Tree.cio that which used earlier was renamed (replaced with) Dump_Tree_MissingEt_Tier23.cio in order to group all the dump_tree files together in alphabetical directory listings. All files were tested, without beam, and without excluding or controlling trigger tower energies. The result is in VME_ACCESS_20030730_V5_0_F.LOG;1. This logfile was carefully scanned to double check that the proper card addresses were used in each command file. ------------------------------------------------------------------------------ DATE: 30-JULY-2003 At: MSU Topics: Recover Luminosity Monitoring Information TCC has been ready since mid-june to save all luminosity monitoring information which it couldn'be deliver to the luminosity server over ITC. Since that time only two instances of Trics have needed to create luminosity recovery files, and a total of 5 blocks have been salvaged. Only 2 of these LBN blocks seem to come from a beam period. It looks like this happened during the luminosity computer problems of 25-July. This may not be a significant amount of information, but was still a good oportunity to verify that we can indeed recover this type of information the day we face a real and prolongated problem. File Luminosity_Recovery_V10_4_H_20030702.dat;1 had 2 LBN blocks 0x25ba94 0x25d920 File Luminosity_Recovery_V10_4_H_20030725.dat;1 had 3 LBN blocks at the time (and a 4th block not listed here by the time Trics was restarted). 0x26a2b4 (*) 0x26a2f2 (*) 0x26b11b (*)=25-july with beam Philippe made a special purpose "LumRecover" application to read such files and send each block to a special purpose "ReacquireTCC" client. Philippe ran LumRecover from MSU's tadpole computer for each file and Michael Begel ran his client application from d0olb. All 5 blocks were successfully sent and received. ------------------------------------------------------------------------------ DATE: 22:25-JULY-2003 At: Fermi Topics: "Fix" the Missing Et sum Px -9,29, And-Or Rates for Missing Et, 3 CIO file to move pages, Return of parts of the Data I/O 2700 programmer, Try cooking PALs for the Distributor Cap, Work on TT's. Cold Start - no change, see the 8:11-JULY-2003 log book entry. Collect some And-Or Rate data for the Missing Et trigger with and without beam. Recall what is on the various Momentum Lookup Pagegs: Momentum Lookup Page Number Symmetric Energy Cuts Implement by this Lookup Page ----------- ----------------------------------------------------- 0 Zero Symmetric Energy Cut at all eta. 1 1.0 GeV Et Symm Energy Cut for TT eta 1:16 2 1.5 GeV Et Symm Energy Cut for TT eta 1:12 2.0 GeV Et Symm Energy Cut for TT eta 13:16 3 2.0 GeV Et Symm Energy Cut for TT eta 1:12 3.0 GeV Et Symm Energy Cut for TT eta 13:16 All data collected between luminosity of 36e30 and 34e30 when the DAQ system was paused, i.e. L1 Accpet rate = 0 Hz. Coverage TT eta 1:16 Page 0 20 GeV 717 kHz 717k 716k 30 GeV 39.9 kHz 39.9k 39.8k 40 GeV 926 Hz 895 928 50 GeV 31 Hz 25 21 Page 1 20 GeV 6423 Hz 6411 6402 6517 30 GeV 67 Hz 77 78 79 40 GeV 36 Hz 45 43 47 50 GeV 34 Hz 41 41 43 Page 2 20 GeV 134 Hz 124 123 131 137 117 30 GeV 18 Hz 19 16 18 17 19 40 GeV 9 Hz 8 6 9 6 8 50 GeV 6 Hz 6 4 6 4 6 Page 3 20 GeV 121 Hz 103 110 109 100 105 30 GeV 15 Hz 14 17 16 12 11 40 GeV 5 Hz 6 7 4 2 3 50 GeV 1 Hz 2 2 2 1 2 Luminosity = 0 DAQ L1 Accept rate = 0 Hz Coverage TT eta 1:16 Page 0 20 GeV 583 kHz 584k 582k 580k 30 GeV 29.3 kHz 29.3k 29.0k 28.8k 40 GeV 793 Hz 753 767 681 50 GeV 28 Hz 22 24 23 Page 1 20 GeV 3256 Hz 3182 3163 3133 30 GeV 73 Hz 71 74 50 40 GeV 34 Hz 42 46 38 50 GeV 32 Hz 40 42 34 Page 2 20 GeV 10 Hz 8 8 20 10 30 GeV 4 Hz 5 3 9 4 40 GeV 2 Hz 3 1 6 2 50 GeV 1 Hz 2 1 5 1 Page 3 20 GeV 7 Hz 8 5 8 5 30 GeV 3 Hz 5 3 3 3 40 GeV 1 Hz 2 3 3 2 50 GeV 0 Hz 2 1 2 1 All data collected between luminosity of 17e30 when the DAQ system was paused, i.e. L1 Accpet rate = 0 Hz. Coverage TT eta 1:16 Page 2 20 GeV 75 Hz 83 30 GeV 20 Hz 17 40 GeV 9 Hz 7 50 GeV 6 Hz 4 Page 3 20 GeV 67 Hz 66 68 30 GeV 15 Hz 13 18 40 GeV 6 Hz 5 10 50 GeV 3 Hz 4 3 In \Scratch\ made 4 short .cio files to move from one Momentum Lookup page to another. These have been tested in beam and are safe to use. Work on the Px problem at -9,29 : -12:29 This problem was the Px bit of value 16 stuck High. Note that it just looked like this bit coming out of the card was stuck High and not this bit associated with a particular Px PROM is stuck High. Check on the backplane and yes you can see this bit stuck high, i.e. it is not a T1 Px CAT2 problem. Make dumps of the Momentum tree using \Scratch\Dump_MissingEt_Tree.cio from VME_Access and yes you can see this bit stuck High. Checked the voltage levels on the backplane for this bit and they are full good ECL levels - so I guess that it is not a static zapped driver chip. Pull the CTFE for -9,29 : -12,29 and try to spot a problem on it. I could not spot any obvious problems with the Fluke Ohm meter. The only thing that looked at all funny was U40 Pin 1 wrt Gnd or Vcc. This looked different than the other F283 outputs - but this is not the Px Board Sum line that has the problem. But it may be an indication that things have been zapped in this general area. Time runs out before the next store so make a card swap with "high eta". Swap the CTFE at -9,29:-12,29 with the CTFE from -17,17:-20,17. CTFE SN# 332 was at -9,29:-12,29 - it is the card with the Px problem. CTFE SN# 373 was at -17,17:-20,17 and is fully OK as far as I know. The swap was complete, i.e. PROM's and Term-Attn-Brds were swapped. Now CTFE SN# 332 the card with the Px problem is at -17,17:-20,17. Fire up, exclude at TT's to value 8 and look with Dump_MissingEt_Tree.cio and thing look OK. Run CTFE PROM check over +-1:16 1:32 and it runs without error on Px (2 passes) Py EM and HD. The Momentum Tree cables are now plugged into the T2 output for both 1:8 and 9:16. At this point I had to give the system back so I will only be able to make some rate checks. A think to note is that we have zero spare CTFE cards. There are 12 to 14 CTFE cards in the white transporter box that need to be checked out and fixed. They already have the Run II modifications on them and probably just need a little work. Aws had a number of TT's that showed bad gain in the Examine run of 300k events. All of these TT's with reported gain problems were at eta 18. Between stores we looked at 3 of these TT's +18,2 HD +18,5 EM and +18,6 EM. It was between stores but all these BLS signals looked alive - we could see SMT noise in the HD signal and SCA switching noise in the EM signals. The direct and complement side of these signals looked balanced. So no reason to suspect missing BLS signal. Pulsed these channels and they looked OK. With the hand pulser you see 544 mV in EM and 472 mV in HD with both sides of the signal being pulsed. So guess that it may be a statistics problem (or missing summer problem that we can not see this way). Once again need to use the Precision Pulser to see this stuff. Receive back from Bob: A total of about 38 245A's and 291A's that failed during programming. A wrist strap. Instruction book for the 2700. Set of spare PROM labels Instruction book for the Chip Lab Set of 3 program disks for the Chip Lab So far he has not found the program disks for the Data I/O 2700. Practice with the Data I/O 2700 programmer and the 21-JULY-2003 version of the new Distributor Cap Configuration and Transmit Sequencer PAL's. Working with the dc-confg.jdc file Device Problem 6033 Vector 36 Expect 10XX XX01 01XN 0HXL LLHL LLLN Find 10XX XX01 01XN 0HXL LHHL HLLN v v Working with the dc_trans.jdc file - no errors. Take the full HSRO "information - design" 3 ring book back to MSU. This is the binder with the HP HDMP-1012 chip data sheets. The basic layout is that Data_00 : Data_19 , DAV* and all that stuff must be valid a minimum of 6 nsec before the rising edge of STRBIN (its frame clock) and there is a 0 nsec hold time requirement. We should be finished with this stuff at Fermi so back to MSU for safe keeping. ------------------------------------------------------------------------------ DATE: 8:11-JULY-2003 At: Fermi Topics: Survey L1_Busy from Cal Trig, Noisy TT TT work: 1x TAB replace, 2x PROM replace, Move to Momentum Page #2, Missing Et Rates using Page #2, Px Py PROM Check, Find_DAC run, Readout from M112, Run IIB Cables, Cal Trig Power Supplies, Trigger List 12.10 Ref Sets, PROM Eraser Cold Start - Same as shown in the log book entry for 20:23-MAY-2003 except: We are running: TRICS V10.4.H and VME_Access V5.0.D The current Gains files are: D0_Config/Gains_17_20_1_32_rev_a.tti # TT eta's 17:20 D0_Config/Gains_1_16_1_32_rev_4.tti # TT eta's 1:16 The current Pedestal files are: D0_Log/Find_DAC_V0_0_D_20030627_Edited.tti;2 # TT eta 17:20 D0_Log/Find_DAC_V5_0_D_20030710.tti;1 # TT eta 1:16 Try watching some more L1_Buzy from the Cal Trig vs rate to get an idea of how long it is asserted for with each event. There is some time that we can trim out of this even without a faster Xmit_Clk. Right now in post_init_auxi_ _L1CT.rio the CT_Readout_Helper is being setup with a long delay between the L1_Acpt and the generation of the Transmit_Trig to the Distributor Cap. L1_Acpt Hz Cal Trig L1_Busy TFW L1_Busy ---------- ---------------- ----------- 1292 2.5 % 1.2% 1285 2.5 1.3 1100 2.1 1.1 1000 1.9 0.9 900 1.7 0.8 826 1.6 0.8 500 0.9 0.5 500 0.8 0.4 Tuesday morning 8-JULY-2003 rates were not stable from the L1 Cal Trig. The examine showed the problem to be -14,13 EM. I Excluded it and the L2 L2 rates then looked fine. There were still L3 rate problems so -14,13 EM must have caused noise in both the trigger pickoff and in the precision readout. Wednesday AM check with scope and -14,13 EM looks fine. Did not Exclude it from the store that started Wednesday AM. No change was made to the Excluded TT file that TRICS uses. TT Work Problems reported based on Run 178483 Store 2736 which was taken 30-JUNE-2003 Channels reported as dead: -14,2 HD Reported to have zero pedestal width. Checked with scope during beam running. The BLS cable signals look fine. The CTFE monitor lemo looks dead. So the problem is at the L1 Cal Trig end. I will (have) fix it. -13,32 HD Reported to have zero pedestal width. Checked with scope during beam running and the BLS cable signal looks OK and the Lemo monitor signal looks OK. Check the log book and find that this was fixed on 2-July-2003. -11,1 HD Reported to have zero pedestal width. Check with scope and there is no signal on either side of the BLS cable. The problem is at the BLS end. I will add this to the "next access work list". +9,21 EM Reported to have zero pedestal width. Checked with scope during beam running and the Lemo monitor signal and everything looks fine. Checked the Find_DAC run and this channel looks fine. It ends up this is just an Excluded channel. Channels reported as too much noise: +6,5 HD Reported to have wider than normal pedestal. Check with scope. This channel has about 1.5 GeV of 10 MHz noise in its BLS cable signal. -6,19 HD Reported to have wider than normal pedestal. Check with scope. This channel has about 1.5 GeV of 10 MHz noise in its BLS cable signal. Thursday AM between stores work -14,2 HD swapped the Term-Attn-Brd. +9,21 EM investigate the BLS and Lemo signals look OK but pedestal has no width problem. It's just Excluded. Replace the PROM's -19,28 EM and -17,29 EM. I verified that the official ones that were in the system were blank. Replaced with parts that Aws and I cooked last trip. Checked the ADC_Clk timing switch on both of these cards. They were set for 2,5 closed I changed them to 1,4 closed. This confirms the 9:13-JUNE-2003 log book entry about the high eta ADC Clk switches being set wrong. Edited the Post_Init_Auxi_L1CT.cio so that the middle address line of the Momentum page select would be High. The intent is to start using Momentum page #2 for the first Momentum lookup (instead of page #0 that we have been using) We Were Using We now are Using Momentum Page Adrs Momentum Page Adrs ------------------ ------------------ First LU 0 = 000 2 = 010 Second LU 4 = 100 6 = 110 We were making the First Momentum Lookup from Page 0 which has a Symmetric Energy Cut of 0.0 GeV for all eta 1:20. Using Page 2 for the First Momentum Lookup then for eta 1:12 we have a Symmetric Energy Cut of 1.5 GeV and for eta 13:16 we have a Symmetric Energy Cut of 2.0 GeV. Page #4 is Large Tile optiont #1 Page #6 is constant 8's. Issues with this: I had to load 255 into the PAL for the Momentum middle address line to get it to go High. Normally we would load either 9 for Low and 25 for High. There must be some strange PAL in this location. I did not pull the MTG to check but I should do this in August. Also verify/document all the MTG output signals. In the Post_Init_Auxi_L1CT.cio we are not defining the outputs that are just used to provide valid ECL levels to the sections of the 10116 chips on the CTMBD's that are not used. But it may be a good idea to start doing this - just so long as an error here can not hang us. PROM Check Ran the CTFE PROM Checker over |eta| 1:16 on Px Py to verify that we actually had Momentum Page #2 selected and to look for errors. All are OK except for Px -9,29 -10,29 -11,29 -12,29. This is basically what was expected. See 9:13-JUNE-2003 and 25:27-JUNE-2003. The one thing that is different is that the PX PROM for +13,27 : +16,27 did not show up as having a problem. This PX problem for +13,27 : +16,27 is believed to only be a readback from T1 CAT2 problem but it did not show up. I need to get Px -9,29 -10,29 -11,29 -12,29 fixed during the next trip. Find_DAC Find_DAC ran over EM HD eta +- 1:16 1:32 2 8 check check Made the file: D0_Log/Find_DAC_V0_0_D_20030710.tti;1 There are no errors or fails in the file. Load it and the pedestals look OK. Now that we have Momentum 1st Lookup from Page #2 look at some AOT rates. Take 3 or 4 looks under each set of conditions. Missing Et DAQ L1_Acpt 450 Hz DAQ L1_Acpt 0 Hz Threshold Zero Beam Zero Beam ---------- ----------------------- --------------------------- 20 GeV 7257 7354 7320 Hz 7.6 8.5 4.8 7.9 Hz 30 GeV 3146 3177 3146 Hz 3.0 3.7 2.4 4.0 Hz 40 GeV 1413 1422 1437 Hz 2.0 1.8 1.3 2.1 Hz 50 GeV 531 526 548 Hz 1.0 1.0 0.5 1.0 Hz Missing Et DAQ L1_Acpt 1250 Hz DAQ L1_Acpt 0 Hz Threshold 32 E30 Beam 33 E30 Beam ---------- -------------------------- ---------------------------- 20 GeV 15.4k 15.2k 15.2k Hz 165 164 164 167 Hz 30 GeV 4.6k 4.6k 4.5k Hz 31 28 24 25 Hz 40 GeV 1129 1097 1085 Hz 7.2 7.4 6.7 7.1 Hz 50 GeV 209 196 194 Hz 2.6 2.8 1.9 2.8 Hz Missing Et DAQ L1_Acpt 0 Hz Threshold 14.9 E30 Beam ---------- ------------------------- 20 GeV 73 78 75 78 Hz 30 GeV 12 15 17 16 Hz 40 GeV 4 4 7 6 Hz 50 GeV 1 1 2 2 Hz Cal Trig Readout from M112 Aws had noticed that the data from M112 had started to act up again. M112 is where we put the 2 ERPB cards that we pulled from M108 because we had concerns about them being the cause of the 1 run readout problem from M108. See the log book for 9:13-JUNE-2003. Of these 2 cards I believe that the problem is in the 4th one down. It appears to get the "GO" and starts to readout but then it sometimes duplicates a word of readout data or has a word of all zeros. Sometimes the screwed up data starts in the middle of this 4th card. Sometimes it starts right at the beginning of the data from the 4th card. The data from the cards after this one look OK - it may be shifted around but it looks just fine. I tried configuring the LCA's in this rack and then immediately looking at the data. The data was still in trouble. A guess is that there is a problem with the Xmit_Clk signal on this card. Run IIB Test Area Cables Installed the 2 SCL Serial Data cables for the Run IIB Cal Trig area and the BLS Trigger Pickoff cable. Power Supplies Used the RM.py to look through the Cal Trig and TFW power supplies. The supplies that are in the most trouble are the Tier 2 supplies. Display "A" channels 12 and 13 are T2 eta 1:8 -4.5 and -2.0 Both of these are still in trouble. Display "B" channels 36 37 39 are T2 eta 9:16 -4.5 -2.0 and -5.2 These are all in trouble. I also looked at these supplies with the Fluke on the front panel test points. The -4.5 looks mostly clean and free of ripple. The -2 (the supplies that were way in trouble before installing capacitor band-aids) are kind of OK but it would be a good idea to swap out clean up all the T2 supplies. I'm still not happy about how we take care of a power supply problem at 4AM. Trigger List 12.10 Cal Trig Ref Sets Philippe pulled out the Reference Set download from COOR on Thursday afternoon from Global Physics Trigger List 12.10 Note that thresholds and count comparisions are not assigned in order. L1CT_Ref_Set EM_Et_Ref_Set 0 Value 3.0 L1CT_Ref_Set EM_Et_Ref_Set 1 Value 6.0 L1CT_Ref_Set EM_Et_Ref_Set 2 Value 11.0 L1CT_Ref_Set EM_Et_Ref_Set 3 Value 9.0 L1CT_Ref_Set TOT_Et_Ref_Set 0 Value 5.0 L1CT_Ref_Set TOT_Et_Ref_Set 1 Value 3.0 L1CT_Ref_Set TOT_Et_Ref_Set 2 Value 7.0 L1CT_Count_Threshold EM_Et_Towers Ref_Set 0 Comparator 0 Value 1 L1CT_Count_Threshold EM_Et_Towers Ref_Set 0 Comparator 1 Value 2 L1CT_Count_Threshold EM_Et_Towers Ref_Set 1 Comparator 0 Value 1 L1CT_Count_Threshold EM_Et_Towers Ref_Set 1 Comparator 1 Value 2 L1CT_Count_Threshold EM_Et_Towers Ref_Set 2 Comparator 0 Value 1 L1CT_Count_Threshold EM_Et_Towers Ref_Set 3 Comparator 0 Value 1 L1CT_Count_Threshold Tot_Et_Towers Ref_Set 0 Comparator 0 Value 1 L1CT_Count_Threshold Tot_Et_Towers Ref_Set 0 Comparator 1 Value 2 L1CT_Count_Threshold Tot_Et_Towers Ref_Set 0 Comparator 2 Value 3 L1CT_Count_Threshold Tot_Et_Towers Ref_Set 0 Comparator 3 Value 4 L1CT_Count_Threshold Tot_Et_Towers Ref_Set 1 Comparator 0 Value 2 L1CT_Count_Threshold Tot_Et_Towers Ref_Set 1 Comparator 1 Value 1 L1CT_Count_Threshold Tot_Et_Towers Ref_Set 2 Comparator 0 Value 4 L1CT_Count_Threshold Tot_Et_Towers Ref_Set 2 Comparator 1 Value 1 PROM Eraser I visited SiDet and got back our PROM eraser. I went through all the parts that people had trouble with during the cook and dumped the bad stuff and cleaned up and labeled the good re-usable stuff. This gives us an OK supply of blank 245's and 291's at Fermi. ------------------------------------------------------------------------------ DATE: 25:27-JUNE-2003 At: Fermi Topics: Cal Trig Readout and Missing Et Tree Testing, Master Clock, Cable runs to the Run IIB Sidewalk Cal Trig Test Area, Find_DAC, Trigger Tower work - Lost Beam, Pager Cold Start - Same as shown in the log book entry for 20:23-MAY-2003 except: We are running: TRICS V10.4.H and VME_Access V5.0.D The current Gains files are: D0_Config/Gains_17_20_1_32_rev_a.tti # TT eta's 17:20 D0_Config/Gains_1_16_1_32_rev_4.tti # TT eta's 1:16 The current Pedestal files are: D0_Log/Find_DAC_V0_0_D_20030627_Edited.tti;2 # TT eta 17:20 D0_Log/Find_DAC_V5_0_C_20030613.tti;2 # TT eta 1:16 L1 Cal Trig Readout Put the Jumper Header into the U23 socket on all 10 running Distributor Caps. U23 is the Fault Shift Register Sequencer PROM. The header pulls all the data lines from this PROM socket to Vcc via 110 Ohm resistor. This will stop the Fault Shift Register from trying to send out its data from the DC to the CRC/Spark/Bougie. Set all DC ID/Mode Switches to Key #1 UP and Keys #2:#8 Down This plus the state of the MTG_* signals that is setup at the end of L1CT_Post_Init_Auxi.rio will prevent writing to a MTG Alternate Register. Set all DC Setup Switches to all Keys Down except Negative Eta DC's have Key #8 UP. This is the correct setup, i.e. do not use MTG Setup and the correct eta order readout. Master Clock Work with Fritz and between stores test that we can get Master Clock mode and status errors into the Alarm System. We check in detail the PCC and the Sequencer #2. We make less of a check on Sequencer #1, i.e. trying not to cause too much trouble. The templates for Sequencer #1 and #2 differ in only one place (I think) - that is the Output Enabled vs Ouput Not Enabled. As part of all this testing I manage to get clean reloads of both Sequencers. Missing Et Tree Testing Setup some Missing Et Thresholds Missing Et Comparator Threshold And-Or Term ---------- --------- ----------- 0 1.0 GeV 152 1 2.0 GeV 153 2 5.0 GeV 154 3 10.0 GeV 155 Exclude all TT's to Value = 8 eta -16:+16 All 4 Missing Et And-Or Term rates are zero. Only eta -8:+8 is plugged into Missing Et Tier 3. Do NOT change the Missing Et Tier 3 Correction Register Play with TT +1,1 EM Loaded ADC Value ------------------- Missing Et Not Fire Does Fire Threshold -------- --------- ---------- 9 10 1.0 GeV 13 14 2.0 GeV 25 26 5.0 GeV 45 46 10.0 GeV Correctly setup the Missing Et Tier 3 CAT3 Correction Registers for a value = 1. 1 is correct because only eta -8:+8 is plugged into Tier 3 MEt. All TT's are Excluded to a value of 8. Play with +1,1 EM Loaded ADC Value ------------------- Missing Et Not Fire Does Fire Threshold -------- --------- ---------- 11 12 1.0 GeV 15 16 2.0 GeV 27 28 5.0 GeV 47 48 10.0 GeV Philippe verifies the expected PROM response: When -1,9 HD is at 47 47 ADC counts is a PROM input of (47+8)/2=27 counts. The Px PROM output should be 10 and the Py output should be 27. This corresponds to Px=9-8= 2 counts and Py=27-8= 19 counts. Px*Px+Py*Py=361+4 which is less than 20*20=400 for a 10 GeV threshold. This should indeed Not Fire When -1,9 HD is at 48 48 ADC counts is a PROM input of (48+8)/2=28 counts. The Px PROM output should be 10 and the Py output should be 28. This corresponds to Px=9-8= 2 counts and Py=28-8= 20 counts. Px*Px+Py*Py=400+4 which is above than 20*20=400 for 10 GeV threshold. This should indeed Fire All TT's are Excluded to a value of 8. Set +1,1 EM to vale 100 Play with +1,17 EM Loaded ADC Value ------------------- Missing Et Not Fire Does Fire Threshold -------- --------- ---------- 103 104 1.0 GeV 107 108 2.0 GeV 119 120 5.0 GeV 139 140 10.0 GeV Now play with +1,17 EM on the other side of the balance at 100 with +1,1 EM Loaded ADC Value ------------------- Missing Et Not Fire Does Fire Threshold -------- --------- ---------- 98 97 1.0 GeV 94 93 2.0 GeV 82 81 5.0 GeV 62 61 10.0 GeV Exclude all channels EM and HD to ADC Value 8 Play with -1,9 HD Loaded ADC Value ------------------- Missing Et Not Fire Does Fire Threshold -------- --------- ---------- 11 12 1.0 GeV 15 16 2.0 GeV 27 28 5.0 GeV 47 48 10.0 GeV Set -1,9 HD to a ADC value of 200. Play with -1,25 HD Loaded ADC Value ------------------- Missing Et Not Fire Does Fire Threshold -------- --------- ---------- 203 204 1.0 GeV 207 208 2.0 GeV 221 222 5.0 GeV 241 242 10.0 GeV Verify the expected PROM response: When -1,9 HD is at 200 200 ADC counts is a PROM input of (200+8)/2=104 counts. The Px PROM output should be 17 and the Py output should be 104. This corresponds to Px=17-8= 9 counts and Py=104-8= 96 counts. When -1,25 HD is at 241 241 ADC counts is a PROM input of (241+8)/2=124 counts. The Px PROM output should be 19 and the Py output should be 123. This corresponds to Px=19-8= 11 counts and Py=123-8= 115 counts. The total is Px=9-11= -2 and Py=96-115= -19 counts Px*Px+Py*Py=361+4 which is less than 20*20=400 for a 10 GeV threshold. This should indeed Not Fire When -1,25 HD is at 242 241 ADC counts is a PROM input of (242+8)/2=125 counts. The Px PROM output should be 19 and the Py output should be 124. This corresponds to Px=19-8= 11 counts and Py=124-8= 116 counts. The total is Px=9-11= -2 and Py=96-116= -20 counts Px*Px+Py*Py=400+4 which is above than 20*20=400 for 10 GeV threshold. This should indeed Fire Now play with -1,25 HD on the other side of the balance point at 200. Loaded ADC Value ------------------- Missing Et Not Fire Does Fire Threshold -------- --------- ---------- 198 197 1.0 GeV 194 193 2.0 GeV 182 181 5.0 GeV 162 161 10.0 GeV When -1,25 HD is at 162 241 ADC counts is a PROM input of (162+8)/2=85 counts. The Px PROM output should be 16 and the Py output should be 85. This corresponds to Px=16-8= 8 counts and Py=85-8= 77 counts. The total is Px=9-8= 1 and Py=96-77= 19 counts Px*Px+Py*Py=361+1 which is less than 20*20=400 for a 10 GeV threshold. This should indeed Not Fire When -1,25 HD is at 161 241 ADC counts is a PROM input of (161+8)/2=84 counts. The Px PROM output should be 15 and the Py output should be 84. This corresponds to Px=15-8= 7 counts and Py=84-8= 76 counts. The total is Px=9-7= 2 and Py=96-76= 20 counts Px*Px+Py*Py=400+4 which is above than 20*20=400 for 10 GeV threshold. This should indeed Fire Want to test at 45 degrees Exclude all channels EM and HD to ADC Value 8 Play with -1,20 EM Loaded ADC Value ------------------- Missing Et Not Fire Does Fire Threshold -------- --------- ---------- 11 12 1.0 GeV 15 16 2.0 GeV 27 28 5.0 GeV 49 50 10.0 GeV Verify the expected PROM response: When -1,20 EM is at 49 49 ADC counts is a PROM input of (49+8)/2=28 counts. The Px PROM output should be 23 and the Py output should be 21. This corresponds to Px=23-8= 15 counts and Py=21-8= 13 counts. Px*Px+Py*Py=225+169=394 which is less than 20*20=400 This should indeed Not Fire When -1,20 EM is at 50 48 ADC counts is a PROM input of (50+8)/2=29 counts. The Px PROM output should be 24 and the Py output should be 28. This corresponds to Px=24-8= 16 counts and Py=21-8= 13 counts. Px*Px+Py*Py=256+169=425 which is more than 20*20=400 This should indeed Fire Now check vector addition Exclude all channels EM and HD to ADC Value 8 Will be playing with +4,1 EM and -4,9 EM so first scan these two individually and verify that they are working individually for the standard set of Missing Et Thresholds. Both show: Loaded ADC Value ------------------- Missing Et Not Fire Does Fire Threshold -------- --------- ---------- 11 12 1.0 GeV 15 16 2.0 GeV 27 28 5.0 GeV 47 48 10.0 GeV Now set +4,1 EM TO ADC Value = 20 Then play with -4,9 EM Loaded ADC Value ------------------- Missing Et Not Fire Does Fire Threshold -------- --------- ---------- always fires 1.0 GeV always fires 2.0 GeV 23 24 5.0 GeV 45 46 10.0 GeV Verify the expected PROM response: When +4,1 EM is at 20 20 ADC counts is a PROM input of (20+8)/2=14 counts or 7.0 GeV. The Px PROM output should be 14 and the Py output should be 9. This corresponds to Px=14-8= 6 counts and Py=9-8= 1 counts. Px and Py are both positive for this tower When -4,9 EM is at 45 45 ADC counts is a PROM input of (45+8)/2=26 counts or 13.0 GeV The Px PROM output should be 10 and the Py output should be 26. This corresponds to Px=10-8= 2 counts and Py=26-8= 18 counts. Px is negative and Py is positive for this tower The total is Px=6-2= 4 and Py=1+18= 19 counts Px*Px+Py*Py=16+361=377 which is less than 20*20=400 This should indeed Not Fire When -4,9 EM is at 46 46 ADC counts is a PROM input of (46+8)/2=27 counts or 13.5 GeV The Px PROM output should be 10 and the Py output should be 27. This corresponds to Px=10-8= 2 counts and Py=27-8= 19 counts. The total is Px=6-2= 4 and Py=1+19= 20 counts Px*Px+Py*Py=16+400=416 which is above 20*20=400 This should indeed Fire Cable Runs to the Run IIB L1 Cal Trig sidewalk test area This is for both the "BLS" cables and for the SCL cables. Outside the MCH want about 25 ft. This is 3' to floor, 10' to run 5 racks and 12' to get to the top and front of the rack. Inside MCH we need 12 ft to reach the cable tray via direct path up the wall. Then the BLS cables need to be 18' to run 9 racks, 3' to reach front, and 7' to run down the front. This is 28 ft. And the SCL cables need 4' to run 2 racks, 3' to the front, 4' to cross the aisle, and 3' to the back of M124, and 4' to the patch panel. This is 18 ft. This makes the BLS cables 25' + 12' + 28' = 65 ft This makes the SCL cables 25' + 12' + 18' = 55 ft Parts for these cables were passed to Tom Dump Missing Et Tree Tests This is all in a VME Access log file that started and ended Thursday morning 26-JUNE-2003. Conditions: Only eta 1:8 Tier 2 is plugged into Tier 3 Eta 9:16 Tier 2 is not plugged into Tier 3 Tier 3 Correction Register is setup for running with eta 1:8 The intent was to first ramp up all eta,phi to a significant ADC Value, in this case 60 using rather big steps, in this case 1 GeV. Then with a big load of high order bits, ramp all eta,phi slowly and verify that low order bits looks OK in smaller steps. Then set all eta,phi to an ADC Value of 8 and crank up just a single TT at a time and do this at 8 different places in eta,phi land to verify the eta,phi to operand mapping. Then pick just a single TT and ramp it up in 1/2 GeV steps to verify what a single TT is doing. Pick this TT in the middle of the area where in Run I we had a flaky paddle card. Every time below where it shows that the registers were dumped, I dumped them twice - just to get two looks at every condition. Touch nothing - just as Cal Trig came out of Initialize - Dump All EM and HD in eta range 1:16 to ADC Value = 8 - Dump All EM and HD in eta range 1:16 to ADC Value = 12 - Dump All EM and HD in eta range 1:16 to ADC Value = 16 - Dump All EM and HD in eta range 1:16 to ADC Value = 20 - Dump All EM and HD in eta range 1:16 to ADC Value = 24 - Dump All EM and HD in eta range 1:16 to ADC Value = 28 - Dump All EM and HD in eta range 1:16 to ADC Value = 32 - Dump All EM and HD in eta range 1:16 to ADC Value = 36 - Dump All EM and HD in eta range 1:16 to ADC Value = 40 - Dump All EM and HD in eta range 1:16 to ADC Value = 44 - Dump All EM and HD in eta range 1:16 to ADC Value = 48 - Dump All EM and HD in eta range 1:16 to ADC Value = 52 - Dump All EM and HD in eta range 1:16 to ADC Value = 56 - Dump All EM and HD in eta range 1:16 to ADC Value = 60 - Dump All EM and HD in eta range 1:16 to ADC Value = 61 - Dump All EM and HD in eta range 1:16 to ADC Value = 62 - Dump All EM and HD in eta range 1:16 to ADC Value = 63 - Dump All EM and HD in eta range 1:16 to ADC Value = 64 - Dump All EM and HD in eta range 1:16 to ADC Value = 65 - Dump All EM and HD in eta range 1:16 to ADC Value = 66 - Dump All EM and HD in eta range 1:16 to ADC Value = 67 - Dump All EM and HD in eta range 1:16 to ADC Value = 68 - Dump All EM and HD in eta range 1:16 to ADC Value = 69 - Dump All EM and HD in eta range 1:16 to ADC Value = 70 - Dump All EM and HD in eta range 1:16 to ADC Value = 71 - Dump All EM and HD in eta range 1:16 to ADC Value = 72 - Dump All EM and HD in eta range 1:16 to ADC Value = 73 - Dump All EM and HD in eta range 1:16 to ADC Value = 74 - Dump All EM and HD in eta range 1:16 to ADC Value = 75 - Dump All EM and HD in eta range 1:16 to ADC Value = 76 - Dump All EM and HD in eta range 1:16 to ADC Value = 77 - Dump All EM and HD in eta range 1:16 to ADC Value = 78 - Dump All EM and HD in eta range 1:16 to ADC Value = 8 - Dump +1,1 EM to ADC Value = 28 - Dump All EM and HD in eta range 1:16 to ADC Value = 8 +1,9 HD to ADC Value = 28 - Dump All EM and HD in eta range 1:16 to ADC Value = 8 +1,17 EM to ADC Value = 28 - Dump All EM and HD in eta range 1:16 to ADC Value = 8 +1,25 HD to ADC Value = 28 - Dump All EM and HD in eta range 1:16 to ADC Value = 8 -8,1 HD to ADC Value = 48 - Dump All EM and HD in eta range 1:16 to ADC Value = 8 -8,9 EM to ADC Value = 48 - Dump All EM and HD in eta range 1:16 to ADC Value = 8 -8,17 HD to ADC Value = 48 - Dump All EM and HD in eta range 1:16 to ADC Value = 8 -8,25 EM to ADC Value = 48 - Dump All EM and HD in eta range 1:16 to ADC Value = 8 +6,29 EM to ADC Value = 10 - Dump +6,29 EM to ADC Value = 12 - Dump +6,29 EM to ADC Value = 14 - Dump +6,29 EM to ADC Value = 16 - Dump +6,29 EM to ADC Value = 18 - Dump +6,29 EM to ADC Value = 20 - Dump +6,29 EM to ADC Value = 22 - Dump +6,29 EM to ADC Value = 24 - Dump +6,29 EM to ADC Value = 26 - Dump +6,29 EM to ADC Value = 28 - Dump +6,29 EM to ADC Value = 30 - Dump +6,29 EM to ADC Value = 32 - Dump +6,29 EM to ADC Value = 34 - Dump +6,29 EM to ADC Value = 36 - Dump +6,29 EM to ADC Value = 38 - Dump +6,29 EM to ADC Value = 40 - Dump +6,29 EM to ADC Value = 42 - Dump +6,29 EM to ADC Value = 44 - Dump +6,29 EM to ADC Value = 46 - Dump +6,29 EM to ADC Value = 48 - Dump +6,29 EM to ADC Value = 50 - Dump +6,29 EM to ADC Value = 52 - Dump +6,29 EM to ADC Value = 54 - Dump +6,29 EM to ADC Value = 56 - Dump +6,29 EM to ADC Value = 58 - Dump +6,29 EM to ADC Value = 60 - Dump +6,29 EM to ADC Value = 62 - Dump +6,29 EM to ADC Value = 64 - Dump +6,29 EM to ADC Value = 66 - Dump All EM and HD in eta range 1:16 to ADC Value = 8 - Dump Philippe examined in detail and annotated this VME_Access logifle, lining it up with the above list of tests. The relevant sections were extracted into a separate file that will be preserved with the archived logfiles. - The operand mapping indeed matches the description at http://www.pa.msu.edu/hep/d0/ftp/run1/l1/caltrig/cards/ tier_2_operand_usage_full_system.txt - We know that we still have two Px problems cf. 9:13-JUNE-2003 > PX PROM for TT(+13:+16,27) There seems to be a bit of value 16 stuck > LOW at the output of this CTFE card. We know this is Operand #3 of +Px and -Py But we don't see anything wrong in the Tier#2 tree dump for this operand, so this may be a readback problem on the Tier#1 Px CAT2 card. This problem does not seem to affect normal operation. > PX PROM for TT(-9:-12,29) and a bit of value 16 stuck HIGH > at the output of this CTFE card. We know this is Operand #6 of +Px and -Py We do see a difference of 16 counts for op#6 of the Tier#2 +Px CAT2 for eta(9:16) for many, but not all, input conditions. This matches the PROM test results and this problem seems to affect normal operation. We still don't know if the problem is on the CTFE (more likely) or the Tier#1 CAT2 card (less likely) or the backplane or connectors (hopefully not!). - The last two test values > +6,29 EM to ADC Value = 64 - Dump > +6,29 EM to ADC Value = 66 - Dump seemed to produce identical Tree Dumps. e.g. > I$ FMLN Op Px : 22 =0x016 =0b000010110 @MBA#159/CA#51 > I$ FMLN Op Py : -18 =0x1ee =0b111101110 @MBA#159/CA#51 Was that correct, or was it an operator error? Verify expected Prom Response: Prom Input (64+8)/2=36 and Px(36)=30=22+8 and Py(36)=26=18+8 Prom Input (66+8)/2=37 and Px(37)=30=22+8 and Py(37)=26=18+8 Other consecutive input values in the list of tested values produced identical Px .OR. Py output values, but these two successive input values produced identical Px .AND. Py output values at the same time. Look at Missing Et Rates Conditions: Only eta 1:8 Tier 2 is plugged into Tier 3 Eta 9:16 Tier 2 is not plugged into Tier 3 Tier 3 Correction Register is setup for running with eta 1:8 MEt GeV Threshold 1-2 Hz L1_Acpt Zero Beam 0 Hz L1_Acpt Zero Beam --------- ------------------------------- -------------------------- 20 267k 268k 267k 267k 269k 269k Hz 30 5.15k 5.24k 5.22k 5.15k 5.11k 5.13k Hz 40 49 55 54 53 47 45 Hz 50 3.3 5.1 3.6 4.6 1.2 1.5 Hz MEt GeV Threshold 0 Hz L1_Acpt 31E30 Beam --------- --------------------------------------------- 20 319k 305k 306k 306k 307k Hz 30 7.72k 7.15k 7.18k 7.19k 7.18k Hz 40 106 93 99 99 98 Hz 50 6.8 5.6 5.7 6.8 3.0 Hz MEt GeV Threshold 750Hz L1_Acpt 31E30 Beam --------- ------------------------------ 20 357k 358k 359k Hz 30 20.1k 20.0k 20.1k Hz 40 4.40k 4.35k 4.33k Hz 50 1.43k 1.40k 1.37k Hz Notes: Cal Trig asserts its And-Or Terms for 3 Ticks. We are running in Page Zero of the Missing Et PROM's no cut. Looking on the scope at the output of the FMLN these look random. Things are not all bunched up right after BOT or when the SVX chips reset. More Missing Et rate Friday morning: And-Or Term Rate And-Or Fired Rate MEt GeV ---------------------------- ---------------------------- Threshold 0 Hz L1_Acpt - Zero Beam 0 Hz L1_Acpt - Zero Beam --------- ---------------------------- ---------------------------- 20 518k 508k 533k Hz 116k 119k 121k 117k 30 20.5k 20.5k 21.8k Hz 4545 4845 5059 4510 40 307 315 328 Hz 64 67 73 59 50 4 3 3 Hz 1 1 1 1 Note that this ratio of rates also leads you to think that the firing of Missing Et is random i.e. not at a fixed point in a turn. Receive a pager from Dmitri. I think its number is 630-266-0935 To use it: Dial the number. You will hear a beep or perhaps three quick beeps (but no verbal prompt). Enter the phone number at which you want to be called back, and then type the pound (#) sign. Then hang up. Find_DAC Runs Ran Find_DAC over just |eta| 17:20 all phi EM-HD keep 2 target 8. Made a first run then pulled the EM-HD swappers on eta +19 phi 9:16. and made a second run. We will keep all eta 19 EM-HD swappers off the system for now. The 2nd Find_DAC run had a pile of very noise towers but just 4 error tags in the tti file: +20,13 EM TAB looks fine, missed the +-0.2 of 8 range, use 3747 -19,9 EM TAB looks OK, missed the +-0.2 of 8 range, use 3708 +20,23 HD TAB looks OK, Ton-O-Noise, Dev about 13, use 3748 -20,4 HD TAB looks OK, missed the +-0.2 of 8 range, use 3737 Edit the tti;2 file to fix up these error tagged entries. The resulting file is: D0_Log/Find_DAC_V0_0_D_20030627_Edited.tti;2 We know that the following 2 PROM's are bad. We have cooked replacements and will install them at the next chance. -17,29 EM -19,28 EM Friday morning Aws and I checked some more high eta TT signals. We found: -17,26 HD BLS signal looks OK. Need to check this channel with pulser. +17,28 HD HD+ looks OK No signal on HD- Cable looks OK and plugged in. +20,1 EM EM- looks OK No signal on EM+ Cable looks OK and plugged in. Captain 12:34:38 PM CDT Jun 27, 2003 At around noon, DAQ problems appeared, which were eventually traced to the fact that the L0 HV power supplies had been turned off by hand (toggle switch in MCH1). DAQ 12:49:26 PM CDT Jun 27, 2003 Somebody manually turned off the power to the lumi crate in the middle of run 178418. this caused a surge in rate to L3 farm causing all the EVB buffers to fill up and therefore the RM started disabling triggers. We had to stop the run with 157388 events. We then had a connection problem in coor with devdnl. Ultimately we had to do a "start_daq comics" and then a reconnect to get it back. We have now started run 178420 with a prescale of 30e30. L1/L2/Tape = 700/461/45 D0Lum = 30.3e30 ------------------------------------------------------------------------------ DATE: 17-JUNE-2003 At: MSU Topics: Trics Revision notes updated. We are currently running Trics V10.4 Rev G. The revision notes for V10.4 Rev D include the description of - the CHTCR PROM Test - the pedestal analysis L1CT diagnostics tool - the CTFE Reference Set output PAL diagnostics tool - the Hot Trigger Tower search tool cf. http://www.pa.msu.edu/hep/d0/ftp/tcc/trics_ii/000_trics_ii_revision_notes.txt ------------------------------------------------------------------------------ DATE: 16-JUNE-2003 At: MSU Topics: Note about Missing Et Operation in Run I Notes about Operation of Missing Et in Run I and the FMLN card: Missing Et was probably first run in the summer of 1992. The following is the only rate information from Run I that I could find right away. 10-July-1992 Luminosity 0.25 10**30 Includes L0_Fast_Z_Good, MR_Cycle, MRBS_Loss veto, Micro_Blank veto 5 GeV 2 kHz 10 GeV 200 Hz 15 GeV 30 Hz 20 GeV 15 Hz 25 GeV 3 Hz 30 GeV 0.1 Hz The flaky paddle board problem involved +Px the bit of value 8 comming from eta +5:+8 Phi 25:32. This is in Tier 2 CAT2 card M105 3rd paddel down from the top. For details see the log book entries for: DATE: 9,10-JAN-1996 & Date: 27,28,29-JUN-1995 & Date: 6-JUL-1994 Date: 30-JUN-1994 & Date: 22-25 MAR 1994 & Date: 16,17,18-FEB-1994 Date: 4-NOV-1993 The problem is not the sending CAT2 or the receiving CAT2. The way to see the problem is to unplug the cable from the sending end, put a 110 Ohm load resistor on the sending end of the cable, check the voltage across the sending end load and see that for bit of value 8 you do not have the expected spread due to the receiving end (Tier 2) bias resistors at the input network. This is not a FMLN problem. ------------------------------------------------------------------------------ DATE: 9:13-JUNE-2003 At: Fermi Topics: Cal Trig Tier 2-3 Power Supplies, Missing Et work, TT work, FPD AOIT, New TRICS, Voltage Monitoring Cables, Master Clock Timing Cables, CHTCR Test Runs, Find_DAC runs, High eta rack ADC Clk Switches, CMT-11.04 load Cold Start Instructions are the same as listed at the end of the 20:23-MAY-2003 log book entry except that we are now running TRICS Version 10.4.G and that the current Gains files are: D0_Config/Gains_17_20_1_32_rev_a.tti # TT eta's 17:20 all phi's D0_Config/Gains_1_16_1_32_rev_4.tti # TT eta's 1:16 all phi's The current Pedestal files are: D0_Log/Find_DAC_V4_1_D_20030307_edited.tti;4 # TT eta 13:20 D0_Log/Find_DAC_V5_0_C_20030613.tti;2 # TT eta 1:16 Started running TRICS V10.4.D:G The new Cal Trig diagnostics tools look super. Power Supply Work Return Pioneer Magnetics 5V bricks MSU# 67 and MSU# 95 to Fermi. They have had the MSU 6 Capacitor modification installed. Return Pioneer Magnetics 2V brick MSU# 54 to Fermi. It has had the MSU 7 Capacitor modification installed. Looked at the Tier 2 and Tier 3 power supplies before installing the Tier 2 Momentum CAT2 cards. The Pioneer Magnets -2V 325A supplies look to be in trouble. Add Capacitor Band-Aids to these supplies before plugging in the Tier 2 CAT2 cards. Did not look at or add a Cap Band-Aid to the high eta Tier 2 -2V supply. Tier 2 |eta| 1:8 After Adding After Adding After all Work Supply Initial Look Cap Band-Aid Momentum CAT's Shea Box Reads ------ ----------------- -------------- -------------- -------------- +5.0V 5.060 0 mV 5.060 0 mV 5.060 0 mV +5.13 V -2.0V 2.040 400/450 mV 2.041 8/50 mV 2.041 9/52 mV -1.92 V -4.5V 4.540 0 mV 4.540 0 mV 4.537 0 mV -4.38 V -5.2V 5.231 0 mV 5.231 0 mV 5.232 0 mV -5.03 V Tier 3 After Adding After all Work Supply Initial Look Cap Band-Aid Shea Box Reads ------ ----------------- -------------- -------------- +5.0V 5.054 0 mV 5.054 0 mV +5.07 V -2.0V 2.041 525/580 mV 2.041 8/50 mV -1.99 V -4.5V 4.574 0 mV 4.574 0 mV -4.51 V -5.2V 5.237 0 mV 5.237 0 mV -5.10 V Tier 2 |eta| 9:16 After Adding After Adding After all Work Supply Initial Look Cap Band-Aid Momentum CAT's Shea Box Reads ------ ----------------- -------------- -------------- -------------- +5.0V 5.062 0 mV 5.063 0 mV 5.063 0 mV +5.15 V -2.0V 2.041 730/750 mV 2.041 11/55 mV 2.041 11/57 mV -1.89 V -4.5V 4.543 0 mV 4.543 0 mV 4.538 0 mV -4.35 V -5.2V 5.223 0 mV 5.223 0 mV 5.223 0 mV -5.00 V The full list of supplies that have received Capacitor Band-Aids is now: Date Power Pan PDM Capacitor Installed Location Number Type Notes --------- ------------- ------ --------- -------------------------- 10-JUN Tier 2 eat 1:8 Sprague on -2.0V 10-JUN Tier 2 eat 9:16 Sprague on -2.0V 10-JUN Tier 3 CDE on -2.0V 29-MAY M112 Upper T1 -4.5 has MSU 6 Cap, Band-Aid -2 & -5.2 8-MAY PDM-13 spare T1 Pan -4.5 & -5.2 have MSU 6 Cap, -2 has Band-Aid 1-APR PDM-09 spare T1 Pan -2 -4.5 -5.2 have Refurbished + Band-Aid 1-APR M110 Upper T1 Sprague on all 3 Taped capacitors 18-MAR M110 Lower T1 Sprague on all 3 Taped capacitors 6-MAR M105 Upper T1 Mallory -4.5 CDE -2.0 & -5.2 6-MAR M106 Upper T1 Mallory -4.5 CDE -2.0 & -5.2 6-MAR M107 Lower T1 Mallory -4.5 CDE -2.0 & -5.2 6-MAR M108 Upper T1 Mallory -4.5 CDE -2.0 & -5.2 6-MAR M109 Lower T1 CDE on all 3 6-MAR M111 Lower T1 CDE on all 3 6-MAR M111 Upper T1 CDE on all 3 6-MAR PDM-13 CDE on all 3 Taped capacitors 19-FEB M103 Lower T1 Mallory on all 3 19-FEB M105 Lower T1 Mallory on all 3 19-FEB M108 Lower T1 Mallory on all 3 19-FEB M106 Lower T1 Mallory on -4.5 & -5.2 only none on -2 Work with the Missing Et Trigger Run the PROM checker on Px and Py over |eta| 1:16. Px is in trouble at eta +13:+16 phi 27 and at eta -9:-12 phi 29. Py looks OK now over this full range. Philippe dug out the log files and the two remaining problems appear to be: PX PROM for TT(+13:+16,27) There seems to be a bit of value 16 stuck LOW at the output of this CTFE card. PX PROM for TT(-9:-12,29) and a bit of value 16 stuck HIGH at the output of this CTFE card. With things as they are, including these problems, take a look at the Missing Et rates: With only |eta| 1:8 plugged in to Tier 3 and no beam Threshold ZB DAQ Stopped ZB running 400 Hz --------- -------------- ----------------- 20 GeV 222 kHz 243 kHz 25 GeV 35 kHz 47 kHz 30 GeV 4650 Hz 11 kHz 35 GeV 740 Hz 4870 Hz With only |eta| 1:16 plugged in to Tier 3 and no beam Threshold ZB DAQ Stopped ZB running 400 Hz --------- -------------- ----------------- 20 GeV 980 kHz 1.04 MHz 25 GeV 323 kHz 365 kHz 30 GeV 84 kHz 106 kHz 35 GeV 18 kHz 29 kHz Look at the LED's on the +-Px and +-Py cards in both Tier 2's. During beam eta 1:8 has 11 LED's ON and a 12th LED glows very dimly During beam eta 9:16 has 11 LED's ON Work and readout problem in rack M108 (eta -9:-12) that appears to once in a while to effect phi's 13 and higher. Note that phi 13 is the first CTFE readout by the 4th ERPB card in the rack. The problem appears that it may have to do with passing the "GO" from the 3rd ERPB to the 4th ERPB. Swap the following M108 ERPB's with their equals in rack M112 which is eta -17:-20 and does not matter for now. ERPB M108 <- swap -> M112 -------- ----------- ----------- 3rd ERPB Pull SN# 30 Pull SN# 90 4th ERPB Pull SN# 28 Pull SN# 91 OK, now M108 looks OK (continues to look OK) and M112 is screwed up all the time. Thursday morning figure out that the only problem with M112 is that it was never initialized to get its ADC Clk running since it was power cycled for the ERPB move. Initialize out to eta 20 and it comes to life. So we are back to zero understanding about whether or not these ERPB's were actually the problem with -9:-12 13:32 readout data. Not good. General summary: 1. Bob, Aws, Joe spotted some funny looking pedestal files in an expert mode run of the L1 Cal Examine on a Physics Beam run file from Friday 6-JUNE-2003. Store 2642 Run 177751 Pedestals looked funny in the range eta -9:-12 phi 13:32. EM was sometimes at zero and HD had bands in it at places other than 8. This was perhaps 10% ??? of the events. 2. Tuesday afternoon 10-JUNE-2003 Dan and Aws looked at raw L1 Cal Trig readout data from 23 events. They saw 1 event that looked very funny. The data from the first 3 ERPB cards in the eta -9:-12 rack looked fine, i.e. out through phi 12 looked fine. The data from the next 2 ERPB cards was all 0x070B - this is phi's 13:20. The data from the last 3 ERPB cards in that rack was all 0x080B except for the first 2 locations which were 0x070B and 0810. These last 3 ERPB cards service phi's 21:32. 3. Tuesday afternoon and evening Aws and Bob ran the Examine online during the store and looked at the EM_Ped_CTFE_Rack_2 and the Had_Ped_CTFE_Rack_2 plots. They looked at a sample with about 1,000 events and at a sample with about 10,000 events. Everything looked OK. 4. Wednesday morning 11-JUNE Aws started an offline Examine run to look at 100,000 events from the store that ran from Tuesday afternoon to early Wednesday morning. Store Number 2671, Run 177945. It will take about 24 hours to process this data. 5. Wednesday morning between stores Dan swapped the 3rd and 4th ERPB cards in rack M108 (i.e. eta -9:-12) with rack M112 (eta -17:-20). During the subsequent ZB run, Aws and Joe ran the expert mode L1 Cal Examine online and saw no problem in "CTFE_Rack_2" plots, i.e. the data from rack M108 looked OK. Dan looked at a number of raw events and M108 data looks fine - but the data from M112 is now solidly screwed up. It is too close to the next store to do any work on M112. But M112 (eta -17:-20) is not used for L1 triggering and is not used by L2 Cal Preprocessor. So no problem - we will run this store with M112 data screwed up. This ends up just being that M112 was never initialized. 7. On Thursday Aws reported to everyone about the results of the 100,000 event analysis of the Tuesday afternoon to Wednesday AM store Store 2671, Run 177945 - all looked OK 8. Thursday afternoon Reinhard reports that the high statistics Global Monitoring plots from L2 Cal only show a fall off in L2 acceptance in the range eta -9:-12 phi 13:32 for that one store and mostly for that one run. 9. Aws will continue to run expert mode L1 Cal Examine during each store for the next week or so. Work with FPD folks. Move their BOT timing signal to a Sync Gap Timing Signal. Plug in their taped up 34 conductor And-Or Term Input cable to the TFW AOIT 112:127. After some adjustments they lock up OK with 16 steps in use in the AOT Display of TrgMon. I verified that the common mode looked OK. For now all is OK with this screw driver tweeked setup. Installed the power supply voltage monitoring cables for M124. They plug into the RT=26 Shea Box channels 24:27. The file that shows the mapping was brought up to date. All this new stuff looks OK in the RM.py display. Recall that the 9 pin "D" connector pinout is: Pin #1 Hi Side Ch relative #1 Pin #6 Low Side Ch relative #1 Pin #2 Hi Side Ch relative #2 Pin #7 Low Side Ch relative #2 Pin #3 Hi Side Ch relative #3 Pin #8 Low Side Ch relative #3 Pin #4 Hi Side Ch relative #4 Pin #9 Low Side Ch relative #4 Pin #6 Shield Worked with Dean Thursday morning to make recorded runs with the Ref Set only on a single TT, i.e. one of the Excluded TT's. We looked at +5,23 EM -4,25 HD and got good data for both and also looked at -4,7 EM/HD and probably got good data. This should help find what layer has the sparks. Worked with Aws on Thursday looking at Trigger Tower signals. Looked at 8 TT's with the following results: -13,18 EM No signal on the + side of the cable. The signal on the - side looks OK. The cable does not look shorted. During access look at this BLS. -13,32 EM There is symmetric noise on each side of the cable so it is not shorted and it is plugged in. There is nothing that looks like a real BLS trigger pickoff signal. During access look at this BLS. -12,18 EM No signal BLS Trigger pickoff signal on either side of the cable. There is some L1_Acpt noise pickup that you can see on both sides of the cable - so the cable is not shorted and it is plugged in. During access look at this BLS. +12,17 EM No signal at all on either side of the cable. The cable is not shorted. The cable appears to be plugged in. During access look at this BLS. +13,2 EM Signals look OK. Will check this CTFE with the pulser before looking at this BLS. Have now used pulser to check CTFE gain - it is correct. During access look at this BLS. +16,8 EM No signal on the - side of the cable. The signal on the + side looks OK. The cable does not look shorted. During access look at this BLS. +17,25 EM No signal on the - side of the cable. The signal on the + side looks OK. The cable does not look shorted. During access look at this BLS. +17,28 EM Signals look OK. Will check this CTFE with the pulser before looking at this BLS. Have now used pulser to check CTFE gain - it is correct. During access look at this BLS. Master Clock Timing Cables Received the 1.5 2.0 2.5 3.0 nsec Master Clock timing cables from Victor and labeled them and put then in the assigned storage box. Replace Term-Attn-Brd's Replaced the 4 Terminator Attenuator Boards that service TT's -15,19 -16,19 -15,25 -16,25 +13,23 +14,23 +15,29 +16,29 Checked and verified the gain of the new cards. Typical output levels to see for these eta's with the Hand Pulser with both sides of the line turned on are the following. eta Typical EM Typical HD --- ---------- ---------- 13 EM 660 mV HD 650 mV 14 EM 560 mV HD 550 mV 15 EM 1280 mV HD 1260 mV 16 EM 1090 mV HD 1090 mV Runs of CHTCR Test Philippe made multiple runs of CHTCR Test. It verified a missing -17,29 EM PROM. There were also two IO errors to MBA#231/CA#41/FA#40 where the first write failed, but the second succeeded. This corresponds to TT -18,21 Tot Et Threshold Comparator #0. In both cases, the LSbit was reading back 0 instead of 1. This only happened twice out of many writes to this register. The more serious problem discovered by CHTCR Test is at -10,11 (which is in M108). -10,11 looked funny for a while and then started looking fine in subsequent runs of CHTCR test. More digging is required here. The errors are very focussed on TT(-10,11). What I thought was weird (PROM#1 fails for EM, while it is PROM#0 that fails for Tot) actually makes sense, and is due to the reverse Phi ordering of EM towers, as opposed to Tot. TT(-10,11) uses address bit#4 of PROM #0 for EM, and bit#9 of PROM #1 for Tot. The errors are focussed, but not systematic. If that tower had been stuck, there would have been way more errors. It is complaining about the tower not being asserted, but never about being negated. Given that the tower threshold gets toggled between 0 and 255, it is unlikely that this is due to an incorrect or missing simulated DAC value during the initialization of the test, even if our little friends were dumping 50 GeV in our inputs... I think. The test was happy with EM Ref Set # 0, and #1, and failed EM #2 and #3 in a similar manner. All Tot Et RS have similar but very few (one) error; probably because this is a high order address bit that gets flipped high only twice, I think. That it passed the first two channels of that card, and failed the other 6 channels, may tell us that something flaky kicked in during the test. It is also peculiar that the complaint is about the CHTCR input, but almost nothing about the CHTCR output. The exception is that Tot RS #2 complained about the output while checking the second pass prom. (This may mean something cf below *). The only other thing I could think of is not pretty. It could however possibly tie in with the readout problem. The CHTCR input is read from the 29525's and, for whatever reason, I decided to read the oldest slice, not the newest. What if this is just telling us that sometimes something doesn't get clocked in for that channel. Could we possibly have Timing Signal distribution (or termination) problems in that crate? Could that hose the ERPBs? (*) For the 3x first pass PROMs, the tester checks the CHTCR input right after having changed the CTFE threshold. For the 1x second pass PROM, the tester skips checking the CHTCR inputs (asumed to be under control) and goes right to the CHTCR output, i.e. CAT2 input. The pattern of errors could thus be consistent with the information not always travelling right away. The first pass PROM tester complains about the CHTCR input, while the CHTCR output is ok by the time it looks at it. The second pass PROM tester gets a better chance at noticing ouptput errors because it gets to it earlier. These are clearly not solid errors at the near DC speed of the PROM tester. They are also not always there, as we only caught this on one test run. Maybe we were just "lucky" and got a glimpse at a flaky problem. Maybe that particular channel is somehow the most sensitive to the flakyness, and that's why the errors seemed so focussed. Check for M108 Timing Signals The 16 Lemo monitor points on the front panel of M107 CTMBD (eta +9:+12) match those in M108 (eta -9:-12) in both cases upper Tier 1. Look at the Upper Tier 1 backplane timing signals. Check A:H in detail and they are all good ECL levels. Check the 2 clock signals to the Tier 1 CAT's and they are good ECL levels. I did not make a detailed check for reflections and such. Look at the end of the Parallel Bus to the ERPB's. All signals look to be OK ECL levels and rations waveforms. So there may be a detailed problem with the M108 timing signals but at the bog picture level all is OK. Find_DAC Runs Made 2 runs on 13-June. The first run is without a historgram file and was just a quick test of the new Term-Attn-Brd's. The second run is full eta 1:16 coverage and has a histogram file. Survey the 4 high eta racks CTFE ADC Clk Switches Look at CTFE ADC Clk Switches in racks M107 : M112. M107 & M108 are fine. The top 1/4 of M109:M112 are correct, i.e. the part that was stacked first in December 2002 is correct - so all the timing measurements that were done based on that are correct. The bottom 3/4 of M110:M112 are screwed up and will have to be pulled apart. M110 has 2&7 closed in the bottom 3/4. M112 and M111 have 2&5 closed in the bottom 3/4. Oh Joy. Capture of the first part of a CMT-11.04 COOR download. L1CT_Ref_Set EM_Et_Ref_Set 0 TT_ETA(-8:8) Value 2.5 L1CT_Ref_Set EM_Et_Ref_Set 1 Value 5.0 L1CT_Ref_Set EM_Et_Ref_Set 2 Value 10.0 L1CT_Ref_Set EM_Et_Ref_Set 3 Value 15.0 L1CT_Ref_Set TOT_Et_Ref_Set 0 Value 5.0 L1CT_Ref_Set TOT_Et_Ref_Set 1 Value 7.0 L1CT_Count_Threshold EM_Et_Towers Ref_Set 0 Comparator 0 Value 2 L1CT_Count_Threshold EM_Et_Towers Ref_Set 0 Comparator 1 Value 3 L1CT_Count_Threshold EM_Et_Towers Ref_Set 1 Comparator 0 Value 1 L1CT_Count_Threshold EM_Et_Towers Ref_Set 1 Comparator 1 Value 2 L1CT_Count_Threshold EM_Et_Towers Ref_Set 2 Comparator 0 Value 1 L1CT_Count_Threshold EM_Et_Towers Ref_Set 2 Comparator 1 Value 2 L1CT_Count_Threshold EM_Et_Towers Ref_Set 3 Comparator 0 Value 1 L1CT_Count_Threshold Tot_Et_Towers Ref_Set 0 Comparator 0 Value 2 L1CT_Count_Threshold Tot_Et_Towers Ref_Set 0 Comparator 1 Value 1 L1CT_Count_Threshold Tot_Et_Towers Ref_Set 0 Comparator 2 Value 3 L1CT_Count_Threshold Tot_Et_Towers Ref_Set 0 Comparator 3 Value 4 L1CT_Count_Threshold Tot_Et_Towers Ref_Set 1 Comparator 0 Value 4 I changed the order of this to make it readable by humans and pulled out all of the L1CT_Ref_Set HD_Veto_Ref_Set 2 Value 10000.0 ------------------------------------------------------------------------------ DATE: 28:31-MAY-2003 At: Fermi Topics: Cal Trig Power Supply Work, Term-Attn Brd work, Find_DAC runs and Gains File, Install Voltage Monitoring into M122 and M123, Power outages, Watch Cal pickup of 13.2 as iron closes Work on Term-Attn-Brd's in eta -9:-12 phi 1:24 phi 1 Both OK phi 2 Replace Both phi 3 Low was OK Replace Hi phi 4 Replace Both phi 5 Both OK phi 6 Replace Both phi 7 Both OK phi 8 Replace Both phi 9 Both OK phi 10 Both OK phi 11 Replace Both pinched green wire under pin 13 phi 12 Replace Both phi 13 Replace Low Hi was OK phi 14 Replace Both phi 15 Low was OK Replace Hi phi 16 Replace Both phi 17 Replace Both phi 18 Both OK phi 19 Replace Low Hi was OK phi 20 Low was OK Replace Hi phi 21 Both OK this CTFE had replaced choke and taped serial cable phi 22 Both OK phi 23 Replace Low Hi was OK phi 24 Replace Low Hi was OK And once again what I found on the boards was an exact match with Bob's plots and then with how gins had been setup in the Gains file. PROM work Verified that the -16,29 HD Energy PROM was OK. Dug into the problem with the Py Momentum PROM's for +16,25 : +16,31. The part labeled phi "N" actually had the contents for phi "N+1". Cooked a new PY part and re-labeled and moved things so that all of this should be OK now. Power Supply work Pull M112 Upper Tier 1 It is PDM-15 It consisted of: +5.0V is MSU# 21, -2.0V is MSU# 69, -4.5V is MSU# 67, -5.2V is MSU# 68. Pull MSU# 67 the -4.5V brick from PDM-15 and ship it back to MSU. Install MSU# 71 as the new -4.5V brick in PDM-15. MSU# 71 has the 6 Capacitor modification. Install Cap Band-Aids on the -2.0V and -5.2V bricks. Noisy Trigger Tower Signals I went through all the TT's that we currently have excluded trying to find them in the act of causing trouble. Everything looks OK - what's going on ? +2,25 EM OK with HV off, HV on sparks 0.1 Hz looks like real signal -4,7 EM&HD OK with HV off, HV on both make sparks -4,25 HD OK with HV off, HV on sparks 0.2 Hz looks like spark ringing +5,23 EM OK with HV off, HV on sparks 1 Hz looks like real signal +8,28 EM OK with HV off, not tested today with HV on -6,22 EM OK with HV off, Hv on pulse every 4 usec 100 mv undulate 30 usec -6,28 EM OK with HV off, HV on bursts of high freq and pulses +9,21 EM OK with HV off, not tested today with HV on -9,22 HD OK with HV off, not tested today with HV on On Thursday 29-May-2003 watch the 13.2 MHz noise as the iron is closed. This is looking at TT +1,2 HD. This is the same TT as we watched in the 24-APR-2003 as the iron was closed (see log book). This is done with the "standard" setup of the scope - spectrum analyzer as described in the log book entry for Thursday East & West both open 100 - 200 uV Close West East full open 46 1/2" 200 - 225 uV Closing East to 41 1/2" 350 - 400 uV 30 1/2" 375 - 410 uV 20" 340 - 390 uV 11 1/2" 300 - 330 uV 5 1/2" 180 - 220 uV East closed 0" 950 - 1000 uV re-open East to 1" 1200 - 1300 uV 2" 1200 - 1300 uV East closed 0" 950 - 1000 uV Saturday morning everything closed up and running 750 - 800 uV Voltage Monitoring Install the Voltage Monitoring cables into racks M122 and M123. They are connected to the 2nd Shea box, i.e. Shea Box Remote Terminal = 26 Shea Box Voltage Monitor Signal Analog --------------------------------------------------------- Channel Power Supply Rack - Crate Function -------- ---------------- --------------- --------------- 16 +5.0 Volt supply M122 All Crates L2 FW & PBS 17 +3.3 Volt supply M122 All Crates L2 FW & PBS 18 -2.0 Volt supply M122 All Crates L2 FW & PBS 19 -4.5 Volt supply M122 All Crates L2 FW & PBS 20 +5.0 Volt supply M123 All Crates L1 FW 21 +3.3 Volt supply M123 All Crates L1 FW 22 -2.0 Volt supply M123 All Crates L1 FW 23 -4.5 Volt supply M123 All Crates L1 FW Power up Saturday morning Turn back on and have one TT stuck above all Ref Sets. Figure out that it is some where in eta -9:-12 phi 1:8. Read the CHTCR at MBA = 204 CA = 2. FA's 1,5,9,13 read 128. FA's 17,21,25,29 read 1. All other FA's read 0. This means relative eta 2 phi 1. So pull the top CTFE and its #2 PAL has a hole burned in the center of its label. Replace this PAL and now everything looks OK. I do not know why this part cooked itself. Make a fast run of Find_DAC over just the region with the new Term-Attn-Brd's. That looks OK. Then make a full run over |eta| = 1:16. Then after the Cal Trig has been running for about 2 1/2 hours make a 3rd run of Find_DAC over |eta| = 1:16. It is the results of this 3rd run that we will use. Cold Start Instructions are the same as last week except that the current Gains files are: D0_Config/Gains_17_20_1_32_rev_a.tti # TT eta's 17:20 all phi's D0_Config/Gains_1_16_1_32_rev_4.tti # TT eta's 1:16 all phi's The current Pedestal files are: D0_Log/Find_DAC_V4_1_D_20030307_edited.tti;4 # TT eta 13:20 D0_Log/Find_DAC_V5_0_C_20030531.tti;3 # TT eta 1:16 ------------------------------------------------------------------------------ DATE: 20:23-MAY-2003 At: Fermi Topics: Cal Trig Power Supply Work, AOT #246 once per 5 seconds, Term-Attn-Brd work, Test of FPD ground, Find_DAC runs and Gains File, Install Voltage Monitoring cables for rack M101, SCL Receivers, Current Cold Start Instructions are at the end of the Log Book entry. On Monday Don Lincoln took the two spare SCL Receivers out of the locked spares cabinet. He said that Ted was going to bring more of them to D-Zero. so far I have none, i.e. zero to support the running Physics experiment. Work on M109 Lower Tier 1 Power Supply Before turning anything off, look at this Power Pan with the Fluke meter and look at the Shea Box Monitoring display. Fluke Front Panel Shea Monitoring ----------------- ----------------- +5.0 5.054 0/ 5.03 - 5.04 -2.0 2.095 1/4 2.02 - 2.04 -4.5 4.605 8/51 4.42 - 4.65 -5.2 5.228 0/2 5.16 - 5.24 Guess that most of the problem is with the -4.5 brick and that if had a lot of ripple that this may show up in the other supplies too. Shut off and pull the pan. It is PDM-21 and consists of: +5.0 MSU # 5 -2.0 MSU # 45 -4.5 MSU # 95 -5.2 MSU # 46 All 3 of the Pioneer Magnetics supplies had Capacotor Band-Aids on them. Then in PDM-21 replace the -4.5 brick MSU #95 with MSU #21 which has the MSU 6 Capacitor modification installed. Do not put the Capacitor Band-Aid back on this supply. Put PDM-21 back into M109 Lower Tier 1. Now with the system cold, i.e. basically just this pan running, after 15 min it reads: Fluke Front Panel Shea Monitoring ----------------- ----------------- +5.0 5.054 0/ 5.03 - 5.05 -2.0 2.092 1/4 2.00 - 2.04 -4.5 4.606 0/ 4.52 - 4.57 -5.2 5.223 0/2 5.16 - 5.20 The remaining question is should I also have replaced the -5.2 brick ? Take back to MSU Pioneer Magnets supplies: MSU# 95 (a 5V supply) and MSU# 54 (a 2V supply) to have their internal capacitors replaced with the MSU 6 Cap setup. Once per 5 Seconds The once per 5 second source is asserted for 4.57 usec when the Cal Trig racks were cold. By mistake I have the Red and Black wires backwards in the cable that bring MTG output #32 to the TFW And-Or Term Input patch panel, i.e. at the TFW end you need to use Black as Direct and Red as the Complement. On Wednesday connect this to the TFW. I put a note into the elog book. This once in 5 seconds source is plugged ito And-Or Term #246 which had been the SCL "spare_marker" term. COOR's name for this term is "spare_Marker" and we can just leave it that way. You can lookup COOR's logical names in /online/data/coor/resources/coor_resources.xml I did check to see if anyone have been using the "spare_marker" term as it was by grepping around the trigger files in the smt and cft sections of /online/data/coor/configurations Using a trigger file that Reinhard wrote that sets up a L1 Trigger based on just this once per 5 seconds spare_marker term and requests that the L2 collect_status Qualifier be asserted each time this trigger fires we can see this Qualifier on the logic analyzer (if we plug the test SCL Receiver into the lower half of the G.S.) and Reinhard can see it in the running L2 processor. This is Qualifier #7 and shows up on the logic analyzer as $0080. You can look up the Qualifier Number to logical name in: /online/data/coor/resources/l1qualifiers.xml We want this Qualifier to appera in both the low half of the G.S. and in the upper half. E.G. soon STT also want to use this Qualifier. To be decided should the trigger file ask for Qualifiers collect_status & qual23 or should TRICS just give you both "N" and N+16 ? Best long term way to live ? Work on Term-Attn-Brd's in eta +9:+12 phi 1:18 phi 1 Replace Both phi 2 Replace Low High was OK phi 3 Low was OK Replace High phi 4 Both were OK phi 5 Low was OK Replace High phi 6 Both were OK phi 7 Replace Both phi 8 Replace Low High was OK phi 9 Low was OK Replace High phi 10 Replace Both phi 11 Replace Both phi 12 Both were OK phi 13 Both were OK - major big nick in the Blk wire next to pin 13 phi 14 Replace Both phi 15 Both were OK phi 16 Replace Low High was OK phi 17 Replace Both phi 18 Replace Both installed the card with the epoxied together connector. So in total checked 18 CTFE and replaced 20 Term-Attn-Brds. There were lots of CTFE cards with the Momentum PROM page select jumper wire all jerked around. I did not catch this right away. Phi 1:8 I did check very carefully. Phi 9:18 may still contain some problems with these jumper wires. People were clearly jerks when they pulled out the PROM's and then blind when they installed the new ones. I assume that these are cards that had parts pulled at D-Zero, i.e. not at SiDet. Edit the gain file. The new gains file for |eta| 1:16 is D0_Config/ Gains_1_16_1_32_rev_3.tti edit this so that all the new correct R80 Term-Attn-Brd's are set back up to their correct gain. Find_DAC Runs: D0_Log/Find_DAC_V5_0_C_20030521.tti;1 over just eta +9:+12 all phi to verify that the new Term-Attn-Brd's were all OK. D0_Log/Find_DAC_V5_0_C_20030521.tti;2 over all |eat| 1:16 keep = 2 the Cal Trig had only been on for about 30 minutes this run found pedestals for all TT's D0_Log/Find_DAC_V5_0_C_20030521.tti;3 over all |eat| 1:16 keep = 2 the Cal Trig had been on for about 1 1/2 hours before this run of Find_DAC started this run found pedestals for all TT's for now will use this run for cold start. Voltage Monitoring in M101 Install the Voltage monitoring cables to rack M101. M101 has two separate supply chassis: one for the bottom crate the Routing Master and one for the Middle & Top crates the L1 Cal Trig Readout. These 8 supplies will go to 8 channels in the first Shea Box. This will be cabled up with the same fancy cables that have the 1N5818 diode || 200 Ohm resistor that we will use in racks M122, M123, M124 to group the power supplies together. The connection of M101 to the Shea Box is the following: Shea Box Remote Terminal = 25 Shea Box Voltage Monitor Signal Analog --------------------------------------------------------- Channel Power Supply Rack - Crate Function -------- ---------------- --------------------- --------------- 32 +5.0 Volt supply M101 Bottom Crate Routing Master 33 +3.3 Volt supply M101 Bottom Crate Routing Master 34 -2.0 Volt supply M101 Bottom Crate Routing Master 35 -4.5 Volt supply M101 Bottom Crate Routing Master 36 +5.0 Volt supply M101 Mid & Top Crates Cal Trig Readout 37 +3.3 Volt supply M101 Mid & Top Crates Cal Trig Readout 38 -2.0 Volt supply M101 Mid & Top Crates Cal Trig Readout 39 -4.5 Volt supply M101 Mid & Top Crates Cal Trig Readout The color code of the wires in the new fancy Voltage monitoring cables is: Red = +5.0V Blue = +3.3V White = -2.0V Orange = -4.5V Green/Black is ground Cold start instructions Rev. 23-MAY-2003 We are running: TRICS V10.3.B and VME_Access V5.0.C See the full instructions at: www.pa.msu.edu/hep/d0/ftp/l1/framework/hardware/rack_crate/ master_clock_instructions.txt www.pa.msu.edu/hep/d0/ftp/l1/framework/hardware/rack_crate/ framework_power_control_procedures.txt www.pa.msu.edu/hep/d0/ftp/l1/cal_trig/hardware/rack_crate/ cal_trig_power_control_procedures.txt Overview: After Master Clock is running, turn on TFW and Cal Trig (one power supply at a time) then: Configure FPGA's The normal Master Command file for this does everything except for the M101 Routing Master Get the Routing Master Configured and Running 1) After the Routing Master is powered up, wait for its SBC to boot (less than 1 minute) 2) Login to the SBC with 'ssh d0runsu@d0sbc001b' The password is same as the online d0run account. 3) On the SBC run 'reset_all.sh stop' which will stop the relevant readout processes. 4) Use TCC's Configure FPGA's menue to execute the dcf that configures the Routing Master FPGA's. This file is D0_Config\M101_Routing_Master_All.dcf 5) On the SBC run 'reset_all.sh start' Tell TRICS that the Cal Trig eta coverage is 1:20 Init TFW and Cal Trig Tell TRICS to completely ignore the Cal Trig By hand using VME Access load the Cal Trig Gains files. (made sure that the "No Write to DAC's" box is not checked) Load then in the order indicated below. By hand using VME Access load the Cal Trig Pedestals files. (made sure that the "No Write to DAC's" box is not checked) Load then in the order indicated below. Tell TRICS that it has control of the Cal Trig again and set the Cal Trig eta coverage to 1:16. Init the TFW and Cal Trig verify that it is a clean init of all parts The current Gains files are: D0_Config/Gains_17_20_1_32_rev_a.tti # TT eta's 17:20 all phi's D0_Config/Gains_1_16_1_32_rev_3.tti # TT eta's 1:16 all phi's The current Pedestal files are: D0_Log/Find_DAC_V4_1_D_20030307_edited.tti;4 # TT eta 13:20 D0_Log/Find_DAC_V5_0_C_20030521.tti;3 # TT eta 1:16 ------------------------------------------------------------------------------ DATE: 7:10-MAY-2003 At: Fermi Topics: Cal Trig Power Supply Work, SCL Status Cables to L2 Glb and L2 Cal, Collect noise data, Survey And-Or Input Terms, SCL Receivers - talk with Ted wrt VRBC For cold start instructions see the log book entry for 22:25-APRIL-2003 Cal Trig Power Supplies Bring Pioneer Magnets supplies MSU SN# 26 and SN# 47 to Fermi. These two supplies have the final version of the 6-Cap modification installed in them. Use these 2 supplies to rebuild PDM-13. PDM-13 now consists of: +5.0 Volt is MSU SN# 6 +5.055 V 0 mV @ 214 A -2.0 Volt is MSU SN# 70 with Capacitor Band-Aid -2.104 V 0 mV @ 161 A -4.5 Volt is MSU SN# 47 -4.608 V 0 mV @ 111 A -5.2 Volt is MSU SN# 26 -5.227 V 0 mV @ 100 A Pioneer Magnetics supplies MSU SN# 71 and SN# 72 were pulled out of PDM-13. Currently these two supplies can only work if they have a Capacitor Band-Aid. Take MSU SN# 71 and SN# 72 back to MSU to receive the 6-Cap modification. Also take back to MSU SN# 21 to receive the 6-Cap modification. MSU SN# 21 is the initial supply that Petr worked on for the proto-type test. www.pnpi47red.narod.ru/index_en.html Check Power Supplies Check again on the aging study to see how the Capacitor Band-Aids are holding up in service. See 10:12-MAR-2003 log book entry for details. Shea Box Power Supply Channel Cap Original 11-MAR 2-APR 9-MAY ---------------------- ---------- --- -------- ------ ------ ------ M111 Top Tier 1 -4.5V rm_B ch_12 CDE 1130 mV 130 mV 110 mV 140 mV M111 Low Tier 1 -4.5V rm_B ch_48 CDE 1050 mV 50 mV 60 mV 70 mV M108 Low Tier 1 -4.5V rm_B ch_32 Mal 720 mV 70 mV 60 mV 60 mV M106 Low Tier 1 -4.5V rm_A ch_60 Mal 680 mV 40 mV 50 mV 50 mV I used RM.py to review what bricks still look like they are in trouble and what I see fits what is written in the log book for 1:4-APRIL-2003 At the next opportunity I need to: T1 M109 Lower replace the -4.5V brick which now has 230 mV ripple T1 M109 Lower replace the -5.2V brick which now has 90 mV ripple T1 M112 Upper replace the -4.5V brick and Cap Band-Aid the rest T1 M109 Upper Cap Band-Aid all around SCL Status Cables to L2 Glb and L2 Cal Reinhard watched and tested the L1_Busy, L2_Busy, L1_Error, and L2_Error from G.S. $20 and $23 the L2_Global and the L2_Cal crates. He found that L1_Busy from $20 (L2_Global) did not make it to the TFF and in $23 (L2 Cal) none of the 4 status signals made it to the TFW. In the other two L2 crates, L2 MUC and L2 MUF all was OK. I looked at the Status Cables that run to $20 and $23 and the connectors look fine. Both cables are de-laminated in the L2 rack so lines could have broken. Make 2 new cables using the orange/white solid wire twist-flat and install them under the floor for the M121 to M124 run. Reinhard re-tested and now all was OK. So we will make these new cables the permanent SCL Status Cables for the M121 L2 rack. Friday morning collect some more noise data. Disk 1 is -9,17 Disk 2 is -12,15 Disk 3 is -4,14 Disk 4 is -1,17 On each disk files 00 and 01 tif and csv are EM. On each disk files 02 and 03 tif and csv are HD. This was the standard setup looking only a the inverting side of the TT signal. Full description of the setup is in the 8:11-APRIL-2003 And-Or Term Patch Panel Survey what is actually plugged into the And-Or Term Patch Panel Input Connector What is actually plugged in and/or Label on the cable --------- ------------------------------------------------------------- 0:15 0,1,2 are L1AL2 Comparators 1,2,3 8:11-APRIL-2003 Log Book 16:31 Dean's Calorimeter Stuff 32:47 SciFi 5 "175:160" 48:63 SciFi 7 "Spare" 64:79 SciFi 4 "111:96" 80:95 L1MU MTM Crate 1 96:111 open 112:127 open 128:143 L1 Cal Trig 144:159 L1 Cal Trig 160:175 open 176:191 open 192:207 L1 Muon 208:223 L0 Luminosity 224:239 open 240:255 Framework Generated And Or Terms SCL Receivers I visited Feynman and traded in 5 SCL Receivers with Sequencer Delay code for 5 normal SCL Receivers. Of these 5 I immediately gave 3 to Reinhard. Thus there are now 2 SCL Receivers in the spares box. I also gave Thinh 2 SCL Receivers that needed testing/repair. Talked with Ted about VRBC code. He understands our problem and what we would like. They are still in the process of getting control over the 10**9 version of everything and fixing other broken things. The basic suggestion is: if we do not want to change a bunch of time and if we are more or less running OK then just hold on as we are for a while. Everything understood, fixed, tested, sorted out by summer shutdown is a goal. ------------------------------------------------------------------------------ DATE: 07-MAY-2003 At: MSU Topics: L1 Cal Trig VRBC/SCL wouldn't acquire lock The D0 Master Clock lost RF input this morning. All crates had recovered, except the L1CT crate wouldn't readout. The amber LED on the SCL mezzanine was OFF. Michiel S. tried running Init_Post_Auxi_L1CT.vio by hand, and that did not help. Bob Kehoe recommended executing the L1CT_Init command by hand, so Michiel called MSU before sending the command. Philippe explained what that did, when it was a safe thing to do with respect to COOR, and that this was not the recommended method during a store. L1CT_Init did not have any more success (no reason why it should). Looking carefully, Michiel could NOT see the amber light come on at all, even shortly, but he could see some LED activity in the crate. Philippe did manual VME read/write IO to the VRBC at address 0x1C4800B2 to also verify this was not a VME IO issue. Michiel tried to issue a VMESYSRESET command with no better success. (note: was this SBC modified to never send sysreset to the backplane?). Then the L1CAL readout crate was powered off for 30s (and the L1CAL bougie crate with it). The VRBC woke up with its amber LED back on. Philippe ran M101_L1CT_All.dcf, initialized L1CT, and Michiel started a run with L1CAL included. So it seems that either the SCL mezzanine had locked up in a way that would not allow it to understand or execute the re-lock request. Or the VRBC had locked up in a way that it did not hear or execute the "reset" command To remember: resetting the VRBC is done by writing 0 to 0x1C48000A from Trics. Resetting the VRBC makes it tell the SCL mezzanine to re-acquire signal lock on the Serial Command Link. ------------------------------------------------------------------------------ DATE: 22:25-APRIL-2003 At: Fermi Topics: L1 Cal Trig Term-Attn-Brd's and Missing Et work, New M106-M107 water leak, Tricks 10.4.A, Noise Study Cold Start Instructions are at the end of this week's entry. Term-Attn-Brd Work Replace because of problem with +3,28 EM & HD. It looks like the card that was in use has a ZER DAC soldering problem. Replace +1,9 EM and +3,9 HD both fail to force mid. Replace +4,32 HD fail to force mid. Replace -1,-2,-3,-4 phi's 17 and 31. These had the special RC network at the input to move the low frequency cutoff down to 1 usec. +9:+12 , 19 Replace Both +9:+12 , 20 Low eta was OK wrt R80 but Replace Both to get R3 C7 +9:+12 , 21 Replace Both +9:+12 , 22 Replace Both +9:+12 , 23 Low eta was OK Replace High eta +9:+12 , 24 Replace Both +9:+12 , 25 Replace Both A DAC Data cable pin was pushed out housing. +9:+12 , 26 Replace Both +9:+12 , 27 Low eta was OK wrt R80 but Replace Both to get R3 C7 +9:+12 , 28 Replace Both +9:+12 , 29 Replace Both +9:+12 , 30 Replace Both +9:+12 , 31 Replace Both +9:+12 , 32 Both were OK -9:-12 , 25 Replace Both -9:-12 , 26 Both were OK Globs of hot glue everywhere -9:-12 , 27 Replace Both -9:-12 , 28 Low eta was OK Replace High eta -9:-12 , 29 Replace Low eta High eta was OK -9:-12 , 30 Low eta was OK Replace High eta -9:-12 , 31 Replace Both -9:-12 , 32 Low eta was OK Replace High eta Pinched BLK wire under P13 Edit the Gains File to set the Gain for all these eta 9:12 phi 25:32 back up to the nominal Gain for correct R80's. The new Gains file is: D0_Config/Gains_1_16_1_32_rev_2.tti Once again the Gains file did have the correct R80's exactly right. The -9,21 EM problem (strange non-linear response) ended up not being the Term-Attn-Brd but rather something to do with the Mot ADC or the 29525. I have replaced both the ADC and the 29525 for -9,21 EM. Find_DAC Runs There were a bunch of short runs to fix problems and verify that all was OK then a couple of full runs: Thursday night at about 20:20 Find_DAC_V5_0_C_20030424.tti;9 (which also has an edited version) eta 1:16 all phi keep 2 This had problems at: +1,27 EM ratio was good 13.1 jumped around 6,7 best guess is this is synchronous noise coming and going. +1,24 HD ratio was good 13.1 just missed the 0.2 count limit. -4,25 HD way too much noise yes this TT is excluded Friday morning at about 1:00 Find_DAC_V5_0_C_20030425.tti;1 no edited version no error tags. Work on Missing Et Tree Problems Px eta -9:-12 phi 29 checked the PROM's, checked the CTFE for obvious visual or Ohm meter visible problem with Px bit of value 16, checked this bit looking from the CTFE backplane socket at the Px CAT2, can't find the cause of this problem so far. Px eta -1:-4 phi 24) The Px PROM for -2,24 did not have its GND pin plugged into its socket. That must be very good for the chip. All 4 Px PROM Verify OK on the PROM Programmer so I'm not going to cook a new -2,24 Px at this time. Noticed pinched Red DAC Serial Data wire under pin 12 on this CTFE. Px +5,2 and Py +5,2 both of these PROM had some junk data in them. Cook new parts. I'm not certain what was in the old parts. Py +16,13 this PROM had some junk data in it. Cook new part. I'm not certain what was in the old part. Work on the T&SS signal mapping on the Tier 2 and Tier 3 MBD's. Set these up so that for the Counter Tree and Missing Et Tree they match the current Run II documentation in /l1/cal_trig/timing/cal_trig_momentum_tree_timing.txt. Leave the old Run I stuff on these cards that was not in the way of the Run II setup. The highest eta Tier 2 MBD was not worked on. Thursday night late Water Leak L1 Cal Trig trips off Thursday night during a cosmic run for CFT. Leak detector says trouble between M106 and M107. Yes, there is a drop of water and you can see another drop in the mud flaps on the bottom radiator. But which radiator is leaking ? Russ Rucinski has set things up with the Operation shifter to look for water once every 4 hours. I have put notes on the M106 back door and put a plastic flashlight there for people to use. Points include: Right where you need to hold the flashlight to look for water is next to terminals on a high current 5V bus - so no metal flashlights. It is best if the rack door is only open for 30 seconds or perhaps a minute at most. The air flow is reduced with the door open and the electronics gets hot. The drip detector between M106 - M107 has been disconnected. If it does spring a big leak and starts flooding then the values to turn OFF this set of radiators are labeled M106. The location of these valves along the under floor center aisle distribution pipe is marked on the aluminum channel at the very bottom of the front of the racks. Run IIB Cal Trig meeting Meeting to help get organized to have things setup for this summers tests. I did talk with Stu about 4 Ethernet lines out to this area. Petros SCL Splitter LMR-200 2x SMB 4x SMA and crimp tool to Petros. Tricks 10.4.A was running for about 24 hours or so. No problems with TFW. Most of this time Cal Trig was off and TRICS was told to ignore Cal Trig. When I did run with Cal Trig ON there was a problem with the FMLN at initialize time. At I could not swap TRICS at that moment and I wanted to work on Cal Trig I had a change to play with: forge ahead vs abandon all hope. Look at the 10.4.A log file for Thursday afternoon at about 17:54 to 18:00 for a bunch of Cal Trig Initialize that will show the FMLN problems. I was stupid and did not try to program a threshold into the FMLN before I went back to TRICS 10.3.B Noise Study What will open is South Iron and East Iron. Cal will work on CC NE and EC S NE PreAmp boxes. We would like to pick phi's that are just East of straight up and due East. Just East of straight up is phi 7 or 8. Due East is phi 1 or 2 or 31 or 32. The 3 TT's that we have been using to look at 13.2 MHz are: -4,26 HD, -8,15 HD, +12,12 23-Apr-03 8:30 PM Disk 1 Before test started TT Amplitude File ------ -------------- -4,26 1.2 mV 00 -8,15 2.0 mV 01 +12,12 0.4 mV 02 +1,2 1.7 mV 03 +4,31 0.7 mV 04 -12,31 0.5 mV 05 -9,8 4.0 mV 06 All TT's except of -9,8 are used to look at the 13.2 MHz line with the scope set to 5 mV/Div. TT -9,8 is used to look at the 10 MHz pickup with the scope set to 10 mV/Div. In all cases this is looking at just the inverted side of the differential signals from the HD part of the TT. Disk 2: 00.tif +1,19 HD 01.tif +8,16 HD 02.tif +12,13 HD Thur 9:30 03.tif +1,2 Thur 21:15 04.tif +1,19 HD 05.tif +8,16 HD 06.tif +12,13 HD Thur 21:15 The following is a scan of the noise pickup in some West side TT's as various parts of the Central Muon system were turned off. This is looking at the 13.2 MHz line in the TT Hadronic channels. West side was used for this because West Iron was closed and no one was working on West side Calorimeter Electronics. Thursday 9:30 Thursday South and East 9:45 10:10 10:50 21:15 Iron are open. Top A Layer All A Layer All Central All ON All PDT's ON. PDT's OFF PDT's OFF PDT's OFF All Closed TT Amplitude Amplitude Amplitude Amplitude Amplitude ------ -------------- ----------- ----------- ----------- --------- +1,19 0.5 mV 0.5 mV < 0.1 mV < 0.1 mV 1.0 mV +8,16 2.0 mV 2.0 mV 0.9 mV 0.5 mV 2.0 mV +12,13 1.4 mV 1.4 mV 0.7 mV 0.2 mV 1.7 mV In the 10:50 measurement the End Cap Muon Electronics was running. Yes, TT's on the West side did see the effect of the East Iron being open - I saw this in a number of cases. The following is a scan of one East side TT Hadronic channel as the Iron was closed. This is TT +1,2 HD Time Conditions Amplitude ------------------ ------------------------ --------------- Wednesday 20:30 All Closed Everything ON 1.7 mV Thursday 7:45 All Open 0.1 - 0.2 mV Thursday afternoon Full Open 46 1/2" 0.2 mV " " 41 1/2" open 0.2 - 0.3 mV " " 20" open 0.4 mv " " 11 1/2" open 0.65 mV " " 5 1/2" open 0.75 mV " " East Closed 0" open 1.2 mV " " South Closed 1.3 mV Thursday 21:15 All Closed Everything ON 1.5 mv Cold start instructions Rev. 25-APR-2003 We are running: TRICS V10.3.B and VME_Access V5.0.C See the full instructions at: www.pa.msu.edu/hep/d0/ftp/l1/framework/hardware/rack_crate/ master_clock_instructions.txt www.pa.msu.edu/hep/d0/ftp/l1/framework/hardware/rack_crate/ framework_power_control_procedures.txt www.pa.msu.edu/hep/d0/ftp/l1/cal_trig/hardware/rack_crate/ cal_trig_power_control_procedures.txt Overview: After Master Clock is running, turn on TFW and Cal Trig (one power supply at a time) then: Configure FPGA's The normal Master Command file for this does everything except for the M101 Routing Master Get the Routing Master Configured and Running 1) After the Routing Master is powered up, wait for its SBC to boot (less than 1 minute) 2) Login to the SBC with 'ssh d0runsu@d0sbc001b' The password is same as the online d0run account. 3) On the SBC run 'reset_all.sh stop' which will stop the relevant readout processes. 4) Use TCC's Configure FPGA's menue to execute the dcf that configures the Routing Master FPGA's. This file is D0_Config\M101_Routing_Master_All.dcf 5) On the SBC run 'reset_all.sh start' Tell TRICS that the Cal Trig eta coverage is 1:20 Init TFW and Cal Trig Tell TRICS to completely ignore the Cal Trig By hand using VME Access load the Cal Trig Gains files. (made sure that the "No Write to DAC's" box is not checked) Load then in the order indicated below. By hand using VME Access load the Cal Trig Pedestals files. (made sure that the "No Write to DAC's" box is not checked) Load then in the order indicated below. Tell TRICS that it has control of the Cal Trig again and set the Cal Trig eta coverage to 1:16. Init the TFW and Cal Trig verify that it is a clean init of all parts The current Gains files are: D0_Config/Gains_17_20_1_32_rev_a.tti # TT eta's 17:20 all phi's D0_Config/Gains_1_16_1_32_rev_2.tti # TT eta's 1:16 all phi's The current Pedestal files are: D0_Log/Find_DAC_V4_1_D_20030307_edited.tti;4 # TT eta 13:20 D0_Log/Find_DAC_V5_0_C_20030425.tti;1 # TT eta 1:16 ------------------------------------------------------------------------------ DATE: 22-APRIL-2003 At: MSU Topics: Analyze Find_DAC result file with zeroed gain Analysis of the Find_DAC result file made last week with gains programmed to zero. Find_DAC_V5_0_B_20030417.tti;1 There is one tower (+ 3,28) where both the EM and HD DAC cannot reach full scale of the ADC. That was happening even with normal gains. There is one tower EM_TT(+ 9,21) that managed to get some energy in some of its histograms. This is the tower that was already detected as weird, and was already suspected to have a reference voltage problem. >> EM_TT(+ 9,21) on all runs with very weird and consistent pattern !! There is indeed a significant effect of the gain on the zero energy response. Out of the 2*2*16*32= 2048 towers: 1373 towers have moved by more than 1 ADC count (13 counts of control DAC) 787 towers have moved by more than 2 ADC count 405 towers have moved by more than 3 ADC count 196 towers have moved by more than 4 ADC count 5 towers have moved by more than 8 ADC count (104 counts of control DAC) The drift goes both ways, but zeroeing the gain tends to require (mildly) a lower Pedestal control value, meaning that they require a higher ADC offset produce an average of 8 counts, meaning that zeroeing the gain had lowered the zero energy ADC response. 830 of the above 1373 towers that moved by more than 1 ADC count required a lower Pedestal DAC control value -> 60% 486 out of the 787 that moved by more than 2 -> 62% 129 out of the 196 that moved by more than 4 -> 65% 4 out of the 5 that moved my more than 8 (-> 80%) For reference: Now Processing EM_TT(+ 3,28) DAC= 0 -> Avr=211.00 Dev= 0.00 ( 100@211 ) EM_TT(+ 3,28) Warning: Failed Forcing Zeresp to 0xff: DAC = 0 Avr = 211.00 DAC= 4095 -> Avr= 0.00 Dev= 0.00 ( 100@0 ) DAC= 2054 -> Avr= 93.00 Dev= 0.00 ( 1000@93 ) EM_TT(+ 3,28) Warning: Failed Forcing Zeresp to Mid Scale: DAC = 2054 Avr = 93.00 DAC= 3010 -> Avr= 39.00 Dev= 0.00 ( 1000@39 ) EM_TT(+ 3,28) Warning: DAC to ADC Ratio Problem, 2054->93.00, 3010->39.00 Computed DAC to ADC ratio = 17.70 Now Processing HD_TT(+ 3,28) DAC= 0 -> Avr=211.00 Dev= 0.00 ( 100@211 ) HD_TT(+ 3,28) Warning: Failed Forcing Zeresp to 0xff: DAC = 0 Avr = 211.00 DAC= 4095 -> Avr= 0.00 Dev= 0.00 ( 100@0 ) DAC= 2054 -> Avr= 94.00 Dev= 0.00 ( 1000@94 ) HD_TT(+ 3,28) Warning: Failed Forcing Zeresp to Mid Scale: DAC = 2054 Avr = 94.00 DAC= 3023 -> Avr= 39.00 Dev= 0.00 ( 1000@39 ) HD_TT(+ 3,28) Warning: DAC to ADC Ratio Problem, 2054->94.00, 3023->39.00 Computed DAC to ADC ratio = 17.62 Now Processing EM_TT(+ 9,21) DAC= 0 -> Avr=255.00 Dev= 0.00 ( 100@255 ) DAC= 4095 -> Avr= 0.00 Dev= 0.00 ( 100@0 ) DAC= 2054 -> Avr=132.00 Dev= 0.00 ( 1000@132 ) DAC= 3521 -> Avr= 20.00 Dev= 0.00 ( 1000@20 ) Computed DAC to ADC ratio = 13.10 DAC= 3691 -> Avr= 7.00 Dev= 0.00 ( 1000@7 ) DAC= 3690 -> Avr= 7.00 Dev= 0.00 ( 1000@7 ) DAC= 3689 -> Avr= 7.00 Dev= 0.00 ( 1000@7 ) DAC= 3688 -> Avr= 7.00 Dev= 0.00 ( 1000@7 ) DAC= 3687 -> Avr= 7.00 Dev= 0.00 ( 1000@7 ) DAC= 3686 -> Avr= 7.00 Dev= 0.00 ( 1000@7 ) DAC= 3685 -> Avr= 7.00 Dev= 0.00 ( 1000@7 ) DAC= 3684 -> Avr= 8.41 Dev= 3.05 ( 820@7 4@8 176@15 ) DAC= 3683 -> Avr= 10.92 Dev= 3.66 ( 184@7 372@8 444@15 ) EM_TT(+ 9,21) Average out of tolerance DAC= 3683 Avr= 10.92 Dev= 3.66 DAC= 3682 -> Avr= 8.11 Dev= 0.88 ( 984@8 16@15 ) DAC= 3681 -> Avr= 8.00 Dev= 0.00 ( 1000@8 ) DAC= 3680 -> Avr= 8.00 Dev= 0.00 ( 1000@8 ) DAC= 3679 -> Avr= 8.00 Dev= 0.00 ( 1000@8 ) DAC= 3678 -> Avr= 8.00 Dev= 0.00 ( 1000@8 ) DAC= 3677 -> Avr= 8.00 Dev= 0.00 ( 1000@8 ) DAC= 3676 -> Avr= 8.00 Dev= 0.00 ( 1000@8 ) DAC= 3675 -> Avr= 8.00 Dev= 0.00 ( 1000@8 ) DAC= 3674 -> Avr= 8.00 Dev= 0.00 ( 1000@8 ) DAC= 3673 -> Avr= 8.01 Dev= 0.11 ( 988@8 12@9 ) DAC= 3672 -> Avr= 8.31 Dev= 0.46 ( 688@8 312@9 ) DAC= 3671 -> Avr= 8.94 Dev= 0.23 ( 56@8 944@9 ) DAC= 3670 -> Avr= 9.00 Dev= 0.00 ( 1000@9 ) DAC= 3669 -> Avr= 9.00 Dev= 0.00 ( 1000@9 ) DAC= 3668 -> Avr= 9.00 Dev= 0.00 ( 1000@9 ) DAC= 3667 -> Avr= 9.00 Dev= 0.00 ( 1000@9 ) DAC= 3666 -> Avr= 9.00 Dev= 0.00 ( 1000@9 ) EM_TT(+ 9,21) Picked Histo#11 DAC= 3681 Avr= 8.00 Dev= 0.00 DevAvr= 0.32 ------------------------------------------------------------------------------ DATE: 17:18-APRIL-2003 At: Fermi Topics: CTFE "Readout Problem", Scaler to count time since previous L1_Acpt, Move Tick Select #3, Find_DAC run, SCL Receivers, Noise Study Cold Start Instructions are at the end of this week's entry. CTFE eta -13:-16 phi 15 Dean reported to me that he had events where it looked like the CTFE card for eta -13:-16 phi 15 was reading out HD for EM and EM for HD. I tested this by cranking down the HD pedestal of one of these 4 TT's and looking at the result in TT_ADC_Mon and in HSRO dump. All OK in TT_ADC_Mon but Dean was exactly right it looked backwards in the raw data. ERPB looked OK so pull the CTFE. All 8 Energy PROM's were in the wrong sockets, swapped EM with HD. Install them correctly and now the data from that CTFE looks OK. Scaler to count time since previous L1_Acpt Want to start using a scaler to count how long it has been since the most recent previous L1_Acpt. The standard Gated Scaler FPGA was made so that it could do this type of operation, i.e. with each L1_Acpt, re-load a count of zero and then count up. This will be an official permanent new use of one of the Gated Scalers in M123B Slot 19. We will use Scaler #0 on MSA FPGA #4. This scaler is currently not used and it does readout in the TFW Data Block. Recall that the MSA FPGA's are 1:16. To implement this "Time Since Previous L1_Acpt" function a copy of L1_Acpt Strobe was routed from the fan-out to Slot #19 P3 MSA_In_84. MSA_In_84 is the first Individual Control Signal to MSA FPGA #4 on a SM card. I checked with Philippe about how TRICS initialized a unused scaler channel and then read all the control registers for this scaler. Yes it did look unused and TRICS initialzes it to a nice safe state. Besides routing the L1_Acpt Strobe to MSA_In_84 the only other work that was necessary to implement this scaler was writing into its control registers at M123B Slot 19 FPGA 4 Reg 130 Write $8000 Gate Control Register Force the gate to this scaler Enabled i.e. it will always try to count up. Reg 131 Write $8008 Load Control Register This allows an asserted signal on this Gated Scaler's Individual Control Signal 0 to cause this scaler to Load. The Load Value register holds all zeros so this just resets this scaler. Reg 132 Write $0002 BX History Shift Register Control Need to look back the correct number of ticks Must Capture the HSRO data before the re-Load. Putting zero in this register caused all zeros in the HSRO. Putting 1 in this register caused rational readout data. Right now have a 2 in this register. Needs investigation. These register writes were added to the Init_Post_Auxi_L1FW.rio file. Edit the gated scaler usage and the single signal files on the web. Note that the gated scaler usage file really has background information about the usage of all the types of scalers. Note also that there is a Gated Scaler Module in Slot 18 of M123B. Slot 18 is Configured with gs.dci but I don't know if this scaler module is actually used for anything. It does not look like TRICS touches it. There is nothing plugged into its front panel P5 or is backplane P2 or P3. There is an optical cable plugged into is HSRO but it is not actually readout by the VRB. More about the BX History Control Register. Yes things are a little funny. By putting an easy to spot value in the LSByte of the Load Value Register it is easy to check what is going on. With a zero in the BX History Control Register what you HSRO is the value in the Load Value Register. So to readout the scaler before you re-Load it you need to have at least a 1 in the BX History Control Reg. Right now I will keep a 2 in that reg. I do not understand why Monit Data and HSRO data do not agree. That is with zero in the BX History Control Reg you see the Load Value Reg data in the HSRO and rational counts in the Monitory readout. Tick Select #3 As requested by Fred Borcherding I edited the Init_Post_Auxi_L1FW.rio file so that Tick Select Comparator #3 is now setup to select tick 28. I made an entry in the Configuration Change log book. SCL Receivers All 5 spare SCL Receivers in the spares cabinet have the label 59803A on them. I think this is the version with delayed Tick Clock for the Sequencers. I need to check with Ted. I needed to and did take Ted 2 SCL Receivers that people say do not work. Check the label numbers with Ted: Current SCL Receivers with simultanious data and clock are: 5980A9 Current SCL Receivers with delayed 7 MHz clock wrt data are: 59803A All SCL Receivers that are programmed by Ted and Thinh have labels on them. SCL Receivers at D-Zero that were reprogrammed by Danile have ?? to identify them. Find_DAC In \Scratch\ make a file Gains_1_16_all_phi_ZERO.tti. This zeros all the Gain DAC's in |eta| 1:16. Execute the file and then run Find_DAC over |eta| 1:16 all phi's keep=2 target=8 make histogram and tti files. This runs to completion OK but I have not looked at the results file. Noise Study On disk #1. This was done using -8,15 HD from the CTFE LEMO and using a Radiation Monitor signal that Ron brought down from the 3rd floor. 00.tif Rad Monitor with SMT readout Noise gated out Thursday 01.tif Rad Monitor with SMT readout noise Thursday 02.tif -8,15 HD CTFE Lemo during Physics running Thursday 03.tif -8,15 HD CTFE Lemo between stores Cal PreAmp OFF Friday 04.tif -8,15 HD CTFE Lemo between stores Cal PreAmp back ON Friday Sent note to Dmitri asking for Muon power off test. Dean has Trigger Pickoff drivers with lower high frequency response in TT Phi 8 of South CC. These TT's do look different on the scope. Cold start instructions Rev. 10-APR-2003 We are running: TRICS V10.3.B and VME_Access V5.0.B See the full instructions at: www.pa.msu.edu/hep/d0/ftp/l1/framework/hardware/rack_crate/ master_clock_instructions.txt www.pa.msu.edu/hep/d0/ftp/l1/framework/hardware/rack_crate/ framework_power_control_procedures.txt www.pa.msu.edu/hep/d0/ftp/l1/cal_trig/hardware/rack_crate/ cal_trig_power_control_procedures.txt Overview: After Master Clock is running, turn on TFW and Cal Trig (one power supply at a time) then: Configure FPGA's The normal Master Command file for this does everything except for the M101 Routing Master Get the Routing Master Configured and Running 1) After the Routing Master is powered up, wait for its SBC to boot (less than 1 minute) 2) Login to the SBC with 'ssh d0runsu@d0sbc001b' The password is same as the online d0run account. 3) On the SBC run 'reset_all.sh stop' which will stop the relevant readout processes. 4) Use TCC's Configure FPGA's menue to execute the dcf that configures the Routing Master FPGA's. This file is D0_Config\M101_Routing_Master_All.dcf 5) On the SBC run 'reset_all.sh start' Tell TRICS that the Cal Trig eta coverage is 1:20 Init TFW and Cal Trig Tell TRICS to completely ignore the Cal Trig By hand using VME Access load the Cal Trig Gains files. (made sure that the "No Write to DAC's" box is not checked) Load then in the order indicated below. By hand using VME Access load the Cal Trig Pedestals files. (made sure that the "No Write to DAC's" box is not checked) Load then in the order indicated below. Tell TRICS that it has control of the Cal Trig again and set the Cal Trig eta coverage to 1:16. Init the TFW and Cal Trig verify that it is a clean init of all parts The current Gains files are: D0_Config/Gains_17_20_1_32_rev_a.tti # TT eta's 17:20 all phi's D0_Config/Gains_1_16_1_32_rev_1.tti # TT eta's 1:16 all phi's The current Pedestal files are: D0_Log/Find_DAC_V4_1_D_20030307_edited.tti;4 # TT eta 13:20 D0_Log/Find_DAC_V5_0_B_20030410_edited.tti;1 # TT eta 1:16 ------------------------------------------------------------------------------ DATE: 16-APRIL-2003 At: MSU Topics: Analyze latest Find_DAC result files Find_DAC_V4_1_D_20030401.Dev_Ave.tti;1 -- 01-Apr-2003 21:14 Find_DAC_V5_0_B_20030409.Dev_Ave.tti;1 -- 09-Apr-2003 14:28 only eta(-8:-5),phi(1:32) for new term-att Find_DAC_V5_0_B_20030410.Dev_Ave.tti;1 -- 10-Apr-2003 07:56 Find_DAC_V5_0_B_20030411.Dev_Ave.tti;1 -- 11-Apr-2003 09:40 Find_DAC_V5_0_B_20030411.Dev_Ave.tti;2 -- 11-Apr-2003 10:20 Edit/moodify the Find_DAC files to associate the Deviation Average value with the "Ped_Deviation:" keyword instead of the value from the chosen histogram. eg. change: TT_Phi: 1 Pedestal_Control_DAC: 3704 Ped_Average: 7.99 Ped_Deviation: 0.80 ! EM_TT(+ 1, 1) Dev_Average: 0.79 into: TT_Phi: 1 Pedestal_Control_DAC: 3704 Ped_Average: 7.99 Ped_Deviation: 0.79 ! EM_TT(+ 1, 1) Histo_Dev: 0.80 Good news: 24 hours apart: between 0411;1 and 0410;1 The noise hasn't changed, and only few pedestal values have moved: Only one tower ot bad with a noise difference greater than 50% and only 3 additional towers show a difference greater than 25 % 0 pedestals had moved by >= one ADC count 12 pedestals had moved by >= one half ADC count evenly in each direction 10 days apart: between 0410;1 and 0401;1 We have good noise stability: If we ignore the 9 towers that were bad and are now good and the 1 tower that was good and is now noisy, there are zero towers where the noise changed by more than 50 % and only 11 with noise that changed more than 25 % The pedestals have drifted somewhat 49 pedestals had moved by >= one ADC count evenly in each direction 6 pedestals had moved by >= two ADC count evenly in each direction Bad News: One hour apart: between 0411;1 and 0411;2 In 0411;2 many towers show more noise, probably oscillations, which also upset the pedestals. 11 more towers had noise that is more than 50% off the eta average. 13 pedestals had moved by >= one ADC count evenly in each direction 58 pedestals had moved by >= one half ADC count evenly in each direction There is a list of towers with consistently much less noise (~zero) than their neighbors. EM_TT(+ 4, 3) EM_TT(+12,17) EM_TT(+13, 2) EM_TT(+13,23) and HD_TT(+13,23) EM_TT(+14,23) and HD_TT(+14,23) HD_TT(-11, 1) EM_TT(-12,18) EM_TT(-13,32) and HD_TT(-13,32) HD_TT(-14, 2) EM_TT(-15,19) and HD_TT(-15,19) EM_TT(-16,19) and HD_TT(-16,19) There is a list of towers with consistently more noise than their neighbors. These are the towers with at least 50% more noise than their eta average. Lowering the detection threshold would make the list longer. EM_TT(+ 2,25) on 0411;1 and less on 0411;2 EM_TT(+ 5,23) ok on 0401, but problem on 0410, 0411;1 and 0411;2 EM_TT(+ 7,13) on 0411;2 EM_TT(+ 7,14) on 0411;2 EM_TT(+ 7,16) on 0411;2 EM_TT(+ 7,20) on 0411;2 EM_TT(+ 8,13) on 0411;2 EM_TT(+ 8,28) ok on 0401, but problem on 0410, 0411;1 and 0411;2 HD_TT(+10,12) on all runs HD_TT(- 6, 7) on 0411;2 HD_TT(- 6, 9) on all HD_TT(- 7, 7) on 0411;2 HD_TT(- 7, 9) on all runs HD_TT(- 7,10) on all HD_TT(- 7,23) on 0411;2 HD_TT(- 7,28) on 0411;2 EM_TT(- 7,32) on 0411;2 noisy on the first 22 out of 28 histograms then normal!! HD_TT(- 8, 9) on all HD_TT(- 8,27) on 0411;2 HD_TT(-10,12) on 0411;2 HD_TT(-10,24) on 0411;2 HD_TT(-11, 9) on all HD_TT(-11,12) on all, especially 0411;2 HD_TT(-11,22) on 0411;2 HD_TT(-12, 9) on all HD_TT(-12,26) on 0411;2 HD_TT(-13, 9) on all HD_TT(-14, 9) on all EM_TT(-15,25) on all HD_TT(-15,24) on 0411;2 HD_TT(-15,26) on 0411;2 Some towers even have sometime weird noise (believed to be extra oscillations) that produce histograms with two bumps EM_TT(+ 9,21) on all runs with very weird and consistent pattern !! EM_TT(- 6,22) on all runs HD_TT(- 8,24) on 0411;2 HD_TT(- 8,26) on all, especially 0411;2 HD_TT(- 9, 4) on all, especially 0411;2 ------------------------------------------------------------------------------ DATE: 8:11-APRIL-2003 At: Fermi Topics: Term-Attn-Brd replacement, Water Leak, Master Clock, Momentum PROM Check, Noise Study, Outstanding_L1_Acpt's comparators Water Leak The leak must have started or been discovered while I was driving to Fermi. Ron was on captain shift and got things turned off. The M109 - M110 drip detector had been disabled because of the trips that happened in March. See 10:12-MAR-2003 log book entry. When I walked in the door people had already cut off the G10 shrouds and were poking around. After some discussion they left. It was the next to the top radiator leaking from the rolled over end of a header tube as is normal. So I had to put the top radiator and then the next to the top one. Pete Simon gave me a tested soldered over radiator to replace the lower one. Pete soldered over and tested the upper one. The front of these radiators now have just one bracket to hange on. The back water connection end has the normal 2 brackets. It takes about 6 hours from start of work to water turned back on looking for leaks. Then another 2 hours to put the G10 shrouds back on. Ran with just the fan over night to dry things out. Master Clock Rich Partridge wants to start using sequencer TL #11 for the Luminosity Monitor TDC Stop signal. This needs to be 2 RF buckets wide and start 13 RF buckets after BX in center of D-Zero. The new Sequencer #1 file is called clk_sequencer_1_10APR03.txt His new time line looks OK but it may be a tick later than he wants. Load the new code into Sequencer #2 that makes the new Missing Et Clocks. Measure the NIM logic to PCC cables. They are both 5 nsec + 8 nec cables that are 101" long This is from extream tip of one LEMO to the extream tip of the other. RG-174 cable is 66% c velocity factor, i.e. 7.795" per nsec or 3.898" per 500 psec. So 101" should be 12.96 nsec so this checks. A 1/2 nsec shorter cable is 97.102" and a 1 nsec shorter cable is 93.205". Send note to Tom Regan asking for the -1/2 and -1 cables. Term-Attn-Brd's Replace Term-Attn-Brd for -7,19 which was a dregs that was put in last week. Replace the following Term-Attn-Brd's in the eta -5:-8 range. phi 1 replace both find a mica cap proken off on the -1.352V supply replace it with a 100 pFd SMD phi 2 replace both phi 3 replace both phi 4 replace both phi 5 replace both phi 6 both were OK as found phi 7 replace both phi 8 Hi eta card was OK as found find a pinched black wire under pin #13 Super Glue and reroute phi 9 Low eta card was OK as found phi 10 Hi eta card was OK as found phi 11 replace both phi 12 replace both phi 13 replace both Once again there was a perfect match between the TT's that we guessed had correct R80's (based on Bob's plots) and what we found when things were pulled apart. Edit the Gains files so that it now knows that all |eta| 5:8 have the correct R80's and all should be set for the nominal gain. Also change the gains files so that we have one for |eta| 1:16 and a separate one for |eta| 17:20. While CTFE cards were pulled out for Term-Attn-Brd swap, on the eta -5:-8 phi 7 card change its comparator output PAL #3. This was to fix a problem with a stuck bit from this PAL. See the log book for 19:25-JULY-2002. Via special cable at the CHTCR output we had surpressed the Tot Et Ref Set #3 from the whole 8 cards serviced by that CHTCR. This now looks OK i.e. there is no longer a stuck hi Tot_Et Ref Set #3 comparator for -7,7 TT's plugged in and TT's Excluded -4,25 HD seen on scope making lots of noise. It oscilates with a period of 830 nsec Exclude -4,25 HD +2,25 has trouble Wednesday night and is Excluded (Dmitri ask) -3,26 Dean fixed this one it looks fine on scope is plugged back in and un-Excluded -4,7 (found unplugged last week) clearly has some type of problem, lots of noise at L1_Acpt time Plug it in and Exclude -4,7 EM & HD Momentum PROM, blow chunks, details: Run Px and Py PROM check on eta (-16:16) phi (1:32) on Page #0. Px(-1,24) Px(-2,24) Px(-3,24) Px(-4,24) all fail the test in a similar way They are all on the same CTFE and are summed together. The sum comes in as operand #7 of the Tier #1 CAT2 card. The CAT2 input jumps all over while Trics ramps the PROM input. EXCEPT while checking Px(-2,24) where the CAT2 input stay fixed.(!?) No idea of where to start looking. ---- Px(+5,2) CAT2 input finds about (not exactly) twice what it expects and rolls over at 255. Could be a problem with the 8vs9 input jumper selection? Py(+5,2) Output is exaclty one-to-one Wrong part (but do we have 1-to-1 on page 0 of any Momentum PROM?) or page select problem. ---- Px(-9,29) Initial CAT2 input value on operand #4 is 24 instead of 8 Some sections are ok (e.g. 0-9, 31-51, 73-92, 114-134, etc). The rest fails. Px(-10,29) Px(-11,29) Px(-12,29) All 4 etas on this CTFE show the exact same symptoms Watching carefully which CAT2 *input* values correspond to errors: It is pretty clear the bit of value 16 is stuck high somewhere between the CTFE adder and the CAT2 input. This bit is on pin 75 and 43 of J2 on the CTFE This bit is on pin 73 and 41 of J2 on the CAT2 ---- Py(+16,13) output constant zero. Probably blank part. ---- Py(+16,25) Py(+16,26) Py(+16,27) Py(+16,28) Py(+16,29) Py(+16,30) Py(+16,31) notice: these are all at the same eta on seven separate CTFE cards notice: all the phi values of one front-end cell except for phi 32 Look at phi 25 in detail: Initial CAT2 input value seems ok, but discrepency starts at input of 20. Then the rest of the PROM increases, but with a slope too small. comparing with phi 26 it seems like the prom that should be at phi 26 is plugged in phi 25 instead. Looking just at the initial and final value of all these proms: E$ PY Tier#1 CAT2 input 244 instead of 254 ==> PROM(255)->254 but seems ->244 E$ Error(s) detected for page #0 of PY PROM for TT(+16,25) E$ Initial PY Tier#1 CAT2 operand #1 is 1 instead of 0 for PROM input of 0 E$Continue test using observed value for initial count ... E$ PY Tier#1 CAT2 input 226 instead of 245 ==> PROM(255)->244 but seems ->225 E$ Error(s) detected for page #0 of PY PROM for TT(+16,26) E$ Initial PY Tier#1 CAT2 operand #2 is 5 instead of 4 for PROM input of 0 E$Continue test using observed value for initial count ... E$ PY Tier#1 CAT2 input 202 instead of 230 ==> PROM(255)->226 but seems ->198 E$ Error(s) detected for page #0 of PY PROM for TT(+16,27) E$ Initial PY Tier#1 CAT2 operand #3 is 9 instead of 8 for PROM input of 0 E$Continue test using observed value for initial count .... E$ PY Tier#1 CAT2 input 171 instead of 206 ==> PROM(255)->199 but seems ->164 E$ Error(s) detected for page #0 of PY PROM for TT(+16,28) E$ Initial PY Tier#1 CAT2 operand #4 is 13 instead of 12 for PROM input of 0 E$Continue test using observed value for initial count ... E$ PY Tier#1 CAT2 input 133 instead of 175 ==> PROM(255)->165 but seems ->123 E$ Error(s) detected for page #0 of PY PROM for TT(+16,29) E$ Initial PY Tier#1 CAT2 operand #5 is 18 instead of 16 for PROM input of 0 E$Continue test using observed value for initial count ... E$ PY Tier#1 CAT2 input 92 instead of 138 ==> PROM(255)->124 but seems ->78 E$ Error(s) detected for page #0 of PY PROM for TT(+16,30) E$ Initial PY Tier#1 CAT2 operand #6 is 25 instead of 24 for PROM input of 0 E$Continue test using observed value for initial count ... E$ PY Tier#1 CAT2 input 50 instead of 99 ==> PROM(255)->80 but seems ->31 E$ Error(s) detected for page #0 of PY PROM for TT(+16,31) This pattern tracks along, the expected value of one phi can be found in the previous phi, with a slight discrepency accountable to the discrepency in initial value. Conclusion: The PROM at phi N actually belongs in the card at phi N+1, except for the CTFE in phi 32 which seems ok. Hunting in the logbook: On 5:7-FEB-2003 you had found the same errors at eta 16 Use TRICS to scan the EM, HD, Px, Py PROM's in eta 13:166 There are problems with the Py PROM at eta +16 for the following phi's: 13 25 26 27 28 29 30 31. All of the other parts check OK. On 18:20-SEPT-2002 we had found the following problems. Tested Px and Py Prom for Page #0 for TTeta(-12:+12) TTphi(1:32) Px (-1,24) \ These towers are all on the same CTFE. Px (-2,24) | In common= CTFE summer/driver, CAT2 operand/receiver. Px (-3,24) | One or a few bits intermitently wrong; mostly bit(256) Px (-4,24) / Note that the eta=-2 test produced a fixed output Px (+5,2) \ Output is twice the input (NOT twice the intended output) Py (+5,2) / Output same as input (instead of long steps for sin(phi)~1) Guess: problem could be wire selecting 8/9 bit vs 4/8 pages wild guess: energy PROM instead of momentum PROM Py (-5,25) Zero for all inputs. Guess: PROM blank, pin missed socket, backwards, etc Px (+11,9) The tests claims the PROM says 7 everywhere but this really only means the test finds the same value everywhere, and asumes it is 7. The initial 4xTT sum for input=0 is found low by 7 counts. This means the response of this Px prom for input=0 which should be output=7 (i.e. cos(phi) is small) is in fact zero and remains zero for all inputs. Px (- 9,29) \ These towers are all on the same CTFE. Px (-10,29) | In common= CTFE summer/driver, CAT2 operand/receiver. Px (-11,29) | Bit of value 16 is always ON. Px (-12,29) / By comparison with today's PROM check: - Px (+11,9) and Py (-5,25) are no longer a problem. - The statement about "Px (+5,2) Output is twice the input" is NOT (or no longer) true. Looking at eta (-1:-4) phi (24) in more details. Common to eta -1, -3, and -4: Trics sees that some input values check ok while most fail. Two different runs produce two different mix of errors. The discrepency is often only by one bit, sometime 2 bits. Eta -2 seems to produce a constant output. One peculiarity though: it looks like Trics first reads one value for input=0 while it figures out the starting conditions, then a different constant value when it reads it again before changing anything, and throughout the same constant value throughout the rest of the PROM input values. Could there be some pull-down resistors missing? In addition to a blank prom at eta -2 ? Recall the current Setup of the Momentum PROM's: Energy Page Adrs Momentum Page Adrs ---------------- ------------------ First LU 7 = 111 0 = 000 Second LU 5 = 101 4 = 100 Currently we are only Clocking the Tier 1 Momentum CAT Cards for the First Lookup so you only see Momentum PROM data from page 0. Note also that the white jumper wire that is used on the Momentum PROM Address Lines is on Adress Line "A0" the LSB Address Line. TSS_K LSB Momentum Page Sel --> PROM A0 via white wire TSS_L Mid Momentum Page Sel --> PROM A9 via trace TSS_M MSB Momentum Page Sel --> PROM A10 via trace Recall what is in the current Run IIA Momentum PROM's: Px Momentum Lookup PROM ======================= +-------+-------+---------+--------+------+----------------------------------+ |TT |Zero |Symmetric|Transfer|Page |Page | |Eta |Energy |Energy |Slope |Number|Usage | |Magn |Resp |Cut | | | | |Range |[Count]|[GeV] |[float] |[0..7]| | +-------+-------+---------+--------+------+----------------------------------+ | (1:20)| 8 | None |Cos(Phi)| 0 |1st Lookup= Px - opt#1 | | | | | | | | | (1:16)| 8 | 1.00 |Cos(Phi)| 1 |1st Lookup= Px - opt#2 | |(17:18)| 8 | None | None | 1 |1st Lookup= Px - opt#2->No Contrib| |(19:20)| 8 | 1.00 |Cos(Phi)| 1 |1st Lookup= Px - opt#2 (ICD+MG)| | | | | | | | | (1:12)| 8 | 1.50 |Cos(Phi)| 2 |1st Lookup= Px - opt#3 | |(13:16)| 8 | 2.00 |Cos(Phi)| 2 |1st Lookup= Px - opt#3 | |(17:18)| 8 | None | None | 2 |1st Lookup= Px - opt#3->No Contrib| |(19:20)| 8 | 2.00 |Cos(Phi)| 2 |1st Lookup= Px - opt#3 (ICD+MG)| | | | | | | | | (1:12)| 8 | 2.00 |Cos(Phi)| 3 |1st Lookup= Px - opt#4 | |(13:16)| 8 | 3.00 |Cos(Phi)| 3 |1st Lookup= Px - opt#4 | |(17:18)| 8 | None | None | 3 |1st Lookup= Px - opt#4->No Contrib| |(19:20)| 8 | 3.00 |Cos(Phi)| 3 |1st Lookup= Px - opt#4 (ICD+MG)| +-------+-------+---------+--------+------+----------------------------------+ | (1:16)| 8 | None | 1.0 | 4 |2nd Lookup= LrgTile-opt#1 | |(17:18)| 8 | None | None | 4 |2nd Lookup= LrgTile-opt#1->NoContr| |(19:20)| 8 | None | 1.0 | 4 |2nd Lookup= LrgTile-opt#1 (ICD+MG)| | | | | | | | | (1:16)| 8 | 1.00 | 1.0 | 5 |2nd Lookup= LrgTile-opt#2 | |(17:18)| 8 | None | None | 5 |2nd Lookup= LrgTile-opt#2->NoContr| |(19:20)| 8 | 1.00 | 1.0 | 5 |2nd Lookup= LrgTile-opt#2 (ICD+MG)| +-------+-------+---------+--------+------+----------------------------------+ | (1:20)| 8 | None | None | 6 |Special/Diagnostics= constant 8's | | (1:20)| 8 | None | 1.0 | 7 |Special/Diagnostics= one-to-one | +-------+-------+---------+--------+------+----------------------------------+ So if we have a problem with the white wire that cirries the LSB Address Line we will see a mix up of Page 0 and Page 1 which in the eta range 1:16 is just a difference of a 1 GeV Symmetric Energy Cut. Find DAQ Runs Four runs this week with VME_Access V5.0.B. A quick run over just the rack with the TT's that have new Term-Attn-Brd's this week. Then a full run over |eta| 1:16 that we are using to run on right now. Then on Friday morning two full runs over |eta| 1:16. These two runs were about 45 minutes apart and were done under the exact same conditions with no change to the hardware between the runs. The purpose of these two consecutive runs was to study how close Find_DAC would come to selecting the same DAC values when nothing had changed between the 2 runs. The VME_Access V5.0.B Find_DAC appears to leave the final DAC value loaded in the Pedestal DAC's. That is after a run of Find_DAC if you then look with TT_ADC_Mon you see mostly 9's and 10's and no 8's. Overall this versio of Find_DAC makes a smoother display in TT_ADC_Mon than previous versions have. Noise Study Thursday 10-Apr-2003 Look at 13.2 MHz and 10.0 MHz Noise with Platform Equipment Turned OFF Looking at 13.2 MHz line Before Test Everything After Test Trigger Everything OFF except Everything Everything Tower ON L1 Muon ON OFF Back ON --------- ---------- ---------- ---------- ---------- -4,26 HD 1.0 mV 1.2 mV 1.1 mV 1.0 mV 00.tif 04.tif 08.tif -8,15 HD 1.5 mV 1.1 mV 2.0 mV 2.0 mV 02.tif 06.tif 10.tif +12,12 HD 0.3 mV 0.3 mV 0.3 mV 0.35 mV 03.tif 07.tif 11.tif Looking at 10.0 MHz line Before Test Everything After Test Trigger Everything OFF except Everything Everything Tower ON L1 Muon ON OFF Back ON --------- ---------- ---------- ---------- ---------- +6,8 HD 1.1 mV 0.25 mV 0.25 mV 0.6 mV 01.tif 05.tif 09.tif Data for the first 3 columns was collected Thursday 10-APR-03 in the morning. Data for the last column (everything back ON) was collected that evening at about 8 PM. Everything OFF means: SMT and CFT Sequencers platform Central OFF L1 Muon platform East was OFF All DFE's platform Center and West were OFF FPD platform West was OFF Mixer was OFF All Clock to All AFE's were OFF The ".tif" files are in: www.pa.msu.edu/hep/d0/ftp/l1/cal_trig/pictures/trig_pickoff/ as filename "tek023xy.tif". The scope and FFT were setup as follows: Use the negative side of the raw BLS signal using the adaptor cable to the scope that correctly terminates the BLS Cable. The scope is set to 5 mV / div with 20 MHz bandwidth limit horizontal is 4 usec / div which causes the scope to sample once every 4 nsec which gives a 125 MHz Nyquest frequency it acquires a 10,000 sample record (40 usec record) it FFT's this full 10,000 sample record using a Hamming window. Outstanding_L1_Acpt's comparators We connected more of the Outstanding_L1_Acpt Comparators to And-OR Terms so that we can fill in more bins of the histogram. Until now only Comparator #0 had been connected to the And-Or Network. Note that Comparators #1,#2,#3 are not connected with corrct "Tick Timing" to be used in the definition of real triggers, they are just connected so we can see what their rate is. These 3 newly connected comparators do have the correct Strobe to Data timing so that the do count correctly (this was checked and verified). The connections are: Outstanding L1_Acpt Comparator Connected to Comparator Number And-Or Term Threshold ----------- ------------ ---------- 1 0 = > 10 2 1 = > 12 3 2 = > 14 0 240 = > 2 This connection is not new. Recall that these are "strictly greater than" comparators, i.e. loading the value N will cause the output to be set when the L1AL2 count is >= N+1. Recall also that the threshold is a 4 bit value, and that two comparators share a 16-bit VME register: The lowest nibble of the lower byte of reg 8 holds threshold comparator #0 The lowest nibble of the upper byte of reg 8 holds threshold comparator #1 The lowest nibble of the lower byte of reg 9 holds threshold comparator #2 The lowest nibble of the upper byte of reg 9 holds threshold comparator #3 Comparator #0 is programmed for >= 2 and connected to Andor Term #240 (official connection, and on time for use in defining triggers) Comparator #1 is programmed for >= 10 and connected to Andor Term #0 Comparator #2 is programmed for >= 12 and connected to Andor Term #1 Comparator #3 is programmed for >= 14 and connected to Andor Term #2 To setup their current programming we use in the TFW post init . rio Vertical_Master: 1 Vertical_Slave: 2 Card_Slot: 19 Chip_Address: 16 Register_Address: 8 Write_Value: 0x0901 ! Comp #0 is > 1, comp#1 is > 9 Register_Address: 9 Write_Value: 0x0d0b ! Comp #2 is > 11, comp#3 is > 13 Bring Terminators to Fermi Bring some blank PROM's to Fermi ------------------------------------------------------------------------------ DATE: 1:4-APRIL-2003 At: Fermi Topics: Capacitor Band-Aids and power supply work, Readout Problem with -16,31 Swap more Term-Attn Brd's, Find_DAC run, Get Ready to Move Master Clock, Unplugged TT's, VME_Access 5.0, Cold start instructions We are running: TRICS V10.3.B and VME_Access V4.0.B After Master Clock is running, turn on TFW and Cal Trig (one power supply at a time) then: Configure FPGA's The normal Master Command file for this does everything except for the M101 Routing Master Get the Routing Master Configured and Running 1) After the Routing Master is powered up, wait for its SBC to boot (less than 1 minute) 2) Login to the SBC with 'ssh d0runsu@d0sbc001b' The password is same as the online d0run account. 3) On the SBC run 'reset_all.sh stop' which will stop the relevant readout processes. 4) Use TCC's Configure FPGA's menue to execute the dcf that configures the Routing Master FPGA's 5) On the SBC run 'reset_all.sh start' Tell TRICS that the Cal Trig eta coverage is 1:20 Init TFW and Cal Trig Tell TRICS to completely ignore the Cal Trig By hand using VME Access load the Cal Trig Gains files. (made sure that the "No Write to DAC's" box is not checked) Load then in the order indicated below. By hand using VME Access load the Cal Trig Pedestals files. (made sure that the "No Write to DAC's" box is not checked) Load then in the order indicated below. Tell TRICS that it has control of the Cal Trig again and set the Cal Trig eta coverage to 1:16. Init the TFW and Cal Trig verify that it is a clean init of all parts The file to configure the Routing Master FPGA's is: D0_Config\M101_Routing_Master_All.dcf The current Gains files are: D0_Config/Gains_13_20_1_32_rev_b.tti # TT eta's 13:20 all phi's D0_Config/Gains_1_12_1_32_rev_11.tti;1 # TT eta's 1:12 all phi's The current Pedestal files are: D0_Log/Find_DAC_V4_1_D_20030307_edited.tti;4 # TT eta 13:20 D0_Log/Find_DAC_V4_1_D_20030401_edited.tti;1 # TT eta 1:16 Power Supply Work Install a Capacitor Band-Aid on the Tier 1 Power Pan in Upper M110. This is a band-aid made from Sprague Capacitors. The -4.5 V brick in Upper Tier 1 M110 had been making 410 mV of ripple. Now with the band-aid it is down to 60 mV. The -5.2 V brick for this crate did not get much better. The full list to date of the Tier 1 Power Pans that have Cap Band-Aids: Date Power Pan PDM Capacitor Installed Location Number Type Notes --------- ------------- ------ --------- -------------------------- 1-APR M110 Upper T1 Sprague on all 3 Taped capacitors 18-MAR M110 Lower T1 Sprague on all 3 Taped capacitors 6-MAR M105 Upper T1 Mallory -4.5 CDE -2.0 & -5.2 6-MAR M106 Upper T1 Mallory -4.5 CDE -2.0 & -5.2 6-MAR M107 Lower T1 Mallory -4.5 CDE -2.0 & -5.2 6-MAR M108 Upper T1 Mallory -4.5 CDE -2.0 & -5.2 6-MAR M109 Lower T1 CDE on all 3 6-MAR M111 Lower T1 CDE on all 3 6-MAR M111 Upper T1 CDE on all 3 6-MAR PDM-13 CDE on all 3 Taped capacitors 19-FEB M103 Lower T1 Mallory on all 3 19-FEB M105 Lower T1 Mallory on all 3 19-FEB M108 Lower T1 Mallory on all 3 19-FEB M106 Lower T1 Mallory on -4.5 & -5.2 only none on -2 I brought a box of Capacitor Band-Aids to Fermi this week. These are made from the Sprague capacitors and after installing one on M110 Upper Tier 1 there still are 19 left, i.e. enough to patch up 6 more Tier 1 Power Pans. The 3 types of capacitors used so far are: Mallory CGO 153 M 7R5 L 15,000 7.5 Short Neg Term Can ?? CDE DCMC 203 U 025 AA2B 20,000 25V Tall Neg Term Can Sprague 36DE 233 G 010 AA2A 23,000 10V Short Isolated Can The following is a list of bricks that still look to be in trouble: Lower Tier 1 M109 -4.5 V brick 230 mV -5.2 V brick 90 mV eta +13:+16 Upper Tier 1 M109 -5.2 V brick 90 mV no band-aid yet eta +13:+16 Upper Tier 1 M110 -5.2 V brick 120 mV eta -13:-16 Upper Tier 1 M111 -4.5 V brick 110 mV eta +17:+20 Upper Tier 1 M112 -4.5 V brick 520 mv no band-aid yet eta -17:-20 Tier 2 eta 17:20 needs to be looked at in general Tier 2 eta 9:16 needs to be looked at in general Tier 2 eta 1:8 needs to be looked at in general Check again on the aging study to see how the Capacitor Band-Aids are holding up in service. See 10:12-MAR-2003 log book entry for details. Shea Box Current Power Supply Channel Cap Original 11-MAR 2-APR ---------------------- ---------- --- -------- ------ ------ M111 Top Tier 1 -4.5V rm_B ch_12 CDE 1130 mV 130 mV 110 mV M111 Low Tier 1 -4.5V rm_B ch_48 CDE 1050 mV 50 mV 60 mV M108 Low Tier 1 -4.5V rm_B ch_32 Mal 720 mV 70 mV 60 mV M106 Low Tier 1 -4.5V rm_A ch_60 Mal 680 mV 40 mV 50 mV Brought to Fermi PowerTec brick MSU SN# 22 that we just got back from repair at ASTEC. Finish assembly and testing of PDM-09. PDM-09 has the following bricks in it: MSU SN# 25 is its -2.0 Volt brick MSU SN# 59 is its -5.2 Volt brick MSU SN# 27 is its -4.5 Volt brick MSU SN# 22 is its +5.0 Volt brick These are all ReFurbished bricks that have just recently come back from Pioneer Magnetics and the repaird brick from ASTEC. There are Capacitor Band-Aids on all the Pioneer Magnetics bricks. It was tested in the standard way and is now in the cabinet as a spare. Peter is returning to Russia on May 11th. Work on the -16,31 always reads $1c1c problem. The CTFE card was just barely plugged in at all. We are lucky that it had not burned up. It was sticking out an obvious distance past the others. I had checked all of these right after they were stacked at the end of January. I tried pushing it in and it would not go in any further. Pulled it out (and pulled a neighbor CTFE out. Verified that the backplane looked OK. Put the cards back in and they all went in just fine and normally. I have no idea what is going on with that socket. I hope it is not a folded pin. Term-Attn-Brd's Work in eta -5:-8 phi 14:32 replacing all the Term-Attn-Brd's with ones that have correct R3, R80, C7. PHI Action Taken - Comments --- ------------------------------- 14 replace both 15 replace both 16 replace both 17 replace both Used dregs TAB's including one that is delaminated 18 replace both Used dregs TAB's 19 replace both Used dregs TAB's 20 replace both Used TAB's that had intermintant ZER control problems 21 replace both 22 replace both 23 replace both 24 replace both 25 replace only lower eta Term-Attn-Brd 26 replace both 27 replace both 28 replace both Loose Momentum Address line to the 283 socket 29 replace both 30 replace both Loose Momentum Address line to the 283 socket 31 replace both Lower eta 74F283 chips were mostly out of their sockets 32 replace both Edited the Gains file to adjust the gains for eta's -6 and -8 phi 14:32 to their nominal values. This makes the file Gains_1_12_1_32_rev_11.tti;1 Find_DAC Ran Find_DAC from VME_Access 4_1_D over the full coverage that we are now using to trigger, i.e. eta -16:+16 and told it to keep every other sample from the 29525 pipeline, i.e. keep = 2. This took about 8.2 hours to run. The result file is: Find_DAC_V4_1_D_20030401_edited.tti;1 TT's that failed in this run of Find_DAC +9,21 EM Too much noise Dev >= 3.0 yes this is an Excluded TT -7,19 EM This is a new Term-Attn-Brd it clearly has a problem -8,9 EM 0.1 "problem" use 3760 +1,29 HD 0.1 "problem" use 3728 +2,2 HD 0.1 "problem" use 3684 +2,18 HD 0.1 "problem" use 3585 +3,20 HD 0.1 "problem" use 3690 +3,32 HD 0.1 "problem" use 3674 +4,30 HD 0.1 "problem" use 3678 +5,17 HD 0.1 "problem" use 3668 +9,24 HD 0.1 "problem" use 3698 -2,1 HD 0.1 "problem" use 3626 -7,19 HD This is a new Term-Attn-Brd it clearly has a problem -7,32 HD 0.1 "problem" use 3733 There were 5 other channels that looked funny to Find_DAC: +1,9 EM failed force ZER to mid scale +3,28 EM failed force ZER to $ff and failed force ZER to mid scale +3,9 HD failed force ZER to mid scale +3,28 HD failed force ZER to $ff and failed force ZER to mid scale +4,32 HD failed force ZER to mid -7,19 is clearly in trouble and needs to be replaced the next time we can turn things off. It is one of the dregs Term-Attn-Brd's. For now -7,19 had to be Excluded. Need to fix the TT next week. Master Clock Get ready to move the Master Clock. Over the fall and winter it drifted in the direction of being early wrt the BX. To now hold it in this position, as it tries to drift later, we will need to use shorter cables between the NIM logic and the PCC inputs. Shorter cables will hold it where it is as it tries to drift later. RG-174 cable is 66% c velocity factor, i.e. 7.795" per nsec or 3.898" per 500 psec. Verified that the current cables that carry TeV_RF and TeV_Sync from the NIM modules to the PCC are 5 nsec plus 8 nsec cables. Restarted TRICS on Wednesday morning. VME_Access 5.0 Thursday try using VME_Access 5.0 between stores. Took Find_DAC sweeps over phi 1,2 all eta using keep 7/8 and keep 4/8. It takes this routine about 2 seconds to download a full eta,phi pedestal file. In /Scratch there is a tti file to push all ZER DAC's to mid scale. I was going to leave the system with Gain's and Ped's loaded using VME_Access 5.0 but one channel looked funny -16,19 HD looked like 15 ADC counts in TT_ADC_Mon. So I tried loading using 4.0.B and this TT still showed 15 ADC counts. So I edited the 0401_edited.tti;1 file to crank down this pedestal and then it looked OK. ---> Keep an eye on -16,19 HD pedestal. Found cables that we can use for now to connect Tier 3 CAT3 Px and Py output to the FMLN input. These cables are 11 sections long. Set them up with new labels. Rework some cables to use for FMLN output to AOT Cable panel. Eximined how some stuff in the Missing Et setup: At Tier 2 the output from the +Px & +Py CAT2 cards comes from their Partial Sum outputs. At Tier 2 the output from the -Px & -Py CAT2 cards comes from their Final Corrected Sum outputs and is passed through a short 2" cable that inverts all the signals before they enter the longer twist-flat cable that takes these inverted signals to the Tier 3 CAT3 inputs. The Tier 2 for eta 17:20 contains only Counter Tree cards, there are no Energy Tree or Momentum Tree cards in this Tier 2 crate. The Tier 1 crates for eta 17:20 contain only Px and Py CAT2 cards, there are no EM or HD CAT2 cards in these Tier 1 crates. There are no cables left to examine but the Run I documentation says that we use the Tier 3 CAT3 Final Corrected Sum outputs to driver the FMLN. I had thought that one could not read the inputs that come into the FMLN but looking at things here it looks like I was just mixed up and you can read the FMLN inputs. This should be a help in makeing a complete static tester of Cal Trig. Look at some TT's Find +12,9 unplugged I do not have a note saying that it should be unplugged. +12,9 EM has some pairs of noise spikes 132 nsec apart. These are about 30 mVpp each spike about 30 nsec wide. They look AC coupled and are common mode signal. There is also some real looking signal about 60 mV total height. +12,9 HD has major SMT pickup which is from preburst to end about 50 usec and 300 mVpp This is 2 or 3 x what the neighbor HD's have. otherwise +12,9 looks normal. Plug +12,9 back in. Find +12,12 unplugged I do not have a note saying that it should be unplugged. +12,12 EM has some SMT pickup about 30 mVpp and can see the 132 nsec switching otherwise it is clean +12,12 HD has nice "clean" SMT pickup 100 to 150 mVpp and some 13.2 MHz spike not much 132 nsec structure otherwise clean Plug +12,12 back in. Find -4,7 unplugged I do have a note from Daniel from 13-MAR-2003 saying that he unplugged it because the shifters said that it caused trouble. I thought that it had been plugged pack in. I was never told whether it was an EM on HD problem. Neither side of -4,7 has been excluded. -4,7 EM has some low level narrow spikes -4,7 HD has 50 mV of SMT pickup Overall -4,7 does not look too sick, but because there was a problem with it, and because we are not running right now (i.e. I can not prove that it is not causing trouble) I'm not going to plug it back in. Record more noise signals: Disk #1 files 0,1 tif & csv are +12,12 EM Disk #1 files 2,3 tif & csv are +12,12 HD Disk #2 files 0,1 tif & csv are -8,15 EM looks clean Disk #2 files 2,3 tif & csv are -8,15 HD nice example of once per turn pickup, 200 mVpp SMT pickup, big 13.2 MHz Disk #3 files 0,1 tif & csv are -4,26 EM Disk #3 files 2,3 tif & csv are -4,26 HD 40 mVpp SMT pickup, very major 13.2 MHz component Talked with Geoff Savage about a display for Cal Trig Power Supplies. He had me look at the following. He also said that it would only take him 15 minutes to do this. setup d0online cd $ONL_APPS_DIR ls Makefile bin doc lib src ups cd src ls Makefile ca cal config ctl daq diag gutil hv lv muo ses smt trg cd ca ls CaChannel.py Makefile ca_event.py ca_gui_cb.py chan_info.py CaDemo.py ca-gui_example.py ca_event_1.py ca_io.py test_records.db CaDisplay.py ca_cb.py ca_event_t.py ca_put_t.py CaGui.py ca_conn_t.py ca_get_t.py ca_wait.py more CaGui.py ...... if __name__ == '__main__': pvname_t = ('CTL_PROC_00', 'CTL_PROC_07') pvattrib_t = ('CPU', 'MEM', 'FD') title_t = ('Device', 'CPU', 'MEM', 'FD') c = CaGui(pvname_t, pvattrib_t, title_t, 'MyGui', __doc__) c.poll() c.mainloop() # this is where it picks up what records it will display ........... ------------------------------------------------------------------------------ DATE: 27-MAR-2003 From: MSU action at Fermi Topics: Move Cal Trig to eta 3.2 Start using L1 Cal Trig out through TT eta 16 (eta 3.2) for generation of L1 Trig's. The instructions for cold starting the Cal Trig are the same as given in the log book entry for 4:8-MAR-2003 except that you now tell TCC that it may use the Cal Trig out through TT eta 16. We discovered a problem that -16,31 was always reading $1c1c via HSRO but that it looked OK as far as actually generating L1 Trig's and it looked OK to TT_ADC_Mon. Excluding it would do nothing so, because it was the bottom ERPB, Bob pulled the cable that connects the bottom ERPB to the one above it. To help be certain that things would remain quiet we Excluded -16,31 and Excluded -13,28. -13,28 is the last channel readout by the next to the bottom ERPB and we want it to always read $0810 which will cause the full block of 16 TT's from the bottom ERPB to also read $0810. I need to repair the readout problem and then stop Excluding these 2 TT's. Eta 3.2 Cal Trig coverage started with trigger list Global CMT-11.00 ------------------------------------------------------------------------------ DATE: 18:20-MAR-2003 At: Fermi Topics: Give TFW tutorial talk, Look at Trigger Pickoff signals, Work on Power Supplies, Check the M109-M110 water leak Between Stores I pulled the M110 Lower Tier 1 Power Pan and added Capacitor band-aids to it. These are capacitor band-aids made with the recent Sprague capacitors. While the M110 Lower Tier 1 pan was pulled out I looked between M109 and M110 for the water leak that caused a problem last week. I could not see any water. While this pan was out I also verified that M110 does have hose clamps on its cooling water distribution manifold. The list to date of the Tier 1 Power Pans that have received Cap Band-Aids: Date Power Pan PDM Capacitor Installed Location Number Type Notes --------- ------------- ------ --------- -------------------------- 18-MAR M110 Lower T1 Sprague on all 3 Taped capacitors 6-MAR M105 Upper T1 Mallory -4.5 CDE -2.0 & -5.2 6-MAR M106 Upper T1 Mallory -4.5 CDE -2.0 & -5.2 6-MAR M107 Lower T1 Mallory -4.5 CDE -2.0 & -5.2 6-MAR M108 Upper T1 Mallory -4.5 CDE -2.0 & -5.2 6-MAR M109 Lower T1 CDE on all 3 6-MAR M111 Lower T1 CDE on all 3 6-MAR M111 Upper T1 CDE on all 3 6-MAR PDM-13 CDE on all 3 Taped capacitors 19-FEB M103 Lower T1 Mallory on all 3 19-FEB M105 Lower T1 Mallory on all 3 19-FEB M108 Lower T1 Mallory on all 3 19-FEB M106 Lower T1 Mallory on -4.5 & -5.2 only none on -2 Bring Bricks MSU SN# 27, 97, 98 to D-Zero. These are refurbished bricks that just came back from Pioneer Magnetics. Later in the week Harry brought SN# 416254 to Fermi - it is also just back from refurbish. Continue putting PDM-09 back together. MSU SN# 25 is its -2.0 Volt brick MSU SN# 59 is its -5.2 Volt brick MSU SN# 27 is its -4.5 Volt brick These are all ReFurbished bricks that have just recently come back from Pioneer Magnetics. Should we put Capacitor Band-Aids on these bricks ?? When starting the L1 Cal Trig back up I had trouble getting +16,19 EM & HD to load a good pedestal. TT_ADC_Mon was reading 255 for +16,19 EM & HD. For a while it looked OK then it when back to 255. So I pulled this CTFE card and installed a new Term-Attn-Brd for eta 15,16. These channels now appear OK. I hand edited the current ped file to get the TT's covered by this replacement Term-Attn-Brd back close to 8. Time spent looking at TT's signals right out the the BLS Cable using the scope and letting it do FFT to look in frequency space. See things at 7.59 10.0 13.275 15.2 MHz Try to correlate this with what channels pick up a lot of SMT noise and that does not fit very well. It may be that 10.0 MHz mostly happens in the crossover and in EC. Scope Pictures -1,17 Using the - side of the raw diff BLS signal on the yellow scope display. This is with the scope cable to plug directly into the BLS cable and properly terminate it. Scope is 5 mV per div 4 usec per div so the full record is 10,000 points and 40 usec. trace is in cvs file, 1st TIF is scope trace, 2nd TIF is FFT Disk 1 is EM and Disk 2 is HD Trigger the scope on L3_Control_Data_Latch_Enable (L2 Decision with Accept) This is the lower purple trace. Yellow is the + side of the BLS signal Blue is the - side this is with the scope cable that looks directly at the BLS cable and properly terminates it. This is looking at TT -4,32 EM. The first (negative energy) bump is at 22 usec. The second positive energy bump is at 306 usec. Their amplitude is about 2 to 3 mV on each side of the line. Disk 5. I looked at two BLS Trigger Tower signals: -4,32 EM and +9,20 EM. I saw no change to the spectrum analyzer output during the time when DAQ was paused compared to traces taken when the system was running. I see the following discreet spectral lines with the DAQ system running or not running. Frequency -4,32 EM +9,20 EM --------- ---------- ---------- 7.59 MHz 650 uV 200 uV 10.00 MHz 0 300 13.28 MHz 850 600 15.17 MHz 700 200 At lower frequencies the spectrum becomes a continuum. For -4,32 EM most of this continuum is below 5 MHz. At 5 MHz it is about 50 uV and at lower frequencies e.g. 1 or 2 MHz it rises to 100 uV. For +9,20 EM the continuum is almost mostly below 5 MHz and is in the 50 to 75 uV range. -4,32 is a "pure" CC Trig Tower and +9,20 is a pure EC Trig Tower. Recall that eta >= 9 is across the first step in the Energy to Volts calibration. So for a given energy deposit, eta 9 has only 35% of the BLS TT signal Volts that eta 4 has. -4,32 EM shows no pickup of SMT noise at L1_Acpt time. +9,20 EM shows a little pickup of SMT noise, its peak-peak width about doubles during the SMT noise pickup blob. When the DAQ system is running and I specifically capture scope sweeps that contains a "blob" of SMT L1 noise, then the discreet spectral lines stay the same amplitude and there is just a new blob of continuum noise mostly in the 7 to 12 MHz range. Dean and I also looked at +6,8 HD. This is a TT that is readout by CC but all of its inputs come from EC. This channel shows a lot of SMT noise pickup. If we look at a sweep taken when there is no observed SMT noise then we see: Frequency +6,8 HD --------- --------- 7.59 MHz 800 uV 10.00 MHz 4000 13.28 MHz 400 15.17 MHz 0 So this TT has a lot of 10 MHz signal in it. If we FFT sweeps that includes the SMT readout noise blob then the discreet spectral lines stay the same amplitude and a continuum in the 7 to 12 MHz range appears. Details of how this data was obtained: Use the negative side of the raw BLS signal using the adaptor cable to the scope that correctly terminates the BLS Cable. The scope is set to 5 mV / div with 20 MHz bandwidth limit horizontal is 4 usec / div which causes the scope to sample once every 4 nsec which gives a 125 MHz Nyquest frequency it acquires a 10,000 sample record (40 usec record) it FFT's this full 10,000 sample record. Tektronix TDS-3054 Scope Duration Period Time per Sample for 10,000 Between Nyquist Division Rate Samples Samples Frequency -------- -------- ---------- --------- --------- 400 usec 2.5 MS/s 4 msec 400 nsec 1.25 MHz 200 usec 5 MS/s 2 msec 200 nsec 2.5 MHz 100 usec 10 MS/s 1 msec 100 nsec 5 Mhz 40 usec 25 MS/s 400 usec 40 nsec 12.5 Mhz 20 usec 50 MS/s 200 usec 20 nsec 25 Mhz 10 usec 100 MS/s 100 usec 10 nsec 50 Mhz 4 usec 250 MS/s 40 usec 4 nsec 125 Mhz 2 usec 500 MS/s 20 usec 2 nsec 250 Mhz 1 usec 1 GS/s 10 usec 1 nsec 500 Mhz 400 nsec 2.5 GS/s 4 usec 400 psec 1.25 Ghz 200 nsec 5 GS/s 2 usec 200 psec 2.5 Ghz For the TDS-3054 Scope the standard sample length is 10,000 points. Before turning on L1 Cal Trig out to 3.2 I need to look at TT -16,31 EM. ------------------------------------------------------------------------------ DATE: 10:12-MAR-2003 At: Fermi Topics: Drip Detector power trips of the L1 Cal Trig, TRICS I/O with TT eta -9:-12 phi 21, New Sign for the 16 Channel Drip Detector, Power Supplies Drip Detector Power Trips Water Leak The Cal Trig tripped off this past Sunday afternoon at about 2 PM. Over the phone we got it running again and it looked all OK. It tripped off again Monday at about 5 AM. Scott was on shift and he looked at the RPSS and let me know that it was a Drip Detector trip. But, the 16 channel Drip Detector panel did not show any current leak problems. Arriving here Monday I could not find any water leaking in the racks and the 16 channel Drip Detector did not show any current leak problems. But if you moved the cables at the back of the Drip Detector there were 2 channels that would show leaks. The cables themselves looked OK but the connectors on the back of the Drip Detector were loose. I took the Drip Detector out of the racks and took it apart and fixed these back panel connectors. Put everything back together and then checked that all Drip Detector strips were working OK. Let the system sit for a couple of hours with no air flow to see if it would trip. No air flow to limit the evaporation and thus perhaps maximize the chance that it would trip. All OK so start the air flow but do not turn on anything in the racks. Run this way over night and it does trip off again. RPSS latches a Drip Detector trip and this time the 16 channel Drip Detector does show a problem on its 4th to the last enabled indicator, i.e. the M109 to M110 detector strip. Look at the M109 to M110 radiators and now I can see a few drops inside the mud flaps. Took off the shockless system covers to look at the mud flaps and you can see a few drops in the process of running down inside the mud flaps. It is probably coming from the lower section of the next to the top radiator but there is so little water that I can not tell for certain. All the water looks contained inside the mud flaps and at this time there is not enough water to say for certain where it is coming from. So pull the detector cable from the strip and put a jumper in it so that the Drip Detector is no longer looking for drips in the M109 to M110 location. Then fire the full system back up. This leak should be OK for a week and by then perhaps there will be enough water to know where it is coming from. The following is the text for the new version of the note that goes on the front panel of the 16 channel Drip Detector. L1 Calorimeter Trigger Water Drip Detector ---------------------------------------------- Original Rev. FEB-1994 Current Rev. 10-MAR-2003 This is the 16 Channel Drip Detector for the Level 1 Calorimeter Trigger. Each channel has 2 LED's and they indicate the following: Red LED If illuminated it implies that this channel is turned off and is not enabled to trip the system power if it detects a leak. Green LED If illuminated it implies that this channel is enabled to trip off the system power if it detects a leak and that currently this channel does not detect any water. If neither LED is illuminated it implies that this channel is enabled to trip off the system power if it detects a leak and that current this channel does detect a water leak. If this drip detector does detect a water leak then, via the L1 Cal Trig Rack Power Safety System, it will drop all power to racks M103 through M112. The Rack Power Safety System is located at the bottom of rack M113. A drip detector trip of system power will show up on the RPSS front panel as a "Air Flow" and "Water Pressure" fault in rack M114. CTFE Card at -9:-12 , 21 For the past couple of months, when the Cal Trig power is first turned back on after it has been off for a long time and things have gotten cold, the system could not correctly do I/O with MBA = 212 CA = 41. When trying to initialize this CTFE card TRICS would say that it could not readback correctly from FA's 0 and 1 and I think sometimes from FA 81. Once things warmed up (1 hour) or something like that then everything was OK. I watched this carefully this morning and the Card Selected LED clearly does flash on so I do not think that it is a CA problem. When this card is cold I have also seen it look funny in the TT_ADC_Mon display - so that kind of hints that it is a read problem and that perhaps the writes are working OK. This is at eta -9:-12 phi 21. Pull CTFE SN# 284 and install CTFE SN# 212. CTFE SN# 212 is a bit famous for having been on fire last fall because of a wire in the serial data cable to its Term-Attn-Brd being pinched under a flash ADC chip. See 18:20-SEPT-2002. Any way CTFE SN# 212 went into eta -9:-12 phi 21 alright and appears to be working OK. I have no idea yet what is wrong with CTFE SN# 284. Bus buffer drivers look OK and visual looks OK. Term-Attn +16,19 I have a couple of times after power up seen +16,19 EM & HD need two downloads of pedestal file to get it loaded into this TT. After the first download the EM & HD for this TT still read 255, then after the second download they move to 8. Power Supplies The next supplies that are pretty seriously in need of Capacitor Band-Aids are both Tier 1 supplies in M110 and the upper Tier 1 supply in M112. I need to look to see if some of the Tier 2-3 supplies can be worked on in place. The following supplies are being looked at periodically to see how well the Capacitor Band-Aids are holding up. To monitor it, watch it with the RM.py for about 1 minute, subtact the low reading from the high reading and write it down in mV. This is not super "scientific" but it will give some idea about how permanent of a fix this is. Recall that the LSDigit on the RM.py display is 10 mV. Shea Box Power Supply Channel Cap Original Current ---------------------- ---------- --- -------- ------- M111 Top Tier 1 -4.5V rm_B ch_12 CDE 1130 mV 130 mV M111 Low Tier 1 -4.5V rm_B ch_48 CDE 1050 mV 50 mV M108 Low Tier 1 -4.5V rm_B ch_32 Mal 720 mV 70 mV M106 Low Tier 1 -4.5V rm_A ch_60 Mal 680 mV 40 mV These were 4 of the worst supplies and we are monitoring them because it is assumed that they will be the first to get into trouble if the Capacitor Band-Aids start to fail. I brought the first 2 repaired bricks (refirbished bricks) from Pioneer Magnetics to Fermi this trip. These are MSU SN# 25 a 2 Volt 250 Amp brick and MSU SN# 59 a 5 Volt 200 Amp brick. I'm not 100% certain that MSU SN# 59 the 5 Volt brick is the one that we sent to them. It has new input and output electrolytic caps, new output rectifier (a Mot MBRP30045CT), new fan, it looks absolutely clean inside and out, all of the semiconductors that I can read are from the original manufacture data, i.e. the switching transistors, the IC, the line rectifier. There are no date codes on the new output filter caps and they are house branded. MSU SN# 25 the 2 Volt brick is the one that we sent to them. It is much the same as above but no new main output rectifier and it is not are clean inside. It's output caps are also house branded with no date code that I can read. The 2 Volt brick uses 4 low post caps and the 5 Volt brick uses 3 high post caps - so the extra post length must make a difference to them. Start to put PDM-09 back together. MSU SN# 25 is its 2 Volt brick and MSU SN# 59 is its -5.2 Volt brick. ------------------------------------------------------------------------------ DATE: 4:8-MAR-2003 At: Fermi Topics: Capacitor Band Aids, swap Term-Attn-Brd Work on TT's, Spare SCL Receivers, Cut Resistor History, Cold start instructions Rev. 8-MAR-2003 We are running: TRICS V10.3.B and VME_Access V4.0.B After Master Clock is running, turn on TFW and Cal Trig (one power supply at a time) then: Configure FPGA's The normal Master Command file for this does everything except for the M101 Routing Master Get the Routing Master Configured and Running 1) After the Routing Master is powered up, wait for its SBC to boot (less than 1 minute) 2) Login to the SBC with 'ssh d0runsu@d0sbc001b' The password is same as the online d0run account. 3) On the SBC run 'reset_all.sh stop' which will stop the relevant readout processes. 4) Use TCC's Configure FPGA's menue to execute the dcf that configures the Routing Master FPGA's 5) On the SBC run 'reset_all.sh start' Tell TRICS that the Cal Trig eta coverage is 1:20 Init TFW and Cal Trig Tell TRICS to completely ignore the Cal Trig By hand using VME Access load the Cal Trig Gains files. (made sure that the "No Write to DAC's" box is not checked) By hand using VME Access load the Cal Trig Pedestals files. Tell TRICS that it has control of the Cal Trig again and set the Cal Trig eta coverage to 1:12. Init the TFW and Cal Trig verify that it is a clean init of all parts The file to configure the Routing Master FPGA's is: D0_Config\M101_Routing_Master_All.dcf The current Gains files are: D0_Config/Gains_13_20_1_32_rev_b.tti # TT eta's 13:20 all phi's D0_Config/Gains_1_12_1_32_rev_10.tti;1 # TT eta's 1:12 all phi's The current Pedestal files are: D0_Log/Find_DAC_V4_1_D_20030307_edited.tti;4 # TT eta 13:20 D0_Log/Find_DAC_V4_1_D_20030307_edited.tti;3 # TT eta 1:12 Power Supplies Put capacitor band-aids on 7 more Tier 1 Power Pans. The Power Pans that received capacitor band-aids on all their Pioneer Magnetics supplies are: The following 4 Power Pans received a Mallory band-aid on their -4.5 Volt supply and CDE band-aids on their -2.0 Volt and -5.2 Volt supplies. M105 Upper Tier 1 M106 Upper Tier 1 # this pan has old style terminal strips and lamp M107 Lower Tier 1 # added water manifold clamps M108 Upper Tier 1 # this pan has old style terminal strip and lamp The following 3 Power Pans received CDE band-aids on their -2.0 Volt -4.5 Volt and -5.2 Volt supplies. M109 Lower Tier 1 # added water manifold clamps M111 Lower Tier 1 # added water manifold clamps M111 Upper Tier 1 # check the ripple on this pans +5.0 Volt brick. M111 Upper Tier 1 +5.0V supply has ripple 9 mV/ 57 mV The big issue that I did not discover until after all of this was done is that the outer can of these capacitors is internally tied to the negative terminal of the capacitor (even though from the outside it looks completely insulated). For the capacitor that is tied to the negative bus this is not a problem. For the capacitor that is tied to the positive bus this means that we are depending on the PVC sleeve to insulate the can from the bus par. I should put some tape either on the bar or else on the side of the capacitor to make an extra layer of insulation. On the CDE DCMD capacitors the negative terminal is connected to the can. I assume the same it true of the Mallory CGO capacitors. Full list of things to check when installing Capacitor Band-Aids Tighten the AC input cable terminal strip connections. Verify that the lug on the black capacitor jumper wire is not touching the case of the capacitor on the positive terminal. Add hose clamps to the ends of the water distribution manifold in the bottom of the rack if necessary & tighten any existing clamps on the ends of these manifolds. Remove any loose paper tags. Verify that fans run OK. Test run the Power Pan before sliding it back in and connecting the power busses to the crate. Check that the power bars have no burs where you put on the caps. Put a layer of tap on the side of the capacitor that attaches to the positive power bus. Verify that the AC power cord has been properly routed. On Upper Tier 1 check the sliders for metal particles before pushing the supply back in. Check that the CHTCR is fully in its socket and that all cables are still well plugged in. Put the Power Pan PDM-13 back together as a Tier 1 spare. Use CDE capacitor band-aids on all 3 of its Pioneer Magnetics supplies. Put a double layer of tape on the side of the capacitor that mounts to the positive power bus. The bricks in PDM-13 are the following: +5.0 V brick MSU SN# 6 from 15-OCT-90 -2.0 V brick MSU SN# 70 from 7-JAN-92 -5.2 V brick MSU SN# 71 from 7-JAN-92 -4.5 V brick MSU SN# 72 from 7-JAN-92 PDM-13 has not yet been tested with the load box. It was run on 7-MAR-2003 and all of its output voltages checked and set without load. The Calorimeter Trigger Control Crate is already running on old, i.e. model 2500A2 power supplies. So all the 2 spare model 2500A2 supplies that I brought to Fermi on this trip are good for is spares for the Cal Trig Control Crate. Term-Attn-Brd Work Swap the Term-Attn-Brd's in the half rack eta +5:+8 phi 1:16 I looked at all of these Term-Attn-Brd's and they all needed to be changed (because of incorrect R80, R3, C7) except for the boards: eta +5,+6 phi 9 eta +5,+6 phi 10 eta +7,+8 phi 12 Once again this exactly matches what had been observed from Bob's plots. The only problem was that the board labeled "Hi eta EM ped control does not work", i.e. the board that worked OK at MSU - did not work here again. Edit the gains file to return the area eta +5:+8 phi 1:16 to normal EM gain. The new gains file is Gains_1_12_1_32_rev_10.tti When powering back up after things were tunred off for cap band-aid and Term- Attn work I had trouble talking with MBA 212 CA 41 which is eta -9:-12 phi 21 (or is it phi 22). After running the system for a while and getting other parts initialized then this card runs OK. I have seen this happen before, 2 weeks ago. Something is wrong - could be a problem at a distance type of thing. SCLR SCL Receiver SN# 206 "SCLR 24-MAY-02 5980A9" to Hal Evans. That leaves the only SCL Receiver in the cabinet SN# 102 "SCLR 7MHZDE Seq Control 8-28-02 0059803A" Ted Zmuda brought over 5 more spare SCL Receivers. So the spares box now has 5 normal SCLR and one Sequencer SCLR. Find_DAC The two significant run were: Find_DAC_V4_1_D_20030307_edited.tti;3 # for eta 1:12 Find_DAC_V4_1_D_20030307_edited.tti;4 # for eta 13:20 I forgot to tell the ;4 run to make a histogram file. The ;4 run had a bunch of failures at 19 and 20 - I just commented these all out. The ;3 run failed at: +4,22 EM +9,21 EM -1,11 EM -3,28 EM +2,29 HD +3,19 HD +5,20 HD +7,10 HD +8,20 HD -2,3 HD -4,1 HD -4,3 HD -8,22 HD +9,21 was because of hugh noise and >0.1 from 8.0 all the rest were because of > 0.1 from 8.0 Work on TT's Looking at the L1 Cal Trig end of the BLS cable Dean and I could see the sparking on +1,24 EM On the scope it was bumps of about 15 GeV max. On the Cal Lego Display it was making 10 to 12 GeV. Dean removed the resistor for Tower 2 Depth 6 and now you can still see it on the scope but it is about 3.0 or 3.5 GeV max. We did not try pulling the neighboring resistors. Dean put a label on this BLS says that it was only for use at +1,24 So +1,24 now officially has this resistor cut for Run II. See also 10:12-JAN-02 for some +1,24 thrashing. We looked at -9,22 HD but could not see any problems on the scope on that TT today. Looked at +3,26 EM and it is making big noise on the scope. Pull the BLS card and we can see the same noise in the test stand. Try all new summers and that does not fix it. Try a new driver and that does not fix it. The number of noise pulses goes down as you remove summers. With no summers we do not see any noise pulses after running for about 5 minutes. It typically takes a few minutes of running before you see the noise pulses. The summers and driver that were in this card look OK in a different BLS card. This is all EM noise pulses - no HD pulses. This card is PC23 bottom crate 10 BLS from the left. Look at +5,23 EM - it is making very big low frequency noise on the scope looking from the MCH end of the cable. It is PC16 top 6th BLS from the left. Put this BLS in the test stand and it looks OK. Put a different BLS from the same eta in the 6th slot and it looks OK i.e. no more big low frequency noise. Looking from the MCH end all the +5 EM signals have a lot of high frequency noise. Examine the summer and driver hybrids on the +5,23 BLS and they all look OK. Put both BLS cards back where they belong. The low frequency noise is gone from the +5,23 BLS (just like it was not there in the test stand). 10 hours later look again and it is still gone. The following are clips from old log books about "cutting resistors" in Run I. Date: 28-FEB & 1,2,3-MAR-1995 At: Fermi Topics: Get things running again after the Feb '95 Shutdown The decision was taken to cut the resistor on the BLS card from the noisy Calorimeter element that drives Trigger Tower +6,23 EM. We watched on a scope plugged into the CTFE monitor LEMO as Joan cut the appropriate resistor and the noisy when away. Jan has removed TT +6,23 EM from the Excluded TT List. See the 30-Nov-1994 log book entry for more details about the noisy Calorimeter element that is in the +6,23 EM TT. See the 12-MAR-1993 log book entry for details about the two other TT's that have a resistor cut: -4,21 HD +1,24 EM. Now with a flat 3 GeV EM Ref Set and requiring one TT the rate is about 1 Hz and with a flat 3 GeV Total Et Ref Set and requiring one TT the rate is about 3 Hz. Date: 29,30-NOV-1994 At: Fermi Topics: Water Leak in M103-M104 radiator, 1-DEC-1994 Look at TT +6,23 EM which has been excluded, DAC Pedestals for TT's +17,15 and +18,15, Added more sensors to the RPSS, Work on getting Pulser runs for L15CT_PROV Look at TT +6,23 EM which has been excluded since Sept shutdown +6,23 EM is rack M105, CBus = 1, MBA = 170, CA = 44:45, "2nd" EM channel on the CTFE card, Clock Control Register is FA = 81, bit of value 4 controls this EM channel. Well, it still looks noisy on the scope and running a L1 Cal Trig with a 3 GeV EM Ref Set threshold everywhere you get about 0.6 Hz if +6,23 EM is excluded and about 5 to 6 Hz if +6,23 EM is not exclude. The decision is to leave it excluded. Should we cut the resistor ? We looked at the Examine from this run and all the noise is coming from eta,phi,depth 11, 45, 2 in the Calorimeter. Date: 12-MAR-1993 At: FERMI Topics: BLS Resistors on Noisy Channels, New Long TimeOut PAL for L15 Control MTG, Official distribution of VTC software for 2 VME Buffers, Understanding Latch-Shift signals. The "HV Spark" noise on Trigger Tower -4,21 HD was/is coming from Calorimeter element: Crate 7, ADC 9, BLS 3, Tower 2, Depth 9. Today Joan cut the BLS resistor for this element. We checked with the scope on the CTFE Monitor output for -4,21 HD before and after the cut and the noise problem DID go away when this resistor was cut. We will leave this resistor cut. The Trigger Tower that has had a resistor cut for some months is +1,24 EM. We checked it with a scope looking at the CTFE Monitor output by plugging in a BLS with all resistors installed. With this BLS we saw noise. With the normal BLS for this Trigger Tower (i.e. a BLS with the resistor for Calorimeter element: Crate 18, ADC 0, BLS 0, Tower 2, Depth 6 cut) there is no noise except for the normal base peak switching noise. We put back the BLS with the cut resistor i.e. we will leave this resistor cut. Restarted TRICS. Finally returned Maxwell-Maxwell. Next Trip The old pressure sensors are Whitman Controls model J205G-10S-C12L. Bring some spare of the special length screws for mounting bricks into pans. Bring voltage monitoring cable and 9 pin connectors and such to Fermi. ------------------------------------------------------------------------------ DATE: 18:20-FEB-2003 At: Fermi Topics: Capacitor Band Aids, swap Term-Attn- Brd's, Cal Trig Voltage Monitoring and Alarms, 200 Amp Pioneer Mag modification, Water Leak The following are the current instructions to start up the TFW and Cal Trig from power up We are running: TRICS V10.3.B and VME_Access V4.0.B After Master Clock is running, turn on TFW and Cal Trig (one power supply at a time) then: Configure FPGA's The normal Master Command file for this does everything except for the M101 Routing Master Get the Routing Master Configured and Running 1) After the Routing Master is powered up, wait for its SBC to boot (less than 1 minute) 2) Login to the SBC with 'ssh d0runsu@d0sbc001b' The password is same as the online d0run account. 3) On the SBC run 'reset_all.sh stop' which will stop the relevant readout processes. 4) Use TCC's Configure FPGA's menue to execute the dcf that configures the Routing Master FPGA's 5) On the SBC run 'reset_all.sh start' Tell TRICS that the Cal Trig eta coverage is 1:20 Init TFW and Cal Trig Tell TRICS to completely ignore the Cal Trig By hand using VME Access load the Cal Trig Gains files. (made sure that the "No Write to DAC's" box is not checked) By hand using VME Access load the Cal Trig Pedestals files. Tell TRICS that it has control of the Cal Trig again and set the Cal Trig eta coverage to 1:12. Init the TFW and Cal Trig verify that it is a clean init of all parts The file to configure the Routing Master FPGA's is: D0_Config\M101_Routing_Master_All.dcf The current Gains files are: D0_Config/Gains_13_20_1_32_rev_b.tti # TT eta's 13:20 all phi's D0_Config/Gains_1_12_1_32_rev_9.tti;1 # TT eta's 1:12 all phi's The current Pedestal files are: D0_Log/Find_DAC_V4_0_B_20030212_edited.tti;4 # TT eta 17:20 D0_Log/Find_DAC_V4_0_B_20030212_edited.tti;5 # TT eta 1:16 D0_Log/Find_DAC_V4_0_B_20030219.tti;2 # TT eta +5:+8 phi 17:32 Voltage Monitoring and Alarms The online machines that we should now be using for our work are: 20, 21 and 30 with perhaps a little emphasis on 30. Voltage monitoring and Alarms To start the SES Alarm Display on your machine you need to: > start_daq alarm_display & You may also need to do a setup d0online before this. The template files to edit to control the High and Low Limits for the alarms are in the following files: /online/ioc/templates/ lvl1.dbt # for normal Tier 1,2,3 Power Pans lvl1c.dbt # for the Cal Trig Control Crate These template files consist of records with fields. You can add a field so that that the monitored value going into alarm condition will pause the run: field(EXPR, "150") #comment severity > 100 --> pause the run These template files make it easy to change the limits on a whole class of supplies (e.g. +5.0 V) all at once. Once edited these template files go into the Oracle database and out comes the information for each channel. Date: Wed, 19 Feb 2003 11:22:40 -0600 From: Geoff Savage To: Dan Edmunds Cc: bartlett@fnal.gov, Stan Krzywdzinski Subject: monitoring L1 cal low voltage supplies Hi Dan, Here is a summary of what we discussed this morning. The alarm limits on the +5V supplies have been added in /online/ioc/templates/lvl1.dbt. Please add limits to the other voltages in this file and the ones in lvl1c.dbt. Once that is done let me know and I'll insert the template into the hardware database (Oracle). Next we extract the new EPICS database and reboot the front end processor. If you want a MAJOR alarm to pause a run one more field needs to be added: field(EXPR, "150"). Summary: 1. Add alarm limits to records. (Dan) 1a. Configure MAJOR alarms to pause runs, field(EXPR, "150"). (Dan) 2. Insert modfied templates into the hardware database. (Geoff) 3. Extract new epics database. (Geoff) 4. Reboot front end processor, d0olctl16. (Geoff) 5. Test (Dan) 6. Put modified templates into cvs. (Geoff) Geoff Work on power supplies. Add "capacitor band aids" to all 3 Pioneer Magnetics bricks in Tier 1 M103B, M105B and M108B. In Tier 1 M106B add capacitor band aids to the Pioneer Magnetics -4.5V and -5.2V bricks but not to the -2.0V brick. That's all the capacitors that I have for now. In the Cal Trig RM.py display this appears to clean things up just fine in these supplies. Things to think about while pulling the Power Pan to install the capacitor band aids: Tighten the AC input cable terminal strip connections. Verify that the lug on the black capacitor jumper wire is not touching the case of the capacitor on the positive terminal. Add hose clamps to the ends of the water distribution manifold in the bottom of the rack if necessary & tighten any existing clamps on the ends of these manifolds. Remove any loose paper tags. Verify that fans run OK. Test run the Power Pan before sliding it back in and connecting the power busses to the crate. While the Power Pan was pulled out of M105 I worked on its water leak. The leak was from the front of the manifold that is on the M104 side. I put hose clamps on all 4 ends. The hose clamp on the leaking end appeared to stop the leak right away. The leak was a drip (not a spray) with one drip every couple of minutes. Peter has done a very nice job of adding 3 more capacitors to the 200 A Pioneer Magnetics bricks and making drawings of all the parts. I will take this brick back to MSU to run it. I need to find a source of caps in this can size with high (normal) lugs (and the same ESR and high ripple current rating). The idea is to replace the 3 caps in the brick and add 3 more. If we need to pull a high eta Tier 1 power pan, M112 Bottom Tier 1 is the best condition supply. Term-Attn-Brd's Work on the Term-Attn-Brd's in the range eta +5:+8 phi 17:32 Pull out these 16 CTFE cards and unplug their Term-Attn-Brd's and verify that they have correct R3's and R80's - replace all Term-Attn-Brd's that have the incorrect resistors. The replacement card are from MSU and have been checked for to verify that they have correct resistors and then tested. The following table list what was found and done to each of these 16 CTFE cards. phi 17 replace neither (both had correct R3 & R80) phi 18 replace neither (both had correct R3 & R80) phi 19 replace neither (both had correct R3 & R80) phi 20 replace neither (both had correct R3 & R80) phi 21 replace neither (both had correct R3 & R80) phi 22 replaced high eta tab phi 23 replace neither (both had correct R3 & R80) phi 24 replace both tabs phi 25 replace both tabs phi 26 replace both tabs phi 27 replace high eta tab phi 28 replace both tabs phi 29 replace high eta tab phi 30 replace high eta tab phi 31 replace high eta tab phi 32 replace neither (both had correct R3 & R80) Looking in the Gain_8 file at the eta +6 and eta +8 channels that had had their gain turned back up because we determined by comparison that they must have correct R80's we find the following list: eta +6 phi's 17 18 19 20 21 22 23 27 29 30 31 32 eta +8 phi's 17 18 19 20 21 23 32 Based on this Gains_8 file information I would have expect to have needed to replace the Term-Attn_brd's at: Lower Eta: 24 25 26 28 Higher Eta: 22 24 25 26 27 28 29 30 31 and that is exactly what was found on the cards themselves. Finally make a 9th version of the Gains file that has the eta +6 and eta +8 EM gain turned back up to be correct for correct R80's for phi's 17:32. There are 13 Term-Attn-Brd's (5-20 type) going back to MSU for resistor work and then testing. There are currently 10 spare Term-Attn-Brd's (5-20 type) here at D-Zero all tested and with correct resistors. There are currently zero spare 1-4 type Term-Attn-Brd's at D-Zero (but there is a stack at MSU ready to ship to D-Zero). Bring more Red DeOxit to Fermi. ------------------------------------------------------------------------------ DATE: 11:14-FEB-2003 At: Fermi Topics: Bug in Data I/O software, Work on MCH end of TT's, Run Find DAC, Power supply Voltage Monitoring, Power supply repair, Test of Splitter noise Bring back the spare SCL Hub Controller To cold start the system the precedure is the same as last week except for new pedestal files which are: TT eta 1:16 20030212_edited.tti;5 TT eta 17:20 20030212_edited.tti;4 Work on the L1CT end of things: Problem -14,12 EM reads 124 on TTADCMonit and $7c84 om HSRO. Its Lemo monitor output reads about -0.025 V. Loading 4095 into its ZER DAC gives -0.0259 at the Lemo. Loading 0 into its ZER DAC gives -0.0248 at the Lemo. So it moves a little bit in the right direction. Work on the Term-Attn-Brd but can not fix it here. Swap. Now OK and peds are in 20030212.tti;3 Problem -13:-16 phi 14 EM & HD read 8 or FF or what ever value it decides to stick at. Reads this in both TTADCMON and HSRO. The active delay line had folded over pins - so just random (and not frequent) Clk's to the FADC. Fix the pins - now OK - peds are in 20030212.tti;2 CA = 22 in the Tier 2 crate for eta 17:20 was having problems at L1CT init time. It would just readback $ff. This is EM Ref Set 0 Counter Tree CAT2 card in the eta 17:20 Tier 2 crate. Push it into its backplane socket better and now it sees its CS and is happy. What is really wrong with it ? Notice that the Tier 2 to Tier 3 Counter Tree cables for eta's 13:16 and 17:20 are not plugged in. So for the first 2 stores (the 30 minute one and the 3 hour one) we were only triggering in eta 1:12. Plug in the cable for eta 13:16 but not for 17:20. Run Find_DAC in just eta's 17:20 results are in 20030212_edited.tti;4 This run had trouble in 3 places: +20,18 EM -19,22 EM -20,22 HD Run Find_DAC in just eta's 1:16 results are in 20030212_edited.tti;5 I stopped Excluding any TT's before this run. It had a number of problems: -8,18 EM ... -8,32 EM all failed -5,17 HD -6,17 HD -7,17 HD -8,17 HD all failed, and +1,24 HD and +5,8 HD both failed. Where the blocks of I just redid -8,18 EM -8,32 EM and that went fine. To get the -5:-8 phi 17 to work I had to first Init the L1CT. I do not understand what happened. While all of this was going on I saw one sweep of TT_ADC_Mon that had crate -5:-8 phi 17:32 all around 230 in red - what happened ? Work on the problem that all 8 pedestals at eta +17:+20 phi 4 are drifting (mostly up a few counts every couple of hours). The -1.0 Volt Ref supply is drifting and is currently putting out about -1.16 Volts. This is CTFE SN# 176. The element in the trim pot on the -1.0 V supply is going open, the wiper and one side are still OK. Pull the trim pot from the -1.352 V and use it on the -1.0 V supply. Just bridge the 3 trim pot contacts together on the -1.352 V supply to keep it happy. Setup the supply and then run Find_DAC 20030212.tti;23 and murge the results into 20030212_edited.tti;4 Just before the store the -2.0 V brick in the bottom Tier 1 crate in M106 died. This is power pan PDM-17. Because there are no tested spare Tier 1 power pan we needed to repair this one on the fly. From PDM-17 pull out the -2.0 V brick SN# 54 and install SN# 63. SN# 63 is a repaired brick from Run I and it was test run during the past couple of months. PDM-17 and the Cal Trig came back to life but some beam time was lost. First steps in Cal Trig voltage monitorining are now available. I think that Stan Krzywdzinski set this up. I have run it from the SUN tube in the following way. Log into d0mino to get a kerberos ticket ssh to the online system e.g. ssh d0l1@d0ol02 # I guess 02 is OK to use for this then setup d0online RM.py L1CAL_RM_MCH1A & # this is the remote termina 25 Shea box # or L1CAL_RM_MCH1A is the RT 26 Shea box. I thought that the display would just come back to the SUN tube without doing anything else but it did not. I needed to xhost d0ol02 and on d0ol02 I needed to setenv DISPLAY d0sunmsu1:0.0 I need to play with all of this some more to verify what really needs to be done. Once running the generic RM.py display you need to switch to Enginering Units by clicking the box at the top. Then RM.py shows you the voltages by Shea Box channel number. You can look in the bottom of the file www.pa.msu.edu/hep/d0/ftp/l1/cal_trig/hardware/rack_crate/ rack_voltage_monitoring_points.txt to see the mapping from Shea Box analog input channel numbers to what power supply in what rack is being monitored by that channel. Peter shows me a bug in the Data I/O programmer software. It is in the number 95 format files, one of the Motorola 32 bit formats. (I think that we use only the number 82 format). He knows about it from his work using the UniSite model 48 programmer. At address 10000 it screws up 4 bytes of data. It may be that this only happens when doing even odd byte stuff. And it is only when reading in files - part to part copies are OK. This bug is fixed in the newest version of the UniSite software. Make another pedestal check of the Splitter card Find_DAC_V4_0_B_20030212.tti;18 Splitter is in the circuit EM_TT(- 9, 1) Ped_DAC: 3719 Ped_Avg: 8.03 Ped_Dev: 0.82 Dev_Avg: 0.78 EM_TT(-10, 1) Ped_DAC: 3654 Ped_Avg: 8.01 Ped_Dev: 0.75 Dev_Avg: 0.65 EM_TT(-11, 1) Ped_DAC: 3659 Ped_Avg: 8.04 Ped_Dev: 0.84 Dev_Avg: 0.78 EM_TT(-12, 1) Ped_DAC: 3642 Ped_Avg: 7.99 Ped_Dev: 0.17 Dev_Avg: 0.45 HD_TT(- 9, 1) Ped_DAC: 3658 Ped_Avg: 7.96 Ped_Dev: 1.60 Dev_Avg: 1.61 HD_TT(-10, 1) Ped_DAC: 3675 Ped_Avg: 8.00 Ped_Dev: 1.71 Dev_Avg: 1.65 HD_TT(-11, 1) Ped_DAC: 3650 Ped_Avg: 7.98 Ped_Dev: 1.99 Dev_Avg: 1.90 HD_TT(-12, 1) Ped_DAC: 3669 Ped_Avg: 8.01 Ped_Dev: 0.72 Dev_Avg: 0.75 Find_DAC_V4_0_B_20030212.tti;19 Splitter is in the circuit EM_TT(- 9, 1) Ped_DAC: 3719 Ped_Avg: 8.03 Ped_Dev: 0.81 Dev_Avg: 0.75 EM_TT(-10, 1) Ped_DAC: 3654 Ped_Avg: 8.00 Ped_Dev: 0.65 Dev_Avg: 0.64 EM_TT(-11, 1) Ped_DAC: 3659 Ped_Avg: 8.01 Ped_Dev: 0.75 Dev_Avg: 0.80 EM_TT(-12, 1) Ped_DAC: 3642 Ped_Avg: 8.00 Ped_Dev: 0.17 Dev_Avg: 0.44 HD_TT(- 9, 1) Ped_DAC: 3657 Ped_Avg: 8.03 Ped_Dev: 1.68 Dev_Avg: 1.60 HD_TT(-10, 1) Ped_DAC: 3676 Ped_Avg: 8.05 Ped_Dev: 1.66 Dev_Avg: 1.66 HD_TT(-11, 1) Ped_DAC: 3650 Ped_Avg: 8.03 Ped_Dev: 1.96 Dev_Avg: 1.90 HD_TT(-12, 1) Ped_DAC: 3669 Ped_Avg: 8.00 Ped_Dev: 0.66 Dev_Avg: 0.74 Find_DAC_V4_0_B_20030212.tti;21 Splitter is out of the circuit EM_TT(- 9, 1) Ped_DAC: 3721 Ped_Avg: 7.95 Ped_Dev: 0.87 Dev_Avg: 0.78 EM_TT(-10, 1) Ped_DAC: 3654 Ped_Avg: 8.02 Ped_Dev: 0.74 Dev_Avg: 0.61 EM_TT(-11, 1) Ped_DAC: 3660 Ped_Avg: 8.02 Ped_Dev: 0.75 Dev_Avg: 0.76 EM_TT(-12, 1) Ped_DAC: 3642 Ped_Avg: 8.02 Ped_Dev: 0.34 Dev_Avg: 0.43 HD_TT(- 9, 1) Ped_DAC: 3658 Ped_Avg: 8.01 Ped_Dev: 1.60 Dev_Avg: 1.56 HD_TT(-10, 1) Ped_DAC: 3674 Ped_Avg: 8.04 Ped_Dev: 1.67 Dev_Avg: 1.65 HD_TT(-11, 1) Ped_DAC: 3653 Ped_Avg: 8.03 Ped_Dev: 1.85 Dev_Avg: 1.84 HD_TT(-12, 1) Ped_DAC: 3670 Ped_Avg: 8.00 Ped_Dev: 0.62 Dev_Avg: 0.75 Find_DAC_V4_0_B_20030212.tti;22 Splitter is out of the circuit EM_TT(- 9, 1) Ped_DAC: 3720 Ped_Avg: 8.00 Ped_Dev: 0.84 Dev_Avg: 0.77 EM_TT(-10, 1) Ped_DAC: 3654 Ped_Avg: 8.01 Ped_Dev: 0.64 Dev_Avg: 0.62 EM_TT(-11, 1) Ped_DAC: 3660 Ped_Avg: 8.03 Ped_Dev: 0.87 Dev_Avg: 0.79 EM_TT(-12, 1) Ped_DAC: 3643 Ped_Avg: 8.00 Ped_Dev: 0.47 Dev_Avg: 0.44 HD_TT(- 9, 1) Ped_DAC: 3660 Ped_Avg: 7.99 Ped_Dev: 1.63 Dev_Avg: 1.57 HD_TT(-10, 1) Ped_DAC: 3674 Ped_Avg: 8.08 Ped_Dev: 1.65 Dev_Avg: 1.64 HD_TT(-11, 1) Ped_DAC: 3653 Ped_Avg: 8.02 Ped_Dev: 1.82 Dev_Avg: 1.83 HD_TT(-12, 1) Ped_DAC: 3670 Ped_Avg: 8.03 Ped_Dev: 0.76 Dev_Avg: 0.74 Dmitri has arranged for Peter Neustroev to help repair some of the 200 Amp bricks. His email is neustr@fanl.gov We looked inside a supply and Peter is thinking about how to install 5 or 6 capacitors inside the supply. I showed him the replacement capacitor from Malory which is CG0153M7R5L and I encouraged him to look for a better replacement part. Currently have 17 spare eta 1:4 Term-Attn-Brds at Fermi. I'm taking all of these back to MSU so that they can be checked - just want to verify their status and get them labeled. Currently have zero spare eta 5:20 Term-Attn-Brds at Fermi. I'm taking back 2 that are here that need repair. Visit Neal over at CD and bring back to D-Zero the spare SCL Hub Controller. It is in the spare cards cabinet. ------------------------------------------------------------------------------ DATE: 5:7-FEB-2003 At: Fermi Topics: Work during the shutdown, L1 Cal Trig optical data path, Drew De-Mux cards, BLS Connector Map, Find_DAC Combination of Bits Feature, Find_DAC Runs, Put the BLS signal Splitter in right side up, new test files The following are the current instructions to start up the TFW and Cal Trig from power up After Master Clock is running, turn on TFW and Cal Trig (one power supply at a time) then: Configure FPGA's The normal Master Command file for this does everything except for the M101 Routing Master Get the Routing Master Configured and Running 1) After the Routing Master is powered up, wait for its SBC to boot (less than 1 minute) 2) Login to the SBC with 'ssh d0runsu@d0sbc001b' The password is same as the online d0run account. 3) On the SBC run 'reset_all.sh stop' which will stop the relevant readout processes. 4) Use TCC's Configure menue item to execute the dcf that configures the Routing Master FPGA's 5) On the SBC run 'reset_all.sh start' Tell TRICS that the Cal Trig eta coverage is 1:20 Init TFW and Cal Trig Tell TRICS to completely ignore the Cal Trig By hand using VME Access load the Cal Trig Gains for eta 13:20 (made sure that the "No Write to DAC's" box is not checked) By hand using VME Access load the Cal Trig Gains for eta 1:12 By hand using VME Access load the Cal Trig Pedestals for eta 1:20 Tell TRICS that it has control of the Cal Trig again and set the Cal Trig eta coverage to 1:16. Init the TFW and Cal Trig verify that it is a clean init of all parts The file to configure the Routing Master FPGA's is: D0_Config\M101_Routing_Master_All.dcf The current Gains files are: D0_Config/Run_II_l1CT_Bob_8_Gain.tti;1 D0_Config/Gains_13_20_1_32_rev_b.tti The current Pedestal file (for eta 1:20) is: D0_Log/Find_DAC_V4_0_B_20030207_edited.tti;2 Work on the L1 Cal Trig optical data output. I thought there was less than the normal level of optical power from M108 to both L2 and L3. I ended up replacing the HSROCB for M108, i.e. on Bougie/Spark ANOM SN# 10 Build A pull off HSROCB SN# 55 (I think it is 55) and put on HSROCB SN# 61. Now L2 and L3 see normal optical power from M108. For M109 there was clearly much less than the normal level of optical power to L2 but L3 received the normal power level from M109. Tried moving cables around to see if there was a problem in the optical path without learning much. Swapped the M109 and M110 cables both at the L2 and and at the L2 output from the splitter and then all looked OK. Swap them back and everything is still OK. Check again hours later and everything is still OK. Wednesday evening at about 21:30 swap the Drew De-Mux cards so that we are now running on the one that should give all 128 L2 Answers to the TFW with timing that the TFW L2 Answer Input FIFO can ingest. Initial test late Wed night said that this De-Mux card was OK. On Thur run all 128 L1 Spec Trig's. Looking into the FRONT of an EC BLS crate. eta eta xx xx 20 xx 19 xx xx BLS xx BLS xx xx xx BLS xx BLS for xx for xx BLS xx BLS xx for xx for eta xx eta xx M xx L xx eta xx eta 8 xx 7 xx xx xx 16 xx 15 xx xx eta xx eta xx xx 18 17 | | | | | | | | | | BLS Cable No BLS Cable BLS Cable BLS Cable Connector BLS Cable Connector Connector Connector for eta's Connector for eta's for eta's for eta's 7,8 at this 19,20 17,18 15,16 Location While running Find_DAC and looking at the histograms we find the "feature" that has been seen before that is if you have relatively low noise so that most of the histogram entries are in two bins and if the bins are something like 8 and 9 then all looks OK, i.e. all histogram entries are 8 and 9. On the other hand if the two bins that are being populated are 7 and 8 then you may also see a few 15's. The assumed reason for this is that to go fast Find_DAC is not toggling the 29525 Read A/B Write A/B control lines. Rather it just asynchronously reads from the same 29525 pipe that is being written to. So once in a while you read some conbination of bits from 2 adjacent samples in the 29525. Trigger Tower Work BLS Cable work at the BLS end: Swap the following pairs of BLS cables to plug them in correctly cable eta -17,-18 phi 8 <--> cable eta -19,-20 phi 8 cable eta -17,-18 phi 18 <--> cable eta -19,-20 phi 18 cable eta -17,-18 phi 19 <--> cable eta -19,-20 phi 19 cable eta -17,-18 phi 20 <--> cable eta -19,-20 phi 20 cable eta +17,+18 phi 10 <--> cable eta +19,+19 phi 10 CTFE PROM Work: Use TRICS to scan the EM, HD, Px, Py PROM's in eta 13:166 There are problems with the Py PROM at eta +16 for the following phi's: 13 25 26 27 28 29 30 31. All of the other parts check OK. Use TRICS to scan the Px and Py PROM's in eta 17:20 The problems are: +17,6 Px +18,6 Px +20,31 Px +20,17 Py -20,1 Px -19,3 Px -20,11 Px -17,15 Px -19,16 Py The PROM PYP2017 was blank as indicated by the TRICS PROM scan. Program it - the check for it is 0001D55C Based on the Examine the following parts need to be checked: -14,12 EM Checks OK with the TRICS PROM scanner -17,29 EM not checked yet -19,28 EM not checked yet +17,4 EM HD Test OK in the PROM Cooker +18,4 EM HD Test OK in the PROM Cooker +17,15 EM HD Test OK in the PROM Cooker +18,15 EM HD Test OK in the PROM Cooker +19,15 EM HD Test OK in the PROM Cooker +20,15 EM HD Test OK in the PROM Cooker +19,17 HD PROM was blank, programmed it, check 00030500 +17,18 HD PROM was blank, programmed it, check 00030500 +20,29 EM PROM was blank, programmed it, check 00038480 The CTFE card for eta +17:+20 phi 15 readout all $0000 in TTADCMon and in the HSRO readout to L2,L3. Make a temporary swap of the CTFE cards for phi's 15 and 16. The problem in the readout moves and follows the CTFE card labeled phi 15. The problem was that the -1.000 Volt Ref supply was sitting at about -0.75 Volts. This was caused by a shorted bypass capacitor. Replace the 0.1 uFd with a SMD and the -1.000 V Ref Supply came back to life and the readout from this card looked rational. BLS eta 6 and 8 noise: A good TT to look at is +6,7 HD. This is the TT that Dean and I were looking at last week for crate pickup noise. Note from Denis, I have the Splitter cabled incorrectly - fixed it and take new pictures. Make some Find_DAC runs just over eta -9:-12 phi 1 i.e. the splitter card. Find_DAC_V4_0_B_20030207.tti;6 Splitter is in the path of the BLS signal EM_TT(- 9, 1) Ped_DAC: 3718 Ped_Avg: 8.03 Ped_Dev: 0.77 Dev_Avg: 0.80 EM_TT(-10, 1) Ped_DAC: 3653 Ped_Avg: 7.97 Ped_Dev: 0.54 Dev_Avg: 0.60 EM_TT(-11, 1) Ped_DAC: 3658 Ped_Avg: 8.05 Ped_Dev: 0.86 Dev_Avg: 0.80 EM_TT(-12, 1) Ped_DAC: 3643 Ped_Avg: 8.00 Ped_Dev: 0.42 Dev_Avg: 0.42 HD_TT(- 9, 1) Ped_DAC: 3654 Ped_Avg: 7.99 Ped_Dev: 1.70 Dev_Avg: 1.62 HD_TT(-10, 1) Ped_DAC: 3674 Ped_Avg: 7.96 Ped_Dev: 1.73 Dev_Avg: 1.69 HD_TT(-11, 1) Ped_DAC: 3648 Ped_Avg: 8.00 Ped_Dev: 1.93 Dev_Avg: 1.94 HD_TT(-12, 1) Ped_DAC: 3670 Ped_Avg: 8.02 Ped_Dev: 0.92 Dev_Avg: 0.75 Find_DAC_V4_0_B_20030207.tti;7 Splitter is in the path of the BLS signal EM_TT(- 9, 1) Ped_DAC: 3718 Ped_Avg: 8.02 Ped_Dev: 0.81 Dev_Avg: 0.80 EM_TT(-10, 1) Ped_DAC: 3653 Ped_Avg: 8.00 Ped_Dev: 0.63 Dev_Avg: 0.62 EM_TT(-11, 1) Ped_DAC: 3659 Ped_Avg: 7.96 Ped_Dev: 0.91 Dev_Avg: 0.81 EM_TT(-12, 1) Ped_DAC: 3643 Ped_Avg: 8.00 Ped_Dev: 0.51 Dev_Avg: 0.49 HD_TT(- 9, 1) Ped_DAC: 3654 Ped_Avg: 8.04 Ped_Dev: 1.80 Dev_Avg: 1.62 HD_TT(-10, 1) Ped_DAC: 3674 Ped_Avg: 8.01 Ped_Dev: 1.74 Dev_Avg: 1.67 HD_TT(-11, 1) Ped_DAC: 3648 Ped_Avg: 7.97 Ped_Dev: 1.94 Dev_Avg: 1.90 HD_TT(-12, 1) Ped_DAC: 3670 Ped_Avg: 7.99 Ped_Dev: 0.76 Dev_Avg: 0.75 Find_DAC_V4_0_B_20030207.tti;8 Splitter is not in the path of the BLS signal EM_TT(- 9, 1) Ped_DAC: 3720 Ped_Avg: 8.00 Ped_Dev: 0.97 Dev_Avg: 0.77 EM_TT(-10, 1) Ped_DAC: 3653 Ped_Avg: 8.05 Ped_Dev: 0.55 Dev_Avg: 0.62 EM_TT(-11, 1) Ped_DAC: 3660 Ped_Avg: 8.00 Ped_Dev: 0.99 Dev_Avg: 0.78 EM_TT(-12, 1) Ped_DAC: 3642 Ped_Avg: 8.00 Ped_Dev: 0.18 Dev_Avg: 0.44 HD_TT(- 9, 1) Ped_DAC: 3654 Ped_Avg: 8.02 Ped_Dev: 1.66 Dev_Avg: 1.61 HD_TT(-10, 1) Ped_DAC: 3674 Ped_Avg: 8.00 Ped_Dev: 1.73 Dev_Avg: 1.66 HD_TT(-11, 1) Ped_DAC: 3654 Ped_Avg: 8.00 Ped_Dev: 1.98 Dev_Avg: 1.91 HD_TT(-12, 1) Ped_DAC: 3670 Ped_Avg: 7.98 Ped_Dev: 0.75 Dev_Avg: 0.76 Find_DAC_V4_0_B_20030207.tti;9 Splitter is not in the path of the BLS signal EM_TT(- 9, 1) Ped_DAC: 3720 Ped_Avg: 7.97 Ped_Dev: 0.89 Dev_Avg: 0.78 EM_TT(-10, 1) Ped_DAC: 3653 Ped_Avg: 8.02 Ped_Dev: 0.50 Dev_Avg: 0.61 EM_TT(-11, 1) Ped_DAC: 3659 Ped_Avg: 8.03 Ped_Dev: 0.72 Dev_Avg: 0.78 EM_TT(-12, 1) Ped_DAC: 3642 Ped_Avg: 8.01 Ped_Dev: 0.15 Dev_Avg: 0.47 HD_TT(- 9, 1) Ped_DAC: 3654 Ped_Avg: 8.04 Ped_Dev: 1.64 Dev_Avg: 1.59 HD_TT(-10, 1) Ped_DAC: 3673 Ped_Avg: 8.01 Ped_Dev: 1.69 Dev_Avg: 1.65 HD_TT(-11, 1) Ped_DAC: 3655 Ped_Avg: 7.95 Ped_Dev: 1.89 Dev_Avg: 1.89 HD_TT(-12, 1) Ped_DAC: 3670 Ped_Avg: 7.98 Ped_Dev: 0.76 Dev_Avg: 0.73 New files to use for various testing purposes: \Scratch\Check_L1CT_Setup_version_1.msg # setup Ref Sets and Count Comp's \Scratch\Check_L1CT_Setup_version_2.msg # setup Ref Sets and Count Comp's \Scratch\Set_DAC_MidScale.tti # All Ped DAC's set to 2047 L1 TCC Switch to Trics V10.3-B and make it the Auto-Start version. This version will not extend the default programming reference sets to the |eta| values 19 and 20 unless explicitely told to. This version is using ITC V02-20-00. 6-Feb-03 at night, we find a Trics V10.3-B access violation pop up window on TCC's screen. Which thread has crashed? Trics' dialog box is still alive, COOR commands are still processed, collecting monitoring is still going on, colecting luminosity data is ok, serving monitoring to new connection still ok, however existing trigmon and TTADC_mon running on TCC no longer receive information, and TCC claims it cannot ship luminosity data out (i.e. it sees no luminosity client). Whatever thread crashed must be an ITC connection thread. Philippe copies the code Trics to TCC to give the debugger a chance to pick up the code. We wait for a chance to try and click "debug" in the pop up window but we learn nothing. The debugger doesn't show us source code, maybe because the error is in ITC. Getting out of the debugger leaves us with a fully functional Trics and the Luminosity Server reconnects to it. L2 TCC Switch L2 TCC's PCI Expansion box with the one we were using at MSU in the hope that it will solve or help the problem where the PC freezes when optical cables are unplugged (or reomote crates powered off). Also add two more bit3 PCI cards taken from of d0ntmsu7 to support L2PS and L2CTT. These crates are not yet going to be part of any run. These crates will typically not even run the L2 Admin application. L2CTT was presenting the crate ID for L2CAL. L2TCC will have to learn how to ignore these two crates. Gave Reinhard the PCI expansion box taken out of L2 TCC. The cable and the PC's PCI card are inside the expansion box. Reinhard said he is taking it to the test stand for storage. One can unplug any one crate and the PC doesn't freeze. Unplugging two crates still freezes the PC, and one needs to plug both back in before the PC recovers. L2RS needs to be updated to make the switch stopping collection of monitoring data also stop the collection of Error messages to effectively totally stop IOs to the DPM (as long as COOR doesn't send explicit messages). Trying unplugging several crates while TCC is NOT trying to access them hasn't been tried. >From the back of the expansion box, the slots are numbered P1 to the right, and P7 to the left. It also appears that the automatic numbering of adapters starts with Adapter#0 being the right most occupied PCI slot. We were using P2-P7. The adapter from P7 was moved to P1 to prevent having the first slot being unused. This didn't seem to produce any difference. cf. L1/L2TCC Logbook. Slot# Adapter S/N Adapt# Cable Label L2 Crate ID P1 197687 #0 LTCC000 L2 Global 0x20 P2 197718 #1 LTCC004 L2 Mu Cen 0x21 P3 197686 #2 LTCC005 L2 Mu Fwd 0x22 P4 183989 #3 LTCC001 L2 Cal 0x23 P5 197674 #4 LTCC003 L2 PS 0x24 P6 197717 #5 LTCC002 L2 CTT 0x25 P7 Unused Note: L2RS doesn't care which orders the cables are plugged. It uses the Crate ID at word 0x0000 of the DPM to dynamically detect which crate is which. Reinhard however needs the crates in the same order as the crate IDs to make it easier to access using their other programs (e.g. Drew's MBT control) Install L2RS V3.1.A on L2 TCC. This version executes a pre-init-auxi command file that we use to write the crate ID 0xff into word 0x0000 of the DPM for bit3 adapters #4 &5. This new version recognizes this crate ID as a request to ignore the crate(s). Prescaler and Luminosity calculation Michael Begel has been analyzing lots of zero bias runs. These runs have no andor requirement besides live_crossing. Some runs have a single SpTrg, others several, up to 128. Some runs all use the same prescale ratio for all SpTrgs while others use different ratios. His reports compute a "unity test" where he compares the number of total L1 Accepts for a run to the product of the number of Exposure Group Exposed (i.e. correlated enable) multiplied by the fraction of Decorrelated Enable. These numbers should match if the decorrelated correction is indeed decorelated from the Exposure Group Exposure. All these tests crosscheck well as long as the DAQ system is not heavily saturated and L3 is not disabling the SpTrg more than 90 or 95%. In particular there is no mutual shadowing of SpTrg and this is independent of the prescale ratio. When the DAQ becomes heavily saturated the ratio of L1 Accept over the computed expectation drops from progressively from 1.0 to 0.25 (for the result we saw, maybe lower). We think the reason for this divergence is that the DeCorrelated Enable (mainly the L3 disable) signal is no longer decorrelated from the Exposure Group Exposed (mainly L1 Busy) when the DAQ is pushed so hard that it disables the trigger for most of the time. When L3 stops disabling the L1FW we start having the L1 Accept come in rapid sequence and with it L1 busy and with it the exposure group rate is low. Then the L3 fills up and has to disable the L1FW for a while and the L1 Busy will thus soon stop. There is thus a direct relationship between when the L3 disable is asserted (Decorrelated Enable Fraction is Zero) and when the Exposure Group is exposed. It is thus no longer acceptable to apply the average decorrelated fraction as a correction to the overall Exposure group exposed count. We don't know how to formulate a quantitative effect or even derive the direction of the error. It is even somewhat surprising that this effect only becomes evident at >95% L3 disable. Follow up 10-Feb-2003: Take snapshot and archive all Trics Run files, Log files. Archive new Trics, VME_Access, and L2RS executables. Also update MSUL1A \Trics files (backup). Also update D0Server4 users\laurens\d0tcc1 files (emergency recovery). Scan all Dec-Jan-Feb Logfiles and verify we don't have more CBus errors since last Mid-December power supply problems. There is only one such instance: 21-Dec-2002 01:09:02.405 E$ Post-Write Check Mismatch: Read = 0x10 / Last Written = 0x11 E$_Read Mask = 0xff / Write Mask = 0xff E$_@MBA#209/CA#59/FA# 40 I$ Recover Attempt: Try Writing Register Again I$ Second Register Write Successful The above error was during executing a COOR request, and it caused an acknowledgement error. Trics V10.3 will not call this a fatal error. There is one more weird error in the logfiles. This looks like a FW IO error, but looking more closely, it is in fact a freak collision between SCL_Init and the Luminosity Pool trying to control the L2 Helper at the same time. 09-Jan-2003 00:32:25.251 E$Post-Write Check Mismatch: Read = 0x8008 / Last Written = 0x0008 E$_Read Mask = 0xffff / Write Mask = 0xffff E$_@Master#0/Slave#2/Slot#20/Chip# 4/Reg# 16 I$Recover Attempt: Try Writing Register Again I$Second Register Write Successful ------------------------------------------------------------------------------ DATE: 27:30-JAN-2003 At: Fermi Topics: Work during the shutdown, BLS Trigger Pickoff signal work, CT Power Supply work, Fritz CMC, HD PROM work The process to start the system from power up is the same as in last week's log book entry. Daniel and Amanda start the BLS Trigger Pick off signal repair work based on Joe's list of problem high eta BLS signals based on looking with the scope before the shutdown. Check some Pioneer Magnetics bricks. This finishes variac ramping up bricks at Fermi. 3 phase variac will go back to MSU. #63 2V 250A Was repaired at end of Run I and then stored. Setup for 2.100 V output @ 163 Amp load 0mV / 0 mV ripple. #79 2V 325A for Tier 2-3 Was repaired at end of Run I then stored. Setup for 2.025 V output @ 157 Amp load 25mV / 75 mV ripple. #21 5V 200A I'm not sure which PDM this brick came from It is OK with a 35 Amp load. With a 50 Amp load it starts to make the classic squelling noise and its regulation fails and it has ripple. @ 100 Amp load 20mV / 85 mV ripple @ 145 Amp load 30mV / 120 mV ripple and its DC output is dowm 50 mV. @ 145 Amp load it is making the classic noise and its output has two components of noise: a slow undulation of aprox sine wave and a faster chopping noise. The fast chopping noise has a period of 19.4 usec i.e. 50 kHz The slow sine wave noise has a period of 154 usec i.e. 6.5 kHz #26 5V 200A This brick is from PDM-09 see 17:20-DEC 2002 log book Noise starts at about 50 Amp load. @ 50 Amp load 2 mV / 15 mV DC is OK @ 100 Amp load 7 mV / 40 mV DC is 10 mV dowm @ 150 Amp load 10 mV / 60 mV DC is 25 mV down #47 5V 200A This was a +5V brick in M114 late in Run I. @ 145 Amp load it is OK with 1 mV of ripple @ 155 Amp load it starts making the classic noise @ 177 Amp load it has lots of noise and 20 mV of ripple. Investigate PDM-13 This is the Power Pan from high eta that tripped a breaker when we first tried to run it last week. This week try running it up in the high bay with the cooker box. PDM-13 (as found) +5.0 V brick MSU SN# 22 from 7-FEB-92 does not run at all -2.0 V brick MSU SN# 70 from 7-JAN-92 2.108 Volts no load @ 162 A load 2.104 V output 0 mV of ripple -5.2 V brick MSU SN# 71 from 7-JAN-92 5.237 Volts no load starts making noise at 50 Amp load @ 136 A load 5.163 V output 40 mV / 130 mV ripple -4.5 V brick MSU SN# 72 from 7-JAN-92 4.613 Volts no load starts making noise at 55 Amp load @ 118 A load 4.505 V output 120 mV / 250 mV ripple At some point during the day on Tuesday the Cal Trig turned itself off or some one turned it off. This happened at about perhaps 11 AM The RPSS did not indicate any reason why it was off. Fritz is working to make code to load the Master Clock from COMICS. So far he has loaded all the status control registers that way. He has not tried the Time Lines yet. I talked with him about how important it is that the clock not get poked at and how it does not need poking at and how stable it has been without people poking at it. Work of TT problems that have been reported by Bob's Examine from Run 169,172 in the eta 1:12 range. +11,18 Problems: EM has about 143% more gain than it should. HD has about 119% more gain than it should. Check the +11,18 Term-Attn-Brd EM sym 800 mV HD sym 940 mV for reference check +11,17 EM sym 790 mV HD sym 940 mV for reference check +11,? EM sym 800 mV HD sym 990 mV So the gains look OK. Pulled the BLS cards for +11,18 and for +11,19 and compare them in the BLS Trigger Pickoff test stand. They look the same. So we do not think that the problem is the analog gain in either the BLS or in the Term-Attn-Brd. -9,22 Problem: HD looks completely dead. Check the -9,22 Term-Attn-Brd EM sym 1.12 V HD sym 1.34 V for reference check -9,17 it looks exactly the same. Shorts check, HD- is shorted at the MCH end. Gain was zero but could not see a HD+ short. --> Turned the repair of the shorted cable over to Shoua Moua. -8,18 Problem: ton of noise on the HD Check the -8,18 Term-Attn-Brd EM not sym 568 mV HD sym 940 mV for reference check -8,17 EM not sym 560 mV HD sym 940 mV Look at raw BLS cable on the scope. See 50 mV pp on both HD+ and HD-, i.e. about 3 or 4 GeV Et. This noise looks like a constant oscillation with a period of 100 nsec (not 132 nsec). -8,25 Problem: EM gain is only 71% of the expected value. Check the -8,25 Term-Attn-Brd EM sym 388 mV HD sym 932 mV for reference check -8,10 EM sym 536 mV HD sym 956 mV for reference check -8,8 EM sym 544 mV HD sym 960 mV So -8,25 EM looks like it has a good R80 but that the hand pulser only gives 388mV instead of the expected 538 mV i.e. only 72% of the expected response. In the gains file -8,25 EM is setup as a bad R80, which turns its gain down 69% of the gain for correct R80's. But 72% x 69% = 50% and this channel responds more than 50% so what is going on. -7,22 Problem: HD gain is only 77% of the expected value. Check the -7,22 Term-Attn-Brd EM sym 424 mV HD sym 752 mV for reference check -7,17 EM sym 428 mV HD sym 756 mV for reference check -7,18 EM sym 424 mV HD sym 752 mV So the gain of -7,22 HD (and EM) looks OK. +8,5 Problems: HD noise, EM gain 31%, HD gain 125% Check the +8,5 Term-Attn-Brd EM not sym 580 mV HD sym 912 mV for reference check -8,17 EM not sym 560 mV HD sym 940 mV So the gains look OK Look at raw BLS cable on the scope. See 40 mV pp on both HD+ and HD-. This noise looks like a constant oscillation with a period of 100 nsec (not 132 nsec). +7,5 Problems: EM gain 152%, HD gain 158% Check the +7,5 Term-Attn-Brd EM sym 440 mV HD sym 772 mV for reference check -7,17 EM sym 428 mV HD sym 756 mV for reference check -7,18 EM sym 424 mV HD sym 752 mV So the gains look OK. +6,5 Problems HD noise, Look at raw BLS cable on the scope. See 50 mV pp on both HD+ and HD-. This noise looks like a constant oscillation with a period of 100 nsec (not 132 nsec). +5,9 Problem HD gain is 153 % Check the +5,9 Term-Attn-Brd EM sym 828 mV HD sym 1.46 V for reference check +5,1 EM sym 828 mV HD sym 1.45 V So the gains look OK. +6,10 Problem HD gain is 51 % Check the +6,10 Term-Attn-Brd EM sym 716 mV HD sym 1.28 V for reference check +6,9 EM sym 724 mV HD sym 1.28 V So the gains look OK. HD- looks like and open cable or a missing Driver output cap. --> Problem was a bent pin at the BLS end. Because no one had been a hero and tried to force the connector in, the pin was not crimped over. It straightened OK and the cable now looks OK to the Ohm meter. +5,19 Problem HD gain is 153 % Check the +5,19 Term-Attn-Brd EM sym 816 mV HD sym 1.48 V for reference check +5,1 EM sym 828 mV HD sym 1.45 V So the gains look OK. +8,28 Problems: EM gain 140%, HD gain 133% Check the +8,28 Term-Attn-Brd EM not sym 568 mV HD sym 944 mV for reference check +8,27 EM not sym 560 mV HD sym 956 mV So the gains look OK. Pulled the BLS cards for +8,28 and for +8,27 and compare them in the BLS Trigger Pickoff test stand. They look the same. +8,28 had been Excluded because of EM noise. We do not see any EM noise. -1,25 Problem: EM gain is 65 % -2,25 Problem: EM gain is 75 % -3,25 Problem: EM gain is 74 % -1,26 Problem: EM gain is 69 % -2,26 Problem: EM gain is 73 % -1,1 -2,1 -3,1 -------- -------- -------- EM 1.19 V 1.12 V 1.06 V HD 2.36 2.20 2.11 -1,18 -2,18 -3,18 -------- -------- -------- EM 1.14 V 1.11 V 1.05 V HD 2.27 2.18 2.09 -1,25 -2,25 -3,25 -------- -------- -------- EM 1.17 V 1.13 V 1.07 V HD 2.35 2.23 2.10 -1,26 -2,26 -3,26 -------- -------- -------- EM 1.17 V 1.08 V 1.06 V HD 2.31 2.22 2.07 Phi 1 and phi 18 were used as reference. So the gains look OK. -2,19 Problem HD gain is 28 % Check the +2,19 Term-Attn-Brd EM sym 1.12 V HD sym 2.24 V for reference see above. Pulled the BLS cards for -2,19 and for -2,18 and compare them in the BLS Trigger Pickoff test stand. They look the same. So we do not think that the problem is the analog gain in either the BLS or in the Term-Attn-Brd. +2,12 Problem EM gain is 71 % Check the +2,12 Term-Attn-Brd EM+ 224 mV EM- 560 mV HD sym 2.19 V for reference see above. So the EM+ input is sick. The EM- input looks OK, i.e. if you had two of these everything would be OK. The problem was a junk 845 Ohm resistor in the EM+ input circuit. --> Replace it and now Check the +2,12 Term-Attn-Brd EM sym 1.10 V HD sym 2.16 V Work with Dean investigating the "noise pickup" or "low level oscillation" that we see in some channels typically at eta 6 or eta 8. The noise pickup is the 100 nsec stuff and looks like it is related to the running of the 132 nsec clock in the crate. We can see it with PreAmps off. The 10 nsec stuff looks like it is from the OpAmp in the summers and is only a problem at the eta's with very few inputs. We have had the BLS's for: +8,28 -6,28 -6,22 +5,23 +9,21 all out and plugged into the test stand looking for noise. These are all TT's that had to be Excluded in the last month or so of the run that ended on 13-JAN-2003 because of high EM noise. We could not see any noise in the test stand (other than the normal test stand noise). Checked the pins on the summers and the drives and found no problem so far. Trigger Tower Pick Off Signal Repair Work at "High Eta" 1. Dead ------------------------------------------------------------------ TT Channel JOB Status ------------------------------------------------------------------ +19, 30 HD - HD driver replaced OK +19, 31 HD - HD driver replaced OK +14, 2 HD+EM - No problem found - +13, 6 HD - No problem found - -15, 7 HD - HD driver replaced, OK summer replaced -20, 9 EM - EM driver replaced OK -20, 15 EM - EM driver replaced OK +16, 12 HD - HD driver replaced OK +16, 17 HD - No problem found* - +13, 23 HD - HD driver replaced OK -14, 12 HD - HD driver replaced OK -13, 18 EM - No problem found - -16, 25 EM - EM driver replaced OK -15, 25 EM - EM driver replaced OK summer replaced -13, 28 HD+EM - Missing HD & EM drivers OK installed *for high eta (>14), the gain on the o-scope may have to be increased. 2. Noisy ----------------------------------------------------------------------- TT Channel JOB Status ----------------------------------------------------------------------- -16, 4 HD - Replaced summer OK +15, 27 HD - Replaced summer OK +16, 32 HD - Replaced summer OK -19. 11 HD - Replaced summer OK +13, 7 HD,Tower 0* - Replaced summer OK *first list had wierd signal on EM, checked OK. 3. Cable problem (?) - BLS Checked Previously ----------------------------------------------------------------------- TT Channel ----------------------------------------------------------------------- +17, 24 HD - HD driver replaced OK +20, 30 EM - No problem found - +13, 2 HD+EM - Wrong cable plugged in, OK Right cable found +16, 8 EM - Signal OK, noise from - o-scope that is seen only at high gain. (see Dead*) Based on what the high eta readout data looks like, Bob has guessed that the following PROM's are bad, i.e. not programmed: -19,1 HD -17,2 HD -18,2 HD -17,4 HD -19,4 HD. Fire up the Data I/O Unisite, yes they are all blank parts. Program them as HD parts. The checksum is 00030500. We are running with the French Splitter in the system and powered up. ------------------------------------------------------------------------------ DATE: 21:24-JAN-2003 At: Fermi Topics: Work during the shutdown System Restart We have built up a long list of things that we need to do by hand to re-start the system. After Master Clock is running, turn on TFW and Cal Trig then Recall that M101 is now on the TFW safety power control system. Configure FPGA's The normal Master Command file for this does everything except for the M101 Routing Master Get the Routing Master Configured and Running 1) After the Routing Master is powered up, wait for its SBC to boot (less than 1 minute) 2) Login to the SBC with 'ssh d0runsu@d0sbc001b' The password is same as the online d0run account. 3) On the SBC run 'reset_all.sh stop' which will stop the relevant readout processes. 4) Use TCC's Configure menue item to execute the dcf that configures the Routing Master FPGA's 5) On the SBC run 'reset_all.sh start' Init TFW and CT Tell TRICS to completely ignore the CT By hand execute the cio file to wake up the eta 13:20 part of CT By hand using VME Access load the CT Gains for eta 13:20 By hand using VME Access load the CT Pedestals for eta 13:20 By hand using VME Access load the CT Gains for eta 1:12 By hand using VME Access load the CT Pedestals for eta 1:12 Tell TRICS that it has control of the CT again and set the eta coverage to the whole ball of wax. Init the TFW and CT verify that it is a clean init of all parts The file to configure the Routing Master FPGA's is D0_Config\M101_Routing_Master_All.dcf The file to wake up the eta 13:20 part of the Cal Trig is D0_Config\Init_L1CT_HighEta_13_20_all_phi.cio Note that this is not necessary any more if you have the TRICS Cal Trig eta coverage set to all eta's. The current Gain and Ped files for eta 13:20 are D0_Config/Gains_13_20_1_32_rev_b.tti D0_Log/Find_DAC_V3_1_h_20030123_Edited.tti;1 The current Gain and Ped files for eta 1:12 are D0_Config/Run_II_l1CT_Bob_8_Gain.tti;1 D0_Log/Find_DAC_V3_1_h_20030108_Edited.tti;1 Clean up some files on TCC to get ready for Wednesday morning's power outage and restart. In D0_Config rename m122_bottom.dcf to M122_Bottom.dcf rename m123_bottom.dcf to M123_Bottom.dcf rename m123_middle.dcf to M123_Middle.dcf ! Now all the dcf filenames have the same capitalization and it ! matches the way the dcf files are called from the higher up files. In D0_Config rename m101_all.dcf to M101_L1CT_All.dcf ! This dcf file was not all the FPGA's in M101 it was just the ! Cal Trig readout in M101 middle. The Routing Master is going ! to be kept separately so start explicitly saying that this file ! is just M101 L1 Cal Trig. In D0_Config Edit Configure_FPGAs.dcf ! Change the last line (of the 3 lines in Configure_FPGAs.dcf) ! from calling %CONFIG%\M101_All.dcf ! to calling %CONFIG%\M101_L1CT_All.dcf Copy \Scratch\Dave_Configure_All.dcf to \D0_Config\M101_Routing_Master_All.dcf ! Give the dcf file for the Routing Master a rational name and put ! it in D0_Config instead of hidding off in scratch. We will still ! call this file by hand. The normal Configure All FPGA's process ! will not call this file or try to configure the Routing Master in ! M101 Bottom. Copy tdm_29_1.exo from DESMO to the appropriate exo file on TCC. In \TRICS\Dcf\ Edit tdm_dci ! Edit tdm.dci so that it now calls tdm_29_1.exo instead of _24_1. ! _24_1 is the standard (pre any random prescaler work) version of ! tdm that has been in use for some years. We are now running TRICS 10.3A and using TDM version _29_1. Move M101 to the TFW power control safety system. It has been setup for now in the following way: In the TFW Contactor Box the 4th and until now unused channel was setup so its contactor will close as soon as the Global Permit is active. Just one phase of AC input power for this new contactor was stolen from an adjacent channel. It goes through a 5 Amp fuse before reaching this new contactor. So on the Hubbell outlet for this new channel we have: Safety Ground, Neutral, on one phase 5 Amps of power that come on as soon as Global Permit is asserted, and two phases that are open circuit. Into this Hubbell outlet for this new channel plug the primary of a control transformer. This transformer is on top of the M125 rack and is bolted to it. The 24 Volt secondary of this new control transformer runs over to the Contactor Box for rack M101 and supplies 24 Volts to the coil of M101 Contactor Box. So as soon as TFW Global Permit is made up, the M101 power contactor will close and power comes down into M101 and its blower will start. The Drip Detector and the Smoke Detector in M101 have been put on the TFW safety system RMI along with these same sensors from M122, M123 and M124. Thus if either the M101 Drip Detector or the M101 Smoke Detector goes off, the Global Permit to the TFW safety system will go away and the power to M101 (and the other 3 racks) will drop. Recall the wiring to the Smoke detectors. Red 1a+ Black and White 5 Green 6 Recall the wiring to the Drip Detectors. ------------------------------------------------------- | Blk | Red | x | Grn | x | Grn | x | Red | Blk | ------------------------------------------------------- To terminate the string of Drip Detectors you jump Green to Red. The black PCI expansion box that I brought here this week does have a copper PCI cable inside it. Connection of the Optical Splitters in the Cal Trig Readout Optical Splitter in Slot 7 (black tags in back) Splitter Input from Cal L3 Readout Uses Channel Trig Rack - Eta VTM Slot Channel -------- --------------- ---------------- 1 M104 -1:-4 Slot 10 Ch 0 2 M103 +1:+4 Slot 10 Ch 1 3 M106 -5:-8 Slot 9 Ch 3 4 M105 +5:+8 Slot 10 Ch 2 5 M108 -9:-12 Slot 9 Ch 2 Optical Splitter in Slot 6 (white tags in back) Splitter Input from Cal L3 Readout Uses Channel Trig Rack - Eta VTM Slot Channel -------- --------------- ---------------- 1 M107 +9:+12 Slot 10 Ch 3 2 M110 -13:-16 Slot 9 Ch 1 3 M109 +13:+16 Slot 11 Ch 0 4 M112 -17:-20 Slot 9 Ch 0 5 M111 +17:+20 Slot 11 Ch 1 Looking at the same information from another angle Splitter Input from Cal L3 Readout Uses Slot Channel Trig Rack - Eta VTM Slot Channel ------------- --------------- ---------------- Slot 6 Ch 4 M112 -17:-20 Slot 9 Ch 0 Slot 6 Ch 2 M110 -13:-16 Slot 9 Ch 1 Slot 7 Ch 5 M108 -9:-12 Slot 9 Ch 2 Slot 7 Ch 3 M106 -5:-8 Slot 9 Ch 3 Slot 7 Ch 1 M104 -1:-4 Slot 10 Ch 0 Slot 7 Ch 2 M103 +1:+4 Slot 10 Ch 1 Slot 7 Ch 4 M105 +5:+8 Slot 10 Ch 2 Slot 6 Ch 1 M107 +9:+12 Slot 10 Ch 3 Slot 6 Ch 3 M109 +13:+16 Slot 11 Ch 0 Slot 6 Ch 5 M111 +17:+20 Slot 11 Ch 1 Optical cables for all 10 eta's are now run over to L2 Cal Preprocessor. Issues during the work to get the new high eta cards all running. The CTFE for eta -13:-16 phi 21 was always reading back $FF. I looked at it, verified the CA dip switch (which may have been wrong) and then this card worked OK. The section of the Term-Attn-Brd that runs -16,31 had no control of its EM or HD pedestals. It looks like this Term-Attn-Brd has been worked on in this area already. On TT_ADC_Mon -14,12 EM reads 124. Either this is a very bad pedestal or something is wrong on this card. In the Find_DAC run over eta 13:20 all phi's on Thursday there were a couple of TT's that Find_DAC failed to find a value for (plus some number of mistakes because to the tons of noise). The outright failures were due to a bad Term-Attn-Brd which I replaced and then get good ped's for. Looking now at TT_ADC_Mon it looks like +17,+18,+19,+20 phi 4 EM and HD all have ped's of about 12. Something moved. Looking now at TT_ADC_Mon it looks like +16,18 EM and HD have ped's of 255. Something fell apart. Thursday evening at about 10 PM the breaker on the roof of the MCH that feeds the transformer for 1st floor MCH poped. All of Cal Trig had been running for some hours when this happened. None of the breakers in the panels on the 1st floor poped - just the transformer primary breaker poped. None of the panels on the first floor were warm but the panel on the roof was warm. The transformer for the first floor is the only one that makes much heat anymore. Tim Martin checked transformer primary currents and they are 125 to 135 Amps and everything looks OK. More work on Friday to look at SMT noise while the South EC was closing. For this we watched +12,9 HD Called Denis Calvet. Talked about splitter and blue cables. His preference to make a clean ADF crate is to use transition cables. This is rational and has several advantages. It may be cost neutral because it saves making the special fancy transition backplane. Installed the splitter card this week. It is in -9:-12 phi 1. I'm running from its own private +5V supply. Promised to get noise width and match to Cal presision measurements to Denis. The fuse in the Pioneer Magnetics power supplies is a BUSS BAF-15. Official Work List for this shutdown: Power Supplies in Cal Trig Repair and then install the M112 Lower Tier 1 power pan. Swap Tier 1 power pans in M103 Lower and M106 Upper Tier 1 and then try to fix the power pans that are removed. Verify that brick repair is getting underway. Master Clock Move the marker that is used to align with PD. Move CMC wrt accelerator BX back to the summer position ? Verify before shutdown and after shutdown numbers with Steve and check with Dmitri. FPD Work with them on their And-Or Term signals. SMT noise in the Cal Help with the measurement of Cal pickup of SMT readout noise on Sunday and Wednesday Voltage Monitoring Work with Stu and Geoff on getting power supply voltage Shea box monitoring working for the TFW and the L1 Cal Trig. This got started today. PreScaler Move to using the new 2nd version of random number prescaler. Simulation work is all done except for writing up a report. Philippe has done a major nice job on this so we now know what we are doing. FPGA work is almost done. We know that it will fit and make timing constraints. TRICS work to setup the random number seeds at TFW Init. De-Mux Card for Answer from L2 Global Two weeks or so before the holidays, Drew did the FPGA work on the DeMux card that sends the L2 Global Answers to the TFW so that it now sends all 128 bits with timing that the TFW can ingest. But I have never tested and tried running with this card and its new FPGA code. If it works then we have the full 128 L1 triggers instead of the 128 - 16 = 112 currently usable L1 triggers. If all is OK then reprogram the firmware in the current DeMux card. This absolutely should get tested during this shutdown. M101 Power Control and Safety System M101 Bottom Crate currently contains the Routing Master. But M101 is on the L1 Cal Trig power control safety system. Thus right now we can not tell the L1 Cal Trig power control safety system to turn off the Cal Trig or we also kill the routing master and thus stop all DAQ activity. Need to move M101 to the TFW power control safety system. Two possible ways to do this and I do not like either one. BLS Signal Splitter We now have a 4 TT BLS signal splitter card from Saclay. I'm to get it installed and verify that it is OK and does not cause trouble for generating correct triggers with the L1 Cal Trig. Run IIB work. Quadrant Terms Must get the detailed specification written for the CHTCR Mezzanine card for generating Quadrant Terms and schedule a meeting with Vince. For now, yes, make 19, 20 into a Region. Must make a full review of how all of this works and make sure that it all holds together as a whole. TCC Work TCC2 PCI Expansion Box work on it or replace it. TCC1 operating system - move to approved version. New TCC2 for support of ? New TCC1 code for support of random number prescaler and for Cal Trig control upgrades, loading DAC's VME_Access vs TRICS and new version of Find_DAC. Clean up the Configuration and deadstart procedure, issues with RM and Cal Trig. Spare standby TCC. Swap Term-Attn-Brd's in eta -12:+12 As the number of Term-Attn-Brds permits, begin the process of swapping Term-Attn-Brd's in eta -12:+12 so that we have only correct R80's and tested Term-Attn-Brd's installed. Routing Master Verify a procedure for dead starting the RM with the issue of who controls the bus to begin with and the issue of no routing messages until the FPGA's are configured. Get this procedure written down. Worked on this this afternoon with Gustaaf and we think that we know what to do. PROM work There is one PROM check issue left in eta -12:+12 investigate this and repair the problem. There is a suspect PROM problem in high eta repair this. Make PROM scans over the whole eta phi. Cables Sort through the salvaged flat cables and get rid of all the stuff that is not needed and get the spares that will be saved boxed up and labeled. Swap 19 EM/HD Bob wants eta 19 EM and HD swapped. Connectors and crimp on pins have been ordered. Test and verify that this is all going to work OK. Specify the cable. Arrange to have the cables made. Feed high eta TT data to L2 Cal PreProcessor Install the fiber optic stuff to feed the data from the last 4 racks of L1 Cal Trig to the L2 Cal PreProcessor. I need my fiber optic splitters back ! VRBC Firmware If stable known OK VRBC firmware becomes available that includes the required SCL Receiver re-syncing management then move TFW and L1 Cal Trig to using this firmware. Will this require and update of all the VRB firmware ? Finish high eta installation. Install the last 96 CTFE cards, i.e. phi's 9:32 in each of the last 4 racks. Finish the last of the timing and control cabling. Enable the last 4 racks to participate in generating L1 triggers. Test and make sure that everything in the last 4 racks is working. BLS Trigger Pickoff Work Fix the 20-30 known BLS trigger pickoff signal problems in eta -12:+12 Fix the known BLS trigger pickoff signal problems in the new high eta. Find and fix all the new BLS trigger pickoff problems that will be made as all the BLS cards are all pulled out, modified, and put back in during this shutdown. Find_DAC Runs Once all the high eta Cal Trig cards are installed make Find-DAC runs over the whole eta phi space. Use the information about the noise in each TT channel to help guide the BLS trigger pickoff repair. Verify that nothing is drifting. Verify that the new fast version and the current version of Find_DAC produce the same results. ------------------------------------------------------------------------------ DATE: 20-JAN-2003 At: MSU Topics: power supply brick factory repairs The following Pioneer Magnetics bricks are going back for repair: Pioneer Original Magnetics Purchase MSU Model Type Serial Number Date Serial Num --------- ------------- ------------- -------- ---------- 2926A-2 5D200-0-4-6-S 398966 MAR-1991 #27 2926A-2 2D250-0-4-S 398988 MAR-1991 #25 2926A-2 5D200-0-4-6-S 416254 JAN-1992 #60 2926A-2 5D200-0-4-6-S 416262 JAN-1992 #59 2926A-2 5D200-0-4-6-S 416268 JAN-1992 #97 2926A-2 5D200-0-4-6-S 416270 JAN-1992 #98 The following PowerTec ASTEC power supplies are going back for repair: ASTEC Original Powertec Purchase MSU Model Number Serial Num Date Problem Serial Num ------------ ---------- -------- ------------ ---------- 9R5-600-381 1723 FEB-1992 No Output #19 9R5-600-381 1725 FEB-1992 No Output #25 These 8 bricks were packaged and ready to ship on 13-JAN-2003 ------------------------------------------------------------------------------ DATE: 13:17-JAN-2003 At: Fermi Topics: January shutdown, Bent Tier 1 backplane pins, Cooked fan motor, CMC to SCL Hub phase, Starting Routing Master, Spare CTFE cards, TFW Redout Amber LED, water leak, Fan Motor, SMT Cal noise, Power Pan work, CBus layout Found rack M110 front door not latched closed. The cabinet keys had been moved (and no one told me) or else some one never put them back in there normal location. Bent L1 Cal Trig backplane pins. In rack M109 in the slot for the CTFE card for eta +13:+16 phi 16 there were a bunch of bent pins on the 3rd connector from the left. In the top row of pins pins 7,8,9,12,16 were bent and in the middle row of pins pins 13,14,15 were bent. Some pins were bent down and some were bent up. They were typically bent so that the tip of the pin was moved by 0.1" but non of them were folded hard and creased. They all bent back into position without feeling like they were weakened. I had a spare circuit board connector so that I could verify that they all plugged in correctly before tryig to put in the CTFE card. I think they are all back in good shape. I think that the signals used by these pins are CTFE Ch 3 Comparators, Ch 4 Comparators, and Ch 3 Total Et outputs. The forward fan motor in rack M124 appears to have burned up. I think that the may have happened when we tried to run the system Monday late afternoon and early evening. I don't think that it is related but this is a time when we had no chilled water and water was leaking down from MCH2. I think that perhaps the motor did not start so it sat there and cooked itself. The armature is now hard to turn and some of the windings inside looked cooked and some look OK. I think that the easiest way to work on this is to pull out the whole TFW readout VIPS crate. CMC - SCL Hub. When we first started the system Tuesday evening (now with chilled water) we had a Red LED on on a couple of the right hand (viewed from the front) SCL FanOut modules. These went out after the fans ran for a while and things got to there normal oerating temperature. But I did try changing the cables that control the 7 MHz vs 53 MHz phase. We had been running with a 1/2 nsec cable in the 7 MHz line. While things were bad, lengthing this to a 1 nsec cable made things worse, i.e. made more Red LED's come on. Once things cooled down everything was fine using the standard 1/2 nsec cable that has been in the circuit for the past "N" months. So I think that: lengthing the 7 MHz line moves things in the direction of turning on Red LED's, that we are operating in just about the right place, and that being hot moves things in the direction of turning on Red LED's. I do not know whose timing changes when it gets hot, CMC or SCL Hub. Starting Routing Master. So far it appears two ways to do this. Crash the SBC code (which appears to happen if there is no configuration in the FPGA's) or be ready with FPGA configure right at the time you power up the RM crate and configure before the SBC boots and starts its application. Neither of these are supper nice so still need to work on this. Spare CTFE cards. 15 Spare CTFE cards were modified by SiDet for Run II. One of these is a Rev A CTFE. Another one is the one that they practiced on and had removed many of the parts (e.g. backplane connectors and sockets and such). This leaves 13 reasonable spare CTFE cards that have been modified for Run II. They are Serial Numbers: 62, 85, 115, 174, 179, 202, 212, 237, 239, 261, 301, 318, 356. I think that 8 of these are from the MSU Test Stand and the other 5 are from ??. At startup on Tuesday evening we had trouble getting the TFW SCL Receiver's amber LED to come on. VRBC reset did not do the job. Finally after the 2nd power cycle (both with good 10+ second off times) and then a VRBC reset it came on. Routing of the CBus M103:M106 Top is a CBus and Bottom is a separate CBus TCC - M106 - M105 - M104 - M103 - Terminator M107:M110 Top is a CBus and Bottom is a separate CBus TCC - M110 - M109 - M108 - M107 - Terminator M111 and M112 have a single CBus servicing top and bottom TCC - M111 Top - M112 Top - M112 Bottom - M111 Bottom - Terminator M110 Tier 1 Bottom Power Pan blows up when it is started. So pull out PDM-13 and install PDM-12. PDM-12 was run and tested on 22-AUG-2002. I don't yet know what brick blew up in PDM-13. While doing this notice that there is a small slow water leak from the distribution manifold under this supply. It is the normal problem and leak point on these manifolds. I put on hose clamps. These are 1 1/16" to 2" clamps Breeze Aero Seal 200 24H. Two clamps on each end and it does the job. SMT noise pick up in Cal Trig Towers test. This was done while the South EC was moved away from the CC. We watched the following TT's +4,7 +4,8 +4,25 +11,22 +11,26 all HD. Next day did it watching the North EC and CC and watching the once per turn stuff soon after BOT. Basically the readout induced noise did not go away when things were opened up the first cm or two. But then as it was opened more the readout noise did fall. By the time it was full open the EC noise was donw by a factor of 4 or 5 and the CC readout noise was down by 2 or 3. This was all in South. Replaced the front fan in rack M124. This is not too bad of a job. The trick is to hot glue the nuts for the mounting screws to the motor fan assembly so that you do not have to reach behind the motor fan when attaching it to the chassis. About one hour to get the old motor fan out and about 2 hours to get the new assembly in and things put back together. While I had the cards out of the VIPA crate I noticed a completely flattened over bent pin on its J3. It is in slot 14 (i.e. the vrbc slot) in row E and is the 12th pin up from the bottom +-1. It does not look like it is touching any other pins. Checked with Daniel and he says that the VRBC only uses the center row of pins on J3 and that row E is a no connect and that row D in the vicinity of the 12th up from the bottom has come GND's. Trigger Framework runs at 1.9% L1_Busy when the L1_Acpt rate is 2300 Hz. Bring more Panduit PV14-6R-M and the same for number 8 screw. The box with the one unused work light says "MAG-125". ------------------------------------------------------------------------------ DATE: 8-JAN-2003 At: MSU Topics: Runs of Find_DAC Over the past 2 days we started to use new Gain files and we ran Find_DAC first over the high eta coverage and then today over eta -12:+12. The following is the set of special startup instructions for the current system. System Restart We have built up a long list of things that we need to do by hand to re-start the system. After Master Clock is running, turn on TFW and Cal Trig then Configure FPGA's By hand configure the RM FPGA's Init TFW and CT Tell TRICS to completely ignore the CT By hand execute the cio file to wake up the eta 13:20 part of CT By hand using VME Access load the CT Gains for eta 13:20 By hand using VME Access load the CT Pedestals for eta 13:20 By hand using VME Access load the CT Gains for eta 1:12 By hand using VME Access load the CT Pedestals for eta 1:12 Tell TRICS that it has control of the CT again Init the TFW and CT verify that it is a clean init of all parts Then either by hand tell the TFW not to bypass L2 Global or tell COOr to issue a "initl1fw" which will tell TFW not to bypass L2. The file to configure the RM FPGA's is Scratch\Dave_Configure_All.dcf The file to wake up the eta 13:20 part of the Cal Trig is D0_Config\Init_L1CT_HighEta.cio The current Gain and Ped files for eta 13:20 are D0_Config/Gains_13_20_1_8_rev_b.tti D0_Log/Find_DAC_V3_1_h_20030107_Edited.tti;2 The current Gain and Ped files for eta 1:12 are D0_Config/Run_II_l1CT_Bob_8_Gain.tti;1 D0_Log/Find_DAC_V3_1_h_20030108_Edited.tti;1 ------------------------------------------------------------------------------ DATE: 6-JAN-2003 At: Fermi Topics: Daniel's study of high eta timing These are the numbers from Daniel Mendoza's very nice study of the current high eta BLS signal timing. Delta T -> time from the falling edge of the ADC clock to the peak of the TP signal + -> the peak occurs later - -> the peak occurs earlier H1 -> Amplitude of the peak signal with respect to the base line H2 -> Difference between the amplitude of the peak of the TP signal and the amplitude at the falling edge of the ADC clock TT Delta T (ns) H1 (mv) H2(mv) --------- ----------- --------- --------- EM +13,4 8 + 460 0 * 10 + 448 0 * 12 + 428 0 * 10 + 404 0 * 0 416 0 * 0 520 0 * 24 + 692 0 * 18 + 640 0 * 0 388 0 * 20 + 476 0 * (*) Widht of the trace 28 mV @ full bandwidth of the oscilloscope TT Delta T (ns) H1 (mv) H2(mv) --------- ----------- --------- --------- HA +13,4 0 368 0 * 0 360 0 * 0 264 0 * 0 356 0 * 0 524 0 * 20 - 356 0 * 20 - 356 0 * 20 - 388 0 * 16 - 524 0 * 0 464 0 * TT Delta T (ns) H1 (mv) H2(mv) --------- ----------- --------- --------- EM +14,4 10 + 460 0 * 16 + 448 0 * 10 + 428 0 * 10 + 404 0 * 0 416 0 * 0 520 0 * 24 + 692 0 * 18 + 640 0 * 0 388 0 * 20 + 476 0 * TT Delta T (ns) H1 (mv) H2(mv) --------- ----------- --------- --------- HA +14,4 20 - 324 0 * 14 - 432 0 * 18 - 264 0 * 18 - 336 0 * 14 - 356 0 * TT Delta T (ns) H1 (mv) H2(mv) --------- ----------- --------- --------- EM +15,4 10 - 552 0 * 18 - 408 0 * 22 - 520 0 * 16 - 456 0 * 18 - 400 0 * 20 - 352 0 * 6 - 356 0 * 20 - 408 0 * 20 - 368 0 * 16 - 520 0 * TT Delta T (ns) H1 (mv) H2(mv) --------- ----------- --------- --------- HA +15,4 40 - 296 0 * 50 - 220 0 * 0 - 272 0 * 0 - 308 0 * 56 - 328 0 * TT Delta T (ns) H1 (mv) H2(mv) --------- ----------- --------- --------- EM +16,4 12 - 400 0 * 10 - 464 0 * 10 - 464 0 * 0 348 0 * 0 476 0 * 0 364 0 * 0 336 0 * 0 310 0 * 0 356 0 * 0 332 0 * Oscilloscope had the trigger threshold set to 300 mV TT Delta T (ns) H1 (mv) H2(mv) --------- ----------- --------- --------- HA +16,4 46 - 288 0 * 60 - 220 0 * 56 - 248 0 * 58 - 212 0 * 68 - 224 0 * Oscilloscope had the trigger threshold set to 200 mV TT Delta T (ns) H1 (mv) H2(mv) --------- ----------- --------- --------- EM -13,4 0 476 0 * 10 - 464 0 * 8 - 484 0 * 0 440 0 * 0 464 0 * 0 704 0 * 0 680 0 * 0 636 0 * 10 + 820 0 * 8 - 460 0 * Oscilloscope had the trigger threshold set to 400 mV TT Delta T (ns) H1 (mv) H2(mv) --------- ----------- --------- --------- HA -13,4 8 - 392 0 * 0 352 0 * 10 - 356 0 * 20 - 352 0 * 14 - 324 0 * Oscilloscope had the trigger threshold set to 300 mV TT Delta T (ns) H1 (mv) H2(mv) --------- ----------- --------- --------- EM -14,4 14 + 632 0 * 0 564 0 * 0 676 0 * 8 + 392 0 * 14 + 716 0 * 12 + 440 0 * 0 354 0 * 0 376 0 * 10 + 448 0 * 10 + 412 0 * Oscilloscope had the trigger threshold set to 350 mV TT Delta T (ns) H1 (mv) H2(mv) --------- ----------- --------- --------- HA -14,4 20 - 332 0 * 26 - 396 0 * 16 - 400 0 * 0 356 0 * 24 - 360 0 * Oscilloscope had the trigger threshold set to 300 mV TT Delta T (ns) H1 (mv) H2(mv) --------- ----------- --------- --------- EM -15,4 0 376 0 * 0 460 0 * 10 - 360 0 * 0 400 0 * 10 - 492 0 * 10 - 344 0 * 0 360 0 * 8 - 364 0 * 6 - 328 0 * 12 - 336 0 * Oscilloscope had the trigger threshold set to 300 mV TT Delta T (ns) H1 (mv) H2(mv) --------- ----------- --------- --------- HA -15,4 30 - 204 0 * 47 - 556 0 * 47 - 232 0 * 46 - 232 0 * 47 - 264 0 * Oscilloscope had the trigger threshold set to 200 mV TT Delta T (ns) H1 (mv) H2(mv) --------- ----------- --------- --------- EM -16,4 12 - 364 0 * 12 - 376 0 * 0 320 0 * 7 - 392 0 * 8 - 316 0 * 20 - 340 0 * 16 - 292 0 * 16 - 384 0 * 18 - 280 0 * 23 - 336 0 * Oscilloscope had the trigger threshold set to 250 mV TT Delta T (ns) H1 (mv) H2(mv) --------- ----------- --------- --------- HA -16,4 51 - 208 0 * 62 - 252 0 * 64 - 252 0 * 58 - 224 0 * 54 - 236 0 * Oscilloscope had the trigger threshold set to 200 mV TT Delta T (ns) H1 (mv) H2(mv) --------- ----------- --------- --------- EM +17,4 20 + 304 0 * 20 + 398 0 * 0 400 0 * 18 + 290 0 * 20 + 284 0 * 20 + 256 0 * 24 + 450 0 * 18 + 260 0 * 12 + 370 0 * 20 + 286 0 * Oscilloscope had the trigger threshold set to 250 mV TT Delta T (ns) H1 (mv) H2(mv) --------- ----------- --------- --------- HA +17,4 18 - 254 0 * 20 - 264 0 * 28 - 200 0 * 28 - 220 0 * 12 - 294 0 * Oscilloscope had the trigger threshold set to 200 mV TT Delta T (ns) H1 (mv) H2(mv) --------- ----------- --------- --------- EM +18,4 0 220 0 * 12 + 240 0 * 16 + 158 0 * 18 + 224 0 * 10 + 174 0 * 0 162 0 * 0 154 0 * 12 + 154 0 * 14 + 168 0 * 16 + 162 0 * Oscilloscope had the trigger threshold set to 150 mV TT Delta T (ns) H1 (mv) H2(mv) --------- ----------- --------- --------- HA +18,4 18 - 178 0 * 32 - 198 0 * 36 - 208 0 * 36 - 340 0 * 36 - 192 0 * Oscilloscope had the trigger threshold set to 150 mV TT Delta T (ns) H1 (mv) H2(mv) --------- ----------- --------- --------- EM +19,4 196 + 696 120 8 + 670 0 * 70 + 600 28 70 + 560 72 36 + 632 16 24 - 796 0 * 70 + 540 68 162 + 560 128 28 + 590 0 * 86 + 584 96 * Oscilloscope had the trigger threshold set to 500 mV TT Delta T (ns) H1 (mv) H2(mv) --------- ----------- --------- --------- HA +19,4 14 - 1910 0 * 0 1580 0 * 26 1670 0 * 8 - 1760 0 * 10 - 1590 0 * Oscilloscope had the trigger threshold set to 1.5 V (*) Widht of the trace 100 mV @ full bandwidth of the oscilloscope TT Delta T (ns) H1 (mv) H2(mv) --------- ----------- --------- --------- EM +20,4 50 + 690 60 70 + 600 60 60 + 730 60 60 + 620 40 56 + 730 40 80 + 930 110 54 + 630 80 50 + 530 60 70 + 630 50 70 + 730 50 Oscilloscope had the trigger threshold set to 500 mV TT Delta T (ns) H1 (mv) H2(mv) --------- ----------- --------- --------- HA +20,4 27 - 670 0 * 10 + 540 0 * 50 + 656 0 * 92 + 548 92 40 + 704 0 * Oscilloscope had the trigger threshold set to 500 mV (*) Widht of the trace 32 mV @ full bandwidth of the oscilloscope TT Delta T (ns) H1 (mv) H2(mv) --------- ----------- --------- --------- EM -17,3 12 + 348 0 * 8 + 336 0 * 0 436 0 * 0 344 0 * 0 352 0 * 0 480 0 * 8 + 356 0 * 0 308 0 * 8 + 348 0 * 0 408 0 * Oscilloscope had the trigger threshold set to 300 mV TT Delta T (ns) H1 (mv) H2(mv) --------- ----------- --------- --------- HA -17,3 16 - 392 0 * 30 - 344 0 * 32 - 300 0 * 44 - 300 0 * 32 - 312 0 * 36 - 292 0 * 24 - 364 0 * Oscilloscope had the trigger threshold set to 250 mV (*) Widht of the trace 32 mV @ full bandwidth of the oscilloscope TT Delta T (ns) H1 (mv) H2(mv) --------- ----------- --------- --------- EM -18,3 8 + 182 0 * 16 + 150 0 * 9 + 158 0 * 10 + 156 0 * 16 + 224 0 * 14 + 164 0 * 16 + 356 0 * 12 + 176 0 * 20 + 156 0 * 18 + 184 0 * Oscilloscope had the trigger threshold set to 150 mV TT Delta T (ns) H1 (mv) H2(mv) --------- ----------- --------- --------- HA -18,3 10 - 188 0 * 0 174 0 * 6 - 164 0 * 0 194 0 * 8 - 314 0 * 8 - 202 0 * Oscilloscope had the trigger threshold set to 150 mV TT Delta T (ns) H1 (mv) H2(mv) --------- ----------- --------- --------- EM -19,3 52 + 512 32 72 + 880 76 30 + 492 36 106 + 500 68 170 + 644 140 76 + 576 64 187 + 564 144 20 + 512 0 * 126 + 580 112 Oscilloscope had the trigger threshold set to 450 mV TT Delta T (ns) H1 (mv) H2(mv) --------- ----------- --------- --------- HA -19,3 16 - 1340 0 * 16 - 1290 0 * 4 - 1400 0 * 12 - 1320 0 * 12 - 1340 0 * Oscilloscope had the trigger threshold set to 1.25 V TT Delta T (ns) H1 (mv) H2(mv) --------- ----------- --------- --------- EM -20,3 124 + 588 120 129 + 612 120 130 + 608 180 130 + 568 108 102 + 668 108 44 + 580 36 35 + 820 40 63 + 660 32 98 + 1005 132 Oscilloscope had the trigger threshold set to 550 mV (*) Widht of the trace 24 mV @ full bandwidth of the oscilloscope TT Delta T (ns) H1 (mv) H2(mv) --------- ----------- --------- --------- HA -20,3 0 536 0 * 82 + 652 56 20 + 556 0 * 36 + 532 0 * 96 + 560 72 Oscilloscope had the trigger threshold set to 450 mV (*) Widht of the trace 24 mV @ full bandwidth of the oscilloscope