Preprocess SPEC format data

Objective

The objective is to obtain usable (i.e. properly corrected) data for the general purpose PDF data processing in section 2.4 Get X-ray PDF.

¶

Do I need to preprocess my data?

This step, preprocess the data, is necessary if any one of the following conditions is met.

• more than one scan exists, and merging is required (see Appendix B)

• the detector counts need to be corrected for dead time effects

• the data need to be normalized point by point, with the normalization factor stored in one column

• the data have been properly preprocessed, but don't have the estimated standard uncertainties.

If none of the above describes your situation, you probably don't need to go through this step, and please jump to section 2.4. If preprocessing is required, the input file needs to be in SPEC format. ASCII files can be easily converted (see Appendix B).

The justification for this data preprocessing is the following. A SPEC data file usually contains more than one scans with each scan covering portions of the measured reciprocal space. Every scan is a n row by m column data set. One row represents a measured data point, and the columns record the Q/2$\theta$, detector counts, monitor counts, collection time, and some other values. However, what's of our interest is the scattering intensity versus Q/2$\theta$ over the entire measured range. This simply means to merge the scans together and extract the useful columns. However, different scans often have different scales due to either varying collection time or incident beam intensity changes. Even within one scan, each point may also require proper normalization due to the small fluctuations of the incident beam. Often more significantly, the non-linear response (counting rate dependence) of detectors needs to be corrected on the raw counts first. Thus each scan needs to be treated individually before being merged together.

Load SPEC data file

First, we need to load the SPEC file. The tutorial raw data from the sample were saved into the SPEC format file, ingaas_tutorial.spec. A simple but sufficient description of the SPEC format can be found in Appendix B.1. In principle, all supported file formats (Appendix B) can be used, but are all converted into SPEC format internally. If your file is not of one of those formats, a format conversion is necessary.

Following the menu Switch-> to Preprocess SPEC format data (shown in Fig. 2.2), you will activate the main preprocess SPEC format data GUI as in Fig. 2.3.

Figure 2.2: Click the Preprocess SPEC Format Data button to switch to the GUI in Fig. 2.3

Figure 2.3: The main preprocess SPEC format data GUI

Clicking on the button SPEC Data File (Fig. 2.3) will bring up a graphic dialog to select the desired file. You may need to go through directories to locate the tutorial file ingaas_tutorial.spec. Either double click on ingaas_tutorial.spec or press the OK button with ingaas_tutorial.spec highlighted will make the selection. After a successful data reading (waiting time depends on computer speed), the top left portion of the GUI should resemble Fig. 2.4 (more details on each GUI section come later).

Figure 2.4: The GUI portion displaying scans information in the SPEC data.

Overview of scans information

In general a SPEC file contains multiple scans. The program reads the scan header line beginning with #S in the SPEC file (Appendix B and reproduces them in the GUI window (Fig. 2.4) allowing you to select the scans you want. Fig. 2.4 shows the GUI portion displaying the scans information. The label on the top tells us that this SPEC file contains five scans. The following 5 Selected shows the total number of scans selected; 5 Highlighted shows the total number of scans being highlighted. Each line/item in this list box represents one scan in the data file. Scan information such as the scan number, number of data points, value range of the column of interest (read on for details), and the selection flag are shown as text contents. We can see that all scans are selected (you can highlight all of them using the mouse), which explains the 5 Selected, 5 Highlighted in the label above the list.

On the last row, the button Toggle Scan(s) Selection Flag will toggle the selection flag of currently highlighted scans. When a scan is flagged as Selected, it will be used when merging scans (which happens at a much later stage). If unselected, it will not be included during merging. You also have the choice to show only the selected or all scans in the list through the droplist list next to the label List Scans. This only becomes necessary when you have many ``bad'' scans in the data.

Set up SPEC columns

The next step is to tell the program several column names of interest. This is done in the upper right side of the GUI as shown in Fig. 2.5. You need to specify the column names for Q/2$\theta$, monitor column, and detector column(s). Only the detector can have more than one columns, in the case that multiple detectors are used during measurements. The allowed maximum number of detectors is 10. For each detector, you need to specify its column name in the SPEC file

In our data, each scan has 13 columns, whose names are listed in the droplist for each column selection. Q/2$\theta$ column should be set to pmQ, while the monitor column set to IC2. Since only one detector was used during the experiment, the number of detectors is 1, whose SPEC column should be ELASTIC. In the end, the GUI should look like Fig. 2.5.

Figure 2.5: The GUI section to set up SPEC columns

Set up dead time correction

Once SPEC column names are set, we are ready to set up the dead time correction on the monitor and detector column(s). They are IC2 and ELASTIC columns, respectively. The GUI for this job is located at the lower left panel, as shown in Fig. 2.6.

¶

Is dead time correction necessary?

This is determined by whether the count rate is within the linear response regime of the detector. Dead time correction is rarely needed for in-house X-ray measurements, while becomes necessary when data are collected at high flux synchrotron sources. If you are not sure, please check the specifications of the detector against your maximum count rate (i.e. the strongest Bragg peak). For more details on why and how detector dead time correction is applied, see the book by Egami and Billinge [5].

From the GUI, the dead time correction setup for ONLY ONE column is directly displayed and editable. Switching between columns (the monitor and detector columns) is done via the Editing Correction SetUp for Column droplist on the top portion of this GUI section. Once a selection is made, the GUI below will be updated by selected column settings. Our monitor counts are from ion chambers, therefore doesn't need any dead time correction, so please select None. For the detector column ELASTIC coming from the solid state detector (SSD), you can either choose to use theory or pulser method (see Chapter 5 of [5]). If you use the theory method, you will need the the total counts in the detector and the measuring time and the detector dead time. Select the data column TOTAL and Seconds for the first two. For this Ge detector, a dead time of 11.6 $\mu$s works well. The pulser method refers to the use of an electronic pulser ([5]) that is propagated with the data. You will need to select the column of data containing the pulser counts ``PULSER''. Either the theory or pulser method will work well for our data, however the theory method is recommended because it introduces less noise. If you are interested, try them both for comparison. The GUI is shown in Fig. 2.6.

Figure 2.6: The GUI to set up dead time corrections for monitor (left) and detector columns (right)

Set up detector(s) merging

In general, when data are collected in more than one scan, the different scans need to be normalized by their respective monitor counts or measurement time before being merged. Therefore, to set up the program for merging scans, we must assign the data column for normalization, also the range of each scan, and scaling between scans We should set lower and upper cutoff values for each scan to exclude the experimental artifacts sometimes appearing at either end of the data. Be aware that you need to set up EVERY detector column in case of multiple detectors. Also for this reason, the GUI to set up each detector is accessed through the Editing Correction SetUp for Column droplist used in setting up the detector dead time corrections. The section for this job is in the lower section of Fig. 2.6.

In our case, we set the normalization column to IC2, which is the monitor counts, and opt not to use auto scale. Qmin should be 0.0 Å^-1, with Qmax of 40.0 Å^-1. The right panel of Fig. 2.6 shows the settings for detector column ELASTIC. Here the range of from 0.0 to 40.0 Å^-1is for the merged data. Setting individually used range for each scan is prompted by clicking on the Edit by Scans button.

Actions and visualization

Now, we have given all the needed information to prepare the raw SPEC data. We can turn to our attention to the GUI section of action and visualization as shown in Fig. 2.7.

Figure 2.7: The GUI section to plot/save data and process data.

The buttons to issue calculation commands are in the bottom row. Button Update Scans is only useful when MCA data are available (under development), though it doesn't hurt to click it. Clicking button Apply Corrections will apply the corrections (the detector dead time correction, normalization, etc) to monitor and detector columns as you specified. Clicking button Merge Scans will take all selected scans (flagged as ``selected'') and merge them into the single merged data, which is what we set out to obtain in the first place. When multiple detectors are used, the merged data for each detector is stored as one scan in the total merged data in SPEC format. The merged data now has more than one scans, however conceptually, this is different from the usually meaning of SPEC data.

In the end, we have data at three stages, raw SPEC data, scans data after correction, and the merged detectors. We can either save or plot them after making our selections. If the Save Data button is clicked, you will be asked for a file name to save the selected data. If the Plot Data button is pressed, the same selected data will be plotted in a separate window, with additional settings described below. The Plot Style should be practically set the Simple Plot View at all times.

Data selection is made as the following. First, choose from the Raw SPEC, Corrected Scans, and Merged Detectors under button group Data Type. When either Raw SPEC or Corrected Scans is selected, the targeted data are those scans currently being highlighted. If the button Compare Detectors is selected, all the detector columns for the first highlighted scan are selected. In all cases, you have the freedom to select the x values to plot via the droplist X Column. Unless Compare Detectors is selected, you can also choose the y data to plot via droplist Y Column. In addition, if you prefer to have a normalization column, you can do so by enabling normalization column (by setting the Norm. Col. button). In this case, the plotted Y Column will be divided by the normalization column. If this sounds rather confusing, please play with the interface for better understanding.

After carrying out the actions (Update Scans, Apply Corrections, Merge Scans, we need to check the correction results. We should not select Compare Detectors since only one detector is present in the tutorial data. Set the X Column as pmQ, the Y Column as ELASTIC, the Norm. Col. as unselected, and highlight all the scans. To visualize the raw SPEC data, select Raw SPEC as the Data Type, then click on the Plot Data button. The plot window appear as Fig. 2.8 left panel. If the Corrected Scans is selected, we should see Fig. 2.8 right panel. All scans match quite nicely after the corrections and normalization.

Figure 2.8: Visualization of the raw SPEC data (left) and the corrected SPEC data (right)

Save merged data and the history

Following the above section, if Merged Data is selected as the Data Type, we can visualize the merged data as in Fig. 2.9. Clicking on Save Data will invoke a dialog to select a file to save the merged data. You can use your favorite plot program to compare what you have just got with the provided reference file ingaas_tutorial.int (under the tutorial directory where we copied the tutorial data).

All the settings can be saved into a history file. Pushing the History -> and then the Save ... button will prompt to select the file to save the history information. The saved history file can be loaded back into the program by clicking on the Load ... button instead. This tutorial also contains an example history file ingaas_specred.xhst. Loading this file will set up what we have gone through in this section.

Figure 2.9: Visualization of the merged data obtained from preprocess SPEC format data