Single-run QA

Single-run QA quantities

Single-run QA is the first place to look to understand in detail the quality for a given run. This explanation of a run's QA page will use an example run (chosen more or less at random), run 1478, rerun 40, which covers a part of strip 12 S. It can be reached from the summary web site, the DRsup summaryQA is here; clicking on the run number gets us to http://das.sdss.org/DR5/data/imaging/QA/1478/40/qa/all-runQA-1478-40.html.

Note that the top of the run's QA page indicates the versions of the astrometric and photometric pipelines used to generate the data that went into this analysis. The run's QA page gives statistics of a wide variety of internal consistency statistics run on the data, with links to specific postscript figures (and in some cases, tables of numbers) giving the gory details. The tests carried out reflect specific concerns and problems we've had with the data and pipelines thereof, and therefore allow us to determine whether for these data, the problem does not appear. This will become clear with the detailed descriptions of the specific tests carried out.

An SDSS run is divided into six scanlines or camera columns, and each camera column is divided into a long series of slightly overlapping fields of 10x13 arcmin². Each field, in turn, is made up of five frames, one in each of the SDSS filters, u,g,r,i,z. There is a condensed version of the QA for the run described below, but to understand it requires familiarity with the basic tests of runQA, which we now describe.

The tests that are carried out on each run fall into six broad categories, which are listed in the table of contents in the beginning of the run's QA page.

• Field Quality Statistics tabulate the distribution of field qualities of each camera column.
• Principal Color Distributions. The constancy of the distribution of stars in color-color diagrams (the stellar locus) allows a test of the photometric uniformity of the data.
• Background Brightness Analysis. Another check of the photometric uniformity comes from comparing the sky brightness from one camera column to another.
• PSF Photometry. There are several algorithms to determine the brightnesses of stars; systematic differences between them are an indication of possible problems in the determination of the point-spread function.
• Relative Astrometry. The relative positions of stars in the five passbands are checked for systematic offsets.
• Flatfield Errors. The principal colors of stars can be used to check for systematic offsets as a function of position on each chip, due to errors in the flatfield vector. Note that the signal-to-noise ratio for this analysis is usually poor for a single run; this analysis is better done on a large group of runs together.

Determining the overall quality of each field

The table lists the fractions of fields in each camera column with each quality classification as defined on thesummaryQA page. In our example run, the vast majority of fields are good or excellent. Tables of relevant numbers are given for each camera column in links in the first column of the table, and (in even more excruciating detail) in the link labelled Here is the QAwizhard's Field Quality Table.

The Principal Colors of the Stellar Locus

The distribution of ordinary stars in SDSS color-color space follows a tight locus with distinctive features that can be used as measures of the photometric calibration. In particular, one can use various regions of these stellar loci that appear straight to define a series of principal colors, whose median values should be essentially constant over the survey (although they are weakly metallicity-dependent). See Helmi et al (2003, ApJ, 586, 195) for an initial description of these principal colors. These colors are defined after correcting for foreground reddening a la Schlegel, Finkbeiner, and Davis (1998); to a very good approximation, at the flux limits of the SDSS, all stars are beyond the absorbing dust. There are four such principal colors defined:

• The s color, s=-0.249u+0.794g-0.555r+0.234, which is perpendicular to the stellar locus in the u-g, g-r diagram.
• The w color, w=-0.227g+0.792r-0.567i+0.050, which is perpendicular to the blue branch of the stellar locus in the r-i vs. g-r diagram.
• The x color, x=0.707g-0.707r-0.983, which is perpendicular to the red branch of the stellar locus in the r-i vs. g-r diagram.
• The y color, y=-0.270r+0.800i-0.534z+0.059, which is perpendicular to the stellar locus in the r-i vs. i-z diagram.

Each of these colors has been offset to be zero over the full SDSS survey. They have been normalized such that the error in the principal colors is comparable to the error in a single band, assuming the errors in each band are the same. These colors are measured for every star brighter than r=19 (which are not flagged as having problematic photometry). The statistics are plotted in a series of figures labelled Click here for the main s color plot (and similarly for w, x, and y). They show the median and rms statistics in each bin, as a function of field number, for each camera column. Here one can see at a glance the extent to which the stellar locus is in agreement with the SDSS global average (the lines in the top panel stay close to zero), and whether there are specific fields or groups of fields that deviate. One can also see whether the width of the stellar locus stays constant. Also shown are the number of stars used in each bin for the principal color determination (this is usually not flat, as the density of stars changes with Galactic latitude). Finally, the rms statistic in bins does not measure the number of extreme outliers from the stellar locus, so also shown is the number of stars more than 2 sigma from the median s, w, x, and y values, on the blue side and red side, respectively.

In addition, these statistics are all made available in the Field Quality Table , and also in links off the flat-field error page linked at the bottom of the runQA page, and are summarized in a series of tables, that look like this:

Color-color and color-magnitude diagrams are given in links to each camera column, showing exactly the stars that go into the definition of each primary color.

The quantities listed, for each camera column, include:

• Median: The median value over all bins of the median principal color. That is, it is the median of the values shown in the top panel of the principal color figure. The principal colors are supposed to be close to zero, so This is the value you want to look for to see if there is some systematic offset of the photometry of the whole camera column from the survey as a whole.
• Max|PC|: The largest deviation of the median principal color in a bin from the median over all bins. That is, it is the most outlying point of the values shown in the top panel of the principal color figure. Look here for an indication of outlying frames with problematic photometry.
• Rms: The RMS scatter of the median PC color in each bin. That is, it is the scatter around the mean of the values shown in the top panel of the principal color figure. Look here for an indication of varying photometric calibration along a run (caused, e.g., by clouds).
• Width: The median value of the rms width of the principal color in each bin. That is, it is the median of the values shown in the second panel of the principal color figure. Typical values are less than 0.05 magnitudes.
• MaxWidth: The maximum value of the rms width of the principal color in each bin. That is, it is the maximum of the values shown in the second panel of the principal color figure. Look here for signs of a single frame with poor photometry.
• Nall (admittedly a confusing name): The median number of stars which went into the determination of the principal color in each bin (i.e., the third panel in the principal color figure). This median is not very interesting, as the number of stars is a strong function of Galactic latitude. Much more meaningful are:
• NblueTail and NredTail, which give the median number of stars per bin which lie more than 2 sigma (on the blue and red sides, respectively) of the median principal color. This is an indication of possible problems with some small fraction of the photometry. This is the median of the quantity shown in the lower two panels of the principal color plot.

Too large a value of any of these quantities indicates a problem. If any of these values is above a threshold, as indicated, it is shown in red, so problems can be recognized at a glance. If a problem is indicated, often the best way to proceed is to look at the principal color plots. For example, in run 1478, the Median x color is about -0.013 (i.e., 13 millimags) in most columns, a bit above the nominal threshold of 0.01. The x color plot does indeed show an offset in the x color of about 1%, which seems to be a real calibration problem.

Background (Sky) Brightness Analysis

Another check of the uniformity of the photometric calibration, from one camera column to another, is the uniformity of the sky brightness. The sky brightness, measured on a frame-by-frame basis, changes with time; here we calculate for each filter a running median as a function of field over the six camera columns. The plot http://das.sdss.org/DR5/data/imaging/QA/1478/40/qa/runQA-1478-40-sky.ps, shows the deviations of each camera column from this median, in units of magnitudes per square arcsecond. Note that the bandpasses of the z chips differ slightly, one from another; this effect is corrected for before taking the medians.

Thus the variation of the sky brightness with time has been taken out; these curves should be quite flat (as indeed they are). In addition, the median of each of these curves has also been taken out, so each should hug zero.

Summary statistics are then given of the quantities in that figure, including medians (i.e., the quantity taken out of the graphs above) in the table Phot. Zeropoint Corr. for Flat Sky, rms (Sky: RMS), and max deviation (Sky: Max Dev)statistics. Note that each of these tables links to the same figure for each of the camera columns.

As this example shows, this is a very powerful statistic, and checks for consistency of the internal photometric calibration and the flat-fields at the level of much better than a percent. It is limited, however, by the presence of scattered light, especially in the u band. An interesting scientific question is whether the sky brightness is expected to differ at this level over the 2.5 degree field of the camera.

PSF Photometry

Errors in the determination of the Point Spread Function (PSF) have been one of the most pernicious in the development of the photometric pipeline. The following tests allow us to determine how well the PSF is calculated, by comparing different measures of the brightnesses of stars.

In particular, the PSF magnitudes of stars (i.e., that determined from a direct fit of the PSF model to the brightness, aperture corrected to a large aperture; see the EDR paper) should agree with the large-aperture (here, a diameter of 7.5 arcsec) magnitude itself. This comparison can be carried out only for bright stars, for which the noise from the sky is negligible. This is carried out for each column of data for each filter, and there are plots such as this example showing the difference between aperture and PSF magnitude (for objects believed to be stars, brighter than 19th magnitude, and with estimated PSF error less than 0.05, in each band considered). Each star is shown as a point, with a running median per field shown as a red line.

The Postage Stamp Pipeline, which determines the PSF in the first place, indicates for each field the quality of its determination of the PSF. There are certainly fields in which the PSF is known to be poorly determined (usually in regions where the PSF is changing rapidly with time; note the bottom panels give the r-band seeing and its derivative); changes in seeing of 0.2 arcsec/field are definitely bad! These fields are flagged as having suspect PSF, and are indicated as such with cyan or magenta lines in the plot. These fields are not included in the summary statistics now described.

These summary statistics consist of:

• The maximum deviation in the median aperture - PSF magnitude per field (i.e., the quantity plotted in red), in m(apert7)-m(psf): Max Dev;
• The median value of the median PSF - aperture magnitude per field, in m(apert7)-m(psf): Med Off;
• The rms scatter of the median PSF - aperture magnitude per field, in m(apert7)-m(psf): RMS.

In run 1478, the median PSF-aperture magnitude hugs zero impressively well, with essentially no offset, and an rms scatter of 0.01 mag in u and z, and substantially better in g, r, and i. Two chips (out of 30!), the u-band in columns 2 and 5, are flagged red in their median offset. As an aperture correction is made to make the PSF and aperture magnitudes agree in the mean, this test is more a consistency check than anything else.

In addition to the PSF, every object is fit to exponential or de Vaucouleurs profile. The better-fitting of these two yields a so-called `model magnitude'. This quantity is aperture-corrected to force the model and PSF magnitudes of stars to agree. Of course, for stars, the model scalesizes are close to zero, but this has really all worked properly with the latest version of the photometric pipeline, as described in detail in the SDSS DR2 paper. The consistency of model and PSF magnitudes for stars is is tested in the next series of plots and tables, m(mod)-m(psf): Max Dev/Med Off/RMS.

Internal Tests of Astrometry

The astrometric calibration is applied to the r-band, and then propagated to each of the other bands in turn. One can ask, on a star-by-star basis, whether the calibrated positions are consistent. This is shown in the last two sets of plots and tables. The photometric pipeline measures a "velocity" for each object. This is done with asteroids in mind: a main-belt asteroid has an apparent proper motion of a few arcseconds over the five-minute interval between the r-band image and the g-band image. Thus the astrometric position of the asteroid varies linearly with filter (in the order, riuzg). The pipeline fits a line to the positions of every object; for all real stars, this line has a slope consistent with zero. The scatter in this slope can be expressed as a relative positional uncertainty between bands in arcseconds; these tables collect the statistics for bright stars (r<19.0).

In particular, the plots under band-to-band astrometric accuracy show the measured distribution of this effective positional uncertainty (in the row and column directions), shown both linearly (upper panels) and logarithmically (middle panels; note the change in scale on the x-axis). The median and sigma (as measured from the interquartile range) are given for each panel. Note the pesky periodic peaks in the upper histograms; those are an artifact of round-off errors in the way the data are stored.

The distribution is roughly Gaussian. One wants to look at outliers in this plot, for example to look for asteroids in the outer solar system. The T3 and T5 statistics shown in the figure give the number of 3-sigma and 5-sigma outliers relative to the number expected in a purely Gaussian distribution, respectively. T3 is typically 5-10, while T5 is meaningless for all but the very longest runs. But note that the distribution cuts off completely at +/- 100 milliarcsec; the internal astrometry is very clean!

If the astrometric errors are estimated correctly, the distribution of the measured errors divided by the errors estimated by the imaging pipelines will be a Gaussian with variance unity. This is tested in the lower panel, Relative Astrometry by Field (for blue and red stars separately). The distributions are not too far off from Gaussian, and indeed have a variance significantly less than unity, implying that the estimated astrometric errors are biased somewhat high.

All these statistics are summarized in the table shown, for row and column statistics separately. All quantities are shown in units of milliarcseconds (except for the chi-squared statistics, which of course are unitless).

One can then study these statistics as a function of field. The astrometry plot (here for column 1) shows the internal astrometric offsets for each star brighter than r=19, together with median per field. The statistics for these quantities (median over the fields, maximum, and rms) are listed in the table. The maximum over a field are a few tens of milliarcsec, with an rms of only a few milliarcsec.

Checks of the flat fields

Note that the results of these tests are given on a separate flat-field QA page, as they are less useful on a run-by-run basis. Also given on the run's flat-field QA page are links to the data files containing detailed information on the principal colors.

The SDSS images are from a drift-scan, thus the flat field is a one-dimensional vector for each chip, as a function of pixel column. If this flat field is in error, the error will propagate into all derived quantities, especially the photometry. The Principal Colors, described above, can be used to check the flat fields. The stellar principal colors are determined for each camera column as a function of pixel column (in bins of 32 pixels). Systematics as a function of pixel column are an indication of flat-field problems. There are four principal colors, and five flat-fields; the equations are closed by including constraints from the PT. The results of solving for the possible correction in each of the u, g, r, i, and z flat-fields is shown in a series of flat-field quality plots like this one). Again, these tend to be noisy for a single run, especially in u (where we've been having the most trouble with the flats). The largest problems in the flats tend to be at their edges. This information is used a posteriori to improve the SDSS flat-fields themselves.

Summary Table

Note that the links off these tables are dummies and do not lead anywhere.

Following that is a table with mean statistics over the run of various additional quantities associated with the PSF determination and the relative astrometry between bands. These of course are taken simply from the detailed QA we have already seen.