The SDSS Data Release 4 (DR4)

DR4 is described in a paper submitted to ApJS (Adelman-McCarthy et al. 2005), available at astro-ph/0507711 and here as DR4.ps.gz, 326 KB.

Contents

• What DR4 contains
• Caveats for imaging data
• Caveats for spectroscopic data
• Advanced features not yet available in DR4

What DR4 contains

The DR4 imaging data cover 6670 square degrees, and include information on roughly 180 million objects. The DR4 spectroscopic data include data from 1052 main survey plates of 640 spectra each, and cover 4681 square degrees. In addition, DR4 for the first time contains 276 "extra" and "special" plates:

• 61 "extra" plate/MJD combinations which are repeat observations of 52 distinct main survey plates,
• 206 distinct "special" plates, which are observations of spectroscopic targets, mostly in the southern galactic cap, which were selected by the collaboration for a series of specialized science programs (note that some of these plates are outside of the DR4 imaging area),
• and 9 "extraspecial" repeat observations of "special" plates.

There is a separate page describing the special plates in DR4.

The DR4 footprint is defined by all non-repeating survey-quality imaging runs within the a priori defined elliptical survey area in the Nothern Galactic Cap, and three stripes in the Southern Galactic Cap obtained prior to 1 July 2004, and the spectroscopy associated with that area as well as the extra and special plates obtained before that date. In fact, 34 square degrees of imaging data in the Nothern Galactic Cap lie outside this ellipse. While the DR4 scans do not repeat a given area of sky, they do overlap to some extent, and the data in the overlaps are included in earlier releases as well. The sky coverage of the imaging and spectroscopic data that make up DR4 are given on the coverage page. The natural unit of imaging data is a run; the DR4 contains data from (about) 200 runs in the best database, and (about) 202 runs in the target database.

A total of 183 square degrees of sky are different runs between target and best, the majority along the Equatorial Stripe in the Fall sky.

Except for the sky coverage, the pipelines and databases are identical in DR4, DR3 and DR2. Thus, DR4 is (very nearly) a proper superset of DR3, which is a superset of DR2. The DR2 included reprocessing of all data included in DR1, and those data in EDR that pass our data-quality criteria for the official survey. For details about what changed from DR2 to DR3, please refer to About DR3 on the DR3 web site.

New in DR4

There is a new table photAuxAll in the CAS that contain errors and covariances for the right ascension and declination as well as galactic coordinates for all objects. See the CAS Schema Browser entry for photoAuxAll and the astrometry algorithms description for the calculation of RA, DEC errors.

Imaging caveats

Overestimation of sky levels in the vicinity of bright objects

A study of weak lensing in the SDSS imaging data (Mandelbaum et al. 2005) found a systematic, 5% decrease in the number density of faint objects within 90′′ of bright (r < 18) galaxies. This was found to be due to a systematic overestimation of the sky levels in the vicinity of bright objects, which of course will affect all the measured photometric quantities of faint objects, including the classification as stars and galaxies. The brighter the object, the larger the overestimation of sky; indeed, restricting to foreground galaxies brighter than r = 16, the number density of fainter galaxies is 10% below the mean. The effect is due to the way in which the sky levels are estimated in the SDSS; they can be biased upward by the faint outer isophotes of a bright galaxy (see the EDR paper [Stoughton et al. 2002]; see also the discussion in Strauss et al. 2002). Closer than 40′′ to the bright objects, the intrinsic clustering of galaxies makes it difficult to assess the problem. This problem is most relevant for applications that involve correlating positions of bright objects with faint ones, yielding a spurious anti-correlation on the affected scales. We are currently investigating changes to the imaging pipeline to address this problem.

Red leak to the u filter and very red objects

The u filter has a natural red leak around 7100 Å which is supposed to be blocked by an interference coating. However, under the vacuum in the camera, the wavelength cutoff of the interference coating has shifted redward (see the discussion in the EDR paper), allowing some of this red leak through. The extent of this contamination is different for each camera column. It is not completely clear if the effect is deterministic; there is some evidence that it is variable from one run to another with very similar conditions in a given camera column. Roughly speaking, however, this is a 0.02 magnitude effect in the u magnitudes for mid-K stars (and galaxies of similar color), increasing to 0.06 magnitude for M0 stars (r-i ~ 0.5), 0.2 magnitude at r-i ~ 1.2, and 0.3 magnitude at r-i = 1.5. There is a large dispersion in the red leak for the redder stars, caused by three effects:

• The differences in the detailed red leak response from column to column, beating with the complex red spectra of these objects.
• The almost certain time variability of the red leak.
• The red-leak images on the u chips are out of focus and are not centered at the same place as the u image because of lateral color in the optics and differential refraction - this means that the fraction of the red-leak flux recovered by the PSF fitting depends on the amount of centroid displacement.

To make matters even more complicated, this is a detector effect. This means that it is not the real i and z which drive the excess, but the instrumental colors (i.e., including the effects of atmospheric extinction), so the leak is worse at high airmass, when the true ultraviolet flux is heavily absorbed but the infrared flux is relatively unaffected. Given these complications, we cannot recommend a specific correction to the u-band magnitudes of red stars, and warn the user of these data about over-interpreting results on colors involving the u band for stars later than K.

Bias in sky determination

There is a slight and only recently recognized downward bias in the determination of the sky level in the photometry, at the level of roughly 0.1 DN per pixel. This is apparent if one compares large-aperture and PSF photometry of faint stars; the bias is of order 29 mag arcsec^-2 in r. This, together with scattered light problems in the u band, can cause of order 10% errors in the u band Petrosian fluxes of large galaxies.

Zeropoint of the photometric system

The SDSS photometry is intended to be on the AB system (Oke & Gunn 1983), by which a magnitude 0 object should have the same counts as a source of F_nu = 3631 Jy. However, this is known not to be exactly true, such that the photometric zeropoints are slightly off the AB standard. We continue to work to pin down these shifts. Our present estimate, based on comparison to the STIS standards of Bohlin, Dickinson, & Calzetti~(2001) and confirmed by SDSS photometry and spectroscopy of fainter hot white dwarfs, is that the u band zeropoint is in error by 0.04 mag, u_AB = u_SDSS - 0.04 mag, and that g, r, and i are close to AB. These statements are certainly not precise to better than 0.01 mag; in addition, they depend critically on the system response of the SDSS 2.5-meter, which was measured by Doi et al. (2004, in preparation). The z band zeropoint is not as certain at this time, but there is mild evidence that it may be shifted by about 0.02 mag in the sense z_AB = z_SDSS + 0.02 mag. The large shift in the u band was expected because the adopted magnitude of the SDSS standard BD+17 in Fukugita et al.(1996) was computed at zero airmass, thereby making the assumed u response bluer than that of the USNO system response.

Holes in the imaging data

About 0.3% of the DR4 imaging footprint area (792 out of 255,167 fields, or 10/square degree) for DR4 are marked as holes. These are indicated in the CAS by setting quality=5 (HOLE) in the tsField file and field table and given in the list of quality holes, which contains further details about the holes and quality flags, including a information about a new (for DR4) table in the CAS which allows one to query for quality information about each field of data.

Problems with one u chip

The u chip in the third column of the camera is read out on two amplifiers. On occasion, electronic problems on this chip caused one of the two amplifiers to fail, meaning that half the chip has no detected objects on it. This was a problem for only two of the 105 imaging runs included in DR4: run 2190, which includes a total of 360 frames in two separate contiguous pieces on strip 12N (centered roughly at delta = +5 degrees in the North Galactic Cap; NGC), and run 2189, which includes 76 frames on stripe 36N near the northern boundary of the contiguous area in the NGC. The relevant frames are flagged as bad in the quality flag; in addition, individual objects in this region have the u band flagged as NOTCHECKED_CENTER (or, for objects which straddle the boundary between the two amplifiers, LOCAL_EDGE). Richards et al (2002) describe how the quasar selection algorithm handles such data; the net effect is that no quasars are selected by the ugri branch of the algorithm for these data.

Spectroscopy caveats

Zero equivalent width of emission lines, especially H alpha

There is a bug in the line-measurement code that has been in use since DR3 which gives some emission lines an equivalent width of zero, even though there is a significant line detection. The aim of the change introducing the bug had been to determine the equivalent width by integrating the spectrum, instead of using the parameters of a fitted Gaussian. The Gauss-fit equivalent width can be recovered from the fit parameters using the usual expression EW = 2.5066 * sigma * height / continuum.

Main survey spectra which are not marked sciencePrimary = 1 in CAS

Due to a bug in the pipelines, there are no tsTargets*.fits files for plates 1617-1620, 1623, and 1652. As a consequence, the objects from this plates do not have entries in the target and targetInfo tables in the CAS. Hence they are not marked sciencePrimary = 1 and do not appear in the default specObj and specPhoto views, which provide a filtered set of unique science spectra and form the basis of all query interfaces. Use the specObjAll and and specPhotoAll tables to access spectra from these plate in the CAS.

Note about galactic extinction correction

In the EDR and DR1, the spectroscopic data were nominally corrected for galactic extinction. The spectrophotometry since DR2 is vastly improved compared to DR1, but the final calibrated spectra in DR2 and beyond are not corrected for foreground Galactic reddening (a relatively small effect; the median E(B-V) over the survey is 0.034). Users of spectra should note that the fractional improvement in spectrophotometry from DR1 to DR2 and beyond was much greater than the extinction correction itself. As the SDSS includes a substantial number of spectra of galactic stars, a decision has been taken not to apply any extinction correction to spectra, since it would only be appropriate for extragalactic objects, but to report the observational result of the SDSS, namely, the spectrum including galactic extinction.

Problematic plates

A small number of plates suffered from a variety of minor problems affecting the quality of the spectrophotometry (but not of redshifts). See the list under Plates with problematic spectrophotometry on the data products page for spectra.

Mismatches between the spectroscopic and imaging data

For various reasons, a small fraction of the spectroscopic objects do not have a counterpart in the best object catalogs. In addition, the DR4 does not contain photometric information for some of the special plates, and the retrieval of photometric data from the CAS database requires special care for objects from the special plates. See the caveat about mismatches between spectra and images on the data products page for spectra.

Advanced features not yet available in DR4

There are a number of advanced features or data products that are not yet available in DR4, but will be in the near future.

• Photometric redshifts for galaxies The DR1 Catalog Archive Server had a table with photometric redshifts for galaxies, including galaxies fainter than those in the spectroscopic survey. The redshifts still need to be computed and loaded into the database for DR4.
• Coverage masks Detailed coverage masks which will allow large-scale structure resarchers to easily calculate power spectrum and related quantities are in preparation.
• IQS/SQS The Imaging Query Server (IQS) and Spectro Query Server (SQS) form interfaces have been re-enginered to use a faster database. They currently do not provide a direct link to the DAS data download form; users need to save .csv files with the necessary information and upload these by hand to the DAS form.