Understanding the image processing flags - Details
In the process of measuring the properties of all objects detected in
the SDSS images, the image-processing software sets an extensive
series of flags that indicate the status of each object, warn of
possible problems with the image itself, and warn of possible problems
in the measurement of various quantities associated with the object.
These flags are described one by one in Robert Lupton's
flags document. There is a separate set of flags (in the quantity
termed 'status') that give geometrical information on how the object
in question is determined to be a unique detection, given the overlap
of the SDSS data. There are further status flags set for each field,
which indicate possible problems in processing that field.
The present document describes the flags in different categories, and
tries to make clear how they can best be used. For full
understanding, one should read this in parallel with Robert Lupton's
flags document containg even more detailed descriptions of the
individual flags.
The "status" of an object
The geometry of the SDSS imaging scans involves overlaps of a variety
of types: the two strips of a stripe can overlap; runs from adjacent
stripes can overlap, two runs along the same stripe can overlap, and
within a scanline, the data are divided up into partially overlapping
fields. We wish to define a unique sample of objects, with no overlap
either from repeat observations of a given object (independent
photons), or objects in the overlap between fields (the same photons).
Bits within the status flag (a quantity available for each object
detected in SDSS) explain how this is done, as described in detail in
the description
of the resolution of overlaps. In addition, these bits use
information from the flags set by the photometric pipeline itself
(described below). Thus the resolving is carried out only on those
objects whose flags indicate them to be "good", in the sense of being
worthy of further investigation. The definition of "good" is:
GOOD = !BRIGHT && (!BLENDED || NODEBLEND || nchild == 0) BRIGHT,
BLENDED, and NODEBLEND are defined below. In brief, objects labelled
BRIGHT are duplicate entries in the SDSS catalog; those flagged as
BLENDED or with nchild > 0 have been deblended into 2 or more
children; to include them would give duplication between a parent and
its children. The GOOD objects are then queried for their
possible overlap with other data. Objects flagged OK_RUN are resolved
as far as the overlap with adjacent fields in the same run/rerun/camCol is
concern. Again, an
object which is not flagged OK_RUN will have a separate detection in
the adjacent field, using the same photons. The division
between fields is artificial. OK_SCANLINE objects are further
resolved as far as the overlap between adjacent scanlines in the two
strips in a stripe is concerned.
Adjacent stripes overlap as well, quite substantially so near the
survey poles. OK_STRIPE resolves this overlap as well. This can
cause confusion, as this is applied purely geometrically, and often
before the adjacent stripe has been observed. Thus there are regions
of the DR1 in the outermost stripes where it might make sense not to
apply this cut. Occasionally, runs will cross over the official
SDSS survey boundaries. This is the last cut; objects that lie within
the survey area, and satisfy all the other cuts described above, are
labelled PRIMARY. For most scientific applications, the PRIMARY
detections are the only ones needed. Objects labelled OK_RUN but
which are duplicate in one of the senses above, are labelled
SECONDARY. The overlap between adjacent scanlines and between stripes
is used extensively by SDSS quality assurance to confirm our
photometric and astrometric accuracy.
RESOLVED simply indicates that this OK_RUN object has been
through this resolve process; all RESOLVED objects should be marked
as either PRIMARY or SECONDARY
The nature of the flags output by the photometric pipeline
The SDSS imaging data consist of images in five photometric bands.
The measurement of the properties of the objects is carried out in the
five bands separately, and flag bits are set in each band. There are
enough flag bits to fill up two 32-bit quantities, thus these are
encoded in two quantities, called generically flags and
flags2. There is one such pair for each of the five bands, u,
g, r, i, and z. These are combined in various ways to make flags
appropriate for the whole object in all five bands; these are called
objc_flags and objc_flags2. Beware of interpreting the
objc_flags blindly! For example, the NOPETRO flag is set in
objc_flags if the Petrosian radius cannot be measured in any of
u, g, r, i, and z. Imagine our horror when the SDSS galaxy
spectroscopic sample (which is defined in terms of Petrosian
magnitudes in r) had 50% of the objects flagged NOPETRO! The reason is
because the u and z bands are low S/N, and many objects had NOPETRO set
in u or z, and therefore in the objc_flags as well. Only 2% of galaxy
targets have NOPETRO set in r.
Flags that affect the object's status
BINNED An object that is detected
as greater than a 5 sigma peak (after smoothing with the local PSF) in
a given band is flagged BINNED1 in that band. The object-finder then
masks all detected objects thus far, bins the image 2x2, and runs the
detection algorithm again; objects discovered at this stage are
flagged BINNED2. This is repeated again (i.e., now binning 4x4);
detections are labelled BINNED4. Many BINNED2 detections include the
outskirts of bright galaxies, and scattered light from bright stars
(as well as genuine low surface brightness galaxies); very few BINNED4
detections seem to be real astrophysical objects.
DETECTED the OR of BINNED1,
BINNED2, and BINNED4, in a given band. Even if the
object is not flagged DETECTED in a given band (usually because
it was detected only in another band), photometry is still carried out
on it, allowing, e.g., a 3-sigma detection of a point source.
BRIGHT As described in the EDR paper, objects detected at more than 200
sigma in the r band have their properties measured twice: once with a
global sky and once with a local sky. The former entry in the SDSS
catalogs is flagged BRIGHT, and in practice is rarely used to do
science. One should always reject such objects. This will be set in
all the bands, as well as objc_flags.
BLENDED, NODEBLEND, CHILD Each detected object is
examined to see if it is composite; if it is, it is flagged BLENDED in
all bands, as well as objc_flags. An attempt is made to deblend it.
If for some reason it is not deblended (usually because it is too
close to the edge), it will be flagged NODEBLEND. Otherwise, its
children will have their properties measured as well, and one wants to
reject BLENDED objects not flagged DEBLENDED in order to avoid
duplicate entries. Note that selecting PRIMARY objects does this, and
the cut on BRIGHT entries, automatically. The children of a
deblend are flagged CHILD. It is possible for multiple peaks to be
detected in the CHILD, and for it to also be labelled BLENDED as well.
There are further flags related to deblending that are mostly
informational in nature; see below.
Flags that indicate problems with the raw data
SATURATED, SATURATED_CENTER The
photometry of saturated objects is questionable, needless to say (in
fact, the total PSF counts of mildly saturated stars should not be too
much in error, as it attempts to include all photons, including those
in the saturated core and the bleed trail). The SATURATED flag is set
in each band that includes saturated pixels; if it is set in any band,
it is also set in objc_flags. Objects that are saturated can be
deblended if they show multiple peaks. Note that a galaxy with a
superposed saturated star in its disk, even if successfully deblended,
will be flagged SATURATED, as some of the pixels in the object
footprint are indeed saturated.
SATURATED_CENTER indicates that the saturated pixels are close to
the center of the object. This can be used to distinguish, e.g.,
galaxies which are flagged SATURATED because of a superposed star,
from those with a very bright Seyfert nucleus, although it still needs
further testing. Note that star-galaxy separation is not very
effective for saturated objects; many saturated stars are
misclassified as galaxies.
EDGE, LOCAL_EDGE, DEBLENDED_AT_EDGE The
photometric pipeline works on one field at a time. An object which is
too close to the edge of a frame is flagged EDGE in that band. Among
PRIMARY objects (which have been resolved in overlaps, and thus should
remove most EDGE objects), only large extended objects should be
flagged EDGE. Thus, if you are interested in point sources, you will
probably not need to worry about the EDGE flag (or at least be
suspicious of objects with EDGE and PRIMARY set). On rare occasions,
it has happened that half of a chip has gone on the blink; objects
close to the new edge there are flagged LOCAL_EDGE in the appropriate
band.
The deblending algorithm has to work harder for objects close to the
edge; it runs only for big objects which otherwise might be missed.
If so, the flag DEBLENDED_AT_EDGE is set. This is an informational
flag; it by itself does not indicate any trouble.
If the photometric pipeline recognizes a pixel as bad (due to a bad
column, a cosmic ray, or a bleed trail), it is interpolated over. If
this is true for any pixel within the object, it is flagged
INTERP. This by itself it just informational; if the
interpolation is over a cosmic ray or a single bad column, for
example, the photometry should be essentially perfect. Further flags
give additional information.
- CR means that the object contained a cosmic ray that was
interpolated over. This does not mean that the object in
question is a cosmic ray! Again, the photometry of objects
flagged CR is usually fine. Thus this is mostly an informational flag.
- MAYBE_CR is an indication that object may be a cosmic ray. It is
not interpolated over; it is set with a threshold lower than
the main cosmic-ray finding algorithm. It is a useful flag to trigger
on if one is looking, e.g., for objects detected in a single band
(such as the high-redshift quasars and T dwarfs, which can show up
only in the z band).
- MAYBE_EGHOST says that the object in question is in a position
that it may be an electronics ghost of a bright star in the given
band. You should be suspicious of objects with this flag that are
faint and detected in only one band.
- INTERP_CENTER means that the interpolated pixel(s) in question
fell within 3 pixels of the center of the object. This is a warning
that perhaps the photometry of this object may be affected. You
should really get concerned when
- PSF_FLUX_INTERP is set. This means that more than 20% of the
PSF flux is interpolated over in the band in question, which may make
one suspicious of the accuracy of the flux. In practice, most objects
with this flag set still appear to have perfectly good PSF photometry,
but the number of outliers (say, in a color-color plot) is definitely
larger than usual.
- You should be especially suspicious if BAD_COUNTS_ERROR is set in
a given band,
which says that the interpolation over bad pixels is so significant
that you should not believe the PSF flux error; it is probably
underestimated.
Flags that indicate problems with the image
CANONICAL_CENTER, PEAKCENTER,
DEBLEND_NOPEAK, NOPROFILE, NOTCHECKED, NOTCHECKED_CENTER, TOO_LARGE,
BADSKY
Often, in deblending, objects near the edge, and at low S/N, various
flags will be set indicating trouble defining the center of the
object. This should make one suspicious of its detailed photometric
properties. In particular:
- CANONICAL_CENTER: The centroid of a given object is usually determined separately
in each band. If in some band, it is impossible to measure the center
(due to being at the edge), one uses the center in another band,
transformed according to the relative astrometry between the bands,
and this flag is set.
- PEAKCENTER: If the centroiding algorithm can't find a better center, it will
often simply report the position of the peak pixel in a given band.
This often happens with little wisps of objects deblended from
something bright; the flag PEAKCENTER is set. It is a hint that this
object may not be real. A related flag is
- DEBLEND_NOPEAK: indicates that after deblending, the child in question doesn't have a
peak. Objects with either of these flags set (especially nominal
point sources in a nominally high S/N band) should be treated with
suspicion.
- An object with NOPROFILE set in a given band had (as the name
implies) zero or one entries in the radial profile; most photometric
quantities measured from it are likely to be suspect.
- NOTCHECKED: An object includes pixels which were not checked for peaks by the
deblender; this can happen close to the edge, and in the cores of
saturated stars. Be suspicious of the
performance of the deblender in this case. If the flag
- NOTCHECKED_CENTER is also set, the situation is worse; the center of
the object is in a not-checked region. This should not happen for any
BINNED1 object.
- The flag TOO_LARGE indicates an object which is still detected at
the outermost point of the radial profile (a radius of over 4 arcmin),
or at least one child in a deblend is larger than 1/2 a frame. This
is indicative either of a very large object, or a poorly determined
sky, or a particularly horrific deblend. The detailed photometry of
this object is questionable.
- BADSKY: If the local sky is badly determined (as occasionally
happens in regions with complex backgrounds), the core of an object
can be strongly negative. This
is bad; the photometry of such objects is meaningless.
Problems associated with specific quantities
Some of these are easy; they simply say that the quantity in question
could not be measured. In particular:
- NOSTOKES: Objects whose Q and U (Stokes parameters) were not measured in a
given band.
- ELLIPFAINT: Objects whose isophotal shape could not be measured.
Three types of measurement generate a lot of flags: Petrosian
quantities, the proper motion of objects, and adaptive shape moments.
These are:
Flags associated with the measurement of
Petrosian quantities
The pipeline measures Petrosian radii, fluxes, 50% and 90% radii,
and errors for all these quantities. This is described in detail in
Appendix A to Strauss
et al. (2002), which discusses the flags as well. The Petrosian
radius (and there can be more than one of them) is often measured at a
fairly low S/N point in the profile of an object. Thus the most
common flags that are set (especially at the faint end) are
- PETROFAINT the
Petrosian radius is measured at a very low surface brightness level,
and
- NOPETRO the code was unable to measure the Petrosian
radius.
These two often appear together. In this case
(and in the absence of other warning flags such as BADSKY or
NOPROFILE, which mean real trouble), the code still returns a
meaningful magnitude (i.e., the total flux within the aperture with
detected counts), so the Petrosian magnitude will still be usable. A
related flag is NOPETRO_BIG, which indicates that the Petrosian
radius appears to be larger than the outermost point of the extracted
radial profile. This can happen in regions in which the background
sky is noisy, or for low S/N objects.
Other Petrosian flags, mostly informational in nature, include:
- MANYPETRO For galaxies with composite profiles, it is
possible for there to be more than one Petrosian radius.
- MANY_R50, MANY_R90 Galaxies which have a radial profile
which dips below zero can have more than one radius including 50%
or 90% of the light. This is a rare
occurrence.
- If the Petrosian radius hits the edge of the frame, the flag
INCOMPLETE_PROFILE is set. The radial profile, and thus the Petrosian
quantities, are still calculated in an unbiased way, and the results
should be reasonable.
Flags associated with moving objects
A main-belt asteroid will show a parallax of a few arcseconds between
the r and g exposure in the SDSS camera. The photometric pipeline,
and in particular, the deblending algorithm, explicitly tests for
this, and measures the motion as appropriate. There are quite a few
flags associated with this process. For most purposes, the
only flag you need to examine is DEBLENDED_AS_MOVING, whose
name should be self-explanatory. If one wants a catalog of moving
objects, for example, cut on this flag, as well as the derived motion
(rowv, colv) and associated errors. It is possible that objects with
small enough motion will have a statistically significant proper
motion, but not trigger this flag; this requires further exploration.
There are no doubt a number of Kuiper Belt objects to be discovered in
the SDSS data!
The remaining flags related to moving objects are mostly informational
in nature, but are useful in understanding why a specific object
was not deblended as moving:
- MOVED indicates that the object is a candidate to be deblended as
moving. This is not a terribly useful flag. In particular, despite
its name, it does
not mean that the object is actually determined to be moving!
- NODEBLEND_MOVING Objects flagged MOVED that are
not deblended as moving. Not terribly useful.
- Objects with detections in only a few bands will not be able to
be tested for motion; they are flagged TOO_FEW_DETECTIONS. Even if
the object passes this threshold, it may turn out that the centroids
are not good in a few of the bands, in which case the flag
TOO_FEW_GOOD_DETECTIONS is also set.
- BAD_MOVING_FIT the motion of the object is inconsistent
with a straight line and it is not deblended as moving. A related
flag is
- BAD_MOVING_FIT_CHILD, which states that in a complicated
deblend putatively involving both a moving and stationary object, a
child's velocity fit is too poor, so the parent wasn't deblended as moving.
- STATIONARY the moving object's velocity is consistent with zero
- CENTER_OFF_AIMAGE the nominal motion moves the object right off the edge of the
atlas image in some band.
Flags associated with the measurement of
adaptive moments
The imaging pipeline carries out the calculations of optimally
weighted second moments of objects in order to determine their shapes
(especially for weak lensing studies). The flags indicate trouble
with this process for a given object in a given band, usually at low
S/N and such moment measurements should not be used.
- AMOMENT_UNWEIGHTED the calculation of the weights failed, the adaptive moments
are calculated without weights
- AMOMENT_SHIFT in the determination of adaptive moments, the centroid is
recalculated; if it shifts too far, the flag AMOMENT_SHIFT is set, and
M_e1 and M_e2 give the value of shift
- AMOMENT_MAXITER the moment calculation did not converge
- AMOMENT_UNWEIGHTED_PSF the adaptive moments for the PSF are unweighted
Further flags related to deblending (all informational)
A complicated object may have many peaks in it (think of the core
of a globular cluster as the worst-case scenario!). However
complicated an object is, the deblender conserves flux, in so much as
the flux in each pixel is split among the children. Still, no effort
is made to ensure that the sum of the children is exactly equal to the
parent, so rounding error could lead to discrepancies of one or two
DN. A number of informational flags point out cases where deblending
was complicated.
If the number of peaks is larger than 25, the flag
DEBLEND_TOO_MANY_PEAKS is set (in the parent, *not* the child),
and only the 25 most significant peaks are deblended.
DEBLEND_UNASSIGNED_FLUX indicates that initially, >5% of the
parent's flux was not assigned to any of the children and that this
flux has been redistributed among them. Thus this is not an indicator
of any problem with the deblender; this is an informational flag
only.
It is occasionally true (especially in complicated deblends) that
some of the peaks are not deblended, for one of two reasons. The
parents in such cases are labelled DEBLEND_PRUNED. The two
reasons are that these peaks lie too close to other peaks (in which
case the flag PEAKS_TOO_CLOSE is set), or that the templates
associated with the peak is degenerate with others (in which case
DEBLEND_DEGENERATE is set - see deblender
description in DR1 changes document).
In a complicated deblend, especially those involving galaxies,
there can be many children, and it is not always obvious (without
looking at the image by eye) which child is the main galaxy. The flag
BRIGHTEST_GALAXY_CHILD flags that object which the code
believes to be the brightest galaxy child.
If the deblending algorithm recognizes a given child as unresolved, it
will use that information in the deblend, and flag it as
DEBLENDED_AS_PSF.
HAS_SATUR_DN indicates that the object is saturated in this
band and the bleed trail doesn't touch the edge of the frame. In such
cases, an attempt is made to add up all the flux in the bleed trails,
and to include it in the object's photometry. Note: some of the CCDs
saturate at over 65535 DN; for these chips, the bled flux will be
underestimated.
After the regular deblender had completed, photo took another pass
looking for some special cases, and the deblend was modified based on
this analysis are flagged DEBLEND_PEEPHOLE. Currently, only one
special case is considered: objects that, when merged together, were
consistent with a moving object that the deblender had missed in the
first pass.
Further informational flags
It is often true that the last bin in the radial profile of an
object goes slightly negative. When this happens, the BAD_RADIAL flag
is set; it can usually be ignored.
The Petrosian and model magnitude calculations make reference to
a canonical band, which defaults to the r band. In the case that the
object is undetected in r, the canonical band is set to the highest
S/N band. The band in question is flagged CANONICAL_BAND.
The extended wings around bright stars are subtracted; such
objects are flagged SUBTRACTED.
For sufficiently extended objects, the PSF-weighted centroid is not optimal,
and the centroid is found using a 2x2-binned images. Such objects are
flagged BINNED_CENTER; such objects probably should not be used, e.g.,
as part of a local astrometric reference frame.
Last modified: Mon Dec 27 17:41:52 CST 2004
|