Directory Structures

The location of files is described in the Directory Structure page.

Survey Interface File Formats

Below are listed all files used by the current set of pipelines/processes, grouped by process. Individual files may be repeated as they may be used by more than one process; in this case, repeated listings include a parenthesized reference to the process which first created it.

Miscellaneous

FTCL Parameter Files

CCD configuration files

Weather log files

Tar tape logfiles

Monitor Telescope

mop

mtframes

excal

Imaging

iop

bias

ssc

ps

runQA

Target Selection

ts

export

tsManual

tile

plate

plug

Zoom image file creation

imaging csv creation

imaging csv loading

plates csv creation

plates csv loading

Miscellaneous

CCD configuration files
- opConfig -- CCD readout configurations
- opECalib -- CCD electronic calibration
- opBC -- CCD bad pixels
- opCCDLog -- CCD temperatures and voltages
Weather log files
- idWeatherLog -- Weather log file
- opCloud -- cloud camera statistics
Tar tape log files
- idTarTape -- Tar tape log

Monitor Telescope

mop
- mopPre outputs
- additional inputs
  - priFieldCenters -- list of field centers
- outputs
  - mdReport -- observing log
  - mdR -- raw frames
  - opConfig (CCD readout configurations)
  - opECalib (CCD electronic calibrations)
  - opBC (CCD bad pixels)
  - opCCDLog (CCD temperatures and voltages)
  - idTarTape (Tar tape logs)
mtframes
- mtframesPre outputs
  - mtPlan -- processing plan
  - mtParam -- tunable parameters
- additional inputs
  - mdReport (mop)
  - mdR (mop)
  - opConfig (mop)
  - opECalib (mop)
  - opBC (mop)
- outputs
  - mtObj -- instrumental object lists
  - mtProcessing -- calibrations list
  - mdReport (mop, modified)
excal
- excaliburPre outputs
  - exPlan -- processing plan
  - exParam -- tunable parameters
- additional inputs
  - mdReport (mop)
  - mtObj (mtframes)
  - metaFC -- MT primary field definitions
- outputs
  - exPhotom -- photometric solutions
  - mdReport (mop, modified)
kali
- kaliPre outputs
  - kaPlan -- processing plan
  - kaParam -- tunable parameters
- additional inputs
  - mdReport (mop)
  - mtObj (mtframes)
  - exPhotom (excal)
  - koCat -- Guide Star Catalog matches
- outputs
  - kaCalObj -- calibrated object lists
  - mdReport (mop, modified)

Imaging

iop
- iopPre outputs
- additional inputs
- outputs
  - idReport -- observing log
  - idR -- raw frames
  - idGang -- frame gang files
  - idFrameLog -- frame log files
  - focus -- focus log
  - idTapeSummary -- nightly log of what run/camcol is on which labeled tape
  - idTapeLog -- raw frame tape
  - idTarTape (Tar tape logs)
  - opConfig (CCD readout configurations)
  - opECalib (CCD electronic calibrations)
  - opBC (CCD bad pixels)
  - opCCDLog (CCD temperatures and voltages)
  - opCamera -- imaging camera geometry
  - idDaParam --- all the DA tunable parameters
bias
- biasPre outputs
  - biPlan -- processing plan
- additional inputs
  - idR (iop)
- outputs
  - idB -- bias vectors
ssc
- sscPre outputs
  - scPlan -- processing plan
  - scParam -- tunable parameters
  - koCat -- catalog of known bright stars
- additional inputs
  - idReport (iop)
  - idGang (iop)
  - idR (iop)
  - opConfig (iop)
  - opECalib (iop)
  - opBC (iop)
  - opCamera (iop)
- outputs
  - scFang -- matching postage stamps
  - scFrameList -- list of very bright star frames
  - scFrame -- very bright star frames
  - scWingList -- list of 200x200 bright star postage stamps
  - scWing -- 200x200 bright star postage stamps
  - scDiag -- diagnostics
  - scTrans -- Rough TRANS from row, column to great circle coordinates
ps
- psPre outputs
  - psPlan -- processing plan
  - psParam -- tunable parameters
- additional inputs
  - idB (bias)
  - opConfig (iop)
  - opECalib (iop)
  - opBC (iop)
  - opCamera (iop)
  - exPhotom (excal)
  - kaCalObj (kali)
  - scFang (ssc)
  - scFrameList (ssc)
  - scFrame (ssc)
  - scWingList (ssc)
  - scWing (ssc)
  - scTrans (ssc)
  - psCT (ps) -- photometric calibration by time (from earlier run)
- outputs
  - psFF -- flat field vector
  - psBB -- corrected bias vector
  - psField -- photometric calibration by field
  - psCT -- photometric calibration by time
  - psFang -- matching postage stamps
astrom
- astromPre outputs
  - asPlan -- processing plan
  - asParam -- tunable parameters
  - koTycho2,koUcac -- Tycho2/UCAC catalog stars
- additional inputs
  - idReport (iop)
  - idWeatherLog (weather)
  - opCamera (iop)
  - opConfig (iop)
  - psFang (ps)
- outputs
  - asTrans -- TRANS from row, column to great circle coordinates
  - asQA -- diagnostics
frames
- framesPre outputs
  - fpPlan -- processing plan
  - fpParam -- tunable parameters
  - koCat -- known objects
- additional inputs
  - idR (iop)
  - opConfig (iop)
  - opECalib (iop)
  - opBC (iop)
  - idB (bias)
  - asTrans (astrom)
  - psFF (ps)
  - psBB -- (ps)
  - psField (ps)
  - psCT (ps)
- outputs
  - fpFieldStat -- summary statistics for each field
  - fpObjc -- limits -- object lists
  - fpC -- corrected frame
  - fpBIN -- binned corrected frame
  - fpM -- mask image
  - fpAtlas -- atlas images
  - fpAtlasObj -- atlas image of one object, extraced from the fpAtlas file by obsolete version of the DAS. These files are no longer created by new versions of the DAS, and the file description has been removed.
fcalib
- fcalibPre outputs
  - fcPlan -- processing plan
  - fcParam -- tunable parameters
- additional inputs
  - idReport (iop)
  - asTrans (astrom)
  - exPhotom (excal)
  - kaCalObj (kali)
  - fpObjc -- limits -- object lists
- outputs
  - fcPCalib -- Photometric Calibrations

runQA outputs
- runQA

Target Selection

ts
- tsPre inputs
  - tsPlan -- processing plan
  - tsMatchParam -- Target Selection Matching Parameters
  - tsQuasarParam -- Quasar Target Selection Parameters
  - tsGalaxyParam -- Galaxy Target Selection Parameters
  - tsStarParam -- Star Target Selection Parameters
  - tsSerendipParam -- Serendipity Target Selection Parameters
  - tsStandardParam -- Standards Target Selection Parameters
  - tsLocusPoints -- Quasar Target Selection Locus Points
  - tsChunk -- list of segments in a resolved chunk of sky
  - tsObj -- limits -- calibrated objects
  - tsFIRST -- FIRST catalog
  - tsROSAT -- ROSAT catalog
- additional inputs
- outputs
  - tsObj -- limits -- calibrated objects
  - tsTargets -- spectroscopic targets selected from a chunk
export
- outputs
  - Directory name
  - tsChunk (ts)
  - tsObj -- limits -- calibrated objects
  - tsField -- limits -- calibrated field info
  - tsTargets -- spectroscopic targets selected from a chunk
  - tsMasks -- culled and bad fields (optional)
tsManual
- outputs
  - tsManual -- manually selected targets
tile
- tilePre outputs
  - tiPlan -- processing plan
  - tiParam -- tunable parameters
  - tiInputTargets -- untiled targets
  - tiBoundariesIn -- targetting chunk boundaries
- additional inputs
- outputs
  - tiOutputTargets -- tiled and untiled targets
  - tiTileCenters -- tile centers
  - tiUnTiledCenters -- tile centers for plates which were never drilled or redesigned
  - tiBoundariesOut -- tiling chunk boundaries
  - tiBoundariesOut -- targetting chunk boundaries in new format for sql DB uploads
  - tilingMask -- targetting chunk mask in new format for sql DB uploads
completeTile
- completeTilePre outputs
  - ctPlan -- processing plan
  - ctParam -- tunable parameters
  - ctObjs -- potential targets in tile area
- additional inputs
  - tiTileCenters (tile)
- outputs
  - plPlugMap -- final target list for tile

Spectroscopy

plate
- platePre outputs
  - plPlan -- processing plan
  - plParam -- tunable parameters
  - plObs -- list of plates to design
- additional inputs
  - plPlugMap -- (completeTile)
- outputs
  - plPlugMap -- (modified)
  - plOverlay -- Plug plate transparency overlay with connect-the-dot drawing
plug
- plugPre outputs
- additional inputs
  - plPlugMap -- (platePrep)
  - plOverlay -- (platePrep)
- outputs
  - plPlugMap -- (modified)
plateDb
- additional inputs
  - tiTileCenters -- (tile)
  - plObs -- (plate)
  - plPlugMap -- (plate)
- outputs
specDA
- sopPre outputs
- additional inputs
  - plPlugMapM -- (plug)
- outputs
  - sdReport -- Spectro observing report
  - sdR -- Raw spectro data frames
  - idTapeSummary -- nightly log of what run/camcol is on which labeled tape
  - idTarTape (Tar tape logs)
  - opConfig (CCD readout configurations)
  - opECalib (CCD electronic calibrations)
  - opBC (CCD bad pixels)
  - opCCDLog (CCD temperatures and voltages)
  - idDaParam --- all the DA tunable parameters
spectro2d
- spectro2dPre outputs
  - spPlan2d -- processing plan
  - spPlancomb -- combining plan
- additional inputs
  - sdReport (specDA)
  - sdR (specDA)
  - plPlugMap (plug)
- outputs
  - spPlate -- combined spectra over all exposures
spectro1d
- spectro1dPre outputs
  - spPlan1d -- processing plan
  - spParam1d -- tunable parameters
- additional inputs
  - spPlate (spectro2d)
  - plPlugMapM (plug)
- outputs
  - spSpec -- limits -- line measurements and redshift determinations
  - spPlot -- gif and ps plots of the spectra
  - spDiag -- diagnostics
spectrodp
- spectrodp inputs
  - tsObj -- the tsObj files
  - fpAtlas -- atlas images
  - spDiag -- spectro1d summary
  - plPlugMapM -- the plug mapping
  - spSpec -- headers of the spSpec file
- outputs
  - spObj -- tsObj+spDiag+plPlugMapM files for each spectro object
  - spAtlas -- atlas images for each spectro object
  - stars-$label -- spObj for all objects spectro classed as stars
  - gals-$label -- spObj for all objects spectro classed as galaxies
  - qsos-$label -- spObj for all objects spectro classed as quasars
  - unk-$label -- spObj for all objects spectro classed as sky or unkown
  - main_gals-$label -- spObj for all objects spectro classed as galaxies and targeted as main galaxies
  - main_qsos-$label -- spObj for all objects spectro classed as quasars and targeted as main quasars
  - main_lrgs-$label -- spObj for all objects spectro classed as galaxies and targeted as LRG

Databases

SQL Database Loading Architecture

Create Zoom Images
- Directory structure
- inputs
  - fpC -- corrected frame
  - tsField -- limits -- calibrated field info
- outputs
  - Zoom -- Zoom frames
Create imaging CSV
- Directory structure
- inputs
  - tsChunk (ts) -- list of runs and fields
  - tsObj -- limits -- calibrated objects, the SDSS incarnate
  - tsField -- limits -- calibrated field info
- outputs
  - sqlField -- imaging field information
  - sqlSegment -- imaging field information
  - sqlPhotoObj -- imaging field information
  - sqlFieldProfile -- imaging field information
  - sqlPhotoProfile -- imaging field information
  - sqlFirst -- imaging field information
  - sqlRosat -- imaging field information
  - sqlUSNO -- imaging field information
  - sqlMask -- imaging mask information
  - csv_created -- "files are ready for loading" semaphore
Load imaging CSV
- inputs
  - sqlField -- imaging field information
  - sqlSegment -- imaging segment information
  - sqlPhotoObj -- imaging objc information
  - sqlFieldProfile -- imaging field profile information
  - sqlPhotoProfile -- imaging object profile information
  - sqlFirst -- imaging FIRST objects information
  - sqlRosat -- imaging ROSAT objects information
  - sqlUSNO -- imaging USNO objects information
  - sqlMask -- imaging mask information
  - csv_created -- "files are ready for loading" semaphore
- output
  - csv_loaded -- "files have been loaded" semaphore
Create spectroscopy CSV
- Directory structure
- inputs
  - spDiag -- list of fibers
  - spSpec -- limits -- line measurements and redshift determinations
  - spPlatesToLoad.par -- which files to load
- outputs
  - sqlPlate -- Plate information
  - sqlSpecObj -- measured information for spectra
  - sqlSpecLine -- measure lines in spectra
  - sqlSpecLineIndex -- spectra line indeces
  - sqlELRedshift -- emission line redshifts
  - sqlXCRedshift -- cross correlation redshifts
  - csv_created -- "files are ready for loading" semaphore
Load spectroscopy CSV
- inputs
  - plate-mjd.par -- plate status from plate-mapper Description to be added
  - spPlatesToLoad.par -- which files to load
  - plPlugMapM -- plugging information
  - sqlPlate -- Plate information
  - sqlSpecObj -- measured information for spectra
  - sqlSpecLine -- measure lines in spectra
  - sqlSpecLineIndex -- spectra line indeces
  - sqlELRedshift -- emission line redshifts
  - sqlXCRedshift -- cross correlation redshifts
  - csv_created -- "files are ready for loading" semaphore
  - Duplicates List -- a list of duplicate observations for a set of plates
- loading output
  - csv_loaded -- "files have been loaded" semaphore
Create Tiling CSV
- Directory structure
- inputs
  - tsTargets -- (ts)
  - tiTileCenters -- (tile)
  - plPlugMapT -- (completeTile)
- outputs
  - tsTargetsCsv -- List of Targets in CSV format
  - csv_created -- "files are ready for loading" semaphore
Load Tiling CSV
- inputs
  - tsTargetsCsv -- List of Targets in CSV format
  - csv_created -- "files are ready for loading" semaphore
- loading output
  - csv_loaded -- "files have been loaded" semaphore
Chris Stoughton < stoughto@fnal.gov >

Steve Kent < skent@fnal.gov >

Jeff Munn < munn@fnal.gov >

Brian Yanny < yanny@fnal.gov >

Jim Annis < annis@fnal.gov >