Database Contents & Structure¶

The vdp database keeps a record of every VLITE image, detected source, association between detections of the same source, and match to sources from other radio catalogs. The Tables section below lists every table in the database along with a description of the columns. A spreadsheet view of the database organization can be seen here.

Tables¶

assoc_source¶

This is the VLITE catalog. It contains a single entry for every source detected by associating detections from multiple images in the same spatial resolution class.

id: unique row id number
ra: weighted average of all right ascension measurements (degrees)
e_ra: error on the right ascension weighted average (degrees)
dec: weighted average of all declination measurements (degrees)
e_dec: error on the declination weighted average (degrees)
res_class: spatial resolution class (see Stage 3: Source Association)
ndetect: number of images in which the source was detected
nmatches: number of matches to other radio catalog sources

catalog_match¶

This table keeps a record of each radio catalog source that was successfully cross-matched with a VLITE source in the assoc_source table.

id: unique row id number
catalog_id: references the id column of the radcat.catalogs table
src_id: references the id column of the radio catalog table (i.e. radcat.nvss) which corresponds to the catalog_id
assoc_id: references the id column of the assoc_source table; deletion of row in assoc_source table cascades to all catalog_match rows with that assoc_id
separation: angular separation between the weighted average VLITE source position and the other radio catalog source position (arcseconds)

corrected_flux¶

All flux measurements from PyBDSF are corrected for VLITE’s primary beam response. A scale factor is applied to account for decreasing sensitivity with increasing distance from the image’s pointing center. A 20% systematic uncertainty is also added (in quadrature) to the PyBDSF statistical uncertainties for every flux measurement error.

src_id: non-unique id number assigned by PyBDSF to each detected source in an image; uniquely referenced by combination of (src_id, image_id), same as in detected_source table (deletions cascade from that table)
isl_id: non-unique id number assigned by PyBDSF to each detected island in an image; uniquely referenced by combination of (isl_id, image_id), same as in the detected_source and detected_island tables (deletions cascade from detected_island)
image_id: unique number identifying the image; references the id column in the image table and the image_id columns in the detected_source and detected_island tables
total_flux: primary beam corrected total integrated flux (mJy)
e_total_flux: 1-sigma statistical + 20% systematic error on the corrected total integrated flux (mJy)
peak_flux: primary beam corrected peak flux density per beam (mJy/beam)
e_peak_flux: 1-sigma statistical + 20% systematic error on the corrected peak flux (mJy/beam)
isl_total_flux: primary beam corrected total integrated flux density of the island in which the source is located (mJy)
isl_e_total_flux: 1-sigma statistical + 20% systematic error on the corrected island integrated flux (mJy)
isl_rms: average background rms noise of the island (mJy/beam)
isl_mean: avearage background mean value of the island (mJy/beam)
isl_resid_rms: average residual background rms noise of the island (mJy/beam)
isl_resid_mean: average residual background mean value of the island (mJy/beam)
distance_from_center: angular separation between the source position and the image pointing center (degrees)
snr: signal-to-noise ratio of the source detection; defined as (peak_flux - isl_mean) / isl_rms

detected_island¶

PyBDSF finds contiguous groups of pixels with flux values above a certain detection threshold which it identifies as islands. Sources are extracted from these islands by fitting them with Gaussians. This table stores all PyBDSF output pertaining to the islands detected in an image (see the PyBDSF documentation for a complete list).

isl_id: non-unique id number assigned by PyBDSF to each detected island in an image; uniquely referenced by combination of (isl_id, image_id) & deletions cascade to corrected_flux table
image_id: unique number identifying the image; references the id column in the image table (deletions cascade from that table)
total_flux: total integrated flux density of the island (mJy); PyBDSF output column Isl_Total_flux
e_total_flux: 1-sigma statistical error on the total integrated flux density (mJy); PyBDSF output column E_Isl_Total_flux
rms: average background rms noise of the island (mJy/beam); PyBDSF output column Isl_rms
mean: average background mean value of the island (mJy/beam); PyBDSF output column Isl_mean
resid_rms: average residual background rms noise of the island (mJy/beam); PyBDSF output column Resid_Isl_rms
resid_mean: average residual background mean value of the island (mJy/beam); PyBDSF output column Resid_Isl_mean

detected_source¶

Properties of sources detected in an image are derived from Gaussian fits to islands of pixels. This table records PyBDSF output pertaining to the sources formed from fitting Gaussians (see the PyBDSF documentation for a complete list). Descriptions are either adapted or straight from their documentation.

src_id: non-unique id number assigned by PyBDSF to each detected source in an image; uniquely referenced by combination of (src_id, image_id) & deletions cascade to corrected_flux table
isl_id: non-unique id number assigned by PyBDSF to each detected island in an image; uniquely referenced by combination of (isl_id, image_id) & deletions cascade to corrected_flux table & from detected_island table
image_id: unique number identifying the image; references the id column in the image table
ra: source right ascension (degrees); PyBDSF output column RA
e_ra: error on the right ascension (degrees); PyBDSF output column E_RA
dec: source declination (degrees); PyBDSF output column DEC
e_dec: error on the declination (degrees); PyBDSF output column E_DEC
total_flux: total integrated flux (mJy); PyBDSF output column Total_flux
e_total_flux: 1-sigma statistical error on the total integrated flux; PyBDSF output column E_Total_flux
peak_flux: peak flux density per beam of the source (mJy/beam); PyBDSF output column Peak_flux
e_peak_flux: 1-sigma statistical error on the peak flux (mJy/beam); PyBDSF output column E_Peak_flux
ra_max: right ascension of the maximum of the source (degrees); PyBDSF output column RA_max
e_ra_max: 1-sigma statistical error on the right ascension of the maximum (degrees); PyBDSF output column E_RA_max
dec_max: declination of the maximum of the source (degrees); PyBDSF output column DEC_max
e_dec_max: 1-sigma statistical error on the declination of the maximum (degrees); PyBDSF output column E_DEC_max
maj: the FWHM of the major axis of the source (arcsec); PyBDSF output column Maj
e_maj: 1-sigma statistical error on the FWHM of the source major axis (arcsec); PyBDSF output column E_Maj
min: the FWHM of the minor axis of the source (arcsec); PyBDSF output column Min
e_min: 1-sigma statistical error on the FWHM of the source minor axis (arcsec); PyBDSF output column E_Min
pa: position angle of the source major axis measured east of north (degrees); PyBDSF output column PA
e_pa: 1-sigma statistical error on the source major axis position angle (degrees); PyBDSF output column E_PA
dc_maj: the FWHM of the deconvolved major axis of the source (arcsec); PyBDSF output column DC_Maj
e_dc_maj: 1-sigma statistical error on the FWHM of the source deconvolved major axis (arcsec); PyBDSF output column E_DC_Maj
dc_min: the FWHM of the deconvolved minor axis of the source (arcsec); PyBDSF output column DC_Min
e_dc_min: 1-sigma statistical error on the FWHM of the source deconvolved minor axis (arcsec); PyBDSF output column E_DC_Min
dc_pa: position angle of the source deconvolved major axis measured east of north (degrees); PyBDSF output column DC_PA
e_dc_pa: 1-sigma statistical error on the source deconvolved major axis position angle (degrees); PyBDSF output column E_DC_PA
code: defines the source structure:
- ‘S’ = a single-Gaussian source that is the only source in the island
- ‘C’ = a single-Gaussian source in an island with other sources
- ‘M’ = a multi-Gaussian source
assoc_id: references the id column of the assoc_source; rows with the same assoc_id value are associated detections of the same source in different images

error¶

This is a look-up table containing explanations for each possible error_id in the image table.

id: referenced by the error_id column in the image table; updates cascade to that table
reason: reason why an image was given that particular error_id

image¶

This table provides a record of every image processed by vdp. The more useful keywords from the image header are summarized in the table, as well.

id: unique identifier for each new image; deletions cascade to the detected_island and vlite_unique tables
filename: image filename with full directory path
imsize: image size in pixels (pixels); header keywords (NAXIS1, NAXIS2)
obs_ra: right ascension of image pointing center (degrees); header keyword OBSRA
obs_dec: declination of image pointing center (degrees); header keyword OBSDEC
pixel_scale: number of arcseconds spanned by each pixel in the image (arcsec/pixel); header keyword CDELT1 or CDELT2
object: name of the object being observed as given by the primary observer; header keyword OBJECT
obs_date: date observations were acquired formatted as yyyy-mm-dd; header keyword DATE-OBS
map_date: date image was created formatted as yyyy-mm-dd; header keyword DATE-MAP
obs_freq: rest frequency of the VLITE observations (MHz); header keyword RESTFREQ or CRVAL3 or CRVAL4
primary_freq: frequency of the primary observations (GHz); taken from VLITE image filename
bmaj: image beam semi-major axis (arcsec); header keyword BMAJ or CLEANBMJ
bmin: image beam semi-minor axis (arcsec); header keyword BMIN or CLEANBMN
bpa: image beam position angle measured east of north (degrees); header keyword BPA or CLEANBPA
noise: estimate of the rms noise measured in the center of the image (mJy/beam); header keyword ACTNOISE
peak: flux value of the brightest pixel in the image (mJy/beam); header keyword PEAK or DATAMAX
config: VLA configuration; header keyword CONFIG
nvis: number of visibilities in the data before imaging; header keyword NVIS
mjdtime: Modified Julian Date at the start of the observations; header keyword MJDTIME + header keyword STARTIME
tau_time: total integration time on source (sec); header keyword TAU_TIME
duration: total duration of the observations (sec); header keyword DURATION
radius: size of the circular field-of-view used in source finding (degrees); calculated as ((NAXIS2 / 2.) * scale) * CDELT2, where scale is defined in the pybdsf_params section of the configuration file
nsrc: number of sources found in the image by PyBDSF
rms_box: size and step size of the box used by PyBDSF to estimate the image background mean and noise (pixels)
stage: value of the highest vdp stage completed on the image:
- 1 = reading the image
- 2 = source finding
- 3 = source association
- 4 = catalog matching
catalogs_checked: list of the names of other radio catalogs that have been checked for sources which can be positionally matched to the sources detected in this image
error_id: value assigned if the image fails one of the quality checks; references the id column of the error table & is updated if that table is updated
nearest_problem: name of the nearest source which is known to cause imaging problems for VLITE
separation: angular separation between the VLITE image pointing center and the nearest known problem source (degrees)

run_config¶

The contents of the configuration file are stored in this table each time the pipeline is executed. Other information about the run of the pipeline like the start time, execution time, and number of images read are is also recorded.

id: unique identifier corresponding to each run of vdp
config_file: name of the configuration file used for the run
log_file: name of the log file
start_time: date and time the run was started; formatted as yyyy-mm-dd hh:mm:ss
execution_time: length of the time taken to execute the full run; formatted as hh:mm:ss.s
nimages: number of images read during the run
stages: contents of the configuration file stages section stored as key-value pairs
options: contents of the configuration file options section stored as key-value pairs
setup: contents of the configuration file setup section stored as key-value pairs
pybdsf_params: contents of the configuration file pybdsf_params section stored as key-value pairs
image_qa_params:contents of the configuration file image_qa_params section stored as key-value pairs

vlite_unique¶

Sources detected in VLITE images which are not successfully matched with any sources in other radio catalogs are deemed to be “VLITE unique”, or VU, sources. These sources are placed into their own table to quickly isolate transient candidates, steep spectrum, or other sources of potential interest. A VLITE source is first added to this table when no matches are found during the catalog matching stage. A new entry for the VU source is added to the table for every image which could have or does contain the same VU source.

id: unique row id number
image_id: unique identifier for the VLITE image; references the id column of the image table & deletions cascade from that table
assoc_id: references the id column of the assoc_source table in which the source has nmatches = 0; deletions cascade from that table
detected: boolean True or False identifying whether or not the source was detected in the image

The “radcat” Schema¶

Cross-matching of VLITE sources with sources in other radio catalogs and surveys is done within the database taking advantage of the Q3C spatial indexing and functions. To enable this, all radio catalogs used for comparison must be stored in the same database as the VLITE sources. Since these catalogs are not part of the same organizational structure as the main VLITE database, they are stored in a separate schema with the name “radcat”. Every survey/catalog is a separate table with the same format in this schema. There is also a table radcat.catalogs which provides a list of every table contained in the schema along with information about that catalog (telescope, frequency, spatial resolution, publication reference).

radcat.catalogs¶

This table contains information about every radio catalog that is available for cross-matching with VLITE sources. It can be used to get the name of the catalog referenced by the catalog_id column in the catalog_match table.

id: unique row id to identify the catalog
name: name of the survey/catalog and table in the “radcat” schema
telescope: name of the radio telescope used to acquire the catalog data
frequency: frequency of the catalog observations (MHz)
resolution: approximate spatial resolution of the catalog observations; used to ensure VLITE sources are only cross-matched with sources from radio catalogs in the same resolution class
reference: reference to the publication containing the catalog

radcat.[catalog_name]¶

Each survey/catalog included in the “radcat” schema is stored in a separate table of the name radcat.[catalog_name], where [catalog_name] is the name of the catalog as it appears in the radcat.catalogs table. Each one of these tables has the same format with the following columns:

id: row id number of the source referenced by the src_id column of the catalog_match table
name: name given to the source
ra: source right ascension (degrees)
e_ra: error on the source right ascension (degrees)
dec: source declination (degrees)
e_dec: error on the source declination (degrees)
total_flux: total integrated flux (mJy)
e_total_flux: error on the total integrated flux (mJy)
peak_flux: peak flux density per beam (mJy/beam)
e_peak_flux: error on the peak flux density (mJy/beam)
maj: size of the source semi-major axis (arcsec)
e_maj: error on the semi-major axis size (arcsec)
min: size of the source semi-minor axis (arcsec)
e_min: error on the semi-minor axis size (arcsec)
pa: source position angle (degrees)
e_pa: error on the position angle
rms: local noise estimate, or the rms noise in the image (mJy/beam)
field: name of the field, or image, where the source was detected
catalog_id: id number of the catalog that the source is in; references the id column of the radcat.catalogs table