From a5a6b3bbcb47c9aa9cff9191da63e9658d701f44 Mon Sep 17 00:00:00 2001 From: Ali Allaoui Date: Fri, 21 Oct 2022 15:03:04 +0200 Subject: [PATCH 1/5] Add warning flags and error codes to redshiftCandidates DM --- datamodel.txt | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/datamodel.txt b/datamodel.txt index ee88c3e7..e73b0489 100644 --- a/datamodel.txt +++ b/datamodel.txt @@ -1087,13 +1087,26 @@ HDU#0 PHDU D1DP_VER KEYWORD Version of the DRP_1DPIPE pipeline DAMD_VER KEYWORD Version of the data model U_PARAM KEYWORD User parameters contents, json - ZWARNING INT64 Quality flag + GALAXY_ZWARNING INT64 Quality flags for galaxy solver + QSO_ZWARNING INT64 Quality flags for QSO solver + STAR_ZWARNING INT64 Quality flags for star solver + GALAXY_LWARNING INT64 Quality flags for galaxy line measurement solver + QSO_LWARNING INT64 Quality flags for QSO line measurement solver + CLASSIFICATION_WARNING INT64 Quality flags for classification solver + GALAXY_ZERROR INT64 Error code for galaxy solver + QSO_ZERROR INT64 Error code for QSO solver + STAR_ZERROR INT64 Error code for star solver + GALAXY_LERROR INT64 Error code for galaxy line measurement solver + QSO_LERROR INT64 Error code for QSO line measurement solver + CLASSIFICATION_ERROR INT64 Error code for classification solver + HDU#1 CLASSIFICATION CLASS KEYWORD Spectro classification: GALAXY, QSO, STAR P_GALAXY KEYWORD Probability to be a galaxy P_QSO KEYWORD Probability to be a QSO P_STAR KEYWORD Probability to be a star + HDU#2 GALAXY_CANDIDATES Binary table [FITS BINARY TABLE] NGALAXY_CANDIDATES Colums for : From a44c92fd09157ddc61a119eb200cd1474b9e11d4 Mon Sep 17 00:00:00 2001 From: Ali Allaoui Date: Wed, 10 Apr 2024 14:12:32 +0200 Subject: [PATCH 2/5] Add error messages --- datamodel.txt | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/datamodel.txt b/datamodel.txt index e73b0489..bdf6d4e5 100644 --- a/datamodel.txt +++ b/datamodel.txt @@ -1098,6 +1098,11 @@ HDU#0 PHDU STAR_ZERROR INT64 Error code for star solver GALAXY_LERROR INT64 Error code for galaxy line measurement solver QSO_LERROR INT64 Error code for QSO line measurement solver + G_ZERR KEYWORD Error message for galaxy solver + Q_ZERR KEYWORD Error message for QSO solver + S_ZERR KEYWORD Error message for star solver + G_LERR KEYWORD Error message for galaxy line measurement solver + Q_LERR KEYWORD Error message for QSO line measurement solver CLASSIFICATION_ERROR INT64 Error code for classification solver From da4adb92a91e66e34753f0d8688e377301118d49 Mon Sep 17 00:00:00 2001 From: Ali Allaoui Date: Wed, 19 Jun 2024 10:53:06 +0200 Subject: [PATCH 3/5] [redshiftCandidates] Add init warning and error --- datamodel.txt | 3 +++ 1 file changed, 3 insertions(+) diff --git a/datamodel.txt b/datamodel.txt index bdf6d4e5..7f881a10 100644 --- a/datamodel.txt +++ b/datamodel.txt @@ -1087,6 +1087,7 @@ HDU#0 PHDU D1DP_VER KEYWORD Version of the DRP_1DPIPE pipeline DAMD_VER KEYWORD Version of the data model U_PARAM KEYWORD User parameters contents, json + INIT_WARNING INT64 Quality flags for spectrum initialization GALAXY_ZWARNING INT64 Quality flags for galaxy solver QSO_ZWARNING INT64 Quality flags for QSO solver STAR_ZWARNING INT64 Quality flags for star solver @@ -1098,11 +1099,13 @@ HDU#0 PHDU STAR_ZERROR INT64 Error code for star solver GALAXY_LERROR INT64 Error code for galaxy line measurement solver QSO_LERROR INT64 Error code for QSO line measurement solver + INIT_ERROR INT46 Error code for spectrum initialization G_ZERR KEYWORD Error message for galaxy solver Q_ZERR KEYWORD Error message for QSO solver S_ZERR KEYWORD Error message for star solver G_LERR KEYWORD Error message for galaxy line measurement solver Q_LERR KEYWORD Error message for QSO line measurement solver + I_ERR KEYWORD Error message for spectrum initialization CLASSIFICATION_ERROR INT64 Error code for classification solver From 8b0cdf708368314c2771e0b2f38732cdd7d201c3 Mon Sep 17 00:00:00 2001 From: Ali Allaoui Date: Tue, 7 Jan 2025 09:56:10 +0100 Subject: [PATCH 4/5] Fix PDF columns definitions --- datamodel.txt | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/datamodel.txt b/datamodel.txt index 7f881a10..09e1715b 100644 --- a/datamodel.txt +++ b/datamodel.txt @@ -1130,7 +1130,7 @@ HDU#2 GALAXY_CANDIDATES Binary table [FITS BINARY TABLE] HDU#3 GALAXY_PDF Binary table [FITS BINARY TABLE] NPIX Colums for : REDSHIFT FLOAT[NPIX] Redshift grid - PDF FLOAT[NPIX] PDF marginalised over all models + ln PDF FLOAT[NPIX] ln(Probability) marginalised over all models HDU#4 GALAXY_LINES Binary table [FITS BINARY TABLE] NLINE Columns for : @@ -1161,7 +1161,7 @@ HDU#5 QSO_CANDIDATES Binary table [FITS BINARY TABLE] HDU#6 QSO_PDF Binary table [FITS BINARY TABLE] NPIX Colums for : REDSHIFT FLOAT[NPIX] Redshift grid - PDF FLOAT[NPIX] PDF marginalised over all models + ln PDF FLOAT[NPIX] ln(Probability) marginalised over all models HDU#7 QSO_LINES Binary table [FITS BINARY TABLE] NLINE Columns for : @@ -1193,7 +1193,7 @@ HDU#8 STAR_CANDIDATES Binary table [FITS BINARY TABLE] HDU#9 STAR_PDF Binary table [FITS BINARY TABLE] NPIX Colums for : VELOCITY FLOAT[NPIX] Radial velocity grid - PDF FLOAT[NPIX] PDF related to best model + ln PDF FLOAT[NPIX] ln(Probability) related to best model -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- From b39a306fc3c40550a920fa9a1be93913c81d7bb6 Mon Sep 17 00:00:00 2001 From: Ali Allaoui Date: Fri, 7 Feb 2025 14:42:48 +0100 Subject: [PATCH 5/5] [redshiftCandidates] Add warning flag description --- datamodel.txt | 164 ++++++++++++++++++++++++++++++++++++++++---------- 1 file changed, 132 insertions(+), 32 deletions(-) diff --git a/datamodel.txt b/datamodel.txt index 09e1715b..a870a96d 100644 --- a/datamodel.txt +++ b/datamodel.txt @@ -1074,39 +1074,39 @@ The path would be The file will have several HDUs: HDU#0 PHDU - tract INT Tract identifier - patch KEYWORD Patch identifier - catId INT Catalog identifier - objId INT Object identifier - nvisit INT Number of visit - vHash KEYWORD 63-bit SHA-1 list of visits - CRPIX1 FLOAT Pixel coordinate of reference point - CRVAL1 FLOAT [m] Coordinate value at reference point - CDELT1 FLOAT [m] Coordinate increment at reference point - D1D_VER KEYWORD Version of the DRP_1D library - D1DP_VER KEYWORD Version of the DRP_1DPIPE pipeline - DAMD_VER KEYWORD Version of the data model - U_PARAM KEYWORD User parameters contents, json - INIT_WARNING INT64 Quality flags for spectrum initialization - GALAXY_ZWARNING INT64 Quality flags for galaxy solver - QSO_ZWARNING INT64 Quality flags for QSO solver - STAR_ZWARNING INT64 Quality flags for star solver - GALAXY_LWARNING INT64 Quality flags for galaxy line measurement solver - QSO_LWARNING INT64 Quality flags for QSO line measurement solver + tract INT Tract identifier + patch KEYWORD Patch identifier + catId INT Catalog identifier + objId INT Object identifier + nvisit INT Number of visit + vHash KEYWORD 63-bit SHA-1 list of visits + CRPIX1 FLOAT Pixel coordinate of reference point + CRVAL1 FLOAT [m] Coordinate value at reference point + CDELT1 FLOAT [m] Coordinate increment at reference point + D1D_VER KEYWORD Version of the DRP_1D library + D1DP_VER KEYWORD Version of the DRP_1DPIPE pipeline + DAMD_VER KEYWORD Version of the data model + U_PARAM KEYWORD User parameters contents, json + INIT_WARNING INT64 Quality flags for spectrum initialization + GALAXY_ZWARNING INT64 Quality flags for galaxy solver + QSO_ZWARNING INT64 Quality flags for QSO solver + STAR_ZWARNING INT64 Quality flags for star solver + GALAXY_LWARNING INT64 Quality flags for galaxy line measurement solver + QSO_LWARNING INT64 Quality flags for QSO line measurement solver CLASSIFICATION_WARNING INT64 Quality flags for classification solver - GALAXY_ZERROR INT64 Error code for galaxy solver - QSO_ZERROR INT64 Error code for QSO solver - STAR_ZERROR INT64 Error code for star solver - GALAXY_LERROR INT64 Error code for galaxy line measurement solver - QSO_LERROR INT64 Error code for QSO line measurement solver - INIT_ERROR INT46 Error code for spectrum initialization - G_ZERR KEYWORD Error message for galaxy solver - Q_ZERR KEYWORD Error message for QSO solver - S_ZERR KEYWORD Error message for star solver - G_LERR KEYWORD Error message for galaxy line measurement solver - Q_LERR KEYWORD Error message for QSO line measurement solver - I_ERR KEYWORD Error message for spectrum initialization - CLASSIFICATION_ERROR INT64 Error code for classification solver + GALAXY_ZERROR INT64 Error code for galaxy solver + QSO_ZERROR INT64 Error code for QSO solver + STAR_ZERROR INT64 Error code for star solver + GALAXY_LERROR INT64 Error code for galaxy line measurement solver + QSO_LERROR INT64 Error code for QSO line measurement solver + INIT_ERROR INT46 Error code for spectrum initialization + G_ZERR KEYWORD Error message for galaxy solver + Q_ZERR KEYWORD Error message for QSO solver + S_ZERR KEYWORD Error message for star solver + G_LERR KEYWORD Error message for galaxy line measurement solver + Q_LERR KEYWORD Error message for QSO line measurement solver + I_ERR KEYWORD Error message for spectrum initialization + CLASSIFICATION_ERROR INT64 Error code for classification solver HDU#1 CLASSIFICATION @@ -1194,6 +1194,106 @@ HDU#9 STAR_PDF Binary table [FITS BINARY TABLE] Colums for : VELOCITY FLOAT[NPIX] Radial velocity grid ln PDF FLOAT[NPIX] ln(Probability) related to best model + + Warning flags : + +**AIR_VACUUM_CONVERSION_IGNORED** + An air to vacuum conversion has been defined in the parameters. + However, it is specified in the input spectrum that the data are in vacuum. + The air to vacuum conversion is therefore ignored. + +**PDF_PEAK_NOT_FOUND** + For a given redshift candidate, in the second pass window, no pdf peak has + been found. + +**ESTIMATED_STD_FAR_FROM_INPUT** + Residual std (obtained from rms) (spectrum - model) very different + (ratio > 1.5 & 1/ 15 <-> 50 + or 50%-) from input spectrum std + +**LINEMATCHING_REACHED_ENDLOOP** + In linematching, when trying to match lines on the spectrum, the maximum + number of iterations has been reached without converging. + +**FORCED_IGNORELINESUPPORT_TO_FALSE** + In line model, for continuum fitting, in the parameters fft processing is + active and ignore line support too (lineModel.continuumFit.ignoreLineSupport). + However masking lines to fit continuum is in this case impossible. + To fix this, ignorelinesupport is forced to False i.e.full spectrum with lines + is used for template fitting. + +**FORCED_CONTINUUM_COMPONENT_TO_FROMSPECTRUM** + In line model, while fitting the continuum, a negative continuum amplitude + has been found. + To fix this, continuumComponent is forced to fromSpectrum. + +**AIR_VACUUM_REACHED_MAX_ITERATIONS** + Wavelengths convertion from air to vacuum is made by iterating an inverse + translation, until a minimal precision is reached or until the max number + of iterations is reached. + The marning means that max precision was not reached at the end ot the iterations. + +**ASYMFIT_NAN_PARAMS** + At least one asymetric fitting param is NaN. All parameters of asymetric line + profile are therefore set to NaN. + +**DELTAZ_COMPUTATION_FAILED** + Some error appeared when trying to calculate Deltaz on a redshift candidate + probability. + +**INVALID_FOLDER_PATH** + Absolute path to piors calibration file does not exist. No priors used + +**FORCED_CONTINUUM_TO_NOCONTINUUM** + The calculated continuum amplitude is weaker than the min continuum amplitude + defined in parameters (`continuumFit.nullThreshold`). + Therefore, continuum is set to noContinuum. + +**FORCED_CONTINUUM_REESTIMATION_TO_NO** + Continuum reestimation was set to "onlyextrema" but it is not possible since + second pass is skipped. It is automatically set to no continuum reestimation. + +**LESS_OBSERVED_SAMPLES_THAN_AMPLITUDES_TO_FIT** + Deficient rank for lines fitting: when trying to fit lines with line model, + the number of parameters to find is greater than the number of observed samples. + +**LBFGSPP_ERROR** + There was an error while fitting whith lbfgspp. We then fix line position and + width and use the initial svd guess for the amplitude. + +**PDF_INTEGRATION_WINDOW_TOO_SMALL** + PDF integration range goes beyond redshift window size. Integration range is + therefore cropped. + If this warning appears many times, consider increasing in the parameter + second pass `halfWindowSize` value. + +**FORCED_POWERLAW_TO_ZERO** + For QSO power law fitting: many samples have fluxes too low compared to SNR to be taken into account. + Power law coefficients are set to zero. + +**UNUSED_PARAMETER** + A parameter has been defined in the parameters file but it is not used. + NB: Some parameters are usefull only under some conditions. + +**SPECTRUM_WAVELENGTH_TIGHTER_THAN_PARAM** + Parameters lambda range goes beyond spectrum range. This means that a part of the lambda range specified in the parameters will not be studied. + +**MULTI_OBS_ARBITRARY_LSF** + Happens in the case of multi obs, for which only one lsf is support. + Only the lsf of the first observation is taken into account and applied + to all observations. + +**NULL_LINES_PROFILE** + One line profile is null (zero flux) on all its samples: the corresponding line is discarded + +**STD_ESTIMATION_FAILED** + The number of samples available to estimate the RMS of the residual is too low. + +**VELOCITY_FIT_RANGE** + The lineMeas velocity fit range does not include the input velocity + (comming from either the previous redhiftSolver stage, input lineMeas catalog). + The range is thus extended to contain this velocity. + + -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-