Synthetic 2MASS data were generated in order to evaluate the performance of the active deblending algorithm in PROPHOT. The data were generated based on instrumental parameters appropriate to 2MASS, in terms of PSF shape, noise model (pixel gain, read noise, PSF variance) and frame-to-frame offsets. Synthetic images of blended source pairs were thereby generated for separations in the range 0.2 - 2.0 pixels (i.e., 0.4 - 4.0 arcsec) and magnitudes in the range 8-15. A constant flux ratio was assumed, corresponding to a sec-pri magnitude difference of 0.8 (the mean value for the blends in John Carpenter's data).
The RMS magnitude errors for the primary and secondary were evaluated, and expressed both in magnitudes and in units of the expected error (a-posteriori standard deviation). These and other quantities of interest were tabulated as a function of source separation and primary magnitude, and the results were as follows:
Percentage of cases in which all blended components were found:
| Source separation [pixels] -->
Mag | 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8
-------------------------------------------------------------
8.0 | 0 0 33 37 66 13 3 13 56
9.0 | 0 6 13 44 63 3 0 16 63
10.0 | 0 3 13 51 56 6 0 23 73
11.0 | 0 10 23 58 73 6 0 30 76
12.0 | 0 3 23 41 73 20 0 13 83
13.0 | 0 0 10 44 90 50 17 40 80
14.0 | 0 0 3 27 50 26 17 33 83
15.0 | 0 0 0 0 0 0 6 6 26
Percentage of the above cases in which active deblending done:
| Source separation [pixels] -->
Mag | 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8
-------------------------------------------------------------
8.0 | 0 0 100 100 100 100 100 0 11
9.0 | 0 100 100 100 100 100 0 40 10
10.0 | 0 100 100 100 100 100 0 42 13
11.0 | 0 100 100 100 100 100 0 55 4
12.0 | 0 100 100 100 100 100 0 25 36
13.0 | 0 0 100 100 100 100 80 75 45
14.0 | 0 0 100 100 100 100 100 60 24
15.0 | 0 0 0 0 0 0 50 50 25
Position error [hundredths of a pixel]:
| Source separation [pixels] -->
Mag | 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8
-------------------------------------------------------------
8.0 | 0 0 24 9 6 9 5 4 5
9.0 | 0 11 8 9 10 3 0 6 5
10.0 | 0 65 10 11 8 23 0 35 9
11.0 | 0 20 12 10 8 66 0 26 5
12.0 | 0 6 22 16 8 7 0 6 14
13.0 | 0 0 21 17 9 13 10 19 23
14.0 | 0 0 19 17 16 18 24 27 21
15.0 | 0 0 0 0 0 0 37 25 22
Primary magnitude error [hundredths of a magnitude]:
| Source separation [pixels] -->
Mag | 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8
-------------------------------------------------------------
8.0 | 0 0 24 24 12 13 78 30 15
9.0 | 0 27 23 20 16 12 0 42 14
10.0 | 0 24 16 24 13 4 0 47 17
11.0 | 0 10 45 22 18 42 0 48 20
12.0 | 0 36 20 23 12 23 0 37 27
13.0 | 0 0 43 33 15 19 19 28 25
14.0 | 0 0 10 32 20 35 59 23 31
15.0 | 0 0 0 0 0 0 19 68 47
Secondary magnitude error [hundredths of a magnitude]:
| Source separation [pixels] -->
Mag | 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8
-------------------------------------------------------------
8.0 | 0 0 37 27 15 11 12 6 26
9.0 | 0 46 15 33 27 2 0 26 27
10.0 | 0 100 41 33 24 30 0 12 8
11.0 | 0 25 46 28 19 73 0 18 13
12.0 | 0 27 66 50 16 15 0 13 39
13.0 | 0 0 57 31 12 23 4 17 36
14.0 | 0 0 1 41 23 20 17 29 26
15.0 | 0 0 0 0 0 0 23 62 42
Position error as a percentage of theoretical value:
| Source separation [pixels] -->
Mag | 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8
-------------------------------------------------------------
8.0 | 0 0 304 202 160 186 82 72 108
9.0 | 0 144 123 173 251 98 0 98 101
10.0 | 0 833 172 210 183 166 0 141 109
11.0 | 0 274 205 170 208 124 0 150 109
12.0 | 0 75 293 205 141 129 0 90 122
13.0 | 0 0 304 235 93 139 122 104 135
14.0 | 0 0 153 107 123 105 118 106 121
15.0 | 0 0 0 0 0 0 116 63 92
Primary magnitude error as a percentage of theoretical value:
| Source separation [pixels] -->
Mag | 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8
-------------------------------------------------------------
8.0 | 0 0 229 268 182 154 748 306 218
9.0 | 0 106 208 211 287 239 0 358 198
10.0 | 0 556 270 322 222 66 0 212 236
11.0 | 0 197 422 244 197 53 0 220 242
12.0 | 0 169 343 260 148 222 0 294 241
13.0 | 0 0 450 258 152 203 257 289 204
14.0 | 0 0 94 127 132 172 192 196 158
15.0 | 0 0 0 0 0 0 125 165 136
Secondary magnitude error as a percentage of theoretical value:
| Source separation [pixels] -->
Mag | 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8
-------------------------------------------------------------
8.0 | 0 0 269 229 222 100 167 97 233
9.0 | 0 201 60 198 473 44 0 259 261
10.0 | 0 511 181 257 307 178 0 63 139
11.0 | 0 221 583 218 239 63 0 131 206
12.0 | 0 124 712 253 145 184 0 120 330
13.0 | 0 0 370 317 75 121 41 175 164
14.0 | 0 0 10 168 96 113 81 474 104
15.0 | 0 0 0 0 0 0 81 118 132
A few things to note from the above:
(1) There appears to be the same trend which was apparent from the real data (i.e., the 2MASS/Carpenter comparison), whereby the relative magnitude error (in units of sigma) increases with decreasing source separation.
(2) The actual magnitude errors are quite consistent with those obtained
from the real data, but are larger than the theoretical values by typically
a factor of 2.
UPDATES:
=======
AUG 14
======
ERRORS OF TOTAL BLEND FLUX
The errors of the total blend flux were analysed both for the synthetic data and real (2MASS+Carpenter) data at K-band.
(a) Comparison of total blend flux for 2MASS (deblended) and Carpenter data:
Figure 5 shows a plot of the magnitude residual (2MASS - Carpenter) as a function of total blend magnitude.
In order to show how these magnitude residuals compare with the theoretical errors, the next figure (Figure 6) shows a plot of magnitude error divided by the expected sigma, taking into account the photometric errors of both 2MASS and the Carpenter results. This figure shows that the magitude residuals are entirely consistent with expectation.
(b) Total blend flux errors for synthetic data:
The total-blend magnitude errors have been evaluated for the K-band synthetic data, both in absolute units and in units of the expected error. The results are tabulated as follows:
Blend magnitude error [hundredths of a magnitude]:
| Source separation [pixels] -->
Mag | 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8
-------------------------------------------------------------
8.0 | 0 0 13 6 7 11 40 18 10
9.0 | 0 8 11 10 6 7 0 35 13
10.0 | 0 3 9 9 8 10 0 25 13
11.0 | 0 7 9 9 9 11 0 32 15
12.0 | 0 11 6 11 7 14 0 24 26
13.0 | 0 0 12 10 9 11 14 18 19
14.0 | 0 0 7 10 13 18 35 17 24
15.0 | 0 0 0 0 0 0 20 24 31
Blend magnitude error as a percentage of theoretical value:
| Source separation [pixels] -->
Mag | 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8
-------------------------------------------------------------
8.0 | 0 0 97 53 91 89 397 139 105
9.0 | 0 35 64 70 83 130 0 271 127
10.0 | 0 16 43 81 80 26 0 131 169
11.0 | 0 68 44 66 93 8 0 159 164
12.0 | 0 42 64 68 69 121 0 151 132
13.0 | 0 0 55 50 64 100 131 118 93
14.0 | 0 0 44 44 51 79 99 111 84
15.0 | 0 0 0 0 0 0 65 50 58
These tables show that the magnitude errors of the total blend are
reasonable, and consistent with theoretical expectation.
AUG 24
======
PROBLEMS RELATED TO COMPLETENESS
The results with synthetic data, as presented in the Aug 7 update, indicated a severe dip in the completeness at a source separation of 1.4 pixels. The relevant portion of the results was as follows:
Percentage of cases in which all blended components were found:
| Source separation [pixels] -->
Mag | 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8
-------------------------------------------------------------
8.0 | 0 0 33 37 66 13 3 13 56
9.0 | 0 6 13 44 63 3 0 16 63
10.0 | 0 3 13 51 56 6 0 23 73
11.0 | 0 10 23 58 73 6 0 30 76
12.0 | 0 3 23 41 73 20 0 13 83
13.0 | 0 0 10 44 90 50 17 40 80
14.0 | 0 0 3 27 50 26 17 33 83
15.0 | 0 0 0 0 0 0 6 6 26
As it turns out, this dip is an indirect consequence of a long-standing typographical error in PROPHOT. This error occurred in the function subroutine FUNK which calculates the chi squared corresponding to an assumed source position (or set of positions in the case of either active or passive deblending). The section in which the fluxes are estimated contains the following piece of code:
c Now calculate the right hand sides. do j = 1,nblend xij = p(2*j-1) etak = p(2*j)
In the 4th line, however, the variable "etak", which represents the y-coordinate of the source, should have been "etaj". In the case of single sources, the location "etak" contained the correct value, and hence the photometry was not affected (which is why the problem has not been detected over the years). In the case of blended sources (either passive or active), however, the typo would have had serious consquences. It would have resulted in an error in flux estimation which depended on the relative y offsets of the components.
In the present tests, if the position angles of the separation vectors of the synthetic blends had been random, the above effect would have given rise to a random error. However, the synthetic data were generated in such a way that the position angle was correlated with the source separation. Specifically, as the scan progressed, the separation vector was systematically rotated and scaled so as to give a range of angles and separations for test purposes. It simply turned out that at a separation of 1.4 pixels, the separation vector was parallel to the y-axis, thus maximizing the y-offset-related error.
After fixing this, the completeness table then looked like:
Percentage of cases in which all blended components were found:
| Source separation [pixels] -->
Mag | 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8
-------------------------------------------------------------
7.0 | 0 0 0 0 3 0 3 13 56
8.0 | 0 3 16 17 20 10 20 36 63
9.0 | 0 0 23 62 70 80 75 80 93
10.0 | 0 3 20 65 80 80 93 90 100
11.0 | 0 0 10 65 90 93 96 100 100
12.0 | 0 0 6 75 96 96 96 96 100
13.0 | 0 0 6 48 80 100 100 96 100
14.0 | 0 0 0 3 6 23 58 70 83
Happily, the completeness dip at 1.4 pixels has now disappeared. Unhappily, though, there is a falloff in completeness with increasing source brightness (the range of magnitudes has been adjusted to go down to 7, so as to better illustrate this effect).
This problem turned out to be a dynamic range effect in the noise-model
variance, resulting from the fact that the noise model for bright sources
is PSF-dominated, and the variance is then proportional to the square
of the flux. This causes precision problems in the computations, especially
in the matrix inversion step. One solution is to convert much of the
parameter-estimation section of the code to double precision. A simpler
solution, which I believe is satisfactory, is to place a ceiling on the
dynamic range of the variance. A value of 1000 seems to preserve the
information content while not running afoul of single-precision limitations.
Based on that modification, the completeness table now looks quite
satisfactory. The complete (no pun intended) set of results of the
synthetic data analysis is as follows:
Note that not only has the completeness been drastically improved, and shows
no spurious trends, but the accuracy of the primary flux estimates has
been improved by a factor of 2.
Additional note: Although the completeness dip turned out not to be due to
an undersized solution radius (i.e. radius of the "data circle" -- the standard
value is 2 pixels), I have
increased this radius to 3 pixels for active deblending in order to avoid
clipping the data for the secondary components. This was an oversight
in the previous runs. For the final version of prophot, we need to retain
the 2-pixel radius for single sources (as a computational economy measure),
but increase it to 3 (or possibly even 4) pixels when the single-source
chi squared indicates the need for active deblending.
APR 10
PROBLEM WITH UNDERESTIMATED FLUX ERROR BARS
Some recent results with real data indicate that the flux errors are being
underestimated. See Figure 4c of:
http://spider.ipac.caltech.edu/staff/laurent/QA/Deblend/.
This behavior is consistent with the results from synthetic
data -- see the last two tables from the Aug 24 update immediately above, which
show the primary and secondary magnitude errors as a percentage of the
theoretical values. These tables show that:
(1) The quoted primary flux errors are reasonable.
(2) The quoted secondary flux errors are substantially underestimated.
(3) The degree by which the secondary flux errors are underestimated
increases with increasing source brightness.
Since the brightest sources are PSF error dominated, items (2) and (3)
suggest a problem with the PSF error term for the secondary component
in the deblending noise model. A check of the code confirmed this, and
the problem was rectified.
APR 13
Running the corrected PROPHOT on the same synthetic data as above produced
the following tables of relative flux errors:
Percentage of cases in which all blended components were found:
| Source separation [pixels] -->
Mag | 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8
-------------------------------------------------------------
7.0 | 0 0 20 75 90 93 96 90 96
8.0 | 0 0 20 62 83 83 100 93 96
9.0 | 0 3 20 65 83 90 96 93 100
10.0 | 0 3 20 65 80 80 93 93 100
11.0 | 0 0 10 65 90 93 96 100 100
12.0 | 0 0 6 75 96 96 96 96 100
13.0 | 0 0 6 48 80 100 100 100 100
14.0 | 0 0 0 3 6 23 58 70 83
Percentage of the above cases in which active deblending done:
| Source separation [pixels] -->
Mag | 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8
-------------------------------------------------------------
7.0 | 0 0 100 100 100 100 96 77 20
8.0 | 0 0 100 100 100 100 100 67 13
9.0 | 0 100 100 100 100 100 100 75 16
10.0 | 0 100 100 100 100 100 100 71 10
11.0 | 0 0 100 100 100 100 100 73 6
12.0 | 0 0 100 100 100 100 96 79 26
13.0 | 0 0 100 100 100 100 100 73 13
14.0 | 0 0 0 100 100 100 94 38 12
Position error [hundredths of a pixel]:
| Source separation [pixels] -->
Mag | 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8
-------------------------------------------------------------
7.0 | 0 0 6 6 8 8 6 4 4
8.0 | 0 0 7 7 8 7 6 4 4
9.0 | 0 6 9 9 8 7 6 4 5
10.0 | 0 7 10 7 7 6 7 4 4
11.0 | 0 0 10 9 8 5 4 4 3
12.0 | 0 0 4 8 7 9 5 6 4
13.0 | 0 0 14 14 12 9 7 9 6
14.0 | 0 0 0 33 19 12 13 13 13
Primary magnitude error [hundredths of a magnitude]:
| Source separation [pixels] -->
Mag | 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8
-------------------------------------------------------------
7.0 | 0 0 8 8 7 9 10 4 7
8.0 | 0 0 10 12 8 7 6 5 5
9.0 | 0 10 11 10 8 10 10 6 8
10.0 | 0 6 17 10 9 7 7 9 5
11.0 | 0 0 15 12 10 7 6 5 5
12.0 | 0 0 10 11 11 6 6 7 5
13.0 | 0 0 9 12 5 7 7 6 7
14.0 | 0 0 0 2 17 6 10 10 16
Secondary magnitude error [hundredths of a magnitude]:
| Source separation [pixels] -->
Mag | 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8
-------------------------------------------------------------
7.0 | 0 0 25 21 21 19 10 6 5
8.0 | 0 0 26 23 22 15 10 6 5
9.0 | 0 21 21 24 18 9 11 5 5
10.0 | 0 15 32 19 18 11 8 5 4
11.0 | 0 0 32 24 21 10 7 7 4
12.0 | 0 0 15 23 15 11 6 10 7
13.0 | 0 0 32 25 15 16 9 14 10
14.0 | 0 0 0 22 26 13 16 19 19
Position error as a percentage of theoretical value:
| Source separation [pixels] -->
Mag | 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8
-------------------------------------------------------------
7.0 | 0 0 100 155 251 308 226 271 179
8.0 | 0 0 126 163 227 245 271 206 120
9.0 | 0 12 146 197 225 219 246 168 123
10.0 | 0 100 154 159 208 208 309 133 106
11.0 | 0 0 120 184 195 158 126 105 99
12.0 | 0 0 41 143 144 155 119 97 77
13.0 | 0 0 158 172 125 108 96 79 99
14.0 | 0 0 0 110 113 79 87 110 105
Primary magnitude error as a percentage of theoretical value:
| Source separation [pixels] -->
Mag | 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8
-------------------------------------------------------------
7.0 | 0 0 127 112 135 147 149 75 91
8.0 | 0 0 142 140 148 110 92 103 85
9.0 | 0 212 132 151 129 145 98 94 116
10.0 | 0 139 147 136 147 118 111 72 83
11.0 | 0 0 63 156 139 101 78 68 88
12.0 | 0 0 196 128 110 88 75 74 68
13.0 | 0 0 92 193 58 74 84 71 62
14.0 | 0 0 0 17 114 47 80 77 142
Secondary magnitude error as a percentage of theoretical value:
| Source separation [pixels] -->
Mag | 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8
-------------------------------------------------------------
7.0 | 0 0 181 182 328 471 227 313 272
8.0 | 0 0 223 166 312 268 310 332 127
9.0 | 0 228 139 245 252 226 345 244 99
10.0 | 0 194 162 190 278 265 284 123 115
11.0 | 0 0 148 183 248 220 163 100 106
12.0 | 0 0 135 193 144 136 123 105 85
13.0 | 0 0 206 211 108 128 71 61 76
14.0 | 0 0 0 84 125 80 93 93 104
======
======
Primary magnitude error as a percentage of theoretical value:
| Source separation [pixels] -->
Mag | 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8
-------------------------------------------------------------
7.0 | 0 0 202 154 160 168 118 108 84
8.0 | 0 0 205 196 154 119 102 85 75
9.0 | 0 0 131 179 143 164 105 94 87
10.0 | 0 0 225 166 137 108 103 82 83
11.0 | 0 0 129 149 129 124 144 107 78
12.0 | 0 0 94 164 102 97 87 107 70
13.0 | 0 0 0 103 61 69 123 87 78
14.0 | 0 0 0 0 4 76 122 70 105
Secondary magnitude error as a percentage of theoretical value:
| Source separation [pixels] -->
Mag | 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8
-------------------------------------------------------------
7.0 | 0 0 307 199 205 234 151 138 173
8.0 | 0 0 271 275 223 136 108 142 150
9.0 | 0 0 230 209 222 140 152 140 159
10.0 | 0 0 266 228 159 159 116 93 127
11.0 | 0 0 230 202 179 175 129 109 111
12.0 | 0 0 67 215 123 112 89 127 91
13.0 | 0 0 0 189 68 111 67 82 86
14.0 | 0 0 0 0 177 216 133 98 99
It is apparent that the secondary flux errors are now much more in line
with expectation.