code and hooks needed for SGA-2024

[moustakas]

@dstndstn I'd like to use this ticket to track some of the (new) code and hooks we're going to need as part of the ongoing SGA-2024 work. At the moment I have two requests, although I'm sure that others will arise:

• It would be great if stage_srcs could produce a catalog of candidate sources (not in the Pickle file) which can then be analyzed, e.g., in a notebook. One suggestion would be to write this catalog to the metrics/cus directory, like the references-*.fits catalog.
• We are going to need the option of using different detection-convolution kernels, e.g., a Gaussian kernel of variable width. Most likely, this should be a command-line option to runbrick, and presumably the choice should be written to the output headers.

@sybenzvi @kadglass @nityar

[dstndstn]

You bet! For the first item, I have a convenient table of sources with a bunch of data (a lot of it coming from reference sources, where they exist). Which of these do you think are worth keeping?

HDU #2  Binary Table:  40 columns x 2 rows
 COL NAME             FORMAT
   1 ra               D
   2 dec              D
   3 ibx              K
(integer brick x coordinate of this source's peak)
   4 iby              K
(integer brick y coordinate)
   5 ref_cat          2A
   6 ref_id           K

  30 jmag             E
   8 hmag             E
  17 kmag             E
(only for Tycho-2)
  15 zguess           E
(Tycho-2 and Gaia: guessed z-band mag)
  35 mag              E
(Tycho-2 / Gaia)

  18 radius           E
  32 radius_pix       K
  10 keep_radius      E
  22 mask_mag         E
(for reference sources, basically how big their masking radii are)

  25 ismedium         L
  27 isbright         L
  29 isgalaxy         B
  11 istycho          L
  16 iscluster        L
  19 islargegalaxy    L
  21 isgaia           L

   9 pmra             E
  26 pmdec            E
  23 parallax         E
  28 phot_g_mean_mag  E
  33 pmra_ivar        E
  12 pmdec_ivar       E
   7 ra_ivar          E
  20 dec_ivar         E
  34 ref_epoch        E
(Gaia only)

  13 donotfit         L
  14 pointsource      L
  24 in_bounds        L
  31 freezeparams     L

  36 blob             J
(integer, which blob is this source in)
  37 blob_y0          K
  38 blob_y1          K
  39 blob_x0          K
  40 blob_x1          K
(integers, brick coordinates of the blob bounding-box)

And what other things would you like to know about sources? S/N or flux in each band?

[moustakas]

It's hard to know what we'll need in the end, but at minimum we'll want central coordinates (ra, dec, ibx, iby) and an estimate of the "size" of the object, which from this table looks like it might just be the blob_* quantities. How do you measure the S/N and flux since there's no source-fitting here, or is it based on the peak flux in each band?

TL;DR, let's just write out something and we'll go from there!

[dstndstn]

Yeah, S/N and flux would just be peak values (in the detection maps, ie, would be the total flux of a point source)

Will push a commit this afternoon!

[dstndstn]

Just pushed 05606ae

[dstndstn]

Reference sources and newly-detected sources differ in some weird ways:

• peaksn is zero for ref sources
• flux_[grz] should be set for both, but for ref sources, they're not from the images, they're from the ref catalog (and may be, for example, the same for each band).

Note that the radius field is only valid for ref sources.

And about size: yeah, currently there's only the blob bounds size. I could try adding in a cheap fwhm measure if you like.

[moustakas]

A very cheap FWHM estimate would be welcome. Could we also write out the blob image / segmentation map during the srcs stage, or will that break something downstream.

[dstndstn]

I'll think about what we can do for FWHM.

For the blob map -- currently it gets written out after fitblobs because sources can get dropped during fitting, so some blob numbers could disappear. Currently, the blob numbers get remapped so that the blob ids are contiguous. I consider that a nice-to-have but not essential feature, so we could just break that and write out the blobmap earlier. OR, you could define a new name for a blob map written out earlier (and probably only write it out when a command-line flag is given).

[moustakas]

Thanks @dstndstn, I'm not convinced that an early blob map is on the critical path, so let's hold off on that for now.