=======================================================
BCC-UFig v0.4 -- release notes (Jun 29 2014)
=======================================================

** Conditions of Use **

These catalogs are provided to DES collaborators with the understanding 
that they can be used for any internal purpose. Results from BCC-UFig may 
be used under the DES Publication Policy. This policy requires collaboration 
with the authors of the simulation, and we strongly encourage anyone 
intending to use the simulations in published work to email Chihway Chang 
(chihway.chang@phys.ethz.ch) when you start your project to let us know 
how they will be used. 

=======================================================

Change from v0.3:

- using new systematics maps from Boris
  (see https://cdcvs.fnal.gov/redmine/projects/des-lss/wiki/Systematics_in_SVA1)\

- calibrate seeing definition between Boris’ maps and true seeing (see 4. below)

- the field centers are aligned with the SV-A1 coadd tiles (see 4. below)

- BCC galaxy sizes are scaled (as a function of mag_i) to match Balrog (see 6. below)

- bug in ufig sersic profile fixed (see detail in wiki page)

- use the "effective gain" when running source extractor

- add galaxy ellipticity in input files 

=======================================================

1. Scope:

- Taking the BCC galaxy catalogs and turning them into UFig 
  images and catalogs similar to the data products released 
  by DESDM in SV-A1

- Input major image properties of that measured in SV-A1

- Simplification in many aspects (see 2.) -- we are NOT making 
  images identicle to data!! But should be good to answer many 
  data-related questions under a controlled environment.    


2. Known issues/simplifications:

- the PSF, background and depth are constant in each image

- we do not have cosmic rays, bad/hot pixels, airplane tracks, 
  satellite tracks, cross-talk, tree rings etc. etc.

- the PSF is a simple Moffat with beta=3.5 

- we simulate directly the coadds and bypass SCAMP and SWARP, 
  but we do try to simulate the correlated noise from stacking

- there are 10% of the galaxies in BCC that are small and given 
  a fixed size of 0.001", which makes them effectively look 
  like stars

- galaxy sersic indicies are assigned based on model in 
  Berge et al. (2010) -- weak dependence on I-band magnitude 
  and fixed across filter bands

- there are regions in BCC that have excess stars, we are not 
  sure whether this is significant in the SV footprint

- all source extractor was run with SEEING=1.0 and 
  MAG_ZEROPOINT=[the input mag zero point]

- noise-level is adjust so that the 2” aperture limiting magnitude 
  is close to that measured in SV at each position, other 
  limiting magnitudes may or may not match

- in both SWARP and Source Extractor processes we do not use a 
  weight map


3. File structure and format:

- all files are located here: 
  http://www.phys.ethz.ch/~ast/cosmo/ufig_bcc/v0.4/sva1/

- the directory structure goes like:
  sva1/survey_${filter}/ufig_${ufigID}

- in each ufig_${ufigID} directory, there are the following files:
  -- ufig_${ufigID}_${filter}.fits.gz —> image (~355M) 
  -- ufig_${ufigID}_${filter}.sexcat.gz —> Source Extractor catalogs (~26M) 
  -- ufig_${ufigID}_${filter}_matching.fits.gz —> matching file (~2.2M)
  -- input_gal_ufig_${ufigID}_${filter}.fits.gz —> input galaxy catalog (~2M)
  -- input_star_ufig_${ufigID}_${filter}.fits.gz —> input star/quasar catalog (~200k)
  (images and other intermediate files can be provided upon request)

- lookup file: matched_ufig_BCC_sva1_25062014.txt

- Source Extractor configuration files: bcc_ufig_config.tar.gz

- download script: download_ufig_0.4.py

- input and measured depth for each field: check_depth.txt
  -- columns are: ufigID, [input limit_mag_aper_4, measured limit_mag_aper_4, error] 
  for grizY in this order

4. Usage of lookup file:

The lookup file contains the main input parameters to the each 
of the images, the columns are: 

- ufigID
- DES coadd tile ID
- ra (deg)
- dec (deg)
- bcc catalog ID
- good tile or not
- depth in grizY (2" aperture magnitude)
- seeing in grizY (arcsec)
- airmass in grizY
- background noise in grizY 
- magzero in grizY

Note that the "seeing" here are taken from the FWHM parameter in 
the IMAGE table in the database (from Boris' maps), which from our 
calibration is ~1.1235 times the seeing input that gives an approximate 
match to data in the FLUX_RADIUS size of the stars. 

5. Usage of matching file:

The matching file contains the matching between the objects 
detected in Source Extractor and the input objects from BCC. 
The columns with "SEXCAT" comes from the source extractor catalogs 
while the colums with "BCCCAT" comes from BCC. The columns are 
mostly self-explanatory, I only explain a few confusing ones:

- SEXCAT_ID: the NUMBER in the source extractor catalog
- CLASS: 0 for galaxies, 1 or stars AND quasars, -1 for not matched
- BCCCAT_CATID: the healpix index for the BCC catalog
- BCCCAT_ID: the object ID in the BCC catalogs, note that galaxies 
  and quasars have indices, but not stars 
- BCCCAT_INPUTID: the index of this object in the ‘input_gal_ufig*’ 
  file and the ‘input_star_ufig*’ catalogs

Note that we match the Source Extractor catalog back to BCC and 
record the _closest_ object, so one can cut on the separation 
between the measured RA/DEC and BCC RA/DEC (eg. 2") to select 
objects that are really matched.


6. Usage of the input file:

The input file records all objects that were put into the UFig 
image. If one mask on the BCCCAT_INPUTID in the matching file 
(see 5.), then the difference is are those that are NOT detected 
because of depth, blending, etc. There is one input file for 
stars/quasars and one for galaxies.

The columns for the both files are mostly self-explanatory, I 
only explain a few confusing ones here:

- BCC_CATID: the healpix index for the BCC catalog
- BCCID: the object ID in the BCC catalogs
- LMAG: lensed magnitude, ie. the input magnitude to UFig
- IMAG: i-band lensed magnitude, used for sersic index model
- SERSIC: sersic index n for galaxies
- SIZE: galaxy intrinsic r50, modelled by “deconvolving” the BCC 
  galaxy “size” parameter and scaling it to match that in Balrog: 
  ((size/0.27)**2-(0.9**2/2.355)/0.27**2)**0.5*(mag_i * (-0.00505744) + 0.80866503)
  when getting negative values in the square-root, assign 0.0001

7. Others:

- the Source Extractor catalogs contain some columns padded with 
  zeros, these are the ones that were in earlier SV-A1 catalogs. 
  Check the Source extractor configuration fiels for detail.

- 48 Y-band images missing, where we don’t have depth, these 
  have "-1.6375e+30" in the Y-band depth in the lookup file


8. Also see wiki page for more details: 
   https://cdcvs.fnal.gov/redmine/projects/simulation/wiki/Simulate_SV-A1_data
   https://cdcvs.fnal.gov/redmine/projects/simulation/wiki/Simulate_SV-A1_data_(continued)

For questions/comments/feedback contact: 
Chihway Chang (chihway.chang@phys.ethz.ch)

Main contributors: Michael Busha, Eli Rykoff, Lukas Gamper, 
Risa Weschler, Claudio Bruderer, Adam Amara, Alexandre Refregier, 
Boris Leistedt, Hiranya Peiris, Matt Becker ...