Astrocut Documentation

Introduction

Astrocut provides tools for making cutouts from sets of astronomical images with shared footprints. It is under active development.

Three main areas of functionality are included:

  • Solving the specific problem of creating image cutouts from Sectors of Transiting Exoplanet Survey Satellite (TESS) full-frame images, and related High-Level Science Product images (TICA).

  • General FITS file cutouts including from single images and sets of images with shared WCS/pixel scale.

  • Cutout post-processing functionality, including centering cutouts along a path (for moving targets) and combining cutouts.

FITS File Image Cutouts

These functions provide general purpose astronomical cutout functionality for FITS files. There are two main cutout functions, fits_cut for creating cutout FITS files, and img_cut for creating cutout JPG or PNG files. An image normalization (normalize_img) function is also available.

Creating FITS Cutouts

The function fits_cut takes one or more FITS files and performs the same cutout on each, returning the result either in a single FITS file or as one FITS file per cutout. It is important to remember that while the expectation is that all input images are aligned and have the same pixel scale, no checking is done by Astrocut.

The cutout FITS file format is described here.

>>> from astrocut import fits_cut
>>> from astropy.io import fits
>>> from astropy.coordinates import SkyCoord

>>> input_files = ["https://archive.stsci.edu/pub/hlsp/candels/cosmos/cos-tot/v1.0/hlsp_candels_hst_acs_cos-tot-sect23_f606w_v1.0_drz.fits",
...                "https://archive.stsci.edu/pub/hlsp/candels/cosmos/cos-tot/v1.0/hlsp_candels_hst_acs_cos-tot-sect23_f814w_v1.0_drz.fits"]

>>> center_coord = SkyCoord("150.0945 2.38681", unit='deg')
>>> cutout_size = [200,300]

>>> cutout_file = fits_cut(input_files, center_coord, cutout_size, single_outfile=True)  
>>> print(cutout_file)    
./cutout_150.094500_2.386810_200-x-300_astrocut.fits

>>> cutout_hdulist = fits.open(cutout_file)  
>>> cutout_hdulist.info() 
Filename: ./cutout_150.094500_2.386810_200-x-300_astrocut.fits
No.    Name      Ver    Type      Cards   Dimensions   Format
  0  PRIMARY       1 PrimaryHDU      11   ()
  1  CUTOUT        1 ImageHDU        44   (200, 300)   float32
  2  CUTOUT        1 ImageHDU        44   (200, 300)   float32

The cutout(s) can also be returned in memory as HDUList object(s).

>>> from astrocut import fits_cut
>>> from astropy.io import fits
>>> from astropy.coordinates import SkyCoord

>>> input_files = ["https://archive.stsci.edu/pub/hlsp/candels/cosmos/cos-tot/v1.0/hlsp_candels_hst_acs_cos-tot-sect23_f606w_v1.0_drz.fits",
...                "https://archive.stsci.edu/pub/hlsp/candels/cosmos/cos-tot/v1.0/hlsp_candels_hst_acs_cos-tot-sect23_f814w_v1.0_drz.fits"]

>>> center_coord = SkyCoord("150.0945 2.38681", unit='deg')
>>> cutout_size = [200,300]

>>> cutout_list = fits_cut(input_files, center_coord, cutout_size,
...                        single_outfile=False, memory_only=True)  
>>> cutout_list[0].info() 
Filename: ./cutout_150.094500_2.386810_200-x-300_astrocut.fits
No.    Name      Ver    Type      Cards   Dimensions   Format
  0  PRIMARY       1 PrimaryHDU      11   ()
  1  CUTOUT        1 ImageHDU        44   (200, 300)   float32
  2  CUTOUT        1 ImageHDU        44   (200, 300)   float32

fits_cut also has the ability to take FITS files from the cloud using S3 URIs.

>>> from astrocut import fits_cut
>>> from astropy.io import fits
>>> from astropy.coordinates import SkyCoord

>>> s3_uri = "s3://stpubdata/hst/public/j8pu/j8pu0y010/j8pu0y010_drz.fits"

>>> center_coord = SkyCoord("150.42838  2.421421", unit='deg')
>>> cutout_size = [100,100]

>>> cutout_file = fits_cut(input_files, center_coord, cutout_size,
...                        single_outfile=True, memory_only=True)  
>>> cutout_hdulist = fits.open(cutout_file)  
>>> cutout_hdulist.info() 
Filename: ./cutout_150.428380_2.421421_100-x-100_astrocut.fits
No.    Name      Ver    Type      Cards   Dimensions   Format
  0  PRIMARY       1 PrimaryHDU      11   ()
  1  CUTOUT        1 ImageHDU        97   (100, 100)   float32

Creating Image Cutouts

The function img_cut takes one or more FITS files and performs the same cutout on each, returning a single JPG or PNG file for each cutout. It is important to remember that while the expectation is that all input images are aligned and have the same pixel scale, no checking is done by Astrocut.

>>> from astrocut import img_cut
>>> from astropy.coordinates import SkyCoord
>>> from PIL import Image

>>> input_files = ["https://archive.stsci.edu/pub/hlsp/candels/cosmos/cos-tot/v1.0/hlsp_candels_hst_acs_cos-tot-sect23_f606w_v1.0_drz.fits",
...                "https://archive.stsci.edu/pub/hlsp/candels/cosmos/cos-tot/v1.0/hlsp_candels_hst_acs_cos-tot-sect23_f814w_v1.0_drz.fits"]

>>> center_coord = SkyCoord("150.0945 2.38681", unit='deg')
>>> cutout_size = [200,300]

>>> png_files = img_cut(input_files, center_coord, cutout_size, img_format='png')    
>>> print(png_files[0])    
./hlsp_candels_hst_acs_cos-tot-sect23_f606w_v1.0_drz_150.094500_2.386810_200-x-300_astrocut.png

>>> Image.open(png_files[1]) 
../_images/png_ex_cutout.png

Color images can also be produced using img_cut, given three input files, which will be treated as the R, G, and B channels, respectively.

>>> from astrocut import img_cut
>>> from astropy.coordinates import SkyCoord
>>> from PIL import Image

>>> input_files = ["https://archive.stsci.edu/pub/hlsp/goods/v2/h_nz_sect14_v2.0_drz_img.fits",
...                "https://archive.stsci.edu/pub/hlsp/goods/v2/h_ni_sect14_v2.0_drz_img.fits",
...                "https://archive.stsci.edu/pub/hlsp/goods/v2/h_nv_sect14_v2.0_drz_img.fits"]

>>> center_coord = SkyCoord("189.51522 62.2865221", unit='deg')
>>> cutout_size = [200,300]

>>> color_image = img_cut(input_files, center_coord, cutout_size, colorize=True)   
>>> print(color_image)    
./cutout_189.515220_62.286522_200-x-300_astrocut.jpg

>>> Image.open(color_image) 
../_images/color_ex_cutout.png

TESS Full-Frame Image Cutouts

Astrocut can be used to create cutouts from TESS full-frame images (FFIs). First, the CubeFactory (if working with SPOC products, or TicaCubeFactory if working with TICA FFIs) class allows you to create a large image cube from a list of FFI files. This is what allows the cutout operation to be performed efficiently. Next, the CutoutFactory class performs the actual cutout and builds a target pixel file (TPF) that is similar to the TESS Mission-produced TPFs. Finally, the cube_cut_from_footprint function generates cutouts from image cube files stored in MAST’s AWS Open Data Bucket.

If you are creating a small number of cutouts, the TESSCut web service may suit your needs: mast.stsci.edu/tesscut

Making Image Cubes

Important

Time-Memory Trade-off

Making an image cube is a simple operation, but comes with an important time-memory trade-off.

The max_memory argument determines the maximum memory in GB that will be used for the image data cube while it is being built. This is the amount of memory required only for the data cube, so is somewhat smaller than the total amount of memory needed for the program to run. You should never set it to your system’s total memory.

Because of this, cube files do not need to allocate their total size in memory all at once. Instead, a smaller memory allocation can be used while the cube file is constructed; however, this will significantly increase the execution time as bytes are swapped into and out of the memory allocation being used. The default value of 50 GB was chosen because it fits all of the TESS FFIs from a single Prime Mission Sector (Sectors 1-26); however, in the current TESS Extended Mission 2, where 6 times more FFIs are observed per Sector (compared to the number of FFIs observed per Sector in the Prime Mission), 50 GB is not enough space to hold all of the FFIs in memory, and the cubes will be written in multiple blocks. With the default settings, on a system with 64 GB of memory, it takes about 3 hours to build a single cube file. On a system with less memory or where max_memory is set to a value less than 50 GB, more passes through the list of files are required, and the time to create a cube can increase significantly.

Assuming that you have set of calibrated TESS (or TICA) FFI files stored locally, you can create a cube using the make_cube method (or make_cube for TICA products). By default, both make_cube and make_cube run in verbose mode and prints out progress; setting verbose to false will silence all output.

Note, you can only make cubes from a set of FFIs with the same product type (i.e., only SPOC or only TICA products) that were observed in the same sector, camera, and CCD.

The output image cube file format is described here.

>>> from astrocut import CubeFactory
>>> from glob import glob
>>> from astropy.io import fits

>>> my_cuber = CubeFactory()
>>> input_files = glob("data/*ffic.fits")

>>> cube_file = my_cuber.make_cube(input_files) 
Completed file 0
Completed file 1
Completed file 2
.
.
.
Completed file 142
Completed file 143
Total time elapsed: 46.42 sec
File write time: 8.82 sec

>>> print(cube_file) 
img-cube.fits

>>> cube_hdu = fits.open(cube_file) 
>>> cube_hdu.info()  
Filename: img-cube.fits
No.    Name      Ver    Type      Cards   Dimensions   Format
0  PRIMARY       1 PrimaryHDU      28   ()
1                1 ImageHDU         9   (2, 144, 2136, 2078)   float32
2                1 BinTableHDU    302   144R x 147C   [24A, J, J, J, J, J, J, D, 24A, J, 24A, 24A, J, J, D, 24A, 24A, 24A, J, D, 24A, D, D, D, D, 24A, 24A, D, D, D, D, D, 24A, D, D, D, D, J, D, D, D, D, D, D, D, D, D, D, D, D, J, J, D, J, J, J, J, J, J, J, J, J, J, D, J, J, J, J, J, J, D, J, J, J, J, J, J, D, J, J, J, J, J, J, D, J, J, J, J, J, J, J, J, 24A, D, J, 24A, 24A, D, D, D, D, D, D, D, D, J, J, D, D, D, D, D, D, J, J, D, D, D, D, D, D, D, D, D, D, D, D, 24A, J, 24A, 24A, J, J, D, 24A, 24A, J, J, D, D, D, D, J, 24A, 24A, 24A]

Making Cutout Target Pixel Files

Assuming that you have a TESS cube file stored locally, you can give the central coordinate of your target of interest and cutout size (in either pixels or angular degrees/arcseconds Quantity) to the cube_cut function.

You can optionally specify an output TPF name; if no output name is provided, the file name will be built as: “<cube_file_base>_<ra>_<dec>_<cutout_size>_astrocut.fits”. You can optionally also specify an output path, the directory in which the TPF will be saved; if unspecified, this will default to the current directory.

The cutout target pixel file format is described here.

>>> from astrocut import CutoutFactory
>>> from astropy.io import fits

>>> my_cutter = CutoutFactory()
>>> cube_file = "img-cube.fits"

>>> cutout_file = my_cutter.cube_cut(cube_file, "251.51 32.36", 5, verbose=True) 
Cutout center coordinate: 251.51,32.36
xmin,xmax: [26 31]
ymin,ymax: [149 154]
Image cutout cube shape: (144, 5, 5)
Uncertainty cutout cube shape: (144, 5, 5)
Target pixel file: img_251.51_32.36_5x5_astrocut.fits
Write time: 0.016 sec
Total time: 0.18 sec

>>> cutout_hdu = fits.open(cutout_file) 
>>> cutout_hdu.info() 
Filename: img_251.51_32.36_5x5_astrocut.fits
No.    Name      Ver    Type      Cards   Dimensions   Format
0  PRIMARY       1 PrimaryHDU      42   ()
1  PIXELS        1 BinTableHDU    222   144R x 12C   [D, E, J, 25J, 25E, 25E, 25E, 25E, J, E, E, 38A]
2  APERTURE      1 ImageHDU        45   (5, 5)   float64

Cloud-based Cutouts

You can generate cutout Target Pixel Files (TPFs) from TESS cube files stored in MAST’s AWS Open Data Bucket using the cube_cut_from_footprint function. Simply provide the target coordinates and cutout size, and the function will match the cutout’s footprint to the footprints of available cube files on the cloud. A cutout will be generated for each matching cube file. To restrict the cutouts to specific sectors, use the sequence parameter with a sector number or a list of sector numbers. If sequence is set to None, cutouts will be returned for all matching cube files.

>>> from astrocut import cube_cut_from_footprint

>>> cube_cut_from_footprint(  
...         coordinates='83.40630967798376 -62.48977125108528',
...         cutout_size=10,
...         sequence=[1, 2],  # TESS sectors
...         product='SPOC')
['./cutouts/tess-s0001-4-4/tess-s0001-4-4_83.406310_-62.489771_10x10_astrocut.fits',
'./cutouts/tess-s0002-4-1/tess-s0002-4-1_83.406310_-62.489771_10x10_astrocut.fits']

Alternatively, you can provide the S3 URI for a cube file directly to the cube_cut function.

Multithreading

Using cube files stored on the cloud allows you the option to implement multithreading to improve cutout generation speed. See below for a multithreaded example, using a TESS cube file stored on AWS.

To use multithreading for cloud-based cutouts, set the threads argument in cube_cut to the number of threads you want to use. Alternatively, you can set threads to "auto", which will set the number of threads based on the CPU count of your machine. Note that Total Time results may vary from machine to machine.

>>> from astrocut import CutoutFactory
>>> from astropy.coordinates import SkyCoord

>>> my_cutter = CutoutFactory()
>>> coord = SkyCoord(217.42893801, -62.67949189, unit="deg", frame="icrs")
>>> cutout_size = 30
>>> cube_file = "s3://stpubdata/tess/public/mast/tess-s0038-2-2-cube.fits"

>>> cut_factory.cube_cut(cube_file, coordinates=coord, cutout_size=cutout_size,
...                      verbose=True, threads="auto") 
Using WCS from row 1852 out of 3705
Cutout center coordinate: 217.42893801,-62.67949189
xmin,xmax: [1572 1602]
ymin,ymax: [852 882]
Image cutout cube shape: (3705, 30, 30)
Uncertainty cutout cube shape: (3705, 30, 30)
Maximum distance between approximate and true location: 3.6009402965268847e-05 deg
Error in approximate WCS (sigma): 0.0003207242331953156
Target pixel file: ./tess-s0038-2-2_217.428938_-62.679492_30x30_astrocut.fits

WARNING: VerifyWarning: Card is too long, comment will be truncated. [astropy.io.fits.card]

Write time: 0.54 sec
Total time: 4.3 sec

The same call made without multithreading enabled will result in a longer processing time, depending on the cutout size. Note that multithreading is disabled by default.

>>> cut_factory.cube_cut(cube_file, coordinates=coord, cutout_size=cutout_size,
...                      verbose=True) 
Using WCS from row 1852 out of 3705
Cutout center coordinate: 217.42893801,-62.67949189
xmin,xmax: [1572 1602]
ymin,ymax: [852 882]
Image cutout cube shape: (3705, 30, 30)
Uncertainty cutout cube shape: (3705, 30, 30)
Maximum distance between approximate and true location: 3.6009402965268847e-05 deg
Error in approximate WCS (sigma): 0.0003207242331953156
Target pixel file: ./tess-s0038-2-2_217.428938_-62.679492_30x30_astrocut.fits

WARNING: VerifyWarning: Card is too long, comment will be truncated. [astropy.io.fits.card]

Write time: 0.56 sec
Total time: 7.8 sec

ASDF File Cutouts

The Nancy Grace Roman Space Telescope will store its data using the Advanced Scientific Data Format (ASDF). With the asdf_cut function, users can create cutouts of Roman mission products.

Creating ASDF Cutouts

The function asdf_cut performs a cutout of an ASDF file and returns the result as either a FITS file or an ASDF file. The format of the cutout is determined by the filename extension of the output_file parameter. In the below example, a cutout is written as a FITS file. The cutout FITS file format is described here.

>>> from astrocut import asdf_cut
>>> from astropy.coordinates import SkyCoord
>>> from astropy.io import fits

>>> input_file = ""  # Path to local ASDF file or URI

>>> center_coord = SkyCoord("80.15189743 29.74561219", unit='deg')

>>> cutout_file = asdf_cut(input_file, center_coord.ra, center_coord.dec, cutout_size=205,
...                        output_file="roman-demo.fits") 

>>> cutout_hdulist = fits.open(cutout_file)  
>>> cutout_hdulist.info() 
Filename: roman-demo.fits
No.    Name      Ver    Type      Cards   Dimensions   Format
  0  PRIMARY       1 PrimaryHDU      25   (205, 205)   float32

Warning

Due to the symmetry of the pixel grid, odd values for cutout_size generally produce cutouts that are more accurately centered on the target coordinates than even values for cutout_size.

asdf_cut also accepts S3 URIs to perform cutouts on ASDF files from the cloud. In this example, a cutout is performed on a cloud file and written as an ASDF file. The cutout ASDF file format is described here.

>>> from astrocut import asdf_cut
>>> from astropy.coordinates import SkyCoord

>>> s3_uri = "s3://..." # Complete URI

>>> center_coord = SkyCoord("80.15189743 29.74561219", unit='deg')

>>> cutout_file = asdf_cut(s3_uri, center_coord.ra, center_coord.dec, cutout_size=205,
...                        output_file="roman-demo.asdf") 

When requesting a cutout that is partially outside of image bounds, the fill_value parameter is used to preserve the cutout shape and fill outside pixels.

Additional Cutout Processing

Path-based Cutouts

The center_on_path function allows the user to take one or more Astrocut cutout TPF(s) and create a single cutout, centered on a moving target that crosses through the file(s). The user can optionally pass in a target object name and FFI WCS object.

The output target pixel file format is described here.

This example starts with a path, and uses several TESScut services to retrieve all of the inputs for the center_on_path function. We also use the helper function path_to_footprints that takes in a path table, cutout size, and WCS object, and returns the cutout location/size(s) necesary to cover the entire path.

>>> import astrocut

>>> import requests  

>>> from astropy.table import Table
>>> from astropy.coordinates import SkyCoord
>>> from astropy.time import Time
>>> from astropy.io import fits
>>> from astropy import wcs

>>> from astroquery.mast import Tesscut  

>>> # The moving target path
>>> path_table = Table({"time": Time([2458468.275827604, 2458468.900827604, 2458469.525827604,
...                                   2458470.150827604, 2458470.775827604], format="jd"),
...                     "position": SkyCoord([82.22813, 82.07676, 81.92551, 81.7746, 81.62425],
...                                          [-1.5821,- 1.54791, -1.5117, -1.47359, -1.43369], unit="deg")
...                    })

>>> # Getting the FFI WCS
>>> resp = requests.get(f"https://mast.stsci.edu/tesscut/api/v0.1/ffi_wcs?sector=6&camera=1&ccd=1")  
>>> ffi_wcs = wcs.WCS(resp.json()["wcs"], relax=True)  
>>> print(ffi_wcs)  
WCS Keywords

Number of WCS axes: 2
CTYPE : 'RA---TAN-SIP'  'DEC--TAN-SIP'
CRVAL : 86.239936828613  -0.87476283311844
CRPIX : 1045.0  1001.0
PC1_1 PC1_2  : 0.0057049915194511  7.5332427513786e-06
PC2_1 PC2_2  : -0.00015248404815793  0.005706631578505
CDELT : 1.0  1.0
NAXIS : 2136  2078

>>> # Making the regular cutout (using astroquery)
>>> size = [15,15]
>>> footprints = astrocut.path_to_footprints(path_table["position"], size, ffi_wcs)  
>>> print(footprints)  
[{'coordinates': <SkyCoord (ICRS): (ra, dec) in deg
     (81.92560877, -1.50880833)>, 'size': (37, 125)}]

>>> manifest = Tesscut.download_cutouts(**footprints[0], sector=6)  
Downloading URL https://mast.stsci.edu/tesscut/api/v0.1/astrocut?ra=81.92560876541987&dec=-1.5088083330171362&y=37&x=125&units=px&sector=6 to ./tesscut_20210707103901.zip ... [Done]
Inflating...

>>> print(manifest["Local Path"][0])  
./tess-s0006-1-1_81.925609_-1.508808_125x37_astrocut.fits

# Centering on the moving target
>>> mt_cutout_fle = astrocut.center_on_path(path_table, size, manifest["Local Path"], target="my_asteroid",
...                                         img_wcs=ffi_wcs, verbose=False)  

>>> cutout_hdu = fits.open(mt_cutout_fle)  
>>> cutout_hdu.info()  
Filename: ./my_asteroid_1468.9120483398438-1470.1412353515625_15-x-15_astrocut.fits
No.    Name      Ver    Type      Cards   Dimensions   Format
  0  PRIMARY       1 PrimaryHDU      56   ()
  1  PIXELS        1 BinTableHDU    152   60R x 16C   [D, E, J, 225J, 225E, 225E, 225E, 225E, J, E, E, 38A, D, D, D, D]
  2  APERTURE      1 ImageHDU        97   (2136, 2078)   int32

Combining Cutouts

The CutoutsCombiner class allows the user to take one or more Astrocut cutout FITS files (as from fits_cut) with a shared WCS object, and combine them into a single cutout. This means that you should request the same cutout size in all of the images you want to combine.

The default setting combines the images with a mean combiner, such that every combined pixel is the mean of all pixels that have data at that point. This mean combiner is made with the build_default_combine_function, which takes the input image HDUs and allows the user to specify a null data value (default is NaN).

Users can write a custom combiner function, either by directly setting the combine_images function, or by writing a custom combiner function builder and passing it to the build_img_combiner function. The main reason to write a function builder is that the combine_images function must work only on the images being combined; any usage of header keywords, for example, must be set in that function. See the build_default_combine_function for an example of how this works.

>>> import astrocut

>>> from astropy.coordinates import SkyCoord

>>> fle_1 = 'hst_skycell-p2381x05y09_wfc3_uvis_f275w-all-all_drc.fits'
>>> fle_2 = 'hst_skycell-p2381x06y09_wfc3_uvis_f275w-all-all_drc.fits'

>>> center_coord = SkyCoord("211.27128477 53.66062066", unit='deg')
>>> size = [30,50]

>>> cutout_1 = astrocut.fits_cut(fle_1, center_coord, size, extension='all',
...                     cutout_prefix="cutout_p2381x05y09", verbose=False)  
>>> cutout_2 = astrocut.fits_cut(fle_2, center_coord, size, extension='all',
...                     cutout_prefix="cutout_p2381x06y09", verbose=False)  

>>> plt.imshow(fits.getdata(cutout_1, 1))  
../_images/hapcut_left.png
>>> plt.imshow(fits.getdata(cutout_2, 1))  
../_images/hapcut_right.png
>>> combined_cutout = astrocut.CutoutsCombiner([cutout_1, cutout_2]).combine("combined_cut.fits")  
>>> plt.imshow(fits.getdata(combined_cutout, 1))  
../_images/hapcut_combined.png

All of the combining can be done in memory, without writing FITS files to disk as well.

>>> import astrocut

>>> from astropy.coordinates import SkyCoord

>>> fle_1 = 'hst_skycell-p2381x05y09_wfc3_uvis_f275w-all-all_drc.fits'
>>> fle_2 = 'hst_skycell-p2381x06y09_wfc3_uvis_f275w-all-all_drc.fits'

>>> center_coord = SkyCoord("211.27128477 53.66062066", unit='deg')
>>> size = [30,50]

>>> cutout_1 = astrocut.fits_cut(fle_1, center_coord, size, extension='all',
...                     cutout_prefix="cutout_p2381x05y09", memory_only=True)[0]  
>>> cutout_2 = astrocut.fits_cut(fle_2, center_coord, size, extension='all',
...                     cutout_prefix="cutout_p2381x06y09", memory_only=True)[0]  

>>> plt.imshow(cutout_1[1].data)  
../_images/hapcut_left.png
>>> plt.imshow(cutout_2[1].data)  
../_images/hapcut_right.png
>>> combined_cutout = astrocut.CutoutsCombiner([cutout_1, cutout_2]).combine(memory_only=True)  
>>> plt.imshow(combined_cutout[1].data)  
../_images/hapcut_combined.png