DavidT3
Generation of Chandra 'photometric' products
This part of the project deals with the implementation of XGA wrappers for the CIAO routine(s?) required to make 'photometric' data products - note that 'photometric' isn't really a term generally applied to X-ray data, but I like to be able to make a distinction between things like images and things like spectra.
For this project task we'll want to generate (all in user-configurable energy bands):
- Images
- Straightforward effective exposure maps in units of seconds
- Flux maps (though they don't have a product class to be put into yet).
- Maybe count-rate maps? But maybe we'll just let XGA do its normal thing of dividing images by exposure maps.
While fluximage is extremely useful, it has the downside that a run of it will produce multiple different files/product types, so that means that the 'execute_cmd' function (common to all telescopes; https://github.com/DavidT3/XGA/blob/ed8552b4d56b7b445cccbaab22644664c57546e0/xga/generate/common.py#L50) will have to be modified in some way to account for that.
For other telescopes we have had separate functions for generating each product, but using fluximage is going to be a time saver so I think we'll just live with having one function make multiple products.
You're going to implement two photometric generating functions; the first will be akin to the 'rate' mode in the DAXA wrapper for flux-image (https://github.com/DavidT3/DAXA/blob/853954ed7a839398747e0254f04ef077f9538562/daxa/process/chandra/generate.py#L26) - this will generate images, 'normal, exposure maps, and rate maps (though again they might be discarded in favour of how XGA already does it.
The second will be akin to the 'flux' mode in the DAXA wrapper, and will make flux maps (also weighted exposure maps but maybe we won't keep them).
Start by setting up a generate.ciao.phot part of the module, then make a 'chandra_image_expmap' function, and start looking at the image making XMM task (https://github.com/DavidT3/XGA/blob/ed8552b4d56b7b445cccbaab22644664c57546e0/xga/generate/sas/phot.py#L25) for inspiration in terms of what setup you need to do, and look at my DAXA wrappers for fluximage as well. Try adding some of the initial checks you might see in the XMM image making function, and then we can talk some more about it.
@rz-wang
Just an FYI, the way you had the chandra_image_expmap function written, specifically this bit:
# Setting up the top level path for the eventual destination of the products to be generated here
dest_dir = os.path.join(OUTPUT, "chandra", obs_id)
# If something got interrupted and the temp directory still exists, this will remove it
if os.path.exists(dest_dir):
rmtree(dest_dir)
os.makedirs(dest_dir)
You would have immediately deleted the entire XGA directory for the current Chandra ObsID - it's useful to be able to be inspired by and take snippets from the product generation methods of other missions, but you need to make sure to read what you've been copying to make sure it all still does what you think it does.
@rz-wang
Oh btw, when writing docstrings for a variable that has a Union type hint, you have to not add the possible types with / in between them, not in a Union. Don't know why but it is a particular foible of Sphinx AutoDoc, which is the package that puts all the docstrings into the documentation (though I have somehow broken a lot of it for the multi-mission version)
So something like this:
:param BaseSource/NullSource/BaseSample sources: A source object, null source, or sample of sources.
The only way I found out about this problem originally was because the docstrings weren't rendering properly
@rz-wang Also, just general advice, when developing code for a module (particularly one you aren't super familiar with yet) you need to try and follow the chain of what will happen when you add new code, to make sure it is going to do what you intended.
For instance, in the chandra_image_expmap function, you add three final output paths for each execution of the command, which of course you were completely right to do as we know it'll make three files, but if you follow the chain of events to the execute_cmd function and look here:
https://github.com/DavidT3/XGA/blob/ef6c2f036e115bebe95baea49c628fb47d29809d/xga/generate/common.py#L50
You'll see that it isn't currently designed to handle multiple files of interest being generated at once, unless they all go into the same product (like spectra) - so it needs to be modified.
This isn't a criticism, I know you've never worked on programming like this, just some software engineering advice :)
We can now generate images, expmaps, and ratemaps (though they aren't loaded yet)! More importantly I think the infrastructure is properly setup to handle the ways that Chandra software can sometimes produce multiple products in a single function call, with different types.
Now that I've made sure this generation function works, we can make more progress on spectra @rz-wang
Some of the fruits of our labours:
Ah well it now doesn't work for multiple ObsIDs in the same source, so I have broken something else.
Haven't yet tested how it behaves with multiple sources
@rz-wang I'm happy-ish now that the chandra_image_expmap function is doing what I expect, both with single ObsIDs and multiple. The products are also being loaded in to the XGA source storage structure properly.
It isn't writing the image/ratemap/expmap files to the write paths yet (but that's only because I haven't been bothered to put move commands into the end of the command in chandra_image_expmap.
Your immediate task at the beginning of the week should be to figure out (from that notebook, and from how eROSITA is set up) how to convert from the regions that XGA has loaded in, to a region file we can pass to specextract so we can generate a spectrum within a specified aperture with contaminating sources removed.
You could also try and complete the specextract command (now that badpix and aspect files are easily available) - and just see if it runs and loads in the spectrum properly, even if that spectrum isn't useful
Note for me - need to make sure the output files are named using the XGA convention before I consider this fully working and done.
Alright, realised that fluximage also needs a mask file passed in to deal with any non-standard chip configurations. Will correct that problem in DAXA quickly, but will also need to be dealt with here. I'll move this back to 'in progress' on the project board
No mask file for fluximage yet, but the output image/expmap/etc. files from running that process should be named in the XGA convention now, which will let us load them back in.
Naming convention is correct, and image/expmap files are being loaded back in correctly