Config Processing From Python

It is also possible to run the config processing from a Python script, rather than using The galsim Executable. An example of this can be found in demo8 .

Running the Whole Script

The following functions are relevant to running the whole config script from Python:

galsim.config.ReadConfig(config_file, file_type=None, logger=None)

Read in a configuration file and return the corresponding dicts.

A YAML file is allowed to define several dicts using multiple documents. The GalSim parser treats this as a set of multiple jobs to be done. The first document is taken to be a “base” dict that has common definitions for all the jobs. Then each subsequent document has the (usually small) modifications to the base dict for each job. See demo6.yaml, demo8.yaml and demo9.yaml in the GalSim/examples directory for example usage.

On output, the returned list will have an entry for each job to be done. If there are multiple documents, then the first dict is a merge of the first two documents, the second a merge of the first and third, and so on. Each job includes the first document merged with each subseqent document in turn. If there is only one document defined, the returned list will have one element, which is this dict.

A JSON file does not have this feature, but to be consistent, we always return a list, which would only have one element in this case.

Also, we actually read in the config file into an OrderedDict. The main advantage of this is for the truth catalog. This lets the columns be in the same order as the entries in the config file. With a normal dict, they get scrambled.

Parameters:

• config_file – The name of the configuration file to read.

• file_type – If given, the type of file to read. [default: None, which mean infer the file type from the extension.]

• logger – If given, a logger object to log progress. [default: None]

Returns:

list of config dicts

galsim.config.CopyConfig(config)

If you want to use a config dict for multiprocessing, you need to deep copy the gal, psf, and pix fields, since they cache values that are not picklable. If you don’t do the deep copy, then python balks when trying to send the updated config dict back to the root process. We do this a few different times, so encapsulate the copy semantics once here.

Parameters:

config – The configuration dict to copy.

Returns:

a deep copy of the config dict.

galsim.config.ImportModules(config, gdict=None)

Import any modules listed in config[‘modules’].

These won’t be brought into the running scope of the config processing, but any side effects of the import statements will persist. In particular, these are allowed to register additional custom types that can then be used in the current config dict.

Parameters:

config – The configuration dict.

galsim.config.ProcessTemplate(config, base, logger=None)

If the config dict has a ‘template’ item, read in the appropriate file and make any requested updates.

Parameters:

• config – The configuration dict.

• base – The base configuration dict.

• logger – If given, a logger object to log progress. [default: None]

galsim.config.ProcessAllTemplates(config, logger=None, base=None)

Check through the full config dict and process any fields that have a ‘template’ item.

Parameters:

• config – The configuration dict.

• logger – If given, a logger object to log progress. [default: None]

• base – The base configuration dict. [default: None]

galsim.config.Process(config, logger=None, njobs=1, job=1, new_params=None, except_abort=False)

Do all processing of the provided configuration dict. In particular, this function handles processing the output field, calling other functions to build and write the specified files. The input field is processed before building each file.

Sometimes, it can be helpful to split up a processing jobs over multiple machines (i.e. not just multiple processes, which can be handled natively with the output.nproc or image.nproc options). In this case, you can ask the Process command to split up the total amount of work into njobs and only do one of those jobs here. To do this, set njobs to be the number of jobs total and job to be which job should be done here.

Parameters:

• config – The configuration dict.

• logger – If given, a logger object to log progress. [default: None]

• njobs – The total number of jobs to split the work into. [default: 1]

• job – Which job should be worked on here (1..njobs). [default: 1]

• new_params – A dict of new parameter values that should be used to update the config dict after any template loading (if any). [default: None]

• except_abort – Whether to abort processing when a file raises an exception (True) or just report errors and continue on (False). [default: False]

Returns:

the final config dict that was used.

Building Files

The following functions are relevant to building one or more files as specified by a config dict:

galsim.config.BuildFiles(nfiles, config, file_num=0, logger=None, except_abort=False)

Build a number of output files as specified in config.

Parameters:

• nfiles – The number of files to build.

• config – A configuration dict.

• file_num – If given, the first file_num. [default: 0]

• logger – If given, a logger object to log progress. [default: None]

• except_abort – Whether to abort processing when a file raises an exception (True) or just report errors and continue on (False). [default: False]

Returns:

the final config dict that was used.

galsim.config.BuildFile(config, file_num=0, image_num=0, obj_num=0, logger=None)

Build an output file as specified in config.

Parameters:

• config – A configuration dict.

• file_num – If given, the current file_num. [default: 0]

• image_num – If given, the current image_num. [default: 0]

• obj_num – If given, the current obj_num. [default: 0]

• logger – If given, a logger object to log progress. [default: None]

Returns:

(file_name, t), a tuple of the file name and the time taken to build file Note: t==0 indicates that this file was skipped.

galsim.config.GetNFiles(config, logger=None)

Get the number of files that will be made, based on the information in the config dict.

Parameters:

• config – The configuration dict.

• logger – If given, a logger object to log progress. [default: None]

Returns:

the number of files

galsim.config.GetNImagesForFile(config, file_num, logger=None)

Get the number of images that will be made for the file number file_num, based on the information in the config dict.

Parameters:

• config – The configuration dict.

• file_num – The current file number.

• logger – If given, a logger object to log progress. [default: None]

Returns:

the number of images

galsim.config.GetNObjForFile(config, file_num, image_num, logger=None, approx=False)

Get the number of objects that will be made for each image built as part of the file file_num, which starts at image number image_num, based on the information in the config dict.

Parameters:

• config – The configuration dict.

• file_num – The current file number.

• image_num – The current image number.

• logger – If given, a logger object to log progress. [default: None]

• approx – Whether an approximate/overestimate is ok [default: False]

Returns:

a list of the number of objects in each image [ nobj0, nobj1, nobj2, … ]

galsim.config.SetupConfigFileNum(config, file_num, image_num, obj_num, logger=None)

Do the basic setup of the config dict at the file processing level.

Includes: - Set config[‘file_num’] = file_num - Set config[‘image_num’] = image_num - Set config[‘obj_num’] = obj_num - Set config[‘index_key’] = ‘file_num’ - Set config[‘start_image_num’] = image_num - Set config[‘start_obj_num’] = obj_num - Make sure config[‘output’] exists - Set default config[‘output’][‘type’] to ‘Fits’ if not specified - Check that the specified output type is valid.

Parameters:

• config – A configuration dict.

• file_num – The current file_num. (If file_num=None, then don’t set file_num or start_obj_num items in the config dict.)

• image_num – The current image_num.

• obj_num – The current obj_num.

• logger – If given, a logger object to log progress. [default: None]

Building Images

The following functions are relevant to building one or more images as specified by a config dict:

galsim.config.BuildImages(nimages, config, image_num=0, obj_num=0, logger=None)

Build a number of postage stamp images as specified by the config dict.

Parameters:

• nimages – How many images to build.

• config – The configuration dict.

• image_num – If given, the current image number. [default: 0]

• obj_num – If given, the first object number in the image. [default: 0]

• logger – If given, a logger object to log progress. [default: None]

Returns:

a list of images

galsim.config.BuildImage(config, image_num=0, obj_num=0, logger=None)

Build an Image according to the information in config.

Parameters:

• config – The configuration dict.

• image_num – If given, the current image number. [default: 0]

• obj_num – If given, the first object number in the image. [default: 0]

• logger – If given, a logger object to log progress. [default: None]

Returns:

the final image

galsim.config.SetupConfigImageSize(config, xsize, ysize, logger=None)

Do some further setup of the config dict at the image processing level based on the provided image size.

• Set config[‘image_xsize’], config[‘image_ysize’] to the size of the image

• Set config[‘image_origin’] to the origin of the image

• Set config[‘image_center’] to the center of the image

• Set config[‘image_bounds’] to the bounds of the image

• Build the WCS based on either config[‘image’][‘wcs’] or config[‘image’][‘pixel_scale’]

• Set config[‘wcs’] to be the built wcs

• If wcs.isPixelScale(), also set config[‘pixel_scale’] for convenience.

• Set config[‘world_center’] to either a given value or based on wcs and image_center

• Create a blank image if possible and store as config[‘current_image’]

Parameters:

• config – The configuration dict.

• xsize – The size of the image in the x-dimension.

• ysize – The size of the image in the y-dimension.

• logger – If given, a logger object to log progress. [default: None]

galsim.config.SetupConfigImageNum(config, image_num, obj_num, logger=None)

Do the basic setup of the config dict at the image processing level.

Includes: - Set config[‘image_num’] = image_num - Set config[‘obj_num’] = obj_num - Set config[‘index_key’] = ‘image_num’ - Make sure config[‘image’] exists - Set default config[‘image’][‘type’] to ‘Single’ if not specified - Check that the specified image type is valid.

Parameters:

• config – The configuration dict.

• image_num – The current image number.

• obj_num – The first object number in the image.

• logger – If given, a logger object to log progress. [default: None]

galsim.config.GetNObjForImage(config, image_num, logger=None, approx=False)

Get the number of objects that will be made for the image number image_num based on the information in the config dict.

Parameters:

• config – The configuration dict.

• image_num – The current image number.

• logger – If given, a logger object to log progress.

• approx – Whether an approximate/overestimate is ok [default: False]

Returns:

the number of objects

galsim.config.FlattenNoiseVariance(config, full_image, stamps, current_vars, logger)

This is a helper function to bring the noise level up to a constant value across the image. If some of the galaxies are RealGalaxy objects and noise whitening (or symmetrizing) is turned on, then there will already be some noise in the stamps that get built. This function goes through and figures out what the maximum current variance is anywhere in the full image and adds noise to the other pixels to bring everything up to that level.

Parameters:

• config – The configuration dict.

• full_image – The full image onto which the noise should be added.

• stamps – A list of the individual postage stamps.

• current_vars – A list of the current variance in each postage stamps.

• logger – If given, a logger object to log progress.

Returns:

the final variance in the image

galsim.config.BuildWCS(config, key, base, logger=None)

Read the wcs parameters from config[key] and return a constructed wcs object.

Parameters:

• config – A dict with the configuration information. (usually base[‘image’])

• key – The key name in config indicating which object to build.

• base – The base dict of the configuration.

• logger – Optionally, provide a logger for logging debug statements. [default: None]

Returns:

a BaseWCS instance

galsim.config.AddSky(config, im)

Add the sky level to the image

Parameters:

• config – The (base) configuration dict

• im – The image onto which to add the sky

galsim.config.AddNoise(config, im, current_var=0.0, logger=None)

Add noise to an image according to the noise specifications in the noise dict.

Parameters:

• config – The (base) configuration dict

• im – The image onto which to add the noise

• current_var – The current noise variance present in the image already [default: 0]

• logger – If given, a logger object to log progress. [default: None]

Returns:

Variance added to the image (units are ADU if gain != 1)

galsim.config.CalculateNoiseVariance(config, full=False)

Calculate the noise variance from the noise specified in the noise dict.

Parameters:

• config – The (base) configuration dict

• full – If the noise is variable across the image, return the full image with the noise variance at every pixel. Otherwise, just return the value at the center.

Returns:

the noise variance (units are ADU if gain != 1)

galsim.config.AddNoiseVariance(config, im, include_obj_var=False, logger=None)

Add the noise variance to an image according to the noise specifications in the noise dict. Typically, this is used for building a weight map, which is typically the inverse variance.

Parameters:

• config – The (base) configuration dict

• im – The image onto which to add the variance values

• include_obj_var – Whether to add the variance from the object photons for noise models that have a component based on the number of photons. Note: if this is True, the returned variance will not include this contribution to the noise variance. [default: False]

• logger – If given, a logger object to log progress. [default: None]

Returns:

the variance in the image (units are ADU if gain != 1)

galsim.config.GetSky(config, base, logger=None, full=False)

Parse the sky information and return either a float value for the sky level per pixel or an image, as needed.

If an image is required (because wcs is not uniform) then it will use the presence of base[‘image_pos’] to determine what size image to return (stamp or full). If there is a current image_pos, then we are doing a stamp. Otherwise a full image.

Parameters:

• config – The configuration field with the sky specification, which can be either base[‘image’] or base[‘image’][‘noise’]

• base – The base configuration dict

• logger – If given, a logger object to log progress. [default: None]

• full – If the sky level is variable across the image, return the full image with the sky at every pixel. Otherwise, just return the sky at the image center.

Returns:

sky, either a float value or an Image. (The latter only if full=True)

Building Stamps

The following functions are relevant to building one or more stamps as specified by a config dict:

galsim.config.BuildStamps(nobjects, config, obj_num=0, xsize=0, ysize=0, do_noise=True, logger=None)

Build a number of postage stamp images as specified by the config dict.

Parameters:

• nobjects – How many postage stamps to build.

• config – A configuration dict.

• obj_num – If given, the current obj_num. [default: 0]

• xsize – The size of a single stamp in the x direction. [default: 0, which means to look first for config.stamp.xsize, then for config.image.stamp_xsize, and if neither are given, then use automatic sizing.]

• ysize – The size of a single stamp in the y direction. [default: 0, which means to look first for config.stamp.ysize, then for config.image.stamp_ysize, and if neither are given, then use automatic sizing.]

• do_noise – Whether to add noise to the image (according to config[‘noise’]). [default: True]

• logger – If given, a logger object to log progress. [default: None]

Returns:

the tuple (images, current_vars). Both are themselves tuples.

galsim.config.BuildStamp(config, obj_num=0, xsize=0, ysize=0, do_noise=True, logger=None)

Build a single stamp image using the given config file

Parameters:

• config – A configuration dict.

• obj_num – If given, the current obj_num [default: 0]

• xsize – The xsize of the stamp to build (if known). [default: 0]

• ysize – The ysize of the stamp to build (if known). [default: 0]

• do_noise – Whether to add noise to the image (according to config[‘noise’]). [default: True]

• logger – If given, a logger object to log progress. [default: None]

Returns:

the tuple (image, current_var)

galsim.config.SetupConfigStampSize(config, xsize, ysize, image_pos, world_pos, logger=None)

Do further setup of the config dict at the stamp (or object) processing level reflecting the stamp size and position in either image or world coordinates.

Note: This is now a StampBuilder method. So this function just calls StampBuilder.locateStamp.

Parameters:

• config – A configuration dict.

• xsize – The size of the stamp in the x-dimension. [may be 0 if unknown]

• ysize – The size of the stamp in the y-dimension. [may be 0 if unknown]

• image_pos – The position of the stamp in image coordinates. [may be None]

• world_pos – The position of the stamp in world coordinates. [may be None]

• logger – If given, a logger object to log progress. [default: None]

galsim.config.SetupConfigObjNum(config, obj_num, logger=None)

Do the basic setup of the config dict at the stamp (or object) processing level.

Includes:

• Set config[‘obj_num’] = obj_num

• Set config[‘index_key’] = ‘obj_num’

• Make sure config[‘stamp’] exists

• Set default config[‘stamp’][‘type’] to ‘Basic’

• Copy over values from config[‘image’] that are allowed there, but really belong in config[‘stamp’].

• Set config[‘stamp’][‘draw_method’] to ‘auto’ if not given.

Parameters:

• config – A configuration dict.

• obj_num – The current obj_num.

• logger – If given, a logger object to log progress. [default: None]

galsim.config.DrawBasic(prof, image, method, offset, config, base, logger, **kwargs)

The basic implementation of the draw command

This function is provided as a free function, rather than just the base class implementation in StampBuilder to make it easier for classes derived from StampBuilder to use to help implement their draw functions. The base class, StampBuilder, just calls this function for its draw method.

This version also allows for additional kwargs, which are passed on to the drawImage function. e.g. you can add add_to_image=True or setup_only=True if these are helpful.

Parameters:

• prof – The profile to draw.

• image – The image onto which to draw the profile (which may be None).

• method – The method to use in drawImage.

• offset – The offset to apply when drawing.

• config – The configuration dict for the stamp field.

• base – The base configuration dict.

• logger – A logger object to log progress.

• **kwargs – Any additional kwargs are passed along to the drawImage function.

Returns:

the resulting image

Building Objects

The following functions are relevant to building individual objects as specified by a config dict:

galsim.config.BuildGSObject(config, key, base=None, gsparams={}, logger=None)

Build a GSObject from the parameters in config[key].

Parameters:

• config – A dict with the configuration information.

• key – The key name in config indicating which object to build.

• base – The base dict of the configuration. [default: config]

• gsparams – Optionally, provide non-default GSParams items. Any gsparams specified at this level will be added to the list. This should be a dict with whatever kwargs should be used in constructing the GSParams object. [default: {}]

• logger – Optionally, provide a logger for logging debug statements. [default: None]

Returns:

the tuple (gsobject, safe), where gsobject is the built object, and safe is a bool that says whether it is safe to use this object again next time.

galsim.config.UpdateGSParams(gsparams, config, base)

Add additional items to the gsparams dict based on config[‘gsparams’].

Parameters:

• gsparams – A dict with whatever kwargs should be used in constructing the GSParams object.

• config – A dict with the configuration information.

• base – The base dict of the configuration.

Returns:

an updated gsparams dict

galsim.config.TransformObject(gsobject, config, base, logger)

Applies ellipticity, rotation, gravitational shearing and centroid shifting to a supplied GSObject, in that order.

Parameters:

• gsobject – The GSObject to be transformed.

• config – A dict with the tranformation information for this object.

• base – The base dict of the configuration.

• logger – A logger for logging debug statements.

Returns:

transformed GSObject.

class galsim.config.SkipThisObject(message=None)

A class that a builder can throw to indicate that nothing went wrong, but for some reason, this particular object should be skipped and just move onto the next object. The constructor takes an optional message that will be output to the logger if logging is active.

Generating Values

The following functions are relevant to generating and accessing individual values as specified by a config dict:

galsim.config.ParseValue(config, key, base, value_type)

Read or generate a parameter value from config.

Parameters:

• config – The config dict from which to parse the value.

• key – The key value in the dict to parse.

• base – The base config dict. [default: None, which means use base=config]

• value_type – The value_type expected. [default: None, which means it won’t check that the value is the right type.]

Returns:

the tuple (value, safe).

galsim.config.GetCurrentValue(key, config, value_type=None, base=None)

Get the current value of another config item given the key name.

Parameters:

• key – The (extended) key value in the dict to get the current value of.

• config – The config dict from which to get the key.

• value_type – The value_type expected. [default: None, which means it won’t check that the value is the right type.]

• base – The base config dict. [default: None, which means use base=config]

Returns:

the current value

galsim.config.EvaluateCurrentValue(key, config, base, value_type=None)

Helper function to evaluate the current value at config[key] where key is no longer an extended key, and config is the local dict where it is relevant.

Parameters:

• key – The key value in the dict to get the current value of.

• config – The config dict from which to get the key.

• base – The base config dict.

• value_type – The value_type expected. [default: None, which means it won’t check that the value is the right type.]

galsim.config.SetDefaultIndex(config, num)

When the number of items in a list is known, we allow the user to omit some of the parameters of a Sequence or Random and set them automatically based on the size of the list, catalog, etc.

Parameters:

• config – The config dict with the field to be updated.

• num – The number to use for the length of the index Sequence if appropriate.

galsim.config.CheckAllParams(config, req={}, opt={}, single=[], ignore=[])

Check that the parameters for a particular item are all valid

Parameters:

• config – The config dict to check

• req – The required items [default: {}]

• opt – The optional items [default: {}]

• single – List of items where exactly one is required [default: []]

• ignore – Items to ignore [default: []]

Returns:

a dict, get, with get[key] = value_type for all keys to get.

galsim.config.GetAllParams(config, base, req={}, opt={}, single=[], ignore=[])

Check and get all the parameters for a particular item

Parameters:

• config – The config dict from which to get items.

• req – The required items [default: {}]

• opt – The optional items [default: {}]

• single – List of items where exactly one is required [default: []]

• ignore – Items to ignore [default: []]

Returns:

the tuple (kwargs, safe).

galsim.config.ParseWorldPos(config, param_name, base, logger)

A helper function to parse the ‘world_pos’ value.

The world_pos can be specified either as a regular RA, Dec (which in GalSim is known as a CelestialCoord) or as Euclidean coordinates in the local tangent plane relative to the image center (a PositionD).

1. For the RA/Dec option, the world_pos field should use the type RADec, which includes two values named ra and dec, each of which should be an Angle type. e.g.:

   world_pos:
       type : RADec
       ra : 37 hours
       dec: -23 degrees
   

   Technically, any other type that results in a CelestialCoord is valid, but RADec is the only one that is defined natively in GalSim.

2. For the relative position in the local tangent plane (where 0,0 is the position of the image center), you can use any PositionD type. e.g.:

   world_pos:
       type : RandomCircle
       radius : 12       # arcsec
       inner_radius : 3  # arcsec
   

Parameters:

• config – The configuration dict for the stamp field.

• param_name – The name of the field in the config dict to parse as a world_pos. Normally, this is just ‘world_pos’.

• base – The base configuration dict.

Returns:

either a CelestialCoord or a PositionD instance.

Using Input Fields

The following functions are relevant to processing and using input fields in a config dict:

galsim.config.ProcessInput(config, logger=None, file_scope_only=False, safe_only=False)

Process the input field, reading in any specified input files or setting up any objects that need to be initialized.

Each item registered as a valid input type will be built and available at the top level of config in config[‘_input_objs’]. Since there is allowed to be more than one of each type of input object (e.g. multilpe catalogs or multiple dicts), these are actually lists. If there is only one e.g. catalog entry in config[‘input’], then this list will have one element.

e.g. config[‘_input_objs’][‘catalog’][0] holds the first catalog item defined in config[‘input’][‘catalog’] (if any).

Parameters:

• config – The configuration dict to process

• logger – If given, a logger object to log progress. [default: None]

• file_scope_only – If True, only process the input items that are marked as being possibly relevant for file- and image-level items. [default: False]

• safe_only – If True, only process the input items whose construction parameters are not going to change every file, so it can be made once and used by multiple processes if appropriate. [default: False]

galsim.config.ProcessInputNObjects(config, logger=None, approx=False)

Process the input field, just enough to determine the number of objects.

Some input items are relevant for determining the number of objects in a file or image. This means we need to have them processed before splitting up jobs over multiple processes (since the seed increments based on the number of objects). So this function builds the input items that have a getNObjects() method and returns the number of objects.

Caveat: This function tries each input type in galsim.config.valid_input_types in

order and returns the nobjects for the first one that works. If multiple input items have nobjects and they are inconsistent, this function may return a number of objects that isn’t what you wanted. In this case, you should explicitly set nobjects or nimages in the configuration dict, rather than relying on this galsim.config “magic”.

Parameters:

• config – The configuration dict to process

• logger – If given, a logger object to log progress. [default: None]

• approx – Whether an approximate count is ok. [default: False]

Returns:

the number of objects to use.

galsim.config.SetupInput(config, logger=None)

Process the input field if it hasn’t been processed yet.

This is mostly useful if the user isn’t running through the full processing and just starting at BuildImage say. This will make sure the input objects are set up in the way that they normally would have been by the first level of processing in a galsim config_file run.

Parameters:

• config – The configuration dict in which to setup the input items.

• logger – If given, a logger object to log progress. [default: None]

galsim.config.SetupInputsForImage(config, logger=None)

Do any necessary setup of the input items at the start of an image.

Parameters:

• config – The configuration dict to process

• logger – If given, a logger object to log progress. [default: None]

galsim.config.GetInputObj(input_type, config, base, param_name, num=0)

Get the input object needed for generating a particular value

Parameters:

• input_type – The type of input object to get

• config – The config dict for this input item

• base – The base config dict

• param_name – The type of value that we are trying to construct (only used for error messages).

• num – Which number in the list of this key, if needed. [default: 0]

Processing Extra Outputs

The following functions are relevant to processing extra output fields in a config dict:

galsim.config.SetupExtraOutput(config, logger=None)

Set up the extra output items as necessary, including building Managers for the work space so they can work safely in multi-processing mode. Each builder will be placed in config[‘extra_builder’][key] where key is the key in galsim.config.valid_extra_outputs.

Parameters:

• config – The configuration dict.

• logger – If given, a logger object to log progress. [default: None]

galsim.config.SetupExtraOutputsForImage(config, logger=None)

Perform any necessary setup for the extra output items at the start of a new image.

Parameters:

• config – The configuration dict.

• logger – If given, a logger object to log progress. [default: None]

galsim.config.ProcessExtraOutputsForStamp(config, skip, logger=None)

Run the appropriate processing code for any extra output items that need to do something at the end of building each object.

This gets called after all the object flux is added to the stamp, but before the sky level and noise are added.

Parameters:

• config – The configuration dict.

• skip – Was the drawing of this object skipped?

• logger – If given, a logger object to log progress. [default: None]

galsim.config.ProcessExtraOutputsForImage(config, logger=None)

Run the appropriate processing code for any extra output items that need to do something at the end of building each image

Parameters:

• config – The configuration dict.

• logger – If given, a logger object to log progress. [default: None]

galsim.config.WriteExtraOutputs(config, main_data, logger=None)

Write the extra output objects to files.

This gets run at the end of the functions for building the regular output files.

Parameters:

• config – The configuration dict.

• main_data – The main file data in case it is needed.

• logger – If given, a logger object to log progress. [default: None]

galsim.config.AddExtraOutputHDUs(config, main_data, logger=None)

Write the extra output objects to either HDUS or images as appropriate and add them to the existing data.

This gets run at the end of the functions for building the regular output files.

Note: the extra items must have hdu numbers ranging continuously (in any order) starting at len(data). Typically first = 1, since the main image is the primary HDU, numbered 0.

Parameters:

• config – The configuration dict.

• main_data – The main file data as a list of images. Usually just [image] where image is the primary image to be written to the output file.

• logger – If given, a logger object to log progress. [default: None]

Returns:

data with additional hdus added

galsim.config.CheckNoExtraOutputHDUs(config, output_type, logger=None)

Check that none of the extra output objects want to add to the HDU list.

Raises an exception if one of them has an hdu field.

Parameters:

• config – The configuration dict.

• output_type – A string to use in the error message to indicate which output type had a problem.

• logger – If given, a logger object to log progress. [default: None]

galsim.config.GetFinalExtraOutput(key, config, main_data=[], logger=None)

Get the finalized output object for the given extra output key

Parameters:

• key – The name of the output field in config[‘output’]

• config – The configuration dict.

• main_data – The main file data in case it is needed. [default: []]

• logger – If given, a logger object to log progress. [default: None]

Returns:

the final data to be output.

Config Utilities

The following functions are used internally by the various galsim.config functions, but they might be useful for some users.

class galsim.config.LoggerWrapper(logger)

A wrap around a Logger object that checks whether a debug or info or warn call will actually produce any output before calling the functions.

This seems like a gratuitous wrapper, and it is if the object being wrapped is a real Logger object. However, we use it to wrap proxy objects (returned from GetLoggerProxy) that would otherwise send the arguments of logger.debug(…) calls through a multiprocessing pipe before (typically) being ignored. Here, we check whether the call will actually produce any output before calling the functions.

Parameters:

logger – The logger object to wrap.

galsim.config.ReadYaml(config_file)

Read in a YAML configuration file and return the corresponding dicts.

A YAML file is allowed to define several dicts using multiple documents. The GalSim parser treats this as a set of multiple jobs to be done. The first document is taken to be a “base” dict that has common definitions for all the jobs. Then each subsequent document has the (usually small) modifications to the base dict for each job. See demo6.yaml, demo8.yaml and demo9.yaml in the GalSim/examples directory for example usage.

The return value will be a list of dicts, one dict for each job to be done.

Parameters:

config_file – The name of the configuration file to read.

Returns:

list of config dicts

galsim.config.ReadJson(config_file)

Read in a JSON configuration file and return the corresponding dicts.

A JSON file only defines a single dict. However to be parallel to the functionality of ReadYaml, the output is a list with a single item, which is the dict defined by the JSON file.

Parameters:

config_file – The name of the configuration file to read.

Returns:

[config_dict]

galsim.config.MergeConfig(config1, config2, logger=None)

Merge config2 into config1 such that it has all the information from either config1 or config2 including places where both input dicts have some of a field defined.

e.g. If config1 has image.pixel_scale, and config2 has image.noise, then the returned dict will have both.

For real conflicts (the same value in both cases), config1’s value takes precedence

galsim.config.ConvertNones(config)

Convert any items whose value is ‘None’ to None.

To allow some parameters to be set to None in the config dict (e.g. in a list, where only some values need to be None), we convert all values == ‘None’ to None.

Parameters:

config – The config dict to process

galsim.config.RemoveCurrent(config, keep_safe=False, type=None, index_key=None)

Remove any “current” values stored in the config dict at any level.

Parameters:

• config – The configuration dict.

• keep_safe – Should current values that are marked as safe be preserved? [default: False]

• type – If provided, only clear the current value of objects that use this particular type. [default: None, which means to remove current values of all types.]

• index_key – If provided, only clear the current value of objects that use this index_key (or start with this index_key, so obj_num also does obj_num_in_file). [default: None]

galsim.config.GetLoggerProxy(logger)

Make a proxy for the given logger that can be passed into multiprocessing Processes and used safely.

Parameters:

logger – The logger to make a copy of

Returns:

a proxy for the given logger

galsim.config.UpdateNProc(nproc, ntot, config, logger=None)

Update nproc

• If nproc < 0, set nproc to ncpu

• Make sure nproc <= ntot

Parameters:

• nproc – The nominal number of processes from the config dict

• ntot – The total number of files/images/stamps to do, so the maximum number of processes that would make sense.

• config – The configuration dict to copy.

• logger – If given, a logger object to log progress. [default: None]

Returns:

the number of processes to use.

galsim.config.ParseRandomSeed(config, param_name, base, seed_offset)

Parse the random_seed field, converting an integer (initial) seed value into a Sequence.

galsim.config.PropagateIndexKeyRNGNum(config, index_key=None, rng_num=None, rng_index_key=None, key=None)

Propagate any index_key or rng_num specification in a dict to all sub-fields

galsim.config.SetupConfigRNG(config, seed_offset=0, logger=None)

Set up the RNG in the config dict.

• Setup config[‘image’][‘random_seed’] if necessary

• Set config[‘rng’] and other related values based on appropriate random_seed

Parameters:

• config – The configuration dict.

• seed_offset – An offset to use relative to what config[‘image’][‘random_seed’] gives. [default: 0]

• logger – If given, a logger object to log progress. [default: None]

Returns:

the seed used to initialize the RNG.

galsim.config.ParseExtendedKey(config, key)

Traverse all but the last item in an extended key and return the resulting config, key.

If key is an extended key like gal.items.0.ellip.e, then this will return the tuple. (config[‘gal’][‘items’][0][‘ellip’], ‘e’).

If key is a regular string, then is just returns the original (config, key).

Parameters:

• config – The configuration dict.

• key – The possibly extended key.

Returns:

the equivalent (config, key) where key is now a regular non-extended key.

galsim.config.GetFromConfig(config, key)

Get the value for the (possibly extended) key from a config dict.

If key is a simple string, then this is equivalent to config[key]. However, key is allowed to be a chain of keys such as ‘gal.items.0.ellip.e’, in which case this function will return config[‘gal’][‘items’][0][‘ellip’][‘e’].

Parameters:

• config – The configuration dict.

• key – The possibly extended key.

Returns:

the value of that key from the config.

galsim.config.SetInConfig(config, key, value, logger=None)

Set the value of a (possibly extended) key in a config dict.

If key is a simple string, then this is equivalent to config[key] = value. However, key is allowed to be a chain of keys such as ‘gal.items.0.ellip.e’, in which case this function will set config[‘gal’][‘items’][0][‘ellip’][‘e’] = value.

Parameters:

• config – The configuration dict.

• key – The possibly extended key.

Returns:

the value of that key from the config.

galsim.config.UpdateConfig(config, new_params, logger=None)

Update the given config dict with additional parameters/values.

Parameters:

• config – The configuration dict to update.

• new_params – A dict of parameters to update. The keys of this dict may be chained field names, such as gal.first.dilate, which will be parsed to update config[‘gal’][‘first’][‘dilate’].

galsim.config.MultiProcess(nproc, config, job_func, tasks, item, logger=None, timeout=900, done_func=None, except_func=None, except_abort=True)

A helper function for performing a task using multiprocessing.

A note about the nomenclature here. We use the term “job” to mean the job of building a single file or image or stamp. The output of each job is gathered into the list of results that is returned. A task is a collection of one or more jobs that are all done by the same processor. For simple cases, each task is just a single job, but for things like a Ring test, the task needs to have the jobs for a full ring.

The tasks argument is a list of tasks. Each task in that list is a list of jobs. Each job is a tuple consisting of (kwargs, k), where kwargs is the dict of kwargs to pass to the job_func and k is the index of this job in the full list of jobs.

Parameters:

• nproc – How many processes to use.

• config – The configuration dict.

• job_func –

  The function to run for each job. It will be called as:

  result = job_func(**kwargs)
  

  where kwargs is from one of the jobs in the task list.

• tasks – A list of tasks to run. Each task is a list of jobs, each of which is a tuple (kwargs, k).

• item – A string indicating what is being worked on.

• logger – If given, a logger object to log progress. [default: None]

• timeout – How many seconds to allow for each task before timing out. [default: 900]

• done_func –

  A function to run upon completion of each job. It will be called as:

  done_func(logger, proc, k, result, t)
  

  where proc is the process name, k is the index of the job, result is the return value of that job, and t is the time taken. [default: None]

• except_func –

  A function to run if an exception is encountered. It will be called as:

  except_func(logger, proc, k, ex, tr)
  

  where proc is the process name, k is the index of the job that failed, ex is the exception caught, and tr is the traceback. [default: None]

• except_abort – Whether an exception should abort the rest of the processing. If False, then the returned results list will not include anything for the jobs that failed. [default: True]

Returns:

a list of the outputs from job_func for each job

galsim.config.GetIndex(config, base, is_sequence=False)

Return the index to use for the current object or parameter and the index_key.

First check for an explicit index_key value given by the user. Then if base[index_key] is other than obj_num, use that. Finally, if this is a sequence, default to ‘obj_num_in_file’, otherwise ‘obj_num’.

Returns:

index, index_key

galsim.config.GetRNG(config, base, logger=None, tag='')

Get the appropriate current rng according to whatever the current index_key is.

If a logger is provided, then it will emit a warning if there is no current rng setup.

Parameters:

• config – The configuration dict for the current item being worked on.

• base – The base configuration dict.

• logger – If given, a logger object to log progress. [default: None]

• tag – If given, an appropriate name for the current item to use in the warning message. [default: ‘’]

Returns:

either the appropriate rng for the current index_key or None

galsim.config.CleanConfig(config, keep_current=False)

Return a “clean” config dict without any leading-underscore values

GalSim config dicts store a lot of ancillary information internally to help improve efficiency. However, some of these are actually pointers to other places in the dict, so printing a config dict, or even what should be a small portion of one, can have infinite loops.

This helper function is useful when debugging config processing to strip out all of these leading-underscore values, so that printing the dict is reasonable.

>>> print(galsim.config.CleanConfig(config_dict))

galsim.config.SetDefaultExt(config, default_ext)

Set a default ext in a config ‘file_name’ field if appropriate.

Parameters:

• config – The configuration dict for the item that might need to be given a default ‘ext’ value.

• default_ext – The default extension to set in the config dict if one is not set.

galsim.config.RetryIO(func, args, ntries, file_name, logger)

A helper function to retry I/O commands

galsim.config.MakeImageTasks(config, jobs, logger)

Turn a list of jobs into a list of tasks.

See the doc string for galsim.config.MultiProcess for the meaning of this distinction.

For most image types, there is just one job per task, so the tasks list is just:

tasks = [ [ (job, k) ] for k, job in enumerate(jobs) ]

But some image types may need groups of jobs to be done sequentially by the same process. The image type=Single for instance uses whatever grouping is needed for the stamp type.

Parameters:

• config – The configuration dict

• jobs – A list of jobs to split up into tasks. Each job in the list is a dict of parameters that includes ‘image_num’ and ‘obj_num’.

• logger – If given, a logger object to log progress.

Returns:

a list of tasks

galsim.config.MakeStampTasks(config, jobs, logger)

Turn a list of jobs into a list of tasks.

See the doc string for galsim.config.MultiProcess for the meaning of this distinction.

For the Basic stamp type, there is just one job per task, so the tasks list is just:

tasks = [ [ (job, k) ] for k, job in enumerate(jobs) ]

But other stamp types may need groups of jobs to be done sequentially by the same process. cf. stamp type=Ring.

Parameters:

• config – The configuration dict

• jobs – A list of jobs to split up into tasks. Each job in the list is a dict of parameters that includes ‘obj_num’.

• logger – A logger object to log progress.

Returns:

a list of tasks

galsim.config.RegisterInputConnectedType(input_type, type_name)

Register that some gsobject or value type is connected to a given input type.

Parameters:

• input_type – The name of the type in config[‘input’]

• type_name – The name of the type that uses this input object.