Added CMFD Functionality - Tally windows, statepoint write/restart, in-memory functions
Created by: shikhar413
This PR adds more functionality to the current CMFD implementation to make it easier to run CMFD functions in memory and on larger benchmark problems with better accuracy. The following features are included as part of the PR:
Numpy 32-bit precision: The
weightfactorsnumpy array passed to OpenMC through the CAPI is now explicitly defined as a 32-bit array. This is because I was finding that originally passing a default 64-bit array led to k-effective values passed back from OpenMC that had slight discrepancies for large test cases that became larger as the cycles progressed. A 32-bit numpy array ensures consistent results for the same test case. This is also the main cause of values changing in the CMFD test suite.
- Restart capabilities from statepoint file: All relevant data needed to restart a CMFD run is now outputtted to statepoint file, and this statepoint file can also be used to restart a CMFD run. More details below.
feedback_begin: Originally, if CMFD is run with feedback, the batch at which CMFD tallies begin and the batch when feedback is turned on had to be the same. However, a lot of times it is beneficial to aggregate CMFD tallies over a few batches before turning CMFD feedback on to ensure that a large enough tally window has been accumulated. This is now implemented by defining
Add functions to allow CMFD to be run in-memory batch-by-batch: Similar to the C API functions, the
CMFDRunclass now has functions
finalizeso that users can run CMFD in-memory batch-by-batch as follows:
from openmc import cmfd cmfd_run = cmfd.CMFDRun() with cmfd_run.run_in_memory(): do_stuff_before_simulation_start() # e.g. define new tallies, set certain tallies to active for _ in cmfd_run.iter_batches(): do_stuff_between_batches() # e.g. observe behavior of tallies, source distribution
Tally window options: Currently, the CMFD matrix system that is solved from batch
tally_beginonwards is based on tallies aggregated from the very first batch. However, tallies from initial batches are usually biased and should not be used to update the source many batches later. While CMFD tallies can be reset at specific batches, this usually causes the CMFD k-effective value to spike suddenly, since the previous CMFD update uses tallies aggregated over a large number of batches, while the following update is based on tallies from only one batch. To smoothen out this behavior, users have the option to choose between a "rolling window" and "expanding window" type for tally aggregation, based on work by @bhermanmit and @lilulu. More details below:
Reference diffusion parameters: Another way to smoothen out CMFD behavior and allow for feedback to start at an earlier batch is to be able to specify reference diffusion coefficients that will be used to set
dtildeat each batch. Normally, the diffusion coefficient is calculated at each batch using the P1 scattering approximation, but the
CMFDRun.ref_dvariable can be used to define reference diffusion coefficients for each group (e.g. from CMM).
CMFD Restart Functionality
The variables outputted to statepoint file can be found in
CMFDRun._write_cmfd_statepoint. While some of these variables are not essential to restart the simulation (such as
indices etc.), many are outputted to overwrite parameters set during a restart run to ensure that users do not change these parameters mid-simulation and lead to unexpected behavior. In terms of how
statepoint_write works, I make sure that statepoint files are written only if this function is called. Typically, a statepoint file is written right after
openmc.capi.next_batch() is called (but before CMFD is run for that batch), but this outputs an incorrect source to the statepoint file that does not have source weights updated through CMFD. Thus, a statepoint is written only after CMFD is run for that batch, using either the function
CMFDRun.statepoint_write() in-memory or by specifying statepoint batches in the settings.xml file. In order to make this work, I create the variable
cmfd_run and function
openmc_is_statepoint_batch() in OpenMC.
Restarting a CMFD simulation from a restart file is also slightly roundabout. CMFD tallies are outputted to statepoint file, but the way OpenMC is currently implemented, tally data is only populated for tallies that have been defined in the tallies.xml file during openmc_simulation_init(). However, since CMFD tallies are defined in-memory, CMFD tally data is never populated since these tallies have not been initialized yet by the time
load_state_point is called in OpenMC. Thus the following workaround is proposed to reset CMFD data and tallies:
openmc.capi.init(args=['-r','res_file.h5']) # Specify restart_run and restart_file openmc.capi.simulation_init() # calls load_state_point CMFDRun._create_cmfd_tally() # allocate CMFD tallies in memory openmc.capi._dll.openmc_load_cmfd_tallies() # populate CMFD tally data from statepoint file
The last two function calls occur in CMFDRun._reset_cmfd which is called by CMFDRun._configure_cmfd in CMFDRun.init()
Description of tally window types
This PR introduces two new types of window types, a rolling window and an expanding window. A rolling window is a fixed window width that moves over by one batch once the window size has been met. On the other hand, an expanding window doubles in size once tallies from previous batches have filled up the entire window. This is based on making sure that more batches are used to compute CMFD parameters at later batches, since there should now be less chance of biased tallies affecting results. The illustration of these two window schemes are given below, taken from @lilulu's Ph.D. thesis (tallies from batch 1 are typically discarded for large problems like 2D BEAVRS). Rolling window: Expanding window (Window at batch 2 is not shown since it is not used for feedback but is of size 1):
To implement these types of windowing schemes, I introduce cross section rate variables like
p1scatt_rate etc that save tally data at each batch in
CMFDRun._compute_xs(), where tallies are reset every batch. The actual cross sections are now defined as the sum of these rate variables over the defined window size. Note that the cross section rate variables grow at the size of the window. By default,
window_type is set to
'none' and this implementation is incorporated within the framework of using cross section rate variables so that
CMFDRun._compute_xs() can be kept as general as possible.