SpEC: waveform extrapolation

Getting Waveform Extrapolation Setup (NSNS, BHNS)

Document is broken up into two parts:

1. First time setup (only needed to run once)
2. Commands needed for every simulation for which you wish to extrapolate.

1.) One Time Setup (for all runs)

Getting python environment set up:

Unload python/anaconda2-4.1.1:

module unload python/anaconda2-4.1.1

Load python/anaconda3-4.4.0:

module load python/anaconda3-4.4.0

(Note: conda versions may be out-of-date, instructions were originally written for Wheeler)

Then:

conda config --add channels conda-forge

conda create --name py3 python=3.7

source activate py3

conda install scri

conda install numpy

conda install numba

Clone CatalogAnalysis repo:

git clone [email protected]:sxs-collaboration/CatalogAnalysis.git

2.) For Every Simulation

For NSNS, files needed from run:

rh_FiniteRadii_CodeUnits.h5
CoM-NSA-InertialFrame.dat
CoM-NSB-InertialFrame.dat
MassAvgs.dat
EvolutionParameters.perl 
Python program CreateMatterH5.py, listed in Appendix below

For BHNS, files needed are:

Run/ApparentHorizon/Horizons.h5
Run/ForWaveExtraction/rh_FiniteRadii_CodeUnits.h5
Run/MatterObservers/MassAvgs.dat
Run/MatterObservers/InertialCenterOfMass.dat
EvID/ID_Params.perl
Python program CreateMatterH5.py, listed 2nd in Appendix below

Modifying files

For NSNS (EvolutionParameters.perl):

• Rename EvolutionParameters.perl ---> ID_Params.perl

• Change InitialAdmEnergy ---> ID_Eadm

• Add three lines, modify values to match your simulation:

  @ID_chiA=(0.,0.,0.);

  @ID_chiB=(0.,0.,0.);

  $ID_Jadm=(0.,0.,0.); (these values are in the FID.dat file in the highest resolution ID run)

For BHNS (ID_Params.perl):

• ID_Params.perl should already exist in the EvID directory

• Change InitialAdmEnergy ---> ID_Eadm

• Add these lines, modify values to match your simulation:

  @ID_chiB=(0.,0.,0.); (This is the spin of NS, usually remains 0,0,0)

  $ID_Jadm=(0.,0.,0.); (these values are in the FID.dat file in the highest resolution ID run)

Arranging files:

(NSNS)

• Create directories with structure

  |__Ev/

  |__EvID/

• Move combined files rh_FiniteRadii_CodeUnits.h5, CoM-NS*, and MassAvgs.dat into the Ev directory, and move ID_Params.perl into EvID, i.e.:

    |__Ev/
  
         |__rh_FiniteRadii_CodeUnits.h5
  
         |__MassAvgs.dat
  
         |__CoM-NS{A/B}-InertialFrame.dat
  
    |__EvID/
  
         |__ID_Params.perl
  

(BHNS)

• Create directories with structure

  |__Ev/

  |__EvID/

• Move combined files rh_FiniteRadii_CodeUnits.h5, CoM-NS*, and MassAvgs.dat into the Ev directory, and move ID_Params.perl into EvID, i.e.:

    |__Ev/
  
         |__rh_FiniteRadii_CodeUnits.h5
  
         |__MassAvgs.dat
  
         |__Horizons.h5
  
         |__InertialCenterOfMass.dat
  
    |__EvID/
  
         |__ID_Params.perl
  

Running CreateMatterH5.py (all run types):

• In Ev/, run CreateMatterH5.py (see Appendix below if you don't already have this script!)

Creating metadata (NSNS/BHNS):

(NSNS)

In Ev/, run

python $SPECLOCATION/Support/Python/WriteMetadataSpEC.py  --alternative-names=SXS-NSNSSPECTRALQ11L0 -t nsns ./

replacing SXS-NSNSSPECTRALQ11L0 with name of simulation.

---

(BHNS)

In Ev/, run

python $SPECLOCATION/Support/Python/WriteMetadataSpEC.py  --alternative-names=SXS-BHNSSPECTRALQ11L0 -t bhns ./

replacing SXS-BHNSSPECTRALQ11L0 with name of simulation.

Note: If an error occurs in regards to Horizon.h5 not existing (used for runs with BH’s), then you will need to modify the WriteMetadataSpEC.py file in your copy of spec. Change line 904 from

   remnant_exists = "AhC.dir" in Utils.ReadH5("Horizons.h5")

to

if os.path.isfile("Horizons.h5"):
    remnant_exists = "AhC.dir" in Utils.ReadH5("Horizons.h5")
else:
  if (args.binarytype=='bbh' or args.binarytype=='bhns'):
    print("ERROR: Horizons.h5 should exist for BBH/BHNS")
    exit()
  remnant_exists = False

Running Extrapolat_scri.py (all run types):

Ensure the python environment is enabled

source activate py3

In the Ev/ directory, run

$CatalogAnalysisLocation/Extrapolation/Extrapolate_scri.py  rh_FiniteRadii_CodeUnits.h5

Extracting ‘extracted-rhOverM_Asymptotic_GeometricUnits.h5’ (all run types):

ExtractFromH5 rhOverM_Asymptotic_GeometricUnits.h5

If the above command fails, then you need to modify $SPEC_HOME/Utils/H5/H5_ObjectVersion.cpp: Comment out lines 60-69 (dealing with version of h5), recompile spec, retry ExtractFromH5 command again

Appendix:

CreateMatterH5.py (NSNS):

#!/usr/bin/env python
import h5py as h
import numpy as np

comA = np.genfromtxt("CoM-NSA-InertialFrame.dat")
comB = np.genfromtxt("CoM-NSB-InertialFrame.dat")
mass = np.genfromtxt("MassAvgs.dat")`

f = h.File('Matter.h5','w')

dset_comA = f.create_dataset("InertialCenterOfMassNS1.dat",data=comA)
dset_comB = f.create_dataset("InertialCenterOfMassNS2.dat",data=comB)
dset_mass = f.create_dataset("BaryonMassOnGrid.dat",data=mass)

f.close()

CreateMatterH5.py (BHNS):

#!/usr/bin/env python
import h5py as h
import numpy as np


comA = np.genfromtxt("InertialCenterOfMass.dat")
mass = np.genfromtxt("MassAvgs.dat")

f = h.File('Matter.h5','w')

dset_comA = f.create_dataset("InertialCenterOfMass.dat",data=comA)
dset_mass = f.create_dataset("RestMass.dat",data=mass)

f.close()

Contact Alexander Knight or Tia Martineau for any issues with this document.