Meeting a grand scientific challenge

Scientific Motivation:

As one of the largest ever full cosmological simulations, the Astrid simulation will provide the best view of the predictions of the standard cosmological model for a wide range of topics in galaxy and structure formation. One of the most timely will be the ability of Astrid with its detailed models of black hole formation and merging, to study the background of gravitational waves indicated by recent Pulsar Timing Array measurements. Another will be predictions for the Rubin Observatory’s Legacy Survey of Space and Time, which will start taking data in 2024. The large dynamic range of Astrid on one hand allows detailed studies of galaxy and massive black hole evolution at high resolution, and on the other hand allows systematic studies of rare massive systems with the large cosmic volume. Astrid is enabling studies of high redshift galaxies and black holes that are valuable for current and future observations such as the James Webb Telescope, LISA gravitational wave observatory, and DESI galaxy surveys.

Physics Models:

The simulations in the ASTRID suite have been run with the MP-Gadget code, which solves gravitational interactions with the TreePM algorithm, hydrodynamics with an entropy-conserving Smoothed Particle Hydrodynamics (SPH) formulation, and incorporates astrophysical processes through a series of subgrid models. ASTRID model radiative cooling and photoionization heating of gas including primordial radiative cooling (Katz et al. 1996) and metal line cooling, with the gas and stellar metallicities traced following Vogelsberger et al. (2014). The cosmic ionizing UV background follows Faucher-Giguere (2020), with patchy reionization implemented in a semi-analytic approach (Battaglia et al. 2013). Hydrogen self-shielding is modeled following Rahmati et al. (2013). Star formation is implemented based on the multi-phase star formation model in Springel & Hernquist (2003), and accounts for the effects of molecular hydrogen based on H2 fraction calculated from the metallicity and local column density (Krumholz & Gnedin 2011). Type II supernova wind feedback is included following Okamoto et al. (2010). ASTRID tracks metal enrichment from AGB stars, Type II SNe, and Type Ia SNe, following 9 individual elements (H, He, C, N, O, Ne, Mg, Si, Fe). Galactic winds driven by stellar feedback are implemented kinetically via temporarily hydrodynamically decoupled particles. Supermassive black holes (SMBHs) are tracked throughout cosmic history, with halo-based BH seeding following a power-law distribution. The dynamics of black holes are modeled using the prescription in Chen(2021). Black hole growth is governed by Eddington-limited Bondi accretion. At high redshift (z>2.3), AGN feedback operates in thermal mode. At z=2.3, AGN feedback is divided into two modes: quasar-mode during high Eddington accretion phases, and jet-mode during low Eddington accretion states for massive BHs.

Supporting Software:

Subgrid Physics

  • Star formation with molecular hydrogen corrections
  • Supernova winds and feedback
  • Stellar evolution and metal return
  • AGN feedback
  • Primordial and metal cooling
  • Inhomogeneous hydrogen and helium reionization models
  • Massive neutrinos
  • Relative CDM/baryon velocities

Non-numerical Algorithms

  • Pencil Fast Fourier Transformation
  • Shared memory neighbor tree
  • Asynchronous MPI communication
  • Uniformly-accessible striped snapshot files
  • Partition-based parallel sorting
  • Friend-of-Friend clustering and object identification

Computational Methods

  • Pressure-entropy smoothed particle hydrodyanmics
  • Tree and Particle Mesh N-Body gravity solver
  • Gadget-4-style hierarchical gravitational timestepping

Programming Environment

  • TACC Frontera Intel Cascade Lake with 56 cores per node
  • OpenMP/MPI Hybrid, most efficient at 1 MPI rank per socket

The Dataset:

We release the full particles in group data from the Astrid simulation through the web portal hosted at the Pittsburgh Supercomputing Center (see the “Data Access” Tab). The particle species in Astrid are: dark matter (1), gas (0), stars (4), and black holes (5). Particles are assigned to FOFGroups (halos) through the Friends-of-Friends halo finder algorithm. Subhalo information is also available for selected snapshots, identified by the Subfind algorithm (Springel, 2001). We will also release the black hole merger files, black hole details files and more post-processed catalogs in the coming months.