The complexity of modern and upcoming computing architectures poses severe challenges for code developers and application specialists, and forces them to expose the highest possible degree of parallelism, in order to make the best use of the available hardware. The Intel$^{(R)}$ Xeon Phi$^{(TM)}$ of second generation (code-named Knights Landing, henceforth KNL) is the latest many-core system, which implements several interesting hardware features like for example a large number of cores per node (up to 72), the 512 bits-wide vector registers and the high-bandwidth memory. The unique features of KNL make this platform a powerful testbed for modern HPC applications. The performance of codes on KNL is therefore a useful proxy of their readiness for future architectures. In this work we describe the lessons learnt during the optimisation of the widely used codes for computational astrophysics P-Gadget-3, Flash and Echo. Moreover, we present results for the visualisation and analysis tools VisIt and yt. These examples show that modern architectures benefit from code optimisation at different levels, even more than traditional multi-core systems. However, the level of modernisation of typical community codes still needs improvements, for them to fully utilise resources of novel architectures.

As modern scientific simulations grow ever more in size and complexity, even their analysis and post-processing becomes increasingly demanding, calling for the use of HPC resources and methods. yt is a parallel, open source post-processing python package for numerical simulations in astrophysics, made popular by its cross-format compatibility, its active community of developers and its integration with several other professional Python instruments. The Intel Distribution for Python enhances yt's performance and parallel scalability, through the optimization of lower-level libraries Numpy and Scipy, which make use of the optimized Intel Math Kernel Library (Intel-MKL) and the Intel MPI library for distributed computing. The library package yt is used for several analysis tasks, including integration of derived quantities, volumetric rendering, 2D phase plots, cosmological halo analysis and production of synthetic X-ray observation. In this paper, we provide a brief tutorial for the installation of yt and the Intel Distribution for Python, and the execution of each analysis task. Compared to the Anaconda python distribution, using the provided solution one can achieve net speedups up to 4.6x on Intel Xeon Scalable processors (codename Skylake).

In this exploratory submission we present the visualization of the largest interstellar turbulence simulations ever performed, unravelling key astrophysical processes concerning the formation of stars and the relative role of magnetic fields. The simulations, including pure hydrodynamical (HD) and magneto-hydrodynamical (MHD) runs, up to a size of $10048^3$ grid elements, were produced on the supercomputers of the Leibniz Supercomputing Centre and visualized using the hybrid parallel (MPI+TBB) ray-tracing engine OSPRay associated with VisIt. Besides revealing features of turbulence with an unprecedented resolution, the visualizations brilliantly showcase the stretching-and-folding mechanisms through which astrophysical processes such as supernova explosions drive turbulence and amplify the magnetic field in the interstellar gas, and how the first structures, the seeds of newborn stars are shaped by this process.

Galaxy clusters are known to be reservoirs of Cosmic Rays (CRs), as inferred from theoretical calculations or detection of CR-derived observables. CR acceleration in clusters is mostly attributed to the dynamical activity that produces shocks. Shocks in clusters emerge out of merger or accretion, but which one is more effective in producing CRs? at which dynamical phase? and why? To this aim, we study the production or injection of CRs through shocks and its evolution in the galaxy clusters using cosmological simulations with the {\sc enzo} code. Particle acceleration model considered here is primarily the Diffusive Shock Acceleration (DSA) of thermal particles, but we also report a tentative study with pre-existing CRs. Defining appropriate dynamical states using the concept of virialization, we studied a sample of merging and non-merging clusters. We report that the merger shocks (with Mach number $\mathcal{M}\sim2-5$) are the most effective CR producers, while high-Mach peripheral shocks (i.e. $\mathcal{M}>5$) are mainly responsible for the brightest phase of CR injection in clusters. Clusters once merged, permanently deviate from CR and X-ray mass scaling of non-merging systems, enabling us to use it as a tool to determine the state of merger. Through a temporal and spatial evolution study, we found a strong correlation between cluster merger dynamics and CR injection. We observed that the brightest phase of X-ray and CR injection from clusters occur respectively at about 1.0 and 1.5 Gyr after every mergers, and CR injection peaks near to the cluster virial radius (i.e $r_{200}$). Delayed CR injection peaks found in this study deserve further investigation for possible impact on the evolution of CR-derived observables from galaxy clusters.

We present recent developments in the parallelization scheme of ECHO-3DHPC, an efficient astrophysical code used in the modelling of relativistic plasmas. With the help of the Intel Software Development Tools, like Fortran compiler and Profile-Guided Optimization (PGO), Intel MPI library, VTune Amplifier and Inspector we have investigated the performance issues and improved the application scalability and the time to solution. The node-level performance is improved by $2.3 \times$ and, thanks to the improved threading parallelisation, the hybrid MPI-OpenMP version of the code outperforms the MPI-only, thus lowering the MPI communication overhead.

We describe a strategy for code modernisation of Gadget, a widely used community code for computational astrophysics. The focus of this work is on node-level performance optimisation, targeting current multi/many-core IntelR architectures. We identify and isolate a sample code kernel, which is representative of a typical Smoothed Particle Hydrodynamics (SPH) algorithm. The code modifications include threading parallelism optimisation, change of the data layout into Structure of Arrays (SoA), auto-vectorisation and algorithmic improvements in the particle sorting. We obtain shorter execution time and improved threading scalability both on Intel XeonR ($2.6 \times$ on Ivy Bridge) and Xeon PhiTM ($13.7 \times$ on Knights Corner) systems. First few tests of the optimised code result in $19.1 \times$ faster execution on second generation Xeon Phi (Knights Landing), thus demonstrating the portability of the devised optimisation solutions to upcoming architectures.

The outskirts of galaxy clusters are characterised by the interplay of gas accretion and dynamical evolution involving turbulence, shocks, magnetic fields and diffuse radio emission. The density and velocity structure of the gas in the outskirts provide an effective pressure support and affect all processes listed above. Therefore it is important to resolve and properly model the turbulent flow in these mildly overdense and relatively large cluster regions; this is a challenging task for hydrodynamical codes. In this work, grid-based simulations of a galaxy cluster are presented. The simulations are performed using adaptive mesh refinement (AMR) based on the regional variability of vorticity, and they include a subgrid scale model (SGS) for unresolved turbulence. The implemented AMR strategy is more effective in resolving the turbulent flow in the cluster outskirts than any previously used criterion based on overdensity. We study a cluster undergoing a major merger, which drives turbulence in the medium. The merger dominates the cluster energy budget out to a few virial radii from the centre. In these regions the shocked intra-cluster medium is resolved and the SGS turbulence is modelled, and compared with diagnostics on larger length scale. The volume-filling factor of the flow with large vorticity is about 60% at low redshift in the cluster outskirts, and thus smaller than in the cluster core. In the framework of modelling radio relics, this point suggests that upstream flow inhomogeneities might affect pre-existing cosmic-ray population and magnetic fields, and the resulting radio emission.

In spring 2015, the Leibniz Supercomputing Centre (Leibniz-Rechenzentrum, LRZ), installed their new Peta-Scale System SuperMUC Phase2. Selected users were invited for a 28 day extreme scale-out block operation during which they were allowed to use the full system for their applications. The following projects participated in the extreme scale-out workshop: BQCD (Quantum Physics), SeisSol (Geophysics, Seismics), GPI-2/GASPI (Toolkit for HPC), Seven-League Hydro (Astrophysics), ILBDC (Lattice Boltzmann CFD), Iphigenie (Molecular Dynamic), FLASH (Astrophysics), GADGET (Cosmological Dynamics), PSC (Plasma Physics), waLBerla (Lattice Boltzmann CFD), Musubi (Lattice Boltzmann CFD), Vertex3D (Stellar Astrophysics), CIAO (Combustion CFD), and LS1-Mardyn (Material Science). The projects were allowed to use the machine exclusively during the 28 day period, which corresponds to a total of 63.4 million core-hours, of which 43.8 million core-hours were used by the applications, resulting in a utilization of 69%. The top 3 users were using 15.2, 6.4, and 4.7 million core-hours, respectively.

Understanding turbulence is critical for a wide range of terrestrial and astrophysical applications. Here we present first results of the world's highest-resolution simulation of turbulence ever done. The current simulation has a grid resolution of 10048^3 points and was performed on 65536 compute cores on SuperMUC at the Leibniz Supercomputing Centre (LRZ). We present a scaling test of our modified version of the FLASH code, which updates the hydrodynamical equations in less than 3 micro seconds per cell per time step. A first look at the column density structure of the 10048^3 simulation is presented and a detailed analysis is provided in a forthcoming paper.

Galaxy clusters are unique laboratories to investigate turbulent fluid motions and large scale magnetic fields. Synchrotron radio halos at the center of merging galaxy clusters provide the most spectacular and direct evidence of the presence of relativistic particles and magnetic fields associated with the intracluster medium. The study of polarized emission from radio halos is extremely important to constrain the properties of intracluster magnetic fields and the physics of the acceleration and transport of the relativistic particles. However, detecting this polarized signal is a very hard task with the current radio facilities.We use cosmological magneto-hydrodynamical simulations to predict the expected polarized surface brightness of radio halos at 1.4 GHz. We compare these expectations with the sensitivity and the resolution reachable with the SKA1. This allows us to evaluate the potential for studying intracluster magnetic fields in the surveys planned for SKA1.

We compare two different codes for simulations of cosmological structure formation to investigate the sensitivity of hydrodynamical instabilities to numerics, in particular, the hydro solver and the application of adaptive mesh refinement (AMR). As a simple test problem, we consider an initially spherical gas cloud in a wind, which is an idealized model for the merger of a subcluster or galaxy with a big cluster. Based on an entropy criterion, we calculate the mass stripping from the subcluster as a function of time. Moreover, the turbulent velocity field is analyzed with a multi-scale filtering technique. We find remarkable differences between the commonly used PPM solver with directional splitting in the Enzo code and an unsplit variant of PPM in the Nyx code, which demonstrates that different codes can converge to systematically different solutions even when using uniform grids. For the test case of an unbound cloud, AMR simulations reproduce uniform-grid results for the mass stripping quite well, although the flow realizations can differ substantially. If the cloud is bound by a static gravitational potential, however, we find strong sensitivity to spurious fluctuations which are induced at the cutoff radius of the potential and amplified by the bow shock. This gives rise to substantial deviations between uniform-grid and AMR runs performed with Enzo, while the mass stripping in Nyx simulations of the subcluster is nearly independent of numerical resolution and AMR. Although many factors related to numerics are involved, our study indicates that unsplit solvers with advanced flux limiters help to reduce grid effects and to keep numerical noise under control, which is important for hydrodynamical instabilities and turbulent flows.

The injection of turbulence in the circum-galactic medium at redshift z = 2 is investigated using the mesh-based hydrodynamic code Enzo and a subgrid-scale (SGS) model for unresolved turbulence. Radiative cooling and heating by a uniform Ultraviolet (UV) background are included in our runs and compared with the effect of turbulence modelling. Mechanisms of gas exchange between galaxies and the surrounding medium, as well as metal enrichment, are not taken into account, and turbulence is here driven solely by structure formation (mergers and shocks). We find that turbulence, both at resolved and SGS scales, impacts mostly the warm-hot intergalactic medium (WHIM), with temperature between 10^5 and 10^7 K, mainly located around collapsed and shock heated structures, and in filaments. Typical values of the ratio of turbulent to thermal pressure is 0.1 in the WHIM, corresponding to a volume-weighted average of the SGS turbulent to thermal Doppler broadening b_t / b_therm = 0.26, on length scales below the grid resolution of 25 kpc/h. In the diffuse intergalactic medium (IGM), defined in a range of baryon overdensity \delta\ between 1 and 50, the importance of turbulence is smaller, but grows as a function of gas density, and the Doppler broadening ratio is fitted by the function b_t / b_therm = 0.023 \delta^{0.58}.

The Square Kilometre Array (SKA) is the most ambitious radio telescope ever planned. With a collecting area of about a square kilometre, the SKA will be far superior in sensitivity and observing speed to all current radio facilities. The scientific capability promised by the SKA and its technological challenges provide an ideal base for interdisciplinary research, technology transfer, and collaboration between universities, research centres and industry. The SKA in the radio regime and the European Extreme Large Telescope (E-ELT) in the optical band are on the roadmap of the European Strategy Forum for Research Infrastructures (ESFRI) and have been recognised as the essential facilities for European research in astronomy. This "White Paper" outlines the German science and R&D interests in the SKA project and will provide the basis for future funding applications to secure German involvement in the Square Kilometre Array.

Merger shocks induce turbulence in the intra-cluster medium (ICM), and, under some circumstances, accelerate electrons to relativistic velocities to form so-called radio relics. Relics are mostly found at the periphery of galaxy clusters and appear to have magnetic fields at the microGauss level. Here we investigate the possible origins of these magnetic fields. Turbulence produced by the shock itself cannot explain the magnitude of these fields. However, we argue that if the turbulent pressure support in the ICM upstream of the merger shock is of the order of 10 to 30 percent of the total pressure on scales of a few times 100 kpc, then vorticity generated by compressive and baroclinic effects across the shock discontinuity can lead to a sufficient amplification of the magnetic field. Compressional amplification can explain the large polarisation of the radio emission more easily than dynamo turbulent amplification. Finally, clumping of the ICM is shown to have a negligible effect on magnetic field amplification.

FEARLESS (Fluid mEchanics with Adaptively Refined Large Eddy SimulationS) is a numerical scheme for modelling subgrid-scale turbulence in cosmological adaptive mesh refinement simulations. In this contribution, the main features of this tool will be outlined. We discuss the application of this method to cosmological simulations of the large-scale structure. The simulations show that the production of turbulence has a different redshift dependence in the intra-cluster medium and the warm-hot intergalactic medium, caused by the distinct stirring mechanisms (mergers and shock interactions) acting in them. Some properties of the non-thermal pressure support in the two baryon phases are also described.

Massive structures like cluster of galaxies, embedded in cosmic filaments, release enormous amount of energy through their interactions. These events are associated with production of Mpc-scale shocks and injection of considerable amount of turbulence, affecting the non-thermal energy budget of the ICM. In order to study this thoroughly, we performed a set of cosmological simulations using the hydrodynamical code Enzo. We studied the formation of clusters undergoing major mergers, the propagation of merger shocks and their interaction with the filamentary cosmic web. This interaction is shown to produce peripheral structures remarkably similar to giant radio relics observed, for example, in Abell 3376 and Abell 3667. We find a relatively long timescale (about 4 Gyr) for turbulence decay in the centre of major merging clusters. This timescale is substantially longer than typically assumed in the turbulent re-acceleration models, invoked for explaining the statistics of observed radio halos.

The injection and evolution of turbulence in the intergalactic medium is studied by means of mesh-based hydrodynamical simulations, including a subgrid scale (SGS) model for small-scale unresolved turbulence. The simulations show that the production of turbulence has a different redshift dependence in the intracluster medium (ICM) and the warm-hot intergalactic medium (WHIM). We show that turbulence in the ICM is produced chiefly by merger-induced shear flows, whereas the production in the WHIM is dominated by shock interactions. Secondly, the effect of dynamical pressure support on the gravitational contraction has been studied. This turbulent support is stronger in the WHIM gas at baryon overdensities 1 < delta < 100, and less relevant for the ICM. Although the relative mass fraction of the gas with large vorticity is considerable (52% in the ICM), we find that for only about 10% in mass this is dynamically relevant, namely not associated to an equally large thermal pressure support. According to this result, a significant non-thermal pressure support counteracting the gravitational contraction is a localised characteristic in the cosmic flow, rather than a widespread feature.

Halo mergers and shock waves play a crucial role in the process of hierarchical clustering. Hydrodynamical simulations are the principal investigation tool in this field for theoreticians, and predict that a by-product of cluster formation and virialisation is the injection of turbulence in the cosmic flow. Here I will summarise results from a series of recent works focused on the main stirring mechanisms acting on baryons: minor and major cluster mergers, and curved shocks. Unresolved turbulence has been treated with the implementation of a subgrid scale model. Recent simulations show that the production of turbulence differs significantly for the warm-hot intergalactic medium (WHIM) and the intra-cluster medium (ICM), because of different stirring mechanisms acting in the two gas phases.

We performed a set of cosmological simulations of major mergers in galaxy clusters to study the evolution of merger shocks and the subsequent injection of turbulence in the post-shock region and in the intra-cluster medium (ICM). The computations were done with the grid-based, adaptive mesh refinement hydro code Enzo, using an especially designed refinement criteria for refining turbulent flows in the vicinity of shocks. A substantial amount of turbulence energy is injected in the ICM due to major merger. Our simulations show that the shock launched after a major merger develops an ellipsoidal shape and gets broken by the interaction with the filamentary cosmic web around the merging cluster. The size of the post-shock region along the direction of shock propagation is about 300 kpc h^-1, and the turbulent velocity dispersion in this region is larger than 100 km s^-1. Scaling analysis of the turbulence energy with the cluster mass within our cluster sample is consistent with M^(5/3), i.e. the scaling law for the thermal energy in the self-similar cluster model. This clearly indicates the close relation between virialization and injection of turbulence in the cluster evolution. We found that the ratio of the turbulent to total pressure in the cluster core within 2 Gyr after the major merger is larger than 10%, and it takes about 4 Gyr to get relaxed, which is substantially longer than typically assumed in the turbulent re-acceleration models, invoked to explain the statistics of observed radio halos. Striking similarities in the morphology and other physical parameters between our simulations and the "symmetrical radio relics" found at the periphery of the merging cluster A3376 are finally discussed. In particular, the interaction between the merger shock and the filaments surrounding the cluster could explain the presence of "notch-like" features at the edges of the double relics.

It is widely accepted that the onset of the explosive carbon burning in the core of a CO WD triggers the ignition of a SN Ia. The features of the ignition are among the few free parameters of the SN Ia explosion theory. We explore the role for the ignition process of two different issues: firstly, the ignition is studied in WD models coming from different accretion histories. Secondly, we estimate how a different reaction rate for C-burning can affect the ignition. Two-dimensional hydrodynamical simulations of temperature perturbations in the WD core ("bubbles") are performed with the FLASH code. In order to evaluate the impact of the C-burning reaction rate on the WD model, the evolution code FLASH_THE_TORTOISE from Lesaffre et al. (2006) is used. In different WD models a key role is played by the different gravitational acceleration in the progenitor's core. As a consequence, the ignition is disfavored at a large distance from the WD center in models with a larger central density, resulting from the evolution of initially more massive progenitors. Changes in the C reaction rate at T < 5e8 K slightly influence the ignition density in the WD core, while the ignition temperature is almost unaffected. Recent measurements of new resonances in the C-burning reaction rate (Spillane et al. 2007) do not affect the core conditions of the WD significantly. This simple analysis, performed on the features of the temperature perturbations in the WD core, should be extended in the framework of the state-of-the-art numerical tools for studying the turbulent convection and ignition in the WD core. Future measurements of the C-burning reactions cross section at low energy, though certainly useful, are not expected to affect dramatically our current understanding of the ignition process.

FEARLESS (Fluid mEchanics with Adaptively Refined Large Eddy SimulationS) is a new numerical scheme arising from the combined use of subgrid scale (SGS) model for turbulence at the unresolved length scales and adaptive mesh refinement (AMR) for resolving the large scales. This tool is especially suitable for the study of turbulent flows in strongly clumped media. In this contribution, the main features of FEARLESS are briefly outlined. We then summarize the main results of FEARLESS cosmological simulations of galaxy cluster evolution. In clusters, the production of turbulence is closely correlated with merger events; for minor mergers, we find that turbulent dissipation affects the cluster energy budget only locally. The level of entropy in the cluster core is enhanced in FEARLESS simulations, in accord with a better modeling of the unresolved flow, and with its feedback on the resolved mixing in the ICM.

We present a numerical scheme for modelling unresolved turbulence in cosmological adaptive mesh refinement codes. As a first application, we study the evolution of turbulence in the intra-cluster medium and in the core of a galaxy cluster. Simulations with and without subgrid scale model are compared in detail. Since the flow in the ICM is subsonic, the global turbulent energy contribution at the unresolved length scales is smaller than 1% of the internal energy. We find that the production of turbulence is closely correlated with merger events occurring in the cluster environment, and its dissipation locally affects the cluster energy budget. Because of this additional source of dissipation, the core temperature is larger and the density is smaller in the presence of subgrid scale turbulence than in the standard adiabatic run, resulting in a higher entropy core value.

The development of turbulent gas flows in the intra-cluster medium and in the core of a galaxy cluster is studied by means of adaptive mesh refinement (AMR) cosmological simulations. A series of six runs was performed, employing identical simulation parameters but different criteria for triggering the mesh refinement. In particular, two different AMR strategies were followed, based on the regional variability of control variables of the flow and on the overdensity of subclumps, respectively. We show that both approaches, albeit with different results, are useful to get an improved resolution of the turbulent flow in the ICM. The vorticity is used as a diagnostic for turbulence, showing that the turbulent flow is not highly volume-filling but has a large area-covering factor, in agreement with previous theoretical expectations. The measured turbulent velocity in the cluster core is larger than 200 km/s, and the level of turbulent pressure contribution to the cluster hydrostatic equilibrium is increased by using the improved AMR criteria.

The problem of the resolution of turbulent flows in adaptive mesh refinement (AMR) simulations is investigated by means of 3D hydrodynamical simulations in an idealised setup, representing a moving subcluster during a merger event. AMR simulations performed with the usual refinement criteria based on local gradients of selected variables do not properly resolve the production of turbulence downstream of the cluster. Therefore we apply novel AMR criteria which are optimised to follow the evolution of a turbulent flow. We demonstrate that these criteria provide a better resolution of the flow past the subcluster, allowing us to follow the onset of the shear instability, the evolution of the turbulent wake and the subsequent back-reaction on the subcluster core morphology. We discuss some implications for the modelling of cluster cold fronts.

The onset of the thermonuclear runaway in a Chandrasekhar-mass white dwarf, leading to the explosion as a type Ia supernova, is studied with hydrodynamical simulations. We investigate the evolution of temperature fluctuations (``bubbles'') in the WD's convective core by means of 2D numerical simulations. We show how the occurrence of the thermonuclear runaway depends on various bubble parameters. The relevance of the progenitor's composition for the ignition process is also discussed.

In the framework of the Chandrasekhar-mass deflagration model for Type Ia supernovae (SNe Ia), a persisting free parameter is the initial morphology of the flame front, which is linked to the ignition process in the progenitor white dwarf. Previous analytical models indicate that the thermal runaway is driven by temperature perturbations (''bubbles'') that develop in the white dwarf's convective core. In order to probe the conditions at ignition (diameters, temperatures and evolutionary timescales), we have performed hydrodynamical 2D simulations of buoyant bubbles in white dwarf interiors. Our results show that fragmentation occurring during the bubble rise affects the outcome of the bubble evolution. Possible implications for the ignition process of SNe Ia are discussed.