Scientists aim to extract simplicity from observations of the complex world. An important component of this process is the exploration of data in search of trends. In practice, however, this tends to be more of an art than a science. Among all trends existing in the natural world, one-dimensional trends, often called sequences, are of particular interest as they provide insights into simple phenomena. However, some are challenging to detect as they may be expressed in complex manners. We present the Sequencer, an algorithm designed to generically identify the main trend in a dataset. It does so by constructing graphs describing the similarities between pairs of observations, computed with a set of metrics and scales. Using the fact that continuous trends lead to more elongated graphs, the algorithm can identify which aspects of the data are relevant in establishing a global sequence. Such an approach can be used beyond the proposed algorithm and can optimize the parameters of any dimensionality reduction technique. We demonstrate the power of the Sequencer using real-world data from astronomy, geology as well as images from the natural world. We show that, in a number of cases, it outperforms the popular t-SNE and UMAP dimensionality reduction techniques. This approach to exploratory data analysis, which does not rely on training nor tuning of any parameter, has the potential to enable discoveries in a wide range of scientific domains. The source code is available on github and we provide an online interface at \url{http://sequencer.org}.

Post starburst E+A galaxies are systems that hosted a powerful starburst that was quenched abruptly. Simulations suggest that these systems provide the missing link between major merger ULIRGs and red and dead ellipticals, where AGN feedback is responsible for the expulsion or destruction of the molecular gas. However, many details remain unresolved and little is known about AGN-driven winds in this short-lived phase. We present spatially-resolved IFU spectroscopy with MUSE/VLT of SDSS J124754.95-033738.6, a post starburst E+A galaxy with a recent starburst that started 70 Myrs ago and ended 30 Myrs ago, with a peak SFR of $\sim 150\,\mathrm{M_{\odot}/yr}$. We detect disturbed gas throughout the entire field of view, suggesting triggering by a minor merger. We detect fast-moving multi-phased gas clouds, embedded in a double-cone face-on outflow, which are traced by ionized emission lines and neutral NaID \emph{emission} and absorption lines. We find remarkable similarities between the kinematics, spatial extents, and line luminosities of the ionized and neutral gas phases, and propose a model in which they are part of the same outflowing clouds, which are exposed to both stellar and AGN radiation. Our photoionization model provides consistent ionized line ratios, NaID absorption optical depths and EWs, and dust reddening. Using the model, we estimate, for the first time, the neutral-to-ionized gas mass ratio (about 20), the sodium neutral fraction, and the size of the outflowing clouds. This is one of the best ever observed direct connections between the neutral and ionized outflow phases in AGN.

We report on the determination of electron densities, and their impact on the outflow masses and rates, measured in the central few hundred parsecs of 11 local luminous active galaxies. We show that the peak of the integrated line emission in the AGN is significantly offset from the systemic velocity as traced by the stellar absorption features, indicating that the profiles are dominated by outflow. In contrast, matched inactive galaxies are characterised by a systemic peak and weaker outflow wing. We present three independent estimates of the electron density in these AGN, discussing the merits of the different methods. The electron density derived from the [SII] doublet is significantly lower than than that found with a method developed in the last decade using auroral and transauroral lines, as well as a recently introduced method based on the ionization parameter. The reason is that, for gas photoionized by an AGN, much of the [SII] emission arises in an extended partially ionized zone where the implicit assumption that the electron density traces the hydrogen density is invalid. We propose ways to deal with this situation and we derive the associated outflow rates for ionized gas, which are in the range 0.001--0.5 M$_{\odot}$yr$^{-1}$ for our AGN sample. We compare these outflow rates to the relation between $\dot{M}_{out}$ and $L_{AGN}$ in the literature, and argue that it may need to be modified and rescaled towards lower mass outflow rates.

We present a statistical analysis of the optical properties of an X-ray selected Type 1 AGN sample, using high signal to-noise ratio ($S/N>20$) spectra of the counterparts of the ROSAT/2RXS sources in the footprint of the SDSS-IV/SPIDERS (Spectroscopic IDentification of eROSITA Sources) programme. The final sample contains 2100 sources. It significantly extends the redshift and luminosity ranges ($\rm z \sim 0.01 - 0.80$ and $\rm L_{0.1-2.4 \,keV}$ ~ $\rm 2.0 \times 10^{41}$ - $\rm 1.0 \times 10^{46} \, erg s^{-1}$ ) used so far in this kind of analysis. By means of a Principal Component Analysis, we derive Eigenvector (EV) 1 and 2 in an eleven dimensional optical and X-ray parameter space, which are consistent with previous results. The validity of the correlations of the Eddington ratio $\rm L/L_{Edd}$ with EV1 and the black hole mass with EV2 are strongly confirmed These results imply that $\rm L/L_{Edd}$ and black hole mass are related to the diversity of the optical properties of Type 1 AGN. Investigating the relation of the width and asymmetry of $\rm H\beta$ and the relative strength of the iron emission $\rm r_{FeII}$ , we show that our analysis supports the presence of a distinct kinematic region: the Very Broad Line Region. Furthermore, comparing sources with a red-asymmetric broad $\rm H\beta$ emission line to sources for which it is blue-asymmetric, we find an intriguing difference in the correlation of the FeII and the continuum emission strengths. We show that this contrasting behaviour is consistent with a flattened, stratified model of the Broad Line Region, in which the FeII emitting region is shielded from the central source.

We report on our combined analysis of HST, VLT/MUSE, VLT/SINFONI, and ALMA observations of the local Seyfert 2 galaxy, NGC 5728 to investigate in detail the feeding and feedback of the AGN. The datasets simultaneously probe the morphology, excitation, and kinematics of the stars, ionized gas, and molecular gas over a large range of spatial scales (10 pc--10 kpc). NGC 5728 contains a large stellar bar which is driving gas along prominent dust lanes to the inner 1 kpc where the gas settles into a circumnuclear ring. The ring is strongly star forming and contains a substantial population of young stars as indicated by the lowered stellar velocity dispersion and gas excitation consistent with HII regions. We model the kinematics of the ring using the velocity field of the CO (2--1) emission and stars and find it is consistent with a rotating disk. The outer regions of the disk, where the dust lanes meet the ring, show signatures of inflow at a rate of 1 M$_{\sun}$ yr$^{-1}$. Inside the ring, we observe three molecular gas components corresponding to the circular rotation of the outer ring, a warped disk, and the nuclear stellar bar. The AGN is driving an ionized gas outflow that reaches a radius of 250 pc with a mass outflow rate of 0.08 M$_{\sun}$ yr$^{-1}$ consistent with its luminosity and scaling relations from previous studies. While we observe distinct holes in CO emission which could be signs of molecular gas removal, we find that largely the AGN is not disrupting the structure of the circumnuclear region.

Astronomy is experiencing a rapid growth in data size and complexity. This change fosters the development of data-driven science as a useful companion to the common model-driven data analysis paradigm, where astronomers develop automatic tools to mine datasets and extract novel information from them. In recent years, machine learning algorithms have become increasingly popular among astronomers, and are now used for a wide variety of tasks. In light of these developments, and the promise and challenges associated with them, the IAC Winter School 2018 focused on big data in Astronomy, with a particular emphasis on machine learning and deep learning techniques. This document summarizes the topics of supervised and unsupervised learning algorithms presented during the school, and provides practical information on the application of such tools to astronomical datasets. In this document I cover basic topics in supervised machine learning, including selection and preprocessing of the input dataset, evaluation methods, and three popular supervised learning algorithms, Support Vector Machines, Random Forests, and shallow Artificial Neural Networks. My main focus is on unsupervised machine learning algorithms, that are used to perform cluster analysis, dimensionality reduction, visualization, and outlier detection. Unsupervised learning algorithms are of particular importance to scientific research, since they can be used to extract new knowledge from existing datasets, and can facilitate new discoveries.

The global influence of AGN-driven outflows remains uncertain, due to a lack of large samples with accurately-determined outflow properties. In the second paper of this series, we determine the mass and energetics of ionized outflows is 234 type II AGN, the largest such sample to date, by combining the infrared emission of the dust in the wind (paper I) with the emission line properties. We provide new general expressions for the properties of the outflowing gas, which depend on the ionization state of the gas. We also present a novel method to estimate the electron density in the outflow, based on optical line ratios and on the known location of the wind. The inferred electron densities, $n_{\mathrm{e}} \sim 10^{4.5}\,\mathrm{(cm^{-3})}$, are two orders of magnitude larger than typically found in most other cases of ionized outflows. We argue that the discrepancy is due to the fact that the commonly-used [SII]-based method underestimates the true density by a large factor. As a result, the inferred mass outflow rates and kinetic coupling efficiencies are $\dot{M}_{\mathrm{out}} \sim 10^{-2}\, \mathrm{(M_{\odot}/yr)}$ and $\epsilon = \dot{E}_{\mathrm{kin}}/L_{\mathrm{bol}} \sim 10^{-5}$ respectively, 1--2 orders of magnitude lower than previous estimates. Our analysis suggests the existence of a significant amount of neutral atomic gas at the back of the outflowing ionized gas clouds, with mass that is a factor of a few larger than the observed ionized gas mass. This has significant implications for the estimated mass and energetics of such flows.

The scaling relations between supermassive black holes and their host galaxy properties are of fundamental importance in the context black hole-host galaxy co-evolution throughout cosmic time. In this work, we use a novel algorithm that identifies smooth trends in complex datasets and apply it to a sample of 2,000 type I active galactic nuclei (AGN) spectra. We detect a sequence in emission line shapes and strengths which reveals a correlation between the narrow L([OIII])/L(H$\beta$) line ratio and the width of the broad H$\alpha$. This scaling relation ties the kinematics of the gas clouds in the broad line region to the ionisation state of the narrow line region, connecting the properties of gas clouds kiloparsecs away from the black hole to material gravitationally bound to it on sub-parsec scales. This relation can be used to estimate black hole masses from narrow emission lines only. It therefore enables black hole mass estimation for obscured type II AGN and allows us to explore the connection between black holes and host galaxy properties for thousands of objects, well beyond the local Universe. Using this technique, we present the $M_{\mathrm{BH}} - \sigma$ and $M_{\mathrm{BH}} - M_{*}$ scaling relations for a sample of about 10,000 type II AGN from SDSS. These relations are remarkably consistent with those observed for type I AGN, suggesting that this new method may perform as reliably as the classical estimate used in non-obscured type I AGN. These findings open a new window for studies of black hole-host galaxy co-evolution throughout cosmic time.

Machine learning (ML) algorithms become increasingly important in the analysis of astronomical data. However, since most ML algorithms are not designed to take data uncertainties into account, ML based studies are mostly restricted to data with high signal-to-noise ratio. Astronomical datasets of such high-quality are uncommon. In this work we modify the long-established Random Forest (RF) algorithm to take into account uncertainties in the measurements (i.e., features) as well as in the assigned classes (i.e., labels). To do so, the Probabilistic Random Forest (PRF) algorithm treats the features and labels as probability distribution functions, rather than deterministic quantities. We perform a variety of experiments where we inject different types of noise to a dataset, and compare the accuracy of the PRF to that of RF. The PRF outperforms RF in all cases, with a moderate increase in running time. We find an improvement in classification accuracy of up to 10% in the case of noisy features, and up to 30% in the case of noisy labels. The PRF accuracy decreased by less then 5% for a dataset with as many as 45% misclassified objects, compared to a clean dataset. Apart from improving the prediction accuracy in noisy datasets, the PRF naturally copes with missing values in the data, and outperforms RF when applied to a dataset with different noise characteristics in the training and test sets, suggesting that it can be used for Transfer Learning.

Large scale outflows of different gas phases are ubiquitous in the host galaxies of active galactic nuclei (AGN). Despite their many differences, they share a common property - they all contain dust. The dust is carried with the outflow, heated by the AGN, and emits at infrared wavelengths. This paper shows that the infrared emission of this dust can be used to detect AGN outflows and derive their properties. We use a sample of about 4000 type II AGN and compare the infrared properties of systems that show spectroscopic signature of ionized gas outflows to systems that do not. We detect an additional mid-infrared emission component in galaxies with spectroscopically discovered winds, and attribute it to the dust in the outflow. This new component offers novel constraints on the outflow properties, such as its mean location and covering factor. We measure the location of the outflow for roughly 1700 systems, with the distribution showing a prominent peak around 500 pc, a tail that extends to large distances (10 kpc), and no objects with a location smaller than 50 pc. The covering factor of the wind shows a wide distribution which is centered around 0.1, with 24% (8%) of the winds showing covering factors larger than 0.2 (0.5). The dust emission is not sensitive to various systematics affecting optically-selected outflows, and can be used to estimate the mass outflow rate in thousands of galaxies with only 1D spectra.

In this work we apply and expand on a recently introduced outlier detection algorithm that is based on an unsupervised random forest. We use the algorithm to calculate a similarity measure for stellar spectra from the Apache Point Observatory Galactic Evolution Experiment (APOGEE). We show that the similarity measure traces non-trivial physical properties and contains information about complex structures in the data. We use it for visualization and clustering of the dataset, and discuss its ability to find groups of highly similar objects, including spectroscopic twins. Using the similarity matrix to search the dataset for objects allows us to find objects that are impossible to find using their best fitting model parameters. This includes extreme objects for which the models fail, and rare objects that are outside the scope of the model. We use the similarity measure to detect outliers in the dataset, and find a number of previously unknown Be-type stars, spectroscopic binaries, carbon rich stars, young stars, and a few that we cannot interpret. Our work further demonstrates the potential for scientific discovery when combining machine learning methods with modern survey data.

Post starburst E+A galaxies show indications of a powerful starburst that was quenched abruptly. Their disturbed, bulge-dominated morphologies suggest that they are merger remnants. The more massive E+A galaxies are suggested to be quenched by AGN feedback, yet little is known about AGN-driven winds in this short-lived phase. We present spatially-resolved IFU spectroscopy by the Keck Cosmic Web Imager of SDSS J003443.68+251020.9, at z=0.118. The system consists of two galaxies, the larger of which is a post starburst E+A galaxy hosting an AGN. Our modelling suggests a 400 Myrs starburst, with a peak star formation rate of 120 Msun/yr. The observations reveal stationary and outflowing gas, photoionized by the central AGN. We detect gas outflows to a distance of 17 kpc from the central galaxy, far beyond the region of the stars (about 3 kpc), inside a conic structure with an opening angle of 70 degrees. We construct self-consistent photoionization and dynamical models for the different gas components and show that the gas outside the galaxy forms a continuous flow, with a mass outflow rate of about 24 Msun/yr. The gas mass in the flow, roughly $10^9$ Msun, is larger than the total gas mass within the galaxy, some of which is outflowing too. The continuity of the flow puts a lower limit of 60 Myrs on the duration of the AGN feedback. Such AGN are capable of removing, in a single episode, most of the gas from their host galaxies and expelling enriched material into the surrounding CGM.

We investigate the limitations of statistical absorption measurements with the SDSS optical spectroscopic surveys. We show that changes in the data reduction strategy throughout different data releases have led to a better accuracy at long wavelengths, in particular for sky line subtraction, but a degradation at short wavelengths with the emergence of systematic spectral features with an amplitude of about one percent. We show that these features originate from inaccuracy in the fitting of modeled F-star spectra used for flux calibration. The best-fit models for those stars are found to systematically over-estimate the strength of metal lines and under-estimate that of Lithium. We also identify the existence of artifacts due to masking and interpolation procedures at the wavelengths of the hydrogen Balmer series leading to the existence of artificial Balmer $\alpha$ absorption in all SDSS optical spectra. All these effects occur in the rest-frame of the standard stars and therefore present Galactic longitude variations due to the rotation of the Galaxy. We demonstrate that the detection of certain weak absorption lines reported in the literature are solely due to calibration effects. Finally, we discuss new strategies to mitigate these issues.

In recent years, the autocorrelation of the hydrogen Lyman-{\alpha} forest has been used to observe the baryon acoustic peak at redshift 2 < z < 3.5 using tens of thousands of QSO spectra from the BOSS survey. However, the interstellar medium of the Milky-Way introduces absorption lines into the spectrum of any extragalactic source. These lines, while weak and undetectable in a single BOSS spectrum, could potentially bias the cosmological signal. In order to examine this, we generate absorption line maps by stacking over a million spectra of galaxies and QSOs. We find that the systematics introduced are too small to affect the current accuracy of the baryon acoustic peak, but might be relevant to future surveys such as the Dark Energy Spectroscopic Instrument (DESI). We outline a method to account for this with future datasets.

Post starburst E+A galaxies are thought to have experienced a significant starburst that was quenched abruptly. Their disturbed, bulge-dominated morphologies suggest that they are merger remnants. We present ESI/Keck observations of SDSS J132401.63+454620.6, a post starburst galaxy at redshift z = 0.125, with a starburst that started 400 Myr ago, and other properties, like star formation rate (SFR) consistent with what is measured in ultra luminous infrared galaxies (ULRIGs). The galaxy shows both zero velocity narrow lines, and blueshifted broader Balmer and forbidden emission lines (FWHM=1350 +- 240 km/s). The narrow component is consistent with LINER-like emission, and the broader component with Seyfert-like emission, both photoionized by an active galactic nucleus (AGN) whose properties we measure and model. The velocity dispersion of the broad component exceeds the escape velocity, and we estimate the mass outflow rate to be in the range 4-120 Mo/yr. This is the first reported case of AGN-driven outflows, traced by ionized gas, in post starburst E+A galaxies. We show, by ways of a simple model, that the observed AGN-driven winds can consistently evolve a ULIRG into the observed galaxy. Our findings reinforce the evolutionary scenario where the more massive ULIRGs are quenched by negative AGN feedback, evolve first to post starburst galaxies, and later become typical red and dead ellipticals.

How can we discover objects we did not know existed within the large datasets that now abound in astronomy? We present an outlier detection algorithm that we developed, based on an unsupervised Random Forest. We test the algorithm on more than two million galaxy spectra from the Sloan Digital Sky Survey and examine the 400 galaxies with the highest outlier score. We find objects which have extreme emission line ratios and abnormally strong absorption lines, objects with unusual continua, including extremely reddened galaxies. We find galaxy-galaxy gravitational lenses, double-peaked emission line galaxies, and close galaxy pairs. We find galaxies with high ionisation lines, galaxies which host supernovae, and galaxies with unusual gas kinematics. Only a fraction of the outliers we find were reported by previous studies that used specific and tailored algorithms to find a single class of unusual objects. Our algorithm is general and detects all of these classes, and many more, regardless of what makes them peculiar. It can be executed on imaging, time-series, and other spectroscopic data, operates well with thousands of features, is not sensitive to missing values, and is easily parallelisable.

The typical optical-UV continuum slopes observed in many type 1 AGN are redder than expected from thin accretion disk models. A possible resolution to this conundrum is that many AGN are reddened by dust along the line of sight. To explore this possibility, we stack 5000 SDSS AGN with luminosity L~10^45erg/s and redshift z~0.4 in bins of optical continuum slope alpha_opt and width of the broad H$\beta$ emission line. We measure the EW of the NaID absorption feature in each stacked spectrum. We find a linear relation between alpha_opt and EW(NaID), such that EW(NaID) increases as alpha_opt becomes redder. In the bin with the smallest H$\beta$ width, objects with the bluest slopes that are similar to accretion disk predictions are found to have EW(NaID)=0, supporting the line-of-sight dust hypothesis. This conclusion is also supported by the dependence of the $H\alpha/H\beta$ line ratio on alpha_opt. The implied relationship between alpha_opt and dust reddening is given by E(B-V)~0.2(-0.1-alpha_opt), and the implied reddening of a typical type 1 AGN with alpha_opt=-0.5 is E(B-V)~0.08mag. Photoionization calculations show that the dusty gas responsible for reddening is too ionized to produce the observed features. Therefore, we argue that the sodium absorption arises in regions of the host ISM which are shielded from the AGN radiation, and the correlation with alpha_opt arises since ISM columns along shielded and non-shielded sightlines are correlated. This scenario is supported by the similarity of the E(B-V)-NaID relation implied by our results with the relation in the Milky-Way found by previous studies.

Using over a million and a half extragalactic spectra we study the correlations of the Diffuse Interstellar Bands (DIBs) in the Milky Way. We measure the correlation between DIB strength and dust extinction for 142 DIBs using 24 stacked spectra in the reddening range E(B-V) < 0.2, many more lines than ever studied before. Most of the DIBs do not correlate with dust extinction. However, we find 10 weak and barely studied DIBs with correlations that are higher than 0.7 with dust extinction and confirm the high correlation of additional 5 strong DIBs. Furthermore, we find a pair of DIBs, 5925.9A and 5927.5A which exhibits significant negative correlation with dust extinction, indicating that their carrier may be depleted on dust. We use Machine Learning algorithms to divide the DIBs to spectroscopic families based on 250 stacked spectra. By removing the dust dependency we study how DIBs follow their local environment. We thus obtain 6 groups of weak DIBs, 4 of which are tightly associated with C2 or CN absorption lines.

Using over a million and a half extragalactic spectra we study the properties of the mysterious Diffuse Interstellar Bands (DIBs) in the Milky Way. These data provide us with an unprecedented sampling of the skies at high Galactic-latitude and low dust-column-density. We present our method, study the correlation of the equivalent width of 8 DIBs with dust extinction and with a few atomic species, and the distribution of four DIBs - 5780.6A, 5797.1A, 6204.3A, and 6613.6A - over nearly 15000 squared degrees. As previously found, DIBs strengths correlate with extinction and therefore inevitably with each other. However, we show that DIBs can exist even in dust free areas. Furthermore, we find that the DIBs correlation with dust varies significantly over the sky. DIB under- or over-densities, relative to the expectation from dust, are often spread over hundreds of square degrees. These patches are different for the four DIBs, showing that they are unlikely to originate from the same carrier, as previously suggested.