We develop techniques of analyzing the unitarity of general Born-Infeld (BI) gravity actions in D-dimensional spacetimes. Determinantal form of the action allows us to find a compact expression quadratic in the metric fluctuations around constant curvature backgrounds. This is highly nontrivial since for the BI actions, in principle, infinitely many terms in the curvature expansion should contribute to the quadratic action in the metric fluctuations around constant curvature backgrounds, which would render the unitarity analysis intractable. Moreover in even dimensions, unitarity of the theory depends only on finite number of terms built from the powers of the curvature tensor. We apply our techniques to some four-dimensional examples.