In N-body simulations the force calculated between particles representing a given mass distribution is usually softened, to diminish the effect of graininess. In this paper we study the effect of such a smoothing, with the aim of finding an optimal value of the softening parameter. As already shown by Merritt (1996), for too small a softening the estimates of the forces will be too noisy, while for too large a softening the force estimates are systematically misrepresented. In between there is an optimal softening, for which the forces in the configuration approach best the true forces. The value of this optimal softening depends both on the mass distribution and on the number of particles used to represent it. For higher number of particles the optimal softening is smaller. More concentrated mass distributions necessitate smaller softening, but the softened forces are never as good an approximation of the true forces as for not centrally concentrated configurations. We give good estimates of the optimal softening for homogeneous spheres, Plummer spheres, and Dehnen spheres. We also give a rough estimate of this quantity for other mass distributions, based on the harmonic mean distance to the $k$th neighbour ($k$ = 1, .., 12), the mean being taken over all particles in the configuration. Comparing homogeneous Ferrers ellipsoids of different shapes we show that the axial ratios do not influence the value of the optimal softening. Finally we compare two different types of softening, a spline softening (Hernquist & Katz 1989) and a generalisation of the standard Plummer softening to higher values of the exponent. We find that the spline softening fares roughly as well as the higher powers of the power-law softening and both give a better representation of the forces than the standard Plummer softening.