In this paper we study possible signatures of binary planets or exomoons on the Rossiter-McLaughlin (R-M) effect. Our analyses show that the R-M effect for a binary planet or exomoon during its complete transit phase can be divided into two parts. The first is the conventional one similar to the R-M effect from the transit of a single planet, of which the mass and the projected area are the combinations of the binary components; and the second is caused by the orbital rotation of the binary components, which may add a sine- or linear-mode deviation to the stellar radial velocity curve. We find that the latter effect can be up to several or several ten m/s. By doing numerical simulations as well as analytical analyses, we illustrate that the distribution and dispersion of the latter effects obtained from multiple transit events can be used to constrain the dynamical configuration of the binary planet, such as, how the inner orbit of the binary planet is inclined to its orbit rotating around the central star. We find that the signatures caused by the orbital rotation of the binary components are more likely to be revealed if the two components of binary planet have different masses and mass densities, especially if the heavy one has a high mass density and the light one has a low density. Similar signature on the R-M effect may also be revealed in a hierarchical triple star system containing a dark compact binary and a tertiary star.