Mapping class groups, Out(F_n) and GL_n(Z) are three important groups in geometric group theory that arise as outer automorphism groups of other groups (respectively, of surface, free and free abelian groups). In general, it can be rather complicated to describe the structure of the group Out(G) for more general groups G, particularly if only certain properties of G are known. I will outline some of the work of Rips and Sela in the 90s, which completely describes Out(G) for any Gromov-hyperbolic group G, based on a canonical JSJ decomposition of G. Then I will talk about a new canonical JSJ decomposition for the compact special groups of Haglund and Wise, and how this can be used to gain information on Out(G) for any virtually special group G.