Academic and political debates over long-run climate policy often invoke "stabilization" of atmospheric concentrations of greenhouse gases (GHGs), but only rarely are non-CO2 greenhouse gases addressed explicitly. Even though the majority of short-term climate policies propose trading between gases on a global warming potential (GWP) basis, discussions of whether CO2 concentrations should be 450, 550, 650, or perhaps as much as 750 ppm leave unstated whether there should be no additional forcing from other GHGs beyond current levels or whether separate concentration targets should be established for each GHG. Here, we use an integrated modeling framework to examine multi-gas stabilization in terms of temperature, economic costs, carbon uptake, and other important consequences. We show that there are significant differences in both costs and climate impacts between different "GWP equivalent" policies and demonstrate the importance of non-CO2 GHG reduction on timescales of up to several centuries.