We refine the Lyapunov–Schmidt analysis from our recent paper (Eichmair and Koerber in Large area-constrained Willmore surfaces in asymptotically Schwarzschild 3-manifolds. arXiv preprint arXiv:2101.12665, 2021) to study the geometric center of mass of the asymptotic foliation by area-constrained Willmore surfaces of initial data for the Einstein field equations. If the scalar curvature of the initial data vanishes at infinity, we show that this geometric center of mass agrees with the Hamiltonian center of mass. By contrast, we show that the positioning of large area-constrained Willmore surfaces is sensitive to the distribution of the energy density. In particular, the geometric center of mass may differ from the Hamiltonian center of mass if the scalar curvature does not satisfy additional asymptotic symmetry assumptions.