The paper reports on all-solution-processed, all-oxide solar cells, based on an electrodeposited Cu2O absorber. The transparent indium-doped zinc oxide (IZO) contact and buffer layers of zinc oxide or zinc magnesium oxide (Zn1−xMg x O) were fabricated by ultrasonic spray pyrolysis. The cells were completed with graphite paste top contacts. The focus was set on using exclusively environment-friendly and low-cost raw materials, deposited from aqueous solutions without organic solvents. The latter is especially important for spray pyrolysis, where high process temperatures restrict the use of flammable solvents. The developed spray pyrolysis recipes yielded conductive (25 Ω/sq.) and transparent IZO and various compositions of transparent Zn1−xMg x O layers, with a linear dependence of the energy band gap (3.28–3.50 eV) as a function of the Mg content (0–16 mol %), as seen for layers deposited by vacuum-based techniques. Solar cells with a Zn0.88Mg0.12O buffer showed an improved photovoltaic performance compared to cells with ZnO buffer or without buffer, reaching a power conversion efficiency of 0.67% with a short-circuit current density of 3.76 mA/cm2, an open-circuit voltage of 0.34 V and a fill factor of 52.7%. The study correlated the improved cell performance with structural and electronic properties of the heterojunction.