Recent progress in integrated-optics technology has made photonics a promising platform for quantum networks and quantum computation protocols. Integrated optical circuits are characterized by small device footprints and unrivalled intrinsic interferometric stability. Here, we take advantage of femtosecond-laser-written waveguides’ ability to process polarization-encoded qubits and present an implementation of a heralded controlled-NOT gate on chip. We evaluate the gate performance in the computational basis and a superposition basis, showing that the gate can create polarization entanglement between two photons. Transmission through the integrated device is optimized using thermally expanded core fibers and adiabatically reduced mode-field diameters at the waveguide facets. This demonstration underlines the feasibility of integrated quantum gates for all-optical quantum networks and quantum repeaters.