Hand-held implementations of recently introduced real-time volumetric tomography approaches represent a promising path toward clinical translation of the optoacoustic technology. To this end, rapid acquisition of optoacoustic image data with spherical matrix arrays has attained exquisite visualizations of three-dimensional vascular morphology and function deep in human tissues. Nevertheless, significant reconstruction inaccuracies may arise from speed of sound (SoS) mismatches between the imaged tissue and the coupling medium used to propagate the generated optoacoustic responses toward the ultrasound sensing elements. Herein, we analyze the effects of SoS variations in three-dimensional hand-held tomographic acquisition geometries. An efficient graphics processing unit (GPU)-based reconstruction framework is further proposed to mitigate the SoS-related image quality degradation without compromising the high-frame-rate volumetric imaging performance of the method, essential for real-time visualization during hand-held scans.