While single-cell technologies have allowed scientists to characterize cell states that emerge during cancer progression through temporal sampling, connecting these samples over time and inferring gene-gene relationships that promote cancer plasticity remains a challenge. To address these challenges, we developed TrajectoryNet, a neural ordinary differential equation network that learns continuous dynamics via interpolation of population flows between sampled time points. By running causality analysis on the output of TrajectoryNet, we compute rich and complex gene-gene networks that drive pathogenic trajectories forward. We applied this pipeline to scRNAseq data generated from in vitro and in-vivo models of breast cancer to identify the temporal transcriptional programs driving cellular plasticity. The temporal in vitro data comprised of scRNA time course of tumourspheres to model CSC differentiation and non-CSC dedifferentiation. We further applied TrajectoryNet to an in vivo xenograft model and demonstrated its ability to elucidate trajectories governing primary tumor metastasis to the lung. Applying the TrajectoryNet pipeline to scRNAseq data generated from in vitro models of breast cancer, we identified a refined CD44hiEPCAM+CAV1 + marker profile that improves the identification and isolation of cancer stem cells (CSCs) from bulk cell populations. Studying the cell plasticity trajectories emerging from this population, we identified comprehensive temporal regulatory networks that drive cell fate decisions between an epithelial-to-mesenchymal (EMT) trajectory, and a mesenchymal-to-epithelial (MET) trajectory. Through these studies, we identified and validated the estrogen-related receptor alpha (ESRRA) as a critical mediator of CSC plasticity. Upon applying TrajectoryNet to our in vivo dataset, we identified a dominant EMT trajectory that includes elements of our newly defined temporal EMT regulatory network from the primary tumor to lung metastasis. Thus, demonstrated here in cancer, the TrajectoryNet pipeline is a transformative approach to uncovering temporal molecular programs operating in dynamic cell systems from static single-cell data.