China is one of the countries in the world with the highest rate of esophageal cancer. Early detection, accurate diagnosis, and treatment of esophageal cancer are critical for improving patients’ prognosis and survival. Machine learning technology has become widely used in cancer, which is benefited from the accumulation of medical images and advancement of artificial intelligence technology. Therefore, the learning model, image type, data type and application efficiency of current machine learning technology in esophageal cancer are summarized in this review. The major challenges are identified, and solutions are proposed in medical image machine learning for esophageal cancer. Machine learning's potential future directions in esophageal cancer diagnosis and treatment are discussed, with a focus on the possibility of establishing a link between medical images and molecular mechanisms. The general rules of machine learning application in the medical field are summarized and forecasted on this foundation. By drawing on the advanced achievements of machine learning in other cancers and focusing on interdisciplinary cooperation, esophageal cancer research will be effectively promoted.