• Medical Faculty, University of Cologne, Cologne, Germany; Insitute of Pathology, University Hospital Cologne, Cologne, Germany.
• Insitute of Pathology, University Hospital Cologne, Cologne, Germany.
• Institute of Pathology, University Hospital Halle, Martin Luther University Halle-Wittenberg, Halle (Salle), Germany.
• Clinic of Internal Medicine, Oncology, University Hospital Cologne, Cologne, Germany.
• Insitute of Pathology, University Hospital Cologne, Cologne, Germany. Electronic address: yuri.tolkach@gmail.com.

The management of colorectal carcinoma (CRC) relies on pathological interpretation. Digital pathology approaches allow for development of new potent artificial intelligence-based prognostic parameters. The study aimed to develop an artificial intelligence-based image analysis platform allowing fully automatized, quantitative, and explainable tumor microenvironment analysis and extraction of prognostic information from hematoxylin and eosin-stained whole-slide images of CRC patients. Three well--characterized, multi-institutional patient cohorts were included (patient n = 1438, whole-slide image n > 2400). The developed image analysis platform implements quality control and established algorithms to segment tissue and detect cell types. It enabled systematic analysis of immune infiltrate, assessing its prognostic relevance, intratumoral heterogeneity, and biological concepts across multiple survival end points. Analyzing single-cell types and their combinations reveals independent, prognostic parameters, highlighting significant intratumoral heterogeneity, especially in the biopsy setting, which must be accounted for. A key morphologic concept related to tumor control by the immune system is described, resulting in a capable, independent prognostic parameter (tumor "out of control"). Our findings have direct clinical implications and can be used as a foundation for updating the existing CRC grading systems.