• Department of Pathology, Northwestern University Feinberg School of Medicine, Chicago, IL, USA.
• Department of Population Science, American Cancer Society, Atlanta, GA, USA.
• Department of Medicine, John H. Stroger, Jr. Hospital of Cook County, Chicago, IL, USA.
• Department of Pathology, Emory University School of Medicine, Atlanta, GA, USA.
• Division of Cancer Epidemiology and Genetics, National Cancer Institute, Bethesda, MD, USA.
• Department of Pathology, Northwestern University Feinberg School of Medicine, Chicago, IL, USA. lee.cooper@northwestern.edu.

Breast cancer is a heterogeneous disease with variable survival outcomes. Pathologists grade the microscopic appearance of breast tissue using the Nottingham criteria, which are qualitative and do not account for noncancerous elements within the tumor microenvironment. Here we present the Histomic Prognostic Signature (HiPS), a comprehensive, interpretable scoring of the survival risk incurred by breast tumor microenvironment morphology. HiPS uses deep learning to accurately map cellular and tissue structures to measure epithelial, stromal, immune, and spatial interaction features. It was developed using a population-level cohort from the Cancer Prevention Study-II and validated using data from three independent cohorts, including the Prostate, Lung, Colorectal, and Ovarian Cancer trial, Cancer Prevention Study-3, and The Cancer Genome Atlas. HiPS consistently outperformed pathologists in predicting survival outcomes, independent of tumor-node-metastasis stage and pertinent variables. This was largely driven by stromal and immune features. In conclusion, HiPS is a robustly validated biomarker to support pathologists and improve patient prognosis.