Mahmood Lab

Multimodal & Generative AI for Pathology

Mahmood Lab at the Brigham and Women’s Hospital aims to utilize machine learning, data fusion, and medical image analysis to develop streamlined workflows for objective diagnosis, prognosis, and biomarker discovery. We are interested in developing automated and objective mechanisms for reducing interobserver and intraobserver variability in cancer diagnosis using artificial intelligence as an assistive tool for pathologists. The lab also focuses on the development of new algorithms and methods to identify clinically relevant morphologic phenotypes and biomarkers associated with response to specific therapeutic agents. We develop multimodal fusion algorithms for combining information from multiple imaging modalities, familial and patient histories and multi-omics data to make more precise diagnostic, prognostic and therapeutic determinations. We are affiliated with the Harvard Data Science Initiative; the Harvard Bioinformatics and Integrative Genomics (BIG) program; the Cancer Data Science Program at the Dana-Farber Cancer Institute and the Cancer Program at the Broad Institute of Harvard and MIT.

Contact:
✉️faisalmahmood<at>bwh.harvard.edu
🏢8002-S, HBTM, 60 Fenwood Rd., Boston, MA 02115

Administrative Assistants
Trisha Ramsey tramsey<at>bwh.harvard.edu
Tarissa A. Mages tarnoldmages<at>bwh.harvard.edu

Recent Select Media Coverage

Harvard News: How AI Can Help Diagnose Rare Diseases
NCI News: NCI-Supported Study Uses Deep Learning for Cancer Prognosis
WebMD: Can AI Deliver a More Accurate Cancer Prognosis?
Inside precision medicine: From Slide to Pixels
Medscape: AI System Helps Spot Signs of Heart Transplant Rejection
Harvard News: Heart Saving AI
ASCO Post: AI System May Aid in Diagnosing Cancer of Unknown Primary
Harvard News: Predicting Cancer’s Epicenter
Nvidia News: Top Pathology Lab Fuses Data Sources to Develop Cancer-Detecting AI
Forbes: Transforming Federal Healthcare with AI
Bioengineering Today: How to Train Your Radiology AI
Auntminnie.com: Cinematic rendering bolsters AI for depth estimation on endoscopy
JHU News: Johns Hopkins School of Medicine Excellence in Research Award

News

Sept’24: Mahmood Lab received an ARPA-H award through the PSI program. Together with the Liu Lab from UW, UW Medicine and Vanderbilt University Medical Center the project aims to develop methods and 3D imaging solutions to assist surgeons remove tumors more precisely and efficiently. Mahmood Lab will lead the computational arm of the project. Read more about the project here.
July’24: New preprints on Deep Learning-based Modeling for Preclinical Drug Safety Assessment and AI-driven Discovery of Morphomolecular Signatures in Toxicology.
June’24: New ICML article on Multimodal Prototyping for cancer outcome prediction.
June’24: New Nature article on PathChat, multimodal generative AI copilot for pathology.
June’24: Mahmood Lab will be presenting three articles at CVPR 2024 main conference, TANGLE (Oral) , PANTHER , and SurvPath . Also attend our keynote at the Foundation Models for Medical Vision Workshop , and check out PathChat and TriPath demos. See more here.
May’24: Together with Jonathan Liu from UW Mahmood Lab received an R01 from the NIDDK for 3D computational pathology for Barrett’s esophagus.
May’24: New Cell article on TriPath a method for weakly supervised AI for 3D pathology. TriPath enables 3D computational pathology via 3D multiple instance learning allowing AI models to capture intricate morphological details from pathology volumes.
April’24: New Nature Medicine article on fairness in computational pathology . We show that pathology AI models can be biased and self-supervised foundation models can reduce such bias.
April’24: Congratulations to Luca Weishaupt for receiving the NSF graduate research fellowship, joining the long list of NSF fellows in the group.
March’24: New Nature Medicine article on UNI , A general purpose foundation model for computational pathology trained on 100 million pathology images from over 100k WSIs and adapted to over 30 different downstream tasks. Code available for academic use here. See Harvard News Coverage and MGB News Coverage coverage.
March’24: New Nature Medicine article on CONCH , A vision-language foundation model for computational pathology trained on 1.17 million pathology image-text pairs and adapted to 14 different downstream tasks. Code available for academic use here. See Harvard News Coverage and MGB News Coverage coverage.
Feburary’24: Three articles accepted at CVPR 2024. More to follow soon.
January’24: New Nature Communications paper on MAPS ML-driven cell annotation for high-plex spatial proteomics data. Code available here and dataset available here.
December’23: We are excited to announce PathChat a vision-language AI assistant for Pathology that can analyze histology images and answer diverse pathology-related queries. See preprint here and demo here . More to follow soon.
October’23: We are excited to welcome HST PhD student Cristina Almagro-Pérez.
June’23: New Nature Reviews Bioengineering article on AI for digital and computational Pathology.
August’23: We are excited to welcome HST PhD student Luca Weishaupt. Congratulations to our superstar postdocs Jana Lipkova and Drew Williamson , both of them will be starting their own research labs as Assistant Professors. Jana will be joining UC Irvine later this year and Drew will move Emory University early next year. Both will be recruiting postdocs and grad students so get in touch with them!
August’23: New pre-print on UNI , a general purpose self-supervised foundation model for computational pathology.
July’23: New pre-print on weakly supervised AI for efficient analysis of 3D pathology samples. Code available here and demo available here .
July’23: New pre-print on CONCH , a large-scale vision + language foundation model for computational pathology.
June’23: New Nature Biomedical Engineering review article discussing algorithmic fairness in AI for medicine and healthcare.
June’23: Visual language pretrained multiple instance zero-shot transfer for histopathology images accepted at CVPR 2023 see article here, video here, and code here.
March’23: Congratulations to Daniel Shao for receiving the NSF GFRP. All eligible PhD students in our group have received this fellowship!
Feb’23: Visual-language representation learning for zeroshot classification of histopathology images was accepted at CVPR 2023.
Jan’23: We are excited to welcome HST PhD student Andrew Zhang to the team and are celebrating Andrew Schaumberg’s exciting new role to lead computational pathology at VA Health.
Dec’22: New Nature Biomedical Engineering paper on transforming the style of Frozen Section to FFPE tissue for real-time AI-driven intraoperative diagnosis.
Nov’22: After a year of planning and renovations Mahmood Lab has moved into our new lab spaces at Thorn 1032 and MRB 502, see more details here.
Oct’22: Bowen Chen, Drew Williamson, and Iain Carmichael receive Brigham excellence in research awards.
Oct’22: New Cancer Cell article on AI for multimodal data integration in oncology.
Oct’22: New Nature Biomedical Engineering paper on fast and scalable WSI retrieval, see read link, HMS news coverage, code.
Aug’22: Congratulations to our Ph.D. student Max Lu for being named a Siebel Scholar for the class of 2023.
Aug’22: Our pan-cancer integrative histology-genomic analysis is published in Cancer Cell (paper, code, demo, animated abstract, news & views, WebMD News,NCI News). The study is also featured on the cover of Cancer Cell.
July’22: Our perspective on digitizing heart transplant reaction w/ Eric Topol is published in The Lancet, see our related study in Nature Medicine. Our comment on identifying morphologic correlates of genomic heterogeneity is published in Cancer Research. Our N&V on the role of context in computational pathology is published in Nature BME.
July’22: Honored to be named on The Pathologist power list 2022.
June’22: The lab is now officially part of MGH Pathology, we are excited about the new opportunities and potential collaborations.
June’22: Our work on Hierarchical Image Pyramid Transformer (HIPT) was accepted at CVPR 2022 (paper, code, oral talk). And our work on incorporating intratumoral heterogeneity into deep models for pathology via variance pooling is accepted at MICCAI 2022 (paper; more soon).
May’22: Congratulations to Mane Williams for passing his qualifying exam.
April’22: Congratulations to our postdocs Iain Carmichael and Tiffany Chen for accepting their next roles. Iain will be joining the faculty at UC Berkeley statistics this fall, and Tiffany Chen will join Bigene as the Director of Pathology.
Mar’22: We are excited to welcome HST PhD student Anurag Vaidya and postdoctoral fellows Guillaume Jaume and Kutsev Bengisu.
Mar’22: Our deep learning-enabled assessment of cardiac transplant rejection study is published in Nature Medicine, see: journal link; demo; code ; HMS News.
Jan’22: HST PhD Student Daniel Shao; BIG PhD Student Mane Williams; Masters students Elea Bach, Tong Ding and research associate Lottie Zhuang joined the lab – welcome.
Dec’21: Our Federated Learning for Computational Pathology study is published in Medical Image Analysis.
Sep’21: We are excited to welcome postdoctoral fellows Iain Carmichael and Andrew Song to the lab.
Aug’21: Honored to be named on The Pathologist power list 2021.
July’21: Multimodal co-attention transformers (MCAT) accepted at ICCV 2021, (read-link; code) . Patch-GCN: Context-aware survival prediction accepted at MICCAI 2021 (read-link; code).
July’21: New pre-prints: SISH for fast and scalable histology image search (pre-print; code); DL-based Frozen Section to FFPE translation (pre-print; code); PORPOISE: Pan-cancer integrative histology-genomic analysis via DL (pre-print; code).
June’21: Congratulations to our PhD students Richard Chen (HMS/BIG) and Sharifa Sahai (HMS/SysBio) for passing their qualifying exams.
June’21: Our comment on synthetic data for machine learning in medicine is published in Nature Biomedical Engineering and our spotlight on using multiplex cPath for treatment response prediction is published in Cancer Cell.
May’21: Our AI-based cancer origin prediction using conventional histology study is published in Nature check out the demo, code and HMS news story.
April’21: Congratulations to Sharifa Sahai for getting the NSF Fellowship and to Max Lu for getting the Tau Beta Pi Fellowship!
March’21: CLAM is now in print at Nature Biomedical Engineering see our opensource code and demo.
Dec’20: Mahmood lab will be presenting one plenary talk and three posters at the AACR AI conference.
Nov’20: Rotation students Mane Williams and Trevor Manz joined the lab – welcome.
Nov’20: Congratulations to Andrew and Bowen for winning awards at Pathology Visions 2020 ; Andrew and Max Lu received Discover Brigham Awards and Max Lu was also awarded the Brigham Research Excellence Award 2020.
Sept’20: New pre-print: Federated Learning for Whole Slide Computational Pathology.
Sept’20: The lab has been awarded a MIRA/R35 Outstanding Investigator Award by NIH/NIGMS for 2020-2025!
Aug’20: Pathomic Fusion accepted to IEEE Transactions on Medical Imaging.
Aug’20: Andrew Schaumberg and Muhammad Shaban joined the lab as postdocs – welcome.
June’20: New demo: TOAD, New code: TOAD.
June’20: New preprint: Tumor Origin Prediction via Deep Learning.
April’20: New code release: CLAM.
April’20: New preprint: CLAM, check our interactive demo here.
April’20: Maha, Dehan, and John joined the lab – welcome.
April’20: Upcoming invited talk at Virtual Pathology Grand Rounds (Hosted by UCSF).
Feb’20: Upcoming invited talk at Nvidia GTC San Jose 2020.
Feb’20: Jana, Sharifa, Matteo, Bowen, and MyeongSeo joined the lab – welcome.
Feb’20: Mahmood lab presenting three oral talks at SPIE Medical Imaging.
Feb’20: Upcoming nano-course on Deep Learning for Biomedical Image Analysis.
Jan’20: Upcoming invited talk at the FDA on 01/15.
Dec’19: New code release: Pathomic Fusion
Dec’19: New pre-print: Pathomic Fusion (Deep Multimodal fusion for integrating histology and -omic data for diagnosis, prognosis).
Dec’19: Article accepted IEEE Transactions on Medical Imaging.
Nov’19: Mahmood Lab will be presenting four abstracts at USCAP 2020 in Los Angeles.
Nov’19: Upcoming Invited Talk at CCDS Grand Rounds.
Nov’19: Mahmood Lab featured in Nvidia News: Under the Microscope: Top Pathology Lab Fuses Data Sources to Develop Cancer-Detecting AI
Nov’19: Mahmood lab will be presenting three posters at Discover Brigham 2019.
Oct’19: New Pre-print GCNs for Prostate TMA classification (ISBI 2020 submission).
Oct’19: New Pre-print MIL+CPC for semi-supervised histology classification (Accepted for presentation at NeurIPS ML4H 2019).
Oct’19: Dr. Mahmood becomes an Associate Member of the Broad Institute of Harvard and MIT.
Oct’19: Max Lu and Richard Chen awarded the NeurIPS 2019 ML4H travel award – congratulations.
Oct’19: Mahmood Lab presenting three oral talks at SPIE Medical Imaging 2019 – Digital Pathology Conference in Houston.
Oct’19: Two papers accepted at NeurIPS ML4H 2019. Pre-prints will be available soon. Visit us in Vancouver to learn more.
Aug’19: Dr. Mahmood will be giving an invited talk at Nvidia GTC DC 2019.
Aug’19: Mahmood Lab giving one oral and two poster presentations at Pathology Visions 2019 in Orlando.
July’19: New pre-print: GANPOP: GAN-based Prediction of Optical Properties from Single Snapshot Wide-field Images.
July’19: Dr. Mahmood becomes a full member of the cancer data sciences program at the Dana-Farber Cancer Institute / Harvard Cancer Center.
July’19: Jingwen Wang, Research Intern joined the lab – welcome.
July’19: Paper accepted at 2019 KDD Workshop on Applied Data Science for Healthcare.
July’19: Dr. Mahmood becomes a member of the Harvard Bioinformatics and Integrative Genomics (BIG) faculty.
June’19: Article accepted in IEEE Robotics and Automation Letters.
June’19: Max Lu, Research Assistant joined the lab – welcome.
June’19: Mahmood Lab welcomes its first member, Richard Chen, Harvard BIG PhD Student joined the lab.
May’19: Article accepted in IEEE Transactions on Medical Imaging.
April’19: Paper accepted in ICLR 2019 Workshop.
April’19: Mahmood lab opens its doors, check out careers for postdoctoral fellowships.

———————————————————-

Mar’19: Article accepted in Biomedical Optics Express.
Dec’18: New Pre-Print: Diffuser Wavefront Sensor
Nov’18: New Pre-Print: Multimodal DenseNet
Oct’18: New Pre-Print: Deep LSR
Oct’18: New Pre-Print: Nuclei Segmentation in Histopathology Data.
Oct’18: 3 oral and 2 poster presentations accepted at SPIE Medical Imaging.
Oct’18: 2 oral and 2 poster presentations accepted at SPIE Photonics West.
Oct’18: Invited talk at Nvidia GTC DC 2018.
Sept’18: Invited talk at SAS Deep Learning Symposium.
Aug’18: Article accepted in Physics in Medicine and Biology.
July’18: Article accepted in Medical Image Analysis.
March’18: Article accepted in IEEE Transactions on Medical Imaging.
March’18: Poster presented at Nvidia GTC 2018.
March’18: Poster presented at SPIE Medical Imaging 2018.
Feb’18: Article accepted in IEEE International Symposium in Biomedical Imaging (ISBI).
Jan’18: Oral talk presented at SPIE Photonics West 2018.
Jan’18: Article accepted in IEEE Signal Processing Letters.
July’18: Article accepted in IEEE Access.
June 2017: Article accepted in International Journal of Reconfigurable Computing.