You are here

Mazurowski, Maciej A

Positions:

Assistant Professor of Radiology

Radiology
School of Medicine

Assistant Professor in the Department of Electrical and Computer Engineering

Electrical and Computer Engineering
Pratt School of Engineering

Member of the Duke Cancer Institute

Duke Cancer Institute
School of Medicine

Education:

Ph.D. 2008

Ph.D. — University of Louisville

Grants:

Breast Cancer Detection Consortium

Administered By
Surgery, Surgical Sciences
AwardedBy
National Institutes of Health
Role
Statistician
Start Date
September 19, 2016
End Date
August 31, 2021

Machine learning and collaborative filtering tools for personalized education in digital breast tomosynthesis

Administered By
Radiology
AwardedBy
National Institutes of Health
Role
Principal Investigator
Start Date
September 01, 2016
End Date
May 31, 2020

Can contrast dynamics in breast MRI predict genomic intra-tumor heterogeneity

Administered By
Radiology
AwardedBy
Bracco Diagnostics, Inc.
Role
Principal Investigator
Start Date
July 18, 2016
End Date
June 30, 2018

Commericialization of a computational imaging-based biomarker for prognostication in breast cancer

Administered By
Radiology
AwardedBy
North Carolina Biotechnology Center
Role
Principal Investigator
Start Date
February 01, 2016
End Date
October 31, 2017

Development of a personalized evidence-based algorithm for the management of suspicious calcifications

Administered By
Radiology, Breast Imaging
AwardedBy
Ge-Aur Radiology Research
Role
Mentor
Start Date
July 01, 2015
End Date
June 30, 2017

Improved education in digital breast tomosynthesis using machine learning and computer vision tools

Administered By
Radiology
AwardedBy
Radiological Society of North America
Role
Principal Investigator
Start Date
July 01, 2014
End Date
March 31, 2016

Information-Theoretic Based CAD in Mammography

Administered By
Radiology
AwardedBy
National Institutes of Health
Role
Research Associate
Start Date
July 01, 2003
End Date
June 30, 2011

Tomosynthesis for Improved Breast Cancer Detection

Administered By
Radiology
AwardedBy
National Institutes of Health
Role
Postdoctoral Associate
Start Date
June 20, 2006
End Date
April 30, 2011
Show More

Publications:

Algorithmic three-dimensional analysis of tumor shape in MRI improves prognosis of survival in glioblastoma: a multi-institutional study.

In this retrospective, IRB-exempt study, we analyzed data from 68 patients diagnosed with glioblastoma (GBM) in two institutions and investigated the relationship between tumor shape, quantified using algorithmic analysis of magnetic resonance images, and survival. Each patient's Fluid Attenuated Inversion Recovery (FLAIR) abnormality and enhancing tumor were manually delineated, and tumor shape was analyzed by automatic computer algorithms. Five features were automatically extracted from the images to quantify the extent of irregularity in tumor shape in two and three dimensions. Univariate Cox proportional hazard regression analysis was performed to determine how prognostic each feature was of survival. Kaplan Meier analysis was performed to illustrate the prognostic value of each feature. To determine whether the proposed quantitative shape features have additional prognostic value compared with standard clinical features, we controlled for tumor volume, patient age, and Karnofsky Performance Score (KPS). The FLAIR-based bounding ellipsoid volume ratio (BEVR), a 3D complexity measure, was strongly prognostic of survival, with a hazard ratio of 0.36 (95% CI 0.20-0.65), and remained significant in regression analysis after controlling for other clinical factors (P = 0.0061). Three enhancing-tumor based shape features were prognostic of survival independently of clinical factors: BEVR (P = 0.0008), margin fluctuation (P = 0.0013), and angular standard deviation (P = 0.0078). Algorithmically assessed tumor shape is statistically significantly prognostic of survival for patients with GBM independently of patient age, KPS, and tumor volume. This shows promise for extending the utility of MR imaging in treatment of GBM patients.

Authors
Czarnek, N; Clark, K; Peters, KB; Mazurowski, MA
MLA Citation
Czarnek, N, Clark, K, Peters, KB, and Mazurowski, MA. "Algorithmic three-dimensional analysis of tumor shape in MRI improves prognosis of survival in glioblastoma: a multi-institutional study." Journal of neuro-oncology 132.1 (March 2017): 55-62.
PMID
28074320
Source
epmc
Published In
Journal of Neuro-Oncology
Volume
132
Issue
1
Publish Date
2017
Start Page
55
End Page
62
DOI
10.1007/s11060-016-2359-7

Predictive Utility of Marketed Volumetric Software Tools in Subjects at Risk for Alzheimer Disease: Do Regions Outside the Hippocampus Matter?

Alzheimer disease is a prevalent neurodegenerative disease. Computer assessment of brain atrophy patterns can help predict conversion to Alzheimer disease. Our aim was to assess the prognostic efficacy of individual-versus-combined regional volumetrics in 2 commercially available brain volumetric software packages for predicting conversion of patients with mild cognitive impairment to Alzheimer disease.Data were obtained through the Alzheimer's Disease Neuroimaging Initiative. One hundred ninety-two subjects (mean age, 74.8 years; 39% female) diagnosed with mild cognitive impairment at baseline were studied. All had T1-weighted MR imaging sequences at baseline and 3-year clinical follow-up. Analysis was performed with NeuroQuant and Neuroreader. Receiver operating characteristic curves assessing the prognostic efficacy of each software package were generated by using a univariable approach using individual regional brain volumes and 2 multivariable approaches (multiple regression and random forest), combining multiple volumes.On univariable analysis of 11 NeuroQuant and 11 Neuroreader regional volumes, hippocampal volume had the highest area under the curve for both software packages (0.69, NeuroQuant; 0.68, Neuroreader) and was not significantly different (P > .05) between packages. Multivariable analysis did not increase the area under the curve for either package (0.63, logistic regression; 0.60, random forest NeuroQuant; 0.65, logistic regression; 0.62, random forest Neuroreader).Of the multiple regional volume measures available in FDA-cleared brain volumetric software packages, hippocampal volume remains the best single predictor of conversion of mild cognitive impairment to Alzheimer disease at 3-year follow-up. Combining volumetrics did not add additional prognostic efficacy. Therefore, future prognostic studies in mild cognitive impairment, combining such tools with demographic and other biomarker measures, are justified in using hippocampal volume as the only volumetric biomarker.

Authors
Tanpitukpongse, TP; Mazurowski, MA; Ikhena, J; Petrella, JR
MLA Citation
Tanpitukpongse, TP, Mazurowski, MA, Ikhena, J, and Petrella, JR. "Predictive Utility of Marketed Volumetric Software Tools in Subjects at Risk for Alzheimer Disease: Do Regions Outside the Hippocampus Matter?." AJNR. American journal of neuroradiology 38.3 (March 2017): 546-552.
PMID
28057634
Source
epmc
Published In
American Journal of Neuroradiology
Volume
38
Issue
3
Publish Date
2017
Start Page
546
End Page
552
DOI
10.3174/ajnr.a5061

Algorithms for prediction of the Oncotype DX recurrence score using clinicopathologic data: a review and comparison using an independent dataset.

Given the potential savings in cost and resource utilization, several algorithms have been proposed to predict Oncotype DX recurrence score (ODX RS) using commonly acquired histopathologic variables. Although it is promising, additional independent validation of these surrogate markers is needed prior to guide the patient management.In this retrospective study, we analyzed 305 patients with invasive breast cancer at our institution who had ODX RS available. We selected five equations that provide a surrogate measure of ODX as previously published by Klein et al. (Magee equations 1-3), Gage et al., and Tang et al. All equations used estrogen receptor status and progesterone receptor status along with different combinations of grade, proliferation indices (Ki-67, mitotic rate), HER2 status, and tumor size.Of all surrogate scores tested, the Magee equation 2 provided the highest correlation with ODX both with regard to raw score (Pearson's correlation coefficient = 0.66 95% CI 0.59-0.72) and categorical correlation (Cohen's kappa = 0.43, 95% CI 0.33-0.53). Although Magee equation 2 provided a way to reliably identify high-risk disease by assigning 95% of the patients with high ODX RS to either the intermediate- or high-risk group, it was unable to reliably identify the potential for patients to have intermediate- or high-risk disease by ODX (66% of such patients identified).Although commonly available surrogates for ODX appear to predict high-risk ODX RS, they are unable to reliably rule out the presence of patients with intermediate-risk disease by ODX. Given the potential benefit of adjuvant chemotherapy in women with intermediate-risk disease by ODX, current surrogates are unable to safely substitute for ODX. Characterizing the true recurrence risk in patients with intermediate-risk disease by ODX is critical to the clinical adoption of current surrogate markers and is an area of ongoing clinical trials.

Authors
Harowicz, MR; Robinson, TJ; Dinan, MA; Saha, A; Marks, JR; Marcom, PK; Mazurowski, MA
MLA Citation
Harowicz, MR, Robinson, TJ, Dinan, MA, Saha, A, Marks, JR, Marcom, PK, and Mazurowski, MA. "Algorithms for prediction of the Oncotype DX recurrence score using clinicopathologic data: a review and comparison using an independent dataset." Breast cancer research and treatment 162.1 (February 2017): 1-10.
PMID
28064383
Source
epmc
Published In
Breast Cancer Research and Treatment
Volume
162
Issue
1
Publish Date
2017
Start Page
1
End Page
10
DOI
10.1007/s10549-016-4093-4

A computer vision-based algorithm to predict false positive errors in radiology trainees when interpreting digital breast tomosynthesis cases

Authors
Wang, M; Wang, M; Grimm, LJ; Mazurowski, MA
MLA Citation
Wang, M, Wang, M, Grimm, LJ, and Mazurowski, MA. "A computer vision-based algorithm to predict false positive errors in radiology trainees when interpreting digital breast tomosynthesis cases." Expert Systems with Applications 64 (December 2016): 490-499.
Source
crossref
Published In
Expert Systems with Applications
Volume
64
Publish Date
2016
Start Page
490
End Page
499
DOI
10.1016/j.eswa.2016.08.023

Predicting false negative errors in digital breast tomosynthesis among radiology trainees using a computer vision-based approach

Authors
Wang, M; Zhang, J; Grimm, LJ; Ghate, SV; Walsh, R; Johnson, KS; Lo, JY; Mazurowski, MA
MLA Citation
Wang, M, Zhang, J, Grimm, LJ, Ghate, SV, Walsh, R, Johnson, KS, Lo, JY, and Mazurowski, MA. "Predicting false negative errors in digital breast tomosynthesis among radiology trainees using a computer vision-based approach." Expert Systems with Applications 56 (September 2016): 1-8.
Source
crossref
Published In
Expert Systems with Applications
Volume
56
Publish Date
2016
Start Page
1
End Page
8
DOI
10.1016/j.eswa.2016.01.053

Interobserver variability in identification of breast tumors in MRI and its implications for prognostic biomarkers and radiogenomics.

To assess the interobserver variability of readers when outlining breast tumors in MRI, study the reasons behind the variability, and quantify the effect of the variability on algorithmic imaging features extracted from breast MRI.Four readers annotated breast tumors from the MRI examinations of 50 patients from one institution using a bounding box to indicate a tumor. All of the annotated tumors were biopsy proven cancers. The similarity of bounding boxes was analyzed using Dice coefficients. An automatic tumor segmentation algorithm was used to segment tumors from the readers' annotations. The segmented tumors were then compared between readers using Dice coefficients as the similarity metric. Cases showing high interobserver variability (average Dice coefficient <0.8) after segmentation were analyzed by a panel of radiologists to identify the reasons causing the low level of agreement. Furthermore, an imaging feature, quantifying tumor and breast tissue enhancement dynamics, was extracted from each segmented tumor for a patient. Pearson's correlation coefficients were computed between the features for each pair of readers to assess the effect of the annotation on the feature values. Finally, the authors quantified the extent of variation in feature values caused by each of the individual reasons for low agreement.The average agreement between readers in terms of the overlap (Dice coefficient) of the bounding box was 0.60. Automatic segmentation of tumor improved the average Dice coefficient for 92% of the cases to the average value of 0.77. The mean agreement between readers expressed by the correlation coefficient for the imaging feature was 0.96.There is a moderate variability between readers when identifying the rectangular outline of breast tumors on MRI. This variability is alleviated by the automatic segmentation of the tumors. Furthermore, the moderate interobserver variability in terms of the bounding box does not translate into a considerable variability in terms of assessment of enhancement dynamics. The authors propose some additional ways to further reduce the interobserver variability.

Authors
Saha, A; Grimm, LJ; Harowicz, M; Ghate, SV; Kim, C; Walsh, R; Mazurowski, MA
MLA Citation
Saha, A, Grimm, LJ, Harowicz, M, Ghate, SV, Kim, C, Walsh, R, and Mazurowski, MA. "Interobserver variability in identification of breast tumors in MRI and its implications for prognostic biomarkers and radiogenomics." Medical physics 43.8 (August 2016): 4558-.
PMID
27487872
Source
epmc
Published In
Medical physics
Volume
43
Issue
8
Publish Date
2016
Start Page
4558
DOI
10.1118/1.4955435

Radiology Trainee Performance in Digital Breast Tomosynthesis: Relationship Between Difficulty and Error-Making Patterns.

The aim of this study was to better understand the relationship between digital breast tomosynthesis (DBT) difficulty and radiology trainee performance.Twenty-seven radiology residents and fellows and three expert breast imagers reviewed 60 DBT studies consisting of unilateral craniocaudal and medial lateral oblique views. Trainees had no prior DBT experience. All readers provided difficulty ratings and final BI-RADS(®) scores. Expert breast imager consensus interpretations were used to determine the ground truth. Trainee sensitivity, specificity, and area under the receiver operating characteristic curve (AUC) were calculated for low- and high-difficulty subsets of cases as assessed by each trainee him or herself (self-assessed difficulty) and consensus expert-assessed difficulty.For self-assessed difficulty, the trainee AUC was 0.696 for high-difficulty and 0.704 for low-difficulty cases (P = .753). Trainee sensitivity was 0.776 for high-difficulty and 0.538 for low-difficulty cases (P < .001). Trainee specificity was 0.558 for high-difficulty and 0.810 for low-difficulty cases (P < .001). For expert-assessed difficulty, the trainee AUC was 0.645 for high-difficulty and 0.816 for low-difficulty cases (P < .001). Trainee sensitivity was 0.612 for high-difficulty and .784 for low-difficulty cases (P < .001). Trainee specificity was 0.654 for high-difficulty and 0.765 for low-difficulty cases (P = .021).Cases deemed difficult by experts were associated with decreases in trainee AUC, sensitivity, and specificity. In contrast, for self-assessed more difficult cases, the trainee AUC was unchanged because of increased sensitivity and compensatory decreased specificity. Educators should incorporate these findings when developing educational materials to teach interpretation of DBT.

Authors
Grimm, LJ; Zhang, J; Lo, JY; Johnson, KS; Ghate, SV; Walsh, R; Mazurowski, MA
MLA Citation
Grimm, LJ, Zhang, J, Lo, JY, Johnson, KS, Ghate, SV, Walsh, R, and Mazurowski, MA. "Radiology Trainee Performance in Digital Breast Tomosynthesis: Relationship Between Difficulty and Error-Making Patterns." Journal of the American College of Radiology : JACR 13.2 (February 2016): 198-202.
PMID
26577878
Source
epmc
Published In
Journal of the American College of Radiology
Volume
13
Issue
2
Publish Date
2016
Start Page
198
End Page
202
DOI
10.1016/j.jacr.2015.09.025

Author's Reply.

Authors
Mazurowski, MA
MLA Citation
Mazurowski, MA. "Author's Reply." Journal of the American College of Radiology : JACR 13.2 (February 2016): 121-122.
PMID
26846389
Source
epmc
Published In
Journal of the American College of Radiology
Volume
13
Issue
2
Publish Date
2016
Start Page
121
End Page
122
DOI
10.1016/j.jacr.2015.11.017

Identification of error making patterns in lesion detection on digital breast tomosynthesis using computer-extracted image features

© 2016 SPIE.Digital breast tomosynthesis (DBT) can improve lesion visibility by eliminating the issue of overlapping breast tissue present in mammography. However, this new modality likely requires new approaches to training. The issue of training in DBT is not well explored. We propose a computer-aided educational approach for DBT training. Our hypothesis is that the trainees' educational outcomes will improve if they are presented with cases individually selected to address their weaknesses. In this study, we focus on the question of how to select such cases. Specifically, we propose an algorithm that based on previously acquired reading data predicts which lesions will be missed by the trainee for future cases (i.e., we focus on false negative error). A logistic regression classifier was used to predict the likelihood of trainee error and computer-extracted features were used as the predictors. Reader data from 3 expert breast imagers was used to establish the ground truth and reader data from 5 radiology trainees was used to evaluate the algorithm performance with repeated holdout cross validation. Receiver operating characteristic (ROC) analysis was applied to measure the performance of the proposed individual trainee models. The preliminary experimental results for 5 trainees showed the individual trainee models were able to distinguish the lesions that would be detected from those that would be missed with the average area under the ROC curve of 0.639 (95% CI, 0.580-0.698). The proposed algorithm can be used to identify difficult cases for individual trainees.

Authors
Wang, M; Zhang, J; Grimm, LJ; Ghate, SV; Walsh, R; Johnson, KS; Lo, JY; Mazurowski, MA
MLA Citation
Wang, M, Zhang, J, Grimm, LJ, Ghate, SV, Walsh, R, Johnson, KS, Lo, JY, and Mazurowski, MA. "Identification of error making patterns in lesion detection on digital breast tomosynthesis using computer-extracted image features." January 1, 2016.
Source
scopus
Published In
Proceedings of SPIE
Volume
9787
Publish Date
2016
DOI
10.1117/12.2201061

Radiogenomics of glioblastoma: A pilot multi-institutional study to investigate a relationship between tumor shape features and tumor molecular subtype

© 2016 SPIE.Genomic subtype has been shown to be an important predictor of therapy response for patients with glioblastomas. Unfortunately, obtaining the genomic subtype is an expensive process that is not typically included in the standard of care. It is therefore of interest to investigate potential surrogates of molecular subtypes that use standard diagnostic data such as magnetic resonance (MR) imaging. In this study, we analyze the relationship between tumor genomic subtypes, proposed by Verhaak et al, 2010, and novel features that capture the shape of abnormalities as seen in fluid attenuated inversion recovery (FLAIR) MR images. In our study, we used data from 54 patients with glioblastomas from four institutions provided by The Cancer Genome Atlas (TCGA) and The Cancer Imaging Archive (TCIA). We explore five shape features calculated by computer algorithms implemented in our laboratory that assess shape both in individual slices and in rendered three-dimensional tumor volumes. The association between each feature and molecular subtype was assessed using area under the receiver operating characteristic curve analysis. We show that the two dimensional measures of edge complexity are significant discriminators between mesenchymal and classical tumors. These preliminary findings show promise for an imaging-based surrogate of molecular subtype and contribute to the understanding of the relationship between tumor biology and its radiology phenotype.

Authors
Czarnek, NM; Clark, K; Peters, KB; Collins, LM; Mazurowski, MA
MLA Citation
Czarnek, NM, Clark, K, Peters, KB, Collins, LM, and Mazurowski, MA. "Radiogenomics of glioblastoma: A pilot multi-institutional study to investigate a relationship between tumor shape features and tumor molecular subtype." January 1, 2016.
Source
scopus
Published In
Proceedings of SPIE
Volume
9785
Publish Date
2016
DOI
10.1117/12.2217084

Predicting outcomes in glioblastoma patients using computerized analysis of tumor shape - Preliminary data

© 2016 SPIE.Glioblastoma (GBM) is the most common primary brain tumor characterized by very poor survival. However, while some patients survive only a few months, some might live for multiple years. Accurate prognosis of survival and stratification of patients allows for making more personalized treatment decisions and moves treatment of GBM one step closer toward the paradigm of precision medicine. While some molecular biomarkers are being investigated, medical imaging remains significantly underutilized for prognostication in GBM. In this study, we investigated whether computer analysis of tumor shape can contribute toward accurate prognosis of outcomes. Specifically, we implemented applied computer algorithms to extract 5 shape features from magnetic resonance imaging (MRI) for 22 GBM patients. Then, we determined whether each one of the features can accurately distinguish between patients with good and poor outcomes. We found that that one of the 5 analyzed features showed prognostic value of survival. The prognostic feature describes how well the 3D tumor shape fills its minimum bounding ellipsoid. Specifically, for low values (less or equal than the median) the proportion of patients that survived more than a year was 27% while for high values (higher than median) the proportion of patients with survival of more than 1 year was 82%. The difference was statistically significant (p < 0.05) even though the number of patients analyzed in this pilot study was low. We concluded that computerized, 3D analysis of tumor shape in MRI may strongly contribute to accurate prognostication and stratification of patients for therapy in GBM.

Authors
Mazurowski, MA; Czarnek, NM; Collins, LM; Peters, KB; Clark, K
MLA Citation
Mazurowski, MA, Czarnek, NM, Collins, LM, Peters, KB, and Clark, K. "Predicting outcomes in glioblastoma patients using computerized analysis of tumor shape - Preliminary data." January 1, 2016.
Source
scopus
Published In
Proceedings of SPIE
Volume
9785
Publish Date
2016
DOI
10.1117/12.2217098

Recurrence-free survival in breast cancer is associated with MRI tumor enhancement dynamics quantified using computer algorithms.

The purpose of this study is to investigate the association between breast cancer recurrence-free survival and breast magnetic resonance imaging (MRI) tumor enhancement dynamics which are quantified semi-automatically using computer algorithms.In this retrospective IRB-approved study, we analyzed data from 275 breast cancer patients at a single institution. Recurrence-free survival data were obtained from the medical record. Routine clinical pre-operative breast MRIs were performed in all patients. The tumors were marked on the MRIs by fellowship-trained breast radiologists. A previously developed computer algorithm was applied to the marked tumors to quantify the enhancement dynamics relative to the automatically assessed background parenchymal enhancement. To establish whether the contrast enhancement feature quantified by the algorithm was associated with recurrence-free survival, we constructed a Cox proportional hazards regression model with the computer-extracted feature as a covariate. We controlled for tumor grade and size (major axis length), patient age, patient race/ethnicity, and menopausal status.The analysis showed that the semi-automatically obtained feature quantifying MRI tumor enhancement dynamics was independently predictive of recurrence-free survival (p=0.024).Semi-automatically quantified tumor enhancement dynamics on MRI are predictive of recurrence-free survival in breast cancer patients.

Authors
Mazurowski, MA; Grimm, LJ; Zhang, J; Marcom, PK; Yoon, SC; Kim, C; Ghate, SV; Johnson, KS
MLA Citation
Mazurowski, MA, Grimm, LJ, Zhang, J, Marcom, PK, Yoon, SC, Kim, C, Ghate, SV, and Johnson, KS. "Recurrence-free survival in breast cancer is associated with MRI tumor enhancement dynamics quantified using computer algorithms." European journal of radiology 84.11 (November 2015): 2117-2122.
PMID
26210095
Source
epmc
Published In
European Journal of Radiology
Volume
84
Issue
11
Publish Date
2015
Start Page
2117
End Page
2122
DOI
10.1016/j.ejrad.2015.07.012

Computational approach to radiogenomics of breast cancer: Luminal A and luminal B molecular subtypes are associated with imaging features on routine breast MRI extracted using computer vision algorithms.

To identify associations between semiautomatically extracted MRI features and breast cancer molecular subtypes.We analyzed routine clinical pre-operative breast MRIs from 275 breast cancer patients at a single institution in this retrospective, Institutional Review Board-approved study. Six fellowship-trained breast imagers reviewed the MRIs and annotated the cancers. Computer vision algorithms were then used to extract 56 imaging features from the cancers including morphologic, texture, and dynamic features. Surrogate markers (estrogen receptor [ER], progesterone receptor [PR], human epidermal growth factor receptor-2 [HER2]) were used to categorize tumors by molecular subtype: ER/PR+, HER2- (luminal A); ER/PR+, HER2+ (luminal B); ER/PR-, HER2+ (HER2); ER/PR/HER2- (basal). A multivariate analysis was used to determine associations between the imaging features and molecular subtype.The imaging features were associated with both luminal A (P = 0.0007) and luminal B (P = 0.0063) molecular subtypes. No association was found for either HER2 (P = 0.2465) or basal (P = 0.1014) molecular subtype and the imaging features. A P-value of 0.0125 (0.05/4) was considered significant.Luminal A and luminal B molecular subtype breast cancer are associated with semiautomatically extracted features from routine contrast enhanced breast MRI.

Authors
Grimm, LJ; Zhang, J; Mazurowski, MA
MLA Citation
Grimm, LJ, Zhang, J, and Mazurowski, MA. "Computational approach to radiogenomics of breast cancer: Luminal A and luminal B molecular subtypes are associated with imaging features on routine breast MRI extracted using computer vision algorithms." Journal of magnetic resonance imaging : JMRI 42.4 (October 2015): 902-907.
PMID
25777181
Source
epmc
Published In
Journal of Magnetic Resonance Imaging
Volume
42
Issue
4
Publish Date
2015
Start Page
902
End Page
907
DOI
10.1002/jmri.24879

Radiogenomics: what it is and why it is important.

In recent years, a new direction in cancer research has emerged that focuses on the relationship between imaging phenotypes and genomics. This direction is referred to as radiogenomics or imaging genomics. The question that subsequently arises is: What is the practical significance of elucidating this relationship in improving cancer patient outcomes. In this article, I address this question. Although I discuss some limitations of the radiogenomic approach, and describe scenarios in which radiogenomic analysis might not be the best choice, I also argue that radiogenomics will play a significant practical role in cancer research. Specifically, I argue that the significance of radiogenomics is largely related to practical limitations of currently available data that often lack complete characterization of the patients and poor integration of individual datasets. Radiogenomics offers a practical way to leverage limited and incomplete data to generate knowledge that might lead to improved decision making, and as a result, improved patient outcomes.

Authors
Mazurowski, MA
MLA Citation
Mazurowski, MA. "Radiogenomics: what it is and why it is important." Journal of the American College of Radiology : JACR 12.8 (August 2015): 862-866.
PMID
26250979
Source
epmc
Published In
Journal of the American College of Radiology
Volume
12
Issue
8
Publish Date
2015
Start Page
862
End Page
866
DOI
10.1016/j.jacr.2015.04.019

Does Breast Imaging Experience During Residency Translate Into Improved Initial Performance in Digital Breast Tomosynthesis?

To determine the initial digital breast tomosynthesis (DBT) performance of radiology trainees with varying degrees of breast imaging experience.To test trainee performance with DBT, we performed a reader study, after obtaining IRB approval. Two medical students, 20 radiology residents, 4 nonbreast imaging fellows, 3 breast imaging fellows, and 3 fellowship-trained breast imagers reviewed 60 unilateral DBT studies (craniocaudal and medio-lateral oblique views). Trainees had no DBT experience. Each reader recorded a final BI-RADS assessment for each case. The consensus interpretations from fellowship-trained breast imagers were used to establish the ground truth. Area under the receiver operating characteristic curve (AUC), sensitivity, and specificity were calculated. For analysis, first- through third-year residents were classified as junior trainees, and fourth-year residents plus nonbreast imaging fellows were classified as senior trainees.The AUCs were .569 for medical students, .721 for junior trainees, .701 for senior trainees, and .792 for breast imaging fellows. The junior and senior trainee AUCs were equivalent (P < .01) using a two one-sided test for equivalence, with a significance threshold of 0.1. The sensitivities and specificities were highest for breast imaging fellows (.778 and .815 respectively), but similar for junior (.631 and .714, respectively) and senior trainees (.678 and .661, respectively).Initial performance with DBT among radiology residents and nonbreast imaging fellows is independent of years of training. Radiology educators should consider these findings when developing educational materials.

Authors
Zhang, J; Grimm, LJ; Lo, JY; Johnson, KS; Ghate, SV; Walsh, R; Mazurowski, MA
MLA Citation
Zhang, J, Grimm, LJ, Lo, JY, Johnson, KS, Ghate, SV, Walsh, R, and Mazurowski, MA. "Does Breast Imaging Experience During Residency Translate Into Improved Initial Performance in Digital Breast Tomosynthesis?." Journal of the American College of Radiology : JACR 12.7 (July 2015): 728-732.
PMID
26143567
Source
epmc
Published In
Journal of the American College of Radiology
Volume
12
Issue
7
Publish Date
2015
Start Page
728
End Page
732
DOI
10.1016/j.jacr.2015.02.025

Modeling false positive error making patterns in radiology trainees for improved mammography education.

While mammography notably contributes to earlier detection of breast cancer, it has its limitations, including a large number of false positive exams. Improved radiology education could potentially contribute to alleviating this issue. Toward this goal, in this paper we propose an algorithm for modeling of false positive error making among radiology trainees. Identifying troublesome locations for the trainees could focus their training and in turn improve their performance.The algorithm proposed in this paper predicts locations that are likely to result in a false positive error for each trainee based on the previous annotations made by the trainee. The algorithm consists of three steps. First, the suspicious false positive locations are identified in mammograms by Difference of Gaussian filter and suspicious regions are segmented by computer vision-based segmentation algorithms. Second, 133 features are extracted for each suspicious region to describe its distinctive characteristics. Third, a random forest classifier is applied to predict the likelihood of the trainee making a false positive error using the extracted features. The random forest classifier is trained using previous annotations made by the trainee. We evaluated the algorithm using data from a reader study in which 3 experts and 10 trainees interpreted 100 mammographic cases.The algorithm was able to identify locations where the trainee will commit a false positive error with accuracy higher than an algorithm that selects such locations randomly. Specifically, our algorithm found false positive locations with 40% accuracy when only 1 location was selected for all cases for each trainee and 12% accuracy when 10 locations were selected. The accuracies for randomly identified locations were both 0% for these two scenarios.In this first study on the topic, we were able to build computer models that were able to find locations for which a trainee will make a false positive error in images that were not previously seen by the trainee. Presenting the trainees with such locations rather than randomly selected ones may improve their educational outcomes.

Authors
Zhang, J; Silber, JI; Mazurowski, MA
MLA Citation
Zhang, J, Silber, JI, and Mazurowski, MA. "Modeling false positive error making patterns in radiology trainees for improved mammography education." Journal of biomedical informatics 54 (April 2015): 50-57.
PMID
25640462
Source
epmc
Published In
Journal of Biomedical Informatics
Volume
54
Publish Date
2015
Start Page
50
End Page
57
DOI
10.1016/j.jbi.2015.01.007

Incorporating breast tomosynthesis into radiology residency: Does trainee experience in breast imaging translate into improved performance with this new modality?

© 2015 SPIE.Digital breast tomosynthesis (DBT) is a powerful new imaging modality that has the potential to transform breast cancer screening practices. The advantages over mammography include improved sensitivity and specificity as well as the detection of additional invasive cancers. While this modality holds many advantages, the best means of incorporating DBT into radiology training programs is currently not well understood. The initial performance of a trainee in DBT might depend on the amount of previous radiology training, in particular breast imaging experience. In our study, we tested the DBT interpretive skills of radiology trainees with different levels of breast imaging training, but with no prior DBT experience. We recruited 16 radiology trainees to review 60 DBT studies. A fellowship-trained expert breast radiologist reviewed the studies and provided the gold standard interpretations. Receiver operating characteristic analysis was used to evaluate the performance of the trainees. Our results show that there is no notable difference in trainee performance in DBT, regardless of the number of years of general radiology experience. These results provide guidance to breast imaging educators as they prepare new curricula to teach DBT. These curricula should provide a base training which will likely be suitable for all trainee levels.

Authors
Grimm, LJ; Zhang, J; Johnson, KS; Lo, JY; Mazurowski, MA
MLA Citation
Grimm, LJ, Zhang, J, Johnson, KS, Lo, JY, and Mazurowski, MA. "Incorporating breast tomosynthesis into radiology residency: Does trainee experience in breast imaging translate into improved performance with this new modality?." January 1, 2015.
Source
scopus
Published In
Proceedings of SPIE
Volume
9416
Publish Date
2015
DOI
10.1117/12.2082810

Recurrence-free survival in breast cancer is associated with MRI tumor enhancement dynamics quantified using computer algorithms

© 2015 Elsevier Ireland Ltd. All rights reserved.Purpose The purpose of this study is to investigate the association between breast cancer recurrence-free survival and breast magnetic resonance imaging (MRI) tumor enhancement dynamics which are quantified semi-automatically using computer algorithms. Methods In this retrospective IRB-approved study, we analyzed data from 275 breast cancer patients at a single institution. Recurrence-free survival data were obtained from the medical record. Routine clinical pre-operative breast MRIs were performed in all patients. The tumors were marked on the MRIs by fellowship-trained breast radiologists. A previously developed computer algorithm was applied to the marked tumors to quantify the enhancement dynamics relative to the automatically assessed background parenchymal enhancement. To establish whether the contrast enhancement feature quantified by the algorithm was associated with recurrence-free survival, we constructed a Cox proportional hazards regression model with the computer-extracted feature as a covariate. We controlled for tumor grade and size (major axis length), patient age, patient race/ethnicity, and menopausal status. Results The analysis showed that the semi-automatically obtained feature quantifying MRI tumor enhancement dynamics was independently predictive of recurrence-free survival (p = 0.024). Conclusion Semi-automatically quantified tumor enhancement dynamics on MRI are predictive of recurrence-free survival in breast cancer patients.

Authors
Mazurowski, MA; Grimm, LJ; Zhang, J; Marcom, PK; Yoon, SC; Kim, C; Ghate, SV; Johnson, KS
MLA Citation
Mazurowski, MA, Grimm, LJ, Zhang, J, Marcom, PK, Yoon, SC, Kim, C, Ghate, SV, and Johnson, KS. "Recurrence-free survival in breast cancer is associated with MRI tumor enhancement dynamics quantified using computer algorithms." European Journal of Radiology 84.11 (2015): 2117-2122.
Source
scival
Published In
European Journal of Radiology
Volume
84
Issue
11
Publish Date
2015
Start Page
2117
End Page
2122
DOI
10.1016/j.ejrad.2015.07.012

Computer-extracted MR imaging features are associated with survival in glioblastoma patients.

Automatic survival prognosis in glioblastoma (GBM) could result in improved treatment planning for the patient. The purpose of this research is to investigate the association of survival in GBM patients with tumor features in pre-operative magnetic resonance (MR) images assessed using a fully automatic computer algorithm. MR imaging data for 68 patients from two US institutions were used in this study. The images were obtained from the Cancer Imaging Archive. A fully automatic computer vision algorithm was applied to segment the images and extract eight imaging features from the MRI studies. The features included tumor side, proportion of enhancing tumor, proportion of necrosis, T1/FLAIR ratio, major axis length, minor axis length, tumor volume, and thickness of enhancing margin. We constructed a multivariate Cox proportional hazards regression model and used a likelihood ratio test to establish whether the imaging features are prognostic of survival. We also evaluated the individual prognostic value of each feature through multivariate analysis using the multivariate Cox model and univariate analysis using univariate Cox models for each feature. We found that the automatically extracted imaging features were predictive of survival (p = 0.031). Multivariate analysis of individual features showed that two individual features were predictive of survival: proportion of enhancing tumor (p = 0.013), and major axis length (p = 0.026). Univariate analysis indicated the same two features as significant (p = 0.021, and p = 0.017 respectively). We conclude that computer-extracted MR imaging features can be used for survival prognosis in GBM patients.

Authors
Mazurowski, MA; Zhang, J; Peters, KB; Hobbs, H
MLA Citation
Mazurowski, MA, Zhang, J, Peters, KB, and Hobbs, H. "Computer-extracted MR imaging features are associated with survival in glioblastoma patients." Journal of neuro-oncology 120.3 (December 2014): 483-488.
PMID
25151504
Source
epmc
Published In
Journal of Neuro-Oncology
Volume
120
Issue
3
Publish Date
2014
Start Page
483
End Page
488
DOI
10.1007/s11060-014-1580-5

Radiogenomic analysis of breast cancer: luminal B molecular subtype is associated with enhancement dynamics at MR imaging.

PURPOSE: To investigate associations between breast cancer molecular subtype and semiautomatically extracted magnetic resonance (MR) imaging features. MATERIALS AND METHODS: Imaging and genomic data from the Cancer Genome Atlas and the Cancer Imaging Archive for 48 patients with breast cancer from four institutions in the United States were used in this institutional review board approval-exempt study. Computer vision algorithms were applied to extract 23 imaging features from lesions indicated by a breast radiologist on MR images. Morphologic, textural, and dynamic features were extracted. Molecular subtype was determined on the basis of genomic analysis. Associations between the imaging features and molecular subtype were evaluated by using logistic regression and likelihood ratio tests. The analysis controlled for the age of the patients, their menopausal status, and the orientation of the MR images (sagittal vs axial). RESULTS: There is an association (P = .0015) between the luminal B subtype and a dynamic contrast material-enhancement feature that quantifies the relationship between lesion enhancement and background parenchymal enhancement. Cancers with a higher ratio of lesion enhancement rate to background parenchymal enhancement rate are more likely to be luminal B subtype. CONCLUSION: The luminal B subtype of breast cancer is associated with MR imaging features that relate the enhancement dynamics of the tumor and the background parenchyma.

Authors
Mazurowski, MA; Zhang, J; Grimm, LJ; Yoon, SC; Silber, JI
MLA Citation
Mazurowski, MA, Zhang, J, Grimm, LJ, Yoon, SC, and Silber, JI. "Radiogenomic analysis of breast cancer: luminal B molecular subtype is associated with enhancement dynamics at MR imaging." Radiology 273.2 (November 2014): 365-372.
PMID
25028781
Source
epmc
Published In
Radiology
Volume
273
Issue
2
Publish Date
2014
Start Page
365
End Page
372
DOI
10.1148/radiol.14132641

Using computer-extracted image features for modeling of error-making patterns in detection of mammographic masses among radiology residents.

Mammography is the most widely accepted and utilized screening modality for early breast cancer detection. Providing high quality mammography education to radiology trainees is essential, since excellent interpretation skills are needed to ensure the highest benefit of screening mammography for patients. The authors have previously proposed a computer-aided education system based on trainee models. Those models relate human-assessed image characteristics to trainee error. In this study, the authors propose to build trainee models that utilize features automatically extracted from images using computer vision algorithms to predict likelihood of missing each mass by the trainee. This computer vision-based approach to trainee modeling will allow for automatically searching large databases of mammograms in order to identify challenging cases for each trainee.The authors' algorithm for predicting the likelihood of missing a mass consists of three steps. First, a mammogram is segmented into air, pectoral muscle, fatty tissue, dense tissue, and mass using automated segmentation algorithms. Second, 43 features are extracted using computer vision algorithms for each abnormality identified by experts. Third, error-making models (classifiers) are applied to predict the likelihood of trainees missing the abnormality based on the extracted features. The models are developed individually for each trainee using his/her previous reading data. The authors evaluated the predictive performance of the proposed algorithm using data from a reader study in which 10 subjects (7 residents and 3 novices) and 3 experts read 100 mammographic cases. Receiver operating characteristic (ROC) methodology was applied for the evaluation.The average area under the ROC curve (AUC) of the error-making models for the task of predicting which masses will be detected and which will be missed was 0.607 (95% CI,0.564-0.650). This value was statistically significantly different from 0.5 (p<0.0001). For the 7 residents only, the AUC performance of the models was 0.590 (95% CI,0.537-0.642) and was also significantly higher than 0.5 (p=0.0009). Therefore, generally the authors' models were able to predict which masses were detected and which were missed better than chance.The authors proposed an algorithm that was able to predict which masses will be detected and which will be missed by each individual trainee. This confirms existence of error-making patterns in the detection of masses among radiology trainees. Furthermore, the proposed methodology will allow for the optimized selection of difficult cases for the trainees in an automatic and efficient manner.

Authors
Zhang, J; Lo, JY; Kuzmiak, CM; Ghate, SV; Yoon, SC; Mazurowski, MA
MLA Citation
Zhang, J, Lo, JY, Kuzmiak, CM, Ghate, SV, Yoon, SC, and Mazurowski, MA. "Using computer-extracted image features for modeling of error-making patterns in detection of mammographic masses among radiology residents." Medical physics 41.9 (September 2014): 091907-.
PMID
25186394
Source
epmc
Published In
Medical physics
Volume
41
Issue
9
Publish Date
2014
Start Page
091907
DOI
10.1118/1.4892173

Radiology resident mammography training: interpretation difficulty and error-making patterns.

RATIONALE AND OBJECTIVES: The purpose of this study was to better understand the concept of mammography difficulty and how it affects radiology resident performance. MATERIALS AND METHODS: Seven radiology residents and three expert breast imagers reviewed 100 mammograms, consisting of bilateral medial lateral oblique and craniocaudal views, using a research workstation. The cases consisted of normal, benign, and malignant findings. Participants identified abnormalities and scored the difficulty and malignant potential for each case. Resident performance (sensitivity, specificity, and area under the receiver operating characteristic curve [AUC]) was calculated for self- and expert-assessed high and low difficulties. RESULTS: For cases classified by self-assessed difficulty, the resident AUCs were 0.667 for high difficulty and 0.771 for low difficulty cases (P = .010). Resident sensitivities were 0.707 for high and 0.614 for low difficulty cases (P = .113). Resident specificities were 0.583 for high and 0.905 for low difficulty cases (P < .001). For cases classified by expert-assessed difficulty, the resident AUCs were 0.583 for high and 0.783 for low difficulty cases (P = .001). Resident sensitivities were 0.558 for high and 0.796 for low difficulty cases (P < .001). Resident specificities were 0.714 for high and 0.740 for low difficulty cases (P = .807). CONCLUSIONS: Increased self- and expert-assessed difficulty is associated with a decrease in resident performance in mammography. However, while this lower performance is due to a decrease in specificity for self-assessed difficulty, it is due to a decrease in sensitivity for expert-assessed difficulty. These trends suggest that educators should provide a mix of self- and expert-assessed difficult cases in educational materials to maximize the effect of training on resident performance and confidence.

Authors
Grimm, LJ; Kuzmiak, CM; Ghate, SV; Yoon, SC; Mazurowski, MA
MLA Citation
Grimm, LJ, Kuzmiak, CM, Ghate, SV, Yoon, SC, and Mazurowski, MA. "Radiology resident mammography training: interpretation difficulty and error-making patterns." Academic radiology 21.7 (July 2014): 888-892.
PMID
24928157
Source
epmc
Published In
Academic Radiology
Volume
21
Issue
7
Publish Date
2014
Start Page
888
End Page
892
DOI
10.1016/j.acra.2014.01.025

Predictors of an academic career on radiology residency applications.

RATIONALE AND OBJECTIVES: To evaluate radiology residency applications to determine if any variables are predictive of a future academic radiology career. MATERIALS AND METHODS: Application materials from 336 radiology residency graduates between 1993 and 2010 from the Department of Radiology, Duke University and between 1990 and 2010 from the Department of Radiology, Stanford University were retrospectively reviewed. The institutional review boards approved this Health Insurance Portability and Accountability Act-compliant study with a waiver of informed consent. Biographical (gender, age at application, advanced degrees, prior career), undergraduate school (school, degree, research experience, publications), and medical school (school, research experience, manuscript publications, Alpha Omega Alpha membership, clerkship grades, United States Medical Licensing Examination Step 1 and 2 scores, personal statement and letter of recommendation reference to academics, couples match status) data were recorded. Listing in the Association of American Medical Colleges Faculty Online Directory and postgraduation publications were used to determine academic status. RESULTS: There were 72 (21%) radiologists in an academic career and 264 (79%) in a nonacademic career. Variables associated with an academic career were elite undergraduate school (P = .003), undergraduate school publications (P = .018), additional advanced degrees (P = .027), elite medical school (P = .006), a research year in medical school (P < .001), and medical school publications (P < .001). A multivariate cross-validation analysis showed that these variables are jointly predictive of an academic career (P < .001). CONCLUSIONS: Undergraduate and medical school rankings and publications, as well as a medical school research year and an additional advanced degree, are associated with an academic career. Radiology residency selection committees should consider these factors in the context of the residency application if they wish to recruit future academic radiologists.

Authors
Grimm, LJ; Shapiro, LM; Singhapricha, T; Mazurowski, MA; Desser, TS; Maxfield, CM
MLA Citation
Grimm, LJ, Shapiro, LM, Singhapricha, T, Mazurowski, MA, Desser, TS, and Maxfield, CM. "Predictors of an academic career on radiology residency applications." Academic radiology 21.5 (May 2014): 685-690.
PMID
24629444
Source
epmc
Published In
Academic Radiology
Volume
21
Issue
5
Publish Date
2014
Start Page
685
End Page
690
DOI
10.1016/j.acra.2013.10.019

A fully automatic extraction of magnetic resonance image features in glioblastoma patients.

Glioblastoma is the most common malignant brain tumor. It is characterized by low median survival time and high survival variability. Survival prognosis for glioblastoma is very important for optimized treatment planning. Imaging features observed in magnetic resonance (MR) images were shown to be a good predictor of survival. However, manual assessment of MR features is time-consuming and can be associated with a high inter-reader variability as well as inaccuracies in the assessment. In response to this limitation, the authors proposed and evaluated a computer algorithm that extracts important MR image features in a fully automatic manner.The algorithm first automatically segmented the available volumes into a background region and four tumor regions. Then, it extracted ten features from the segmented MR imaging volumes, some of which were previously indicated as predictive of clinical outcomes. To evaluate the algorithm, the authors compared the extracted features for 73 glioblastoma patients to the reference standard established by manual segmentation of the tumors.The experiments showed that their algorithm was able to extract most of the image features with moderate to high accuracy. High correlation coefficients between the automatically extracted value and reference standard were observed for the tumor location, minor and major axis length as well as tumor volume. Moderately high correlation coefficients were also observed for proportion of enhancing tumor, proportion of necrosis, and thickness of enhancing margin. The correlation coefficients for all these features were statistically significant (p < 0.0001).The authors proposed and evaluated an algorithm that, given a set of MR volumes of a glioblastoma patient, is able to extract MR image features that correlate well with their reference standard. Future studies will evaluate how well the computer-extracted features predict survival.

Authors
Zhang, J; Barboriak, DP; Hobbs, H; Mazurowski, MA
MLA Citation
Zhang, J, Barboriak, DP, Hobbs, H, and Mazurowski, MA. "A fully automatic extraction of magnetic resonance image features in glioblastoma patients." Medical physics 41.4 (April 2014): 042301-.
PMID
24694151
Source
epmc
Published In
Medical physics
Volume
41
Issue
4
Publish Date
2014
Start Page
042301
DOI
10.1118/1.4866218

Predicting error in detecting mammographic masses among radiology trainees using statistical models based on BI-RADS features.

The purpose of this study is to explore Breast Imaging-Reporting and Data System (BI-RADS) features as predictors of individual errors made by trainees when detecting masses in mammograms.Ten radiology trainees and three expert breast imagers reviewed 100 mammograms comprised of bilateral medial lateral oblique and craniocaudal views on a research workstation. The cases consisted of normal and biopsy proven benign and malignant masses. For cases with actionable abnormalities, the experts recorded breast (density and axillary lymph nodes) and mass (shape, margin, and density) features according to the BI-RADS lexicon, as well as the abnormality location (depth and clock face). For each trainee, a user-specific multivariate model was constructed to predict the trainee's likelihood of error based on BI-RADS features. The performance of the models was assessed using area under the receive operating characteristic curves (AUC).Despite the variability in errors between different trainees, the individual models were able to predict the likelihood of error for the trainees with a mean AUC of 0.611 (range: 0.502-0.739, 95% Confidence Interval: 0.543-0.680,p < 0.002).Patterns in detection errors for mammographic masses made by radiology trainees can be modeled using BI-RADS features. These findings may have potential implications for the development of future educational materials that are personalized to individual trainees.

Authors
Grimm, LJ; Ghate, SV; Yoon, SC; Kuzmiak, CM; Kim, C; Mazurowski, MA
MLA Citation
Grimm, LJ, Ghate, SV, Yoon, SC, Kuzmiak, CM, Kim, C, and Mazurowski, MA. "Predicting error in detecting mammographic masses among radiology trainees using statistical models based on BI-RADS features." Medical physics 41.3 (March 2014): 031909-.
PMID
24593727
Source
epmc
Published In
Medical physics
Volume
41
Issue
3
Publish Date
2014
Start Page
031909
DOI
10.1118/1.4866379

Predicting error in detecting mammographic masses among radiology trainees using statistical models based on BI-RADS features

Authors
Grimm, LJ; Ghate, SV; Yoon, SC; Kuzmiak, CM; Kim, C; Mazurowski, MA
MLA Citation
Grimm, LJ, Ghate, SV, Yoon, SC, Kuzmiak, CM, Kim, C, and Mazurowski, MA. "Predicting error in detecting mammographic masses among radiology trainees using statistical models based on BI-RADS features." Medical Physics 41.3 (February 26, 2014): 031909-031909.
Source
crossref
Published In
Medical physics
Volume
41
Issue
3
Publish Date
2014
Start Page
031909
End Page
031909
DOI
10.1118/1.4866379

Predictors of an academic career on radiology residency applications

Rationale and Objectives: To evaluate radiology residency applications to determine if any variables are predictive of a future academic radiology career. Materials and Methods: Application materials from 336 radiology residency graduates between 1993 and 2010 from the Department of Radiology, Duke University and between 1990 and 2010 from the Department of Radiology, Stanford University were retrospectively reviewed. The institutional review boards approved this Health Insurance Portability and Accountability Act-compliant study with a waiver of informed consent. Biographical (gender, age at application, advanced degrees, prior career), undergraduate school (school, degree, research experience, publications), and medical school (school, research experience, manuscript publications, Alpha Omega Alpha membership, clerkship grades, United States Medical Licensing Examination Step 1 and 2 scores, personal statement and letter of recommendation reference to academics, couples match status) data were recorded. Listing in the Association of American Medical Colleges Faculty Online Directory and postgraduation publications were used to determine academic status. Results: There were 72 (21%) radiologists in an academic career and 264 (79%) in a nonacademic career. Variables associated with an academic career were elite undergraduate school (P=.003), undergraduate school publications (P=.018), additional advanced degrees (P=.027), elite medical school (P=.006), a research year in medical school (P<.001), and medical school publications (P<.001). A multivariate cross-validation analysis showed that these variables are jointly predictive of an academic career (P<.001). Conclusions: Undergraduate and medical school rankings and publications, as well as a medical school research year and an additional advanced degree, are associated with an academic career. Radiology residency selection committees should consider these factors in the context of the residency application if they wish to recruit future academic radiologists. © 2014 AUR.

Authors
Grimm, LJ; Shapiro, LM; Singhapricha, T; Mazurowski, MA; Desser, TS; Maxfield, CM
MLA Citation
Grimm, LJ, Shapiro, LM, Singhapricha, T, Mazurowski, MA, Desser, TS, and Maxfield, CM. "Predictors of an academic career on radiology residency applications." Academic Radiology 21.5 (January 1, 2014): 685-690.
Source
scopus
Published In
Academic Radiology
Volume
21
Issue
5
Publish Date
2014
Start Page
685
End Page
690
DOI
10.1016/j.acra.2013.10.019

Modeling resident error-making patterns in detection of mammographic masses using computer-extracted image features: Preliminary experiments

Providing high quality mammography education to radiology trainees is essential, as good interpretation skills potentially ensure the highest benefit of screening mammography for patients. We have previously proposed a computer-aided education system that utilizes trainee models, which relate human-assessed image characteristics to interpretation error. We proposed that these models be used to identify the most difficult and therefore the most educationally useful cases for each trainee. In this study, as a next step in our research, we propose to build trainee models that utilize features that are automatically extracted from images using computer vision algorithms. To predict error, we used a logistic regression which accepts imaging features as input and returns error as output. Reader data from 3 experts and 3 trainees were used. Receiver operating characteristic analysis was applied to evaluate the proposed trainee models. Our experiments showed that, for three trainees, our models were able to predict error better than chance. This is an important step in the development of adaptive computer-aided education systems since computer-extracted features will allow for faster and more extensive search of imaging databases in order to identify the most educationally beneficial cases. © 2014 SPIE.

Authors
Mazurowski, MA; Zhang, J; Lo, JY; Kuzmiak, CM; Ghate, SV; Yoon, S
MLA Citation
Mazurowski, MA, Zhang, J, Lo, JY, Kuzmiak, CM, Ghate, SV, and Yoon, S. "Modeling resident error-making patterns in detection of mammographic masses using computer-extracted image features: Preliminary experiments." January 1, 2014.
Source
scopus
Published In
Proceedings of SPIE
Volume
9037
Publish Date
2014
DOI
10.1117/12.2044404

Computer-extracted MR imaging features are associated with survival in glioblastoma patients

© 2014, Springer Science+Business Media New York.Automatic survival prognosis in glioblastoma (GBM) could result in improved treatment planning for the patient. The purpose of this research is to investigate the association of survival in GBM patients with tumor features in pre-operative magnetic resonance (MR) images assessed using a fully automatic computer algorithm. MR imaging data for 68 patients from two US institutions were used in this study. The images were obtained from the Cancer Imaging Archive. A fully automatic computer vision algorithm was applied to segment the images and extract eight imaging features from the MRI studies. The features included tumor side, proportion of enhancing tumor, proportion of necrosis, T1/FLAIR ratio, major axis length, minor axis length, tumor volume, and thickness of enhancing margin. We constructed a multivariate Cox proportional hazards regression model and used a likelihood ratio test to establish whether the imaging features are prognostic of survival. We also evaluated the individual prognostic value of each feature through multivariate analysis using the multivariate Cox model and univariate analysis using univariate Cox models for each feature. We found that the automatically extracted imaging features were predictive of survival (p = 0.031). Multivariate analysis of individual features showed that two individual features were predictive of survival: proportion of enhancing tumor (p = 0.013), and major axis length (p = 0.026). Univariate analysis indicated the same two features as significant (p = 0.021, and p = 0.017 respectively). We conclude that computer-extracted MR imaging features can be used for survival prognosis in GBM patients.

Authors
Mazurowski, MA; Zhang, J; Peters, KB; Hobbs, H
MLA Citation
Mazurowski, MA, Zhang, J, Peters, KB, and Hobbs, H. "Computer-extracted MR imaging features are associated with survival in glioblastoma patients." Journal of Neuro-Oncology 120.3 (January 1, 2014): 483-488.
Source
scopus
Published In
Journal of Neuro-Oncology
Volume
120
Issue
3
Publish Date
2014
Start Page
483
End Page
488
DOI
10.1007/s11060-014-1580-5

Imaging descriptors improve the predictive power of survival models for glioblastoma patients.

BACKGROUND: Because effective prediction of survival time can be highly beneficial for the treatment of glioblastoma patients, the relationship between survival time and multiple patient characteristics has been investigated. In this paper, we investigate whether the predictive power of a survival model based on clinical patient features improves when MRI features are also included in the model. METHODS: The subjects in this study were 82 glioblastoma patients for whom clinical features as well as MR imaging exams were made available by The Cancer Genome Atlas (TCGA) and The Cancer Imaging Archive (TCIA). Twenty-six imaging features in the available MR scans were assessed by radiologists from the TCGA Glioma Phenotype Research Group. We used multivariate Cox proportional hazards regression to construct 2 survival models: one that used 3 clinical features (age, gender, and KPS) as the covariates and 1 that used both the imaging features and the clinical features as the covariates. Then, we used 2 measures to compare the predictive performance of these 2 models: area under the receiver operating characteristic curve for the 1-year survival threshold and overall concordance index. To eliminate any positive performance estimation bias, we used leave-one-out cross-validation. RESULTS: The performance of the model based on both clinical and imaging features was higher than the performance of the model based on only the clinical features, in terms of both area under the receiver operating characteristic curve (P < .01) and the overall concordance index (P < .01). CONCLUSIONS: Imaging features assessed using a controlled lexicon have additional predictive value compared with clinical features when predicting survival time in glioblastoma patients.

Authors
Mazurowski, MA; Desjardins, A; Malof, JM
MLA Citation
Mazurowski, MA, Desjardins, A, and Malof, JM. "Imaging descriptors improve the predictive power of survival models for glioblastoma patients." Neuro Oncol 15.10 (October 2013): 1389-1394.
PMID
23396489
Source
pubmed
Published In
Neuro-Oncology
Volume
15
Issue
10
Publish Date
2013
Start Page
1389
End Page
1394
DOI
10.1093/neuonc/nos335

Estimating confidence of individual rating predictions in collaborative filtering recommender systems

Collaborative filtering algorithms predict a rating for an item based on the user's previous ratings for other items as well as ratings of other users. This approach has gained notable popularity both in academic research and in commercial applications. One aspect of collaborative filtering systems that received interest, but little systematic analysis, is confidence of the rating predictions by collaborative filtering algorithms. In this paper, I address this issue. Specifically: (1) I offer a discussion on the definition of confidence, (2) I propose a method for evaluating performance of confidence estimation algorithms, and (3) I evaluate six different confidence estimation algorithms. Three of those algorithms are introduced in this paper and three have been previously suggested for this purpose. The comparative experimental evaluation demonstrates that two of the algorithms proposed in this study: one using resampling of available ratings and one using noise injection to the available ratings provide the best performance in terms of separation between predictions of high and low confidence. The algorithms that use only the number of ratings available for the user of interest or for the item of interest turned out to be of limited use for confidence estimation. © 2012 Elsevier Ltd. All rights reserved.

Authors
Mazurowski, MA
MLA Citation
Mazurowski, MA. "Estimating confidence of individual rating predictions in collaborative filtering recommender systems." Expert Systems with Applications 40.10 (2013): 3847-3857.
Source
scival
Published In
Expert Systems with Applications
Volume
40
Issue
10
Publish Date
2013
Start Page
3847
End Page
3857
DOI
10.1016/j.eswa.2012.12.102

Difficulty of mammographic cases in the context of resident training: Preliminary experimental data

We are currently developing an intelligent data-driven educational system for mammography. Since our system attempts to predict which cases will be difficult for the trainees, it is important to better understand the concept of case difficulty. While the concept of difficulty is central to our efforts on adaptive education, its importance extends to radiology education in general as well as to image perception research. In this study, we tested some hypotheses that related to difficulty. Specifically, we performed a preliminary reader study to evaluate relationship between the error rate (an objective measure of difficulty), individual assessment of case difficulty by a resident and expert's assessment of case difficulty (two subjective measures of difficulty). Furthermore, we investigated the relationship between individual and expert's assessment of difficulty and time that the residents took to interpret the case. Time taken to interpret a case by a resident related well with the individual assessment of difficulty but its relationship with the expert's assessment of difficulty was weaker. The analysis of the difficulty assessments showed that an increase in individual assessment of difficulty made by a resident relates well to an increase in his/her false positive errors but not to an increase in false negative errors. Interestingly, the expert's assessment of difficulty was related to false negative errors in the trainees but not to false positive errors. These results offer additional guidance in our efforts to construct an adaptive education system as well as provide insight into important aspects of radiology education in general. © 2013 SPIE.

Authors
Mazurowski, MA
MLA Citation
Mazurowski, MA. "Difficulty of mammographic cases in the context of resident training: Preliminary experimental data." Proceedings of SPIE - The International Society for Optical Engineering 8673 (2013).
Source
scival
Published In
Proceedings of SPIE - The International Society for Optical Engineering
Volume
8673
Publish Date
2013
DOI
10.1117/12.2008550

Identifying error-making patterns in assessment of mammographic BI-RADS descriptors among radiology residents using statistical pattern recognition.

RATIONALE AND OBJECTIVE: The objective of this study is to test the hypothesis that there are patterns in erroneous assessment of BI-RADS features among radiology trainees when interpreting mammographic masses and that these patterns can be captured in individualized statistical user models. Identifying these patterns could be useful in personalizing and adapting educational material to complement the individual weaknesses of each trainee during his or her mammography education. MATERIALS AND METHODS: Reading data of 33 mammographic cases containing masses was used. The cases were individually described by 10 radiology residents using four BI-RADS features: mass shape, mass margin, mass density and parenchyma density. For each resident, an individual model was automatically constructed that predicts likelihood (HIGH or LOW) of erroneously assigning each BI-RADS descriptor by the resident. Error was defined as deviation of the resident's assessment from the expert assessments. We evaluated the predictive performance of the models using leave-one-out crossvalidation. RESULTS: The user models were able to predict which assessments have higher likelihood of error. The proportion of actual errors to the number of situations in which these errors could potentially occur was significantly higher (P < .05) when user-model assigned HIGH likelihood of error than when LOW likelihood of error was assigned for three of the four BI-RADS features. Overall, the difference between the HIGH and LOW likelihood of error groups was statistically significant (P < .0001) combining all four features. CONCLUSION: Error making in BI-RADS descriptor assessment appears to follow patterns that can be captured with statistical pattern recognition-based user models.

Authors
Mazurowski, MA; Barnhart, HX; Baker, JA; Tourassi, GD
MLA Citation
Mazurowski, MA, Barnhart, HX, Baker, JA, and Tourassi, GD. "Identifying error-making patterns in assessment of mammographic BI-RADS descriptors among radiology residents using statistical pattern recognition." Acad Radiol 19.7 (July 2012): 865-871.
PMID
22459643
Source
pubmed
Published In
Academic Radiology
Volume
19
Issue
7
Publish Date
2012
Start Page
865
End Page
871
DOI
10.1016/j.acra.2012.01.012

The effect of class imbalance on case selection for case-based classifiers: an empirical study in the context of medical decision support.

Case selection is a useful approach for increasing the efficiency and performance of case-based classifiers. Multiple techniques have been designed to perform case selection. This paper empirically investigates how class imbalance in the available set of training cases can impact the performance of the resulting classifier as well as properties of the selected set. In this study, the experiments are performed using a dataset for the problem of detecting breast masses in screening mammograms. The classification problem was binary and we used a k-nearest neighbor classifier. The classifier's performance was evaluated using the receiver operating characteristic (ROC) area under the curve (AUC) measure. The experimental results indicate that although class imbalance reduces the performance of the derived classifier and the effectiveness of selection at improving overall classifier performance, case selection can still be beneficial, regardless of the level of class imbalance.

Authors
Malof, JM; Mazurowski, MA; Tourassi, GD
MLA Citation
Malof, JM, Mazurowski, MA, and Tourassi, GD. "The effect of class imbalance on case selection for case-based classifiers: an empirical study in the context of medical decision support." Neural Netw 25.1 (January 2012): 141-145.
PMID
21820273
Source
pubmed
Published In
Neural Networks
Volume
25
Issue
1
Publish Date
2012
Start Page
141
End Page
145
DOI
10.1016/j.neunet.2011.07.002

Mutual information-based template matching scheme for detection of breast masses: from mammography to digital breast tomosynthesis.

Development of a computational decision aid for a new medical imaging modality typically is a long and complicated process. It consists of collecting data in the form of images and annotations, development of image processing and pattern recognition algorithms for analysis of the new images and finally testing of the resulting system. Since new imaging modalities are developed more rapidly than ever before, any effort for decreasing the time and cost of this development process could result in maximizing the benefit of the new imaging modality to patients by making the computer aids quickly available to radiologists that interpret the images. In this paper, we make a step in this direction and investigate the possibility of translating the knowledge about the detection problem from one imaging modality to another. Specifically, we present a computer-aided detection (CAD) system for mammographic masses that uses a mutual information-based template matching scheme with intelligently selected templates. We presented principles of template matching with mutual information for mammography before. In this paper, we present an implementation of those principles in a complete computer-aided detection system. The proposed system, through an automatic optimization process, chooses the most useful templates (mammographic regions of interest) using a large database of previously collected and annotated mammograms. Through this process, the knowledge about the task of detecting masses in mammograms is incorporated in the system. Then, we evaluate whether our system developed for screen-film mammograms can be successfully applied not only to other mammograms but also to digital breast tomosynthesis (DBT) reconstructed slices without adding any DBT cases for training. Our rationale is that since mutual information is known to be a robust inter-modality image similarity measure, it has high potential of transferring knowledge between modalities in the context of the mass detection task. Experimental evaluation of the system on mammograms showed competitive performance compared to other mammography CAD systems recently published in the literature. When the system was applied "as-is" to DBT, its performance was notably worse than that for mammograms. However, with a simple additional preprocessing step, the performance of the system reached levels similar to that obtained for mammograms. In conclusion, the presented CAD system not only performed competitively on screen-film mammograms but it also performed robustly on DBT showing that direct transfer of knowledge across breast imaging modalities for mass detection is in fact possible.

Authors
Mazurowski, MA; Lo, JY; Harrawood, BP; Tourassi, GD
MLA Citation
Mazurowski, MA, Lo, JY, Harrawood, BP, and Tourassi, GD. "Mutual information-based template matching scheme for detection of breast masses: from mammography to digital breast tomosynthesis." J Biomed Inform 44.5 (October 2011): 815-823.
PMID
21554985
Source
pubmed
Published In
Journal of Biomedical Informatics
Volume
44
Issue
5
Publish Date
2011
Start Page
815
End Page
823
DOI
10.1016/j.jbi.2011.04.008

Comparative analysis of instance selection algorithms for instance-based classifiers in the context of medical decision support.

When constructing a pattern classifier, it is important to make best use of the instances (a.k.a. cases, examples, patterns or prototypes) available for its development. In this paper we present an extensive comparative analysis of algorithms that, given a pool of previously acquired instances, attempt to select those that will be the most effective to construct an instance-based classifier in terms of classification performance, time efficiency and storage requirements. We evaluate seven previously proposed instance selection algorithms and compare their performance to simple random selection of instances. We perform the evaluation using k-nearest neighbor classifier and three classification problems: one with simulated Gaussian data and two based on clinical databases for breast cancer detection and diagnosis, respectively. Finally, we evaluate the impact of the number of instances available for selection on the performance of the selection algorithms and conduct initial analysis of the selected instances. The experiments show that for all investigated classification problems, it was possible to reduce the size of the original development dataset to less than 3% of its initial size while maintaining or improving the classification performance. Random mutation hill climbing emerges as the superior selection algorithm. Furthermore, we show that some previously proposed algorithms perform worse than random selection. Regarding the impact of the number of instances available for the classifier development on the performance of the selection algorithms, we confirm that the selection algorithms are generally more effective as the pool of available instances increases. In conclusion, instance selection is generally beneficial for instance-based classifiers as it can improve their performance, reduce their storage requirements and improve their response time. However, choosing the right selection algorithm is crucial.

Authors
Mazurowski, MA; Malof, JM; Tourassi, GD
MLA Citation
Mazurowski, MA, Malof, JM, and Tourassi, GD. "Comparative analysis of instance selection algorithms for instance-based classifiers in the context of medical decision support." Phys Med Biol 56.2 (January 21, 2011): 473-489.
PMID
21191152
Source
pubmed
Published In
Physics in Medicine and Biology
Volume
56
Issue
2
Publish Date
2011
Start Page
473
End Page
489
DOI
10.1088/0031-9155/56/2/012

Exploring the potential of collaborative filtering for user-adaptive mammography education

Specialized training in breast imaging is critical to ensure high diagnostic accuracy of the radiologists who read screening mammograms in their daily practice. Previously, we proposed a framework for an individualized computer-aided mammography training system as a time-efficient and effective support for radiology education. The system utilizes the concept of user modeling to adapt the training protocol to meet the individual needs of the radiologists-in-training. User models are derived to predict the difficulty that a previously unseen case will pose to the modeled user. Constructing accurate models of the trainees is crucial for the overall effectiveness of the proposed training. In this paper we explore the potential of collaborative filtering for this task. Collaborative filtering is based on the assumption that the relation between ratings of different users or between ratings of different items observed for previous items will translate to new items and users. In the context of radiology trainee modeling we use this approach to predict errors that the trainees will make for unseen cases. These predicted errors can serve as the basis to identify challenging cases that are expected to be more beneficial when included in the training of a given trainee. We performed an experimental evaluation of the algorithm using data collected at Duke University Medical Center from 10 radiology residents for the problem of determining the malignancy status of masses based on their mammographic appearance. Our experiments showed that the collaborative filtering algorithm is able to distinguish cases of high and low difficulty and therefore demonstrated the promise of this approach in building adaptive computer-aided educational systems in radiology education. © 2011 IEEE.

Authors
Mazurowski, MA; Tourassi, GD
MLA Citation
Mazurowski, MA, and Tourassi, GD. "Exploring the potential of collaborative filtering for user-adaptive mammography education." Proceedings of the 2011 Biomedical Sciences and Engineering Conference: Image Informatics and Analytics in Biomedicine, BSEC 2011 (2011).
Source
scival
Published In
Proceedings of the 2011 Biomedical Sciences and Engineering Conference: Image Informatics and Analytics in Biomedicine, BSEC 2011
Publish Date
2011
DOI
10.1109/BSEC.2011.5872325

Modeling error in assessment of mammographic image features for improved computer-aided mammography training: Initial experience

In this study we investigate the hypothesis that there exist patterns in erroneous assessment of BI-RADS image features among radiology trainees when performing diagnostic interpretation of mammograms. We also investigate whether these error making patterns can be captured by individual user models. To test our hypothesis we propose a user modeling algorithm that uses the previous readings of a trainee to identify whether certain BI-RADS feature values (e.g. "spiculated" value for "margin" feature) are associated with higher than usual likelihood that the feature will be assessed incorrectly. In our experiments we used readings of 3 radiology residents and 7 breast imaging experts for 33 breast masses for the following BI-RADS features: parenchyma density, mass margin, mass shape and mass density. The expert readings were considered as the gold standard. Rule-based individual user models were developed and tested using the leave one-one-out crossvalidation scheme. Our experimental evaluation showed that the individual user models are accurate in identifying cases for which errors are more likely to be made. The user models captured regularities in error making for all 3 residents. This finding supports our hypothesis about existence of individual error making patterns in assessment of mammographic image features using the BI-RADS lexicon. Explicit user models identifying the weaknesses of each resident could be of great use when developing and adapting a personalized training plan to meet the resident's individual needs. Such approach fits well with the framework of adaptive computer-aided educational systems in mammography we have proposed before. © 2011 SPIE.

Authors
Mazurowski, MA; Tourassi, GD
MLA Citation
Mazurowski, MA, and Tourassi, GD. "Modeling error in assessment of mammographic image features for improved computer-aided mammography training: Initial experience." Progress in Biomedical Optics and Imaging - Proceedings of SPIE 7966 (2011).
Source
scival
Published In
Proceedings of SPIE
Volume
7966
Publish Date
2011
DOI
10.1117/12.878737

Exploring the potential of context-sensitive CADe in screening mammography.

PURPOSE: Conventional computer-assisted detection (CADe) systems in screening mammography provide the same decision support to all users. The aim of this study was to investigate the potential of a context-sensitive CADe system which provides decision support guided by each user's focus of attention during visual search and reporting patterns for a specific case. METHODS: An observer study for the detection of malignant masses in screening mammograms was conducted in which six radiologists evaluated 20 mammograms while wearing an eye-tracking device. Eye-position data and diagnostic decisions were collected for each radiologist and case they reviewed. These cases were subsequently analyzed with an in-house knowledge-based CADe system using two different modes: Conventional mode with a globally fixed decision threshold and context-sensitive mode with a location-variable decision threshold based on the radiologists' eye dwelling data and reporting information. RESULTS: The CADe system operating in conventional mode had 85.7% per-image malignant mass sensitivity at 3.15 false positives per image (FPsI). The same system operating in context-sensitive mode provided personalized decision support at 85.7%-100% sensitivity and 0.35-0.40 FPsI to all six radiologists. Furthermore, context-sensitive CADe system could improve the radiologists' sensitivity and reduce their performance gap more effectively than conventional CADe. CONCLUSIONS: Context-sensitive CADe support shows promise in delineating and reducing the radiologists' perceptual and cognitive errors in the diagnostic interpretation of screening mammograms more effectively than conventional CADe.

Authors
Tourassi, GD; Mazurowski, MA; Harrawood, BP; Krupinski, EA
MLA Citation
Tourassi, GD, Mazurowski, MA, Harrawood, BP, and Krupinski, EA. "Exploring the potential of context-sensitive CADe in screening mammography." Med Phys 37.11 (November 2010): 5728-5736.
PMID
21158284
Source
pubmed
Published In
Medical physics
Volume
37
Issue
11
Publish Date
2010
Start Page
5728
End Page
5736
DOI
10.1118/1.3501882

Individualized computer-aided education in mammography based on user modeling: concept and preliminary experiments.

PURPOSE: The authors propose the framework for an individualized adaptive computer-aided educational system in mammography that is based on user modeling. The underlying hypothesis is that user models can be developed to capture the individual error making patterns of radiologists-in-training. In this pilot study, the authors test the above hypothesis for the task of breast cancer diagnosis in mammograms. METHODS: The concept of a user model was formalized as the function that relates image features to the likelihood/extent of the diagnostic error made by a radiologist-in-training and therefore to the level of difficulty that a case will pose to the radiologist-in-training (or "user"). Then, machine learning algorithms were implemented to build such user models. Specifically, the authors explored k-nearest neighbor, artificial neural networks, and multiple regression for the task of building the model using observer data collected from ten Radiology residents at Duke University Medical Center for the problem of breast mass diagnosis in mammograms. For each resident, a user-specific model was constructed that predicts the user's expected level of difficulty for each presented case based on two BI-RADS image features. In the experiments, leave-one-out data handling scheme was applied to assign each case to a low-predicted-difficulty or a high-predicted-difficulty group for each resident based on each of the three user models. To evaluate whether the user model is useful in predicting difficulty, the authors performed statistical tests using the generalized estimating equations approach to determine whether the mean actual error is the same or not between the low-predicted-difficulty group and the high-predicted-difficulty group. RESULTS: When the results for all observers were pulled together, the actual errors made by residents were statistically significantly higher for cases in the high-predicted-difficulty group than for cases in the low-predicted-difficulty group for all modeling algorithms (p < or = 0.002 for all methods). This indicates that the user models were able to accurately predict difficulty level of the analyzed cases. Furthermore, the authors determined that among the two BI-RADS features that were used in this study, mass margin was the most useful in predicting individual user errors. CONCLUSIONS: The pilot study shows promise for developing individual user models that can accurately predict the level of difficulty that each case will pose to the radiologist-in-training. These models could allow for constructing adaptive computer-aided educational systems in mammography.

Authors
Mazurowski, MA; Baker, JA; Barnhart, HX; Tourassi, GD
MLA Citation
Mazurowski, MA, Baker, JA, Barnhart, HX, and Tourassi, GD. "Individualized computer-aided education in mammography based on user modeling: concept and preliminary experiments." Med Phys 37.3 (March 2010): 1152-1160.
PMID
20384251
Source
pubmed
Published In
Medical physics
Volume
37
Issue
3
Publish Date
2010
Start Page
1152
End Page
1160
DOI
10.1118/1.3301575

User modeling for improved computer-aided training in radiology: Initial experience

Although mammography is an efficient screening modality for breast cancer, interpretation of mammographic images is a difficult task and notable variability between radiologists performance has been documented. A significant factor impacting radiologists diagnostic performance is adequate training. In this study we propose a new paradigm for computer-assisted training in radiology based on constructing user models for radiologists-in-training that capture individual error making patterns. Such user models are developed and trained to use image features for prediction of the extent of error made by a particular radiologist for variety of cases and therefore estimate difficulty of different types of cases for that radiologist. The constructed user model can be used to develop a personalized training protocol for the radiologist-in-training that focuses on cases that may pose a particular difficulty to the trainee. We initially demonstrate the concept of building individual user models for the task of breast mass diagnosis. Data collected from three resident observers at Duke University was used for the experiments. The result indicate that the proposed models are capable of learning to distinguish difficult and easy cases for each observer with moderate accuracy which shows promise for the proposed concept. © 2010 Copyright SPIE - The International Society for Optical Engineering.

Authors
Mazurowski, MA; Lo, JY; Tourassi, GD
MLA Citation
Mazurowski, MA, Lo, JY, and Tourassi, GD. "User modeling for improved computer-aided training in radiology: Initial experience." 2010.
Source
scival
Published In
Proceedings of SPIE
Volume
7627
Publish Date
2010
DOI
10.1117/12.843863

Perception-Driven IT-CADe Analysis for the Detection of Masses in Screening Mammography: Initial Investigation

Authors
Tourassi, GD; Mazurowski, MA; Krupinski, EA
MLA Citation
Tourassi, GD, Mazurowski, MA, and Krupinski, EA. "Perception-Driven IT-CADe Analysis for the Detection of Masses in Screening Mammography: Initial Investigation." 2010.
Source
wos-lite
Published In
Proceedings of SPIE - The International Society for Optical Engineering
Volume
7624
Publish Date
2010
DOI
10.1117/12.845495

An adaptive incremental approach to constructing ensemble classifiers: application in an information-theoretic computer-aided decision system for detection of masses in mammograms.

Ensemble classifiers have been shown efficient in multiple applications. In this article, the authors explore the effectiveness of ensemble classifiers in a case-based computer-aided diagnosis system for detection of masses in mammograms. They evaluate two general ways of constructing subclassifiers by resampling of the available development dataset: Random division and random selection. Furthermore, they discuss the problem of selecting the ensemble size and propose two adaptive incremental techniques that automatically select the size for the problem at hand. All the techniques are evaluated with respect to a previously proposed information-theoretic CAD system (IT-CAD). The experimental results show that the examined ensemble techniques provide a statistically significant improvement (AUC = 0.905 +/- 0.024) in performance as compared to the original IT-CAD system (AUC = 0.865 +/- 0.029). Some of the techniques allow for a notable reduction in the total number of examples stored in the case base (to 1.3% of the original size), which, in turn, results in lower storage requirements and a shorter response time of the system. Among the methods examined in this article, the two proposed adaptive techniques are by far the most effective for this purpose. Furthermore, the authors provide some discussion and guidance for choosing the ensemble parameters.

Authors
Mazurowski, MA; Zurada, JM; Tourassi, GD
MLA Citation
Mazurowski, MA, Zurada, JM, and Tourassi, GD. "An adaptive incremental approach to constructing ensemble classifiers: application in an information-theoretic computer-aided decision system for detection of masses in mammograms." Med Phys 36.7 (July 2009): 2976-2984.
PMID
19673196
Source
pubmed
Published In
Medical physics
Volume
36
Issue
7
Publish Date
2009
Start Page
2976
End Page
2984
DOI
10.1118/1.3132304

The effect of class imbalance on case selection for case-based classifiers, with emphasis on computer-aided diagnosis systems

In this paper the effect of class imbalance in the case base of a case-based classifier is investigated as it pertains to case base reduction and the resulting classifier performance. A k-nearest neighbor algorithm is used as a classifier and the Random Mutation Hill Climbing (RMHC) algorithm is used for case base reduction. The effects at various levels of positive class prevalence are tested in a binary classification problem. The results indicate that class imbalance is detrimental to both case base reduction and classifier performance. Selection with RMHC generally improves the classification performance regardless of the case base prevalence. ©2009 IEEE.

Authors
Malof, JM; Mazurowski, MA; Tourassi, GD
MLA Citation
Malof, JM, Mazurowski, MA, and Tourassi, GD. "The effect of class imbalance on case selection for case-based classifiers, with emphasis on computer-aided diagnosis systems." Proceedings of the International Joint Conference on Neural Networks (2009): 1975-1980.
Source
scival
Published In
Proceedings of the International Joint Conference on Neural Networks
Publish Date
2009
Start Page
1975
End Page
1980
DOI
10.1109/IJCNN.2009.5178759

Evaluating classifiers: Relation between area under the receiver operator characteristic curve and overall accuracy

In this study, we investigated the relation between two popular classifier performance measures: area under the receiver operator characteristic curve and overall accuracy. We also evaluated the impact of class imbalance and number of examples in test set on this relation. We perform a set of experiments in which we train multiple neural networks and test them in various, well controlled conditions. The experimental results show that given a large and balanced test set, increase in one performance measure is a very good indicator of increase in the other measure. Furthermore increasing the total number of examples, while keeping the positive class prevalence constant generally increases the correlation between the two measures. Our results also indicate that increasing the extent of class imbalance in the test set has a detrimental effect on this correlation. ©2009 IEEE.

Authors
Mazurowski, MA; Tourassi, GD
MLA Citation
Mazurowski, MA, and Tourassi, GD. "Evaluating classifiers: Relation between area under the receiver operator characteristic curve and overall accuracy." Proceedings of the International Joint Conference on Neural Networks (2009): 2045-2049.
Source
scival
Published In
Proceedings of the International Joint Conference on Neural Networks
Publish Date
2009
Start Page
2045
End Page
2049
DOI
10.1109/IJCNN.2009.5178752

Building virtual community in computational intelligence and machine learning

Researchers are making efforts to build a virtual community In computational intelligence (CI) and machine learning (ML). A virtual organization is a group of geographically distributed individuals or institutions that cooperate with each other concurrently. These organizations have become feasible and more convenient than traditional forms of organizations due to the introduction of information technologies to help facilitate them. A number of significant factors drive the development and establishment of these organizations. These organizations are being established to help CI and ML researchers review research papers frequently. These organizations help in interacting with other researchers, quickly obtain relevant reference material, or find programs that are easily adaptable to the project. These organizations are playing a key role in solving many problems in the fields of medicine, genomics, earth sciences, and some engineering areas.

Authors
Zurada, JM; Mazurowski, MA; Ragade, R; Abdullin, A; Wojtudiak, J; Gentle, J
MLA Citation
Zurada, JM, Mazurowski, MA, Ragade, R, Abdullin, A, Wojtudiak, J, and Gentle, J. "Building virtual community in computational intelligence and machine learning." IEEE Computational Intelligence Magazine 4.1 (2009): 43-46+54.
Source
scival
Published In
IEEE Computational Intelligence Magazine
Volume
4
Issue
1
Publish Date
2009
Start Page
43
End Page
46+54
DOI
10.1109/MCI.2008.930986

Relational representation for improved decisions with an information-theoretic CADe system: Initial experience

Our previously presented information-theoretic computer-aided detection (IT-CADe) system for distinguishing masses and normal parenchyma in mammograms is an example of a case-based system. IT-CAD makes decisions by evaluating the querys average similarity with known mass and normal examples stored in the systems case base. Pairwise case similarity is measured in terms of their normalized mutual information. The purpose of this study was to evaluate whether incorporating a new machine learning concept of relational representation to IT-CAD is a more effective strategy than the decision algorithm that is currently in place. A trainable relational representation classifier builds a decision rule using the relational representation of cases. Instead of describing a case by a vector of intrinsic features, the case is described by its NMI-based similarity to a set of known examples. For this study, we first applied random mutation hill climbing algorithm to select the concise set of knowledge cases and then we applied a support vector machine to derive a decision rule using the relational representation of cases. We performed the study with a database of 600 mammographic regions of interest (300 with masses and 300 with normal parenchyma). Our experiments indicate that incorporating the concept of relational representation with a trainable classifier to IT-CAD provides an improvement in performance as compared with the original decision rule. Therefore, relational representation is a promising strategy for IT-CADe. © 2009 SPIE.

Authors
Mazurowski, MA; Tourassi, GD
MLA Citation
Mazurowski, MA, and Tourassi, GD. "Relational representation for improved decisions with an information-theoretic CADe system: Initial experience." Progress in Biomedical Optics and Imaging - Proceedings of SPIE 7260 (2009).
Source
scival
Published In
Proceedings of SPIE
Volume
7260
Publish Date
2009
DOI
10.1117/12.812965

A comparative study of database reduction methods for case-based computer-aided detection systems: Preliminary results

In case-based computer-aided decision systems (CB-CAD) a query case is compared to known examples stored in the systems case base (also called a reference library). These systems offer competitive classification performance and are easy to expand. However, they also require efficient management of the case base. As CB-CAD systems are becoming more popular, the problem of case base optimization has recently attracted interest among CAD researchers. In this paper we present preliminary results of a study comparing several case base reduction techniques. We implemented six techniques previously proposed in machine learning literature and applied it to the classification problem of distinguishing masses and normal tissue in mammographic regions of interest. The results show that the random mutation hill climbing technique offers a drastic reduction of the number of case base examples while providing a significant improvement in classification performance. Random selection allowed for reduction of the case base to 30% without notable decrease in performance. The remaining techniques (i.e., condensed nearest neighbor, reduced nearest neighbor, edited nearest neighbor, and All k-NN) resulted in moderate reduction (to 50-70% of the original size) at the cost of decrease in CB-CAD performance.©2009 SPIE.

Authors
Mazurowski, MA; Malof, JM; Zurada, JM; Tourassi, GD
MLA Citation
Mazurowski, MA, Malof, JM, Zurada, JM, and Tourassi, GD. "A comparative study of database reduction methods for case-based computer-aided detection systems: Preliminary results." Progress in Biomedical Optics and Imaging - Proceedings of SPIE 7260 (2009).
Source
scival
Published In
Proceedings of SPIE
Volume
7260
Publish Date
2009
DOI
10.1117/12.812442

Selection of examples in case-based computer-aided decision systems.

Case-based computer-aided decision (CB-CAD) systems rely on a database of previously stored, known examples when classifying new, incoming queries. Such systems can be particularly useful since they do not need retraining every time a new example is deposited in the case base. The adaptive nature of case-based systems is well suited to the current trend of continuously expanding digital databases in the medical domain. To maintain efficiency, however, such systems need sophisticated strategies to effectively manage the available evidence database. In this paper, we discuss the general problem of building an evidence database by selecting the most useful examples to store while satisfying existing storage requirements. We evaluate three intelligent techniques for this purpose: genetic algorithm-based selection, greedy selection and random mutation hill climbing. These techniques are compared to a random selection strategy used as the baseline. The study is performed with a previously presented CB-CAD system applied for false positive reduction in screening mammograms. The experimental evaluation shows that when the development goal is to maximize the system's diagnostic performance, the intelligent techniques are able to reduce the size of the evidence database to 37% of the original database by eliminating superfluous and/or detrimental examples while at the same time significantly improving the CAD system's performance. Furthermore, if the case-base size is a main concern, the total number of examples stored in the system can be reduced to only 2-4% of the original database without a decrease in the diagnostic performance. Comparison of the techniques shows that random mutation hill climbing provides the best balance between the diagnostic performance and computational efficiency when building the evidence database of the CB-CAD system.

Authors
Mazurowski, MA; Zurada, JM; Tourassi, GD
MLA Citation
Mazurowski, MA, Zurada, JM, and Tourassi, GD. "Selection of examples in case-based computer-aided decision systems." Phys Med Biol 53.21 (November 7, 2008): 6079-6096.
PMID
18854606
Source
pubmed
Published In
Physics in Medicine and Biology
Volume
53
Issue
21
Publish Date
2008
Start Page
6079
End Page
6096
DOI
10.1088/0031-9155/53/21/013

Training neural network classifiers for medical decision making: the effects of imbalanced datasets on classification performance.

This study investigates the effect of class imbalance in training data when developing neural network classifiers for computer-aided medical diagnosis. The investigation is performed in the presence of other characteristics that are typical among medical data, namely small training sample size, large number of features, and correlations between features. Two methods of neural network training are explored: classical backpropagation (BP) and particle swarm optimization (PSO) with clinically relevant training criteria. An experimental study is performed using simulated data and the conclusions are further validated on real clinical data for breast cancer diagnosis. The results show that classifier performance deteriorates with even modest class imbalance in the training data. Further, it is shown that BP is generally preferable over PSO for imbalanced training data especially with small data sample and large number of features. Finally, it is shown that there is no clear preference between oversampling and no compensation approach and some guidance is provided regarding a proper selection.

Authors
Mazurowski, MA; Habas, PA; Zurada, JM; Lo, JY; Baker, JA; Tourassi, GD
MLA Citation
Mazurowski, MA, Habas, PA, Zurada, JM, Lo, JY, Baker, JA, and Tourassi, GD. "Training neural network classifiers for medical decision making: the effects of imbalanced datasets on classification performance." Neural Netw 21.2-3 (March 2008): 427-436.
PMID
18272329
Source
pubmed
Published In
Neural Networks
Volume
21
Issue
2-3
Publish Date
2008
Start Page
427
End Page
436
DOI
10.1016/j.neunet.2007.12.031

Decision optimization of case-based computer-aided decision systems using genetic algorithms with application to mammography.

This paper presents an optimization framework for improving case-based computer-aided decision (CB-CAD) systems. The underlying hypothesis of the study is that each example in the knowledge database of a medical decision support system has different importance in the decision making process. A new decision algorithm incorporating an importance weight for each example is proposed to account for these differences. The search for the best set of importance weights is defined as an optimization problem and a genetic algorithm is employed to solve it. The optimization process is tailored to maximize the system's performance according to clinically relevant evaluation criteria. The study was performed using a CAD system developed for the classification of regions of interests (ROIs) in mammograms as depicting masses or normal tissue. The system was constructed and evaluated using a dataset of ROIs extracted from the Digital Database for Screening Mammography (DDSM). Experimental results show that, according to receiver operator characteristic (ROC) analysis, the proposed method significantly improves the overall performance of the CAD system as well as its average specificity for high breast mass detection rates.

Authors
Mazurowski, MA; Habas, PA; Zurada, JM; Tourassi, GD
MLA Citation
Mazurowski, MA, Habas, PA, Zurada, JM, and Tourassi, GD. "Decision optimization of case-based computer-aided decision systems using genetic algorithms with application to mammography." Phys Med Biol 53.4 (February 21, 2008): 895-908.
PMID
18263947
Source
pubmed
Published In
Physics in Medicine and Biology
Volume
53
Issue
4
Publish Date
2008
Start Page
895
End Page
908
DOI
10.1088/0031-9155/53/4/005

Learning in networks: Complex-valued neurons, pruning, and rule extraction

This paper focuses on neural networks with complex-valued (CV) neurons as well as on selected aspects of neural networks learning, pruning and rule extraction. CV neurons can be used as versatile substitutes in real-valued perceptron networks. Learning of CV layers is discussed in context of traditional, multilayer feedforward architecture. Such learning is derivative-free and it usually requires networks of reduced size. Selected exampies and applications of CV-networks in bioinformatics and pattern recognition are discussed. The paper also covers specialized learning techniques for logic rule extraction. Such techniques include learning with pruning, and can be used in expert systems, and other applications that rely on models developed to fit measured data. © 2008 IEEE.

Authors
Zurada, JM; Aizenberg, I; Mazurowski, MA
MLA Citation
Zurada, JM, Aizenberg, I, and Mazurowski, MA. "Learning in networks: Complex-valued neurons, pruning, and rule extraction." 2008 4th International IEEE Conference Intelligent Systems, IS 2008 1 (2008): 115-120.
Source
scival
Published In
2008 4th International IEEE Conference Intelligent Systems, IS 2008
Volume
1
Publish Date
2008
Start Page
115
End Page
120
DOI
10.1109/IS.2008.4670394

Reliability assessment of ensemble classifiers: Application in mammography

In classifier ensembles predictions of different classifiers regarding a query are combined into one final decision. It was previously shown that using ensemble techniques can significantly improve classification performance. In this study we build upon this result and propose to use variability in the predictions of classifiers contributing to the final decision as an indicator of its reliability. The study hypothesis is tested with respect to previously proposed information-theoretic computer-aided decision (IT-CAD) system for detection of masses in mammograms. A database of 1820 regions of interest (ROIs) extracted from digital database of screening mammography (DDSM) is used. Experimental results show that the proposed reliability assessment successfully identifies decisions that can not be trusted. Further, a low correlation between reliability and the classifier output is noted. This opens a possibility of combining reliability and ensemble output into one improved decision. © 2008 Springer-Verlag Berlin Heidelberg.

Authors
Mazurowski, MA; Zurada, JM; Tourassi, GD
MLA Citation
Mazurowski, MA, Zurada, JM, and Tourassi, GD. "Reliability assessment of ensemble classifiers: Application in mammography." Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 5116 LNCS (2008): 366-370.
Source
scival
Published In
Lecture notes in computer science
Volume
5116 LNCS
Publish Date
2008
Start Page
366
End Page
370
DOI
10.1007/978-3-540-70538-3_51

Computational intelligence virtual community: Framework and implementation issues

This paper discusses the framework for virtual collaborative environment for researchers, practitioners, users and learners in the areas of computational intelligence and machine learning (CIML) that is currently developed by our group. It also outlines main features of the community portal under construction that will support communication and sharing of computational resources. In particular, selected aspects of structure of the portal such as common formats of data, models, software, publications and software documentation are discussed. The preliminary portal is available at UKL: www.cimlcommunity.org. © 2008 IEEE.

Authors
Zurada, JM; Wojtusiak, J; Chowdhury, F; Gentle, JE; Jeannot, CJ; Mazurowski, MA
MLA Citation
Zurada, JM, Wojtusiak, J, Chowdhury, F, Gentle, JE, Jeannot, CJ, and Mazurowski, MA. "Computational intelligence virtual community: Framework and implementation issues." Proceedings of the International Joint Conference on Neural Networks (2008): 3153-3157.
Source
scival
Published In
Proceedings of the International Joint Conference on Neural Networks
Publish Date
2008
Start Page
3153
End Page
3157
DOI
10.1109/IJCNN.2008.4634244

Database decomposition of a knowledge-based CAD system in mammography; An ensemble approach to improve detection

Although ensemble techniques have been investigated in supervised machine learning, their potential with knowledge-based systems is unexplored. The purpose of this study is to investigate the ensemble approach with a knowledge-based (KB) CAD system for the detection of masses in screening mammograms. The system is designed to determine the presence of a mass in a query mammographic region of interest (ROI) based on its similarity with previously acquired examples of mass and normal cases. Similarity between images is assessed using normalized mutual information. Two different approaches of knowledge database decomposition were investigated to create the ensemble. The first approach was random division of the knowledge database into a pre-specified number of equal size, separate groups. The second approach was based on k-means clustering of the knowledge cases according to common texture features extracted from the ROIs. The ensemble components were fused using a linear classifier. Based on a database of 1820 ROIs (901 masses and 919 and the leave-one-out crossvalidation scheme, the ensemble techniques improved the performance of the original KB-CAD system (Az = 0.86±0.01). Specifically, random division resulted in ROC area index of Az = 0.90 ± 0.01 while k-means clustering provided further improvement (A z = 0.91 ± 0.01). Although marginally better, the improvement was statistically significant. The superiority of the k-means clustering scheme was robust regardless of the number of clusters. This study supports the idea of incorporation of ensemble techniques with knowledge-based systems in mammography.

Authors
Mazurowski, MA; Zurada, JM; Tourassi, GD
MLA Citation
Mazurowski, MA, Zurada, JM, and Tourassi, GD. "Database decomposition of a knowledge-based CAD system in mammography; An ensemble approach to improve detection." Progress in Biomedical Optics and Imaging - Proceedings of SPIE 6915 (2008).
Source
scival
Published In
Proceedings of SPIE
Volume
6915
Publish Date
2008
DOI
10.1117/12.771556

Toward perceptually driven image retrieval in mammography: A pilot observer study to assess visual similarity of masses

Development of a fully automated system retrieving visually similar images is a task that could be helpful as the basis of a computer-assisted diagnostic (CADx) tool in mammography. Our study aims at a better understanding of the concept of visual similarity as it pertains to mammographic masses. Such understanding is a necessary step for building effective perceptually-driven image retrieval systems. In our study we deconstruct the concept of visual mass similarity into three components: similarity of size, similarity of shape, and similarity of margin. We present the results of a pilot observer study to determine the importance of each component when human observers assess the overall similarity of two masses. Seven observers of various expertise participated in the study: 1 highly experienced mammographer, 1 expert in visual perception, 3 CAD researchers, and 2 novices. Each observer assessed the similarity between 100 pairs of mammographic regions of interest (ROIs) depicting benign and malignant masses. Visual similarity was assessed in four categories (shape, size, margin, overall) using a web-based interface and a 10-point rating scale. Preliminary analysis of the results suggests the following. First, there is a moderate agreement between observers in similarity assessment for all mentioned categories. Second, all components substantially affect the overall similarity rating, with mass margin having the highest significance and mass size having the lowest significance relatively to the other factors. These findings varied somewhat based on the observer's expertise. Third, some low-level morphological features extracted from the masses can be used to mimic the overall visual similarity ratings and its specific components.

Authors
Mazurowski, MA; Harrawood, BP; Zurada, JM; Tourassi, GD
MLA Citation
Mazurowski, MA, Harrawood, BP, Zurada, JM, and Tourassi, GD. "Toward perceptually driven image retrieval in mammography: A pilot observer study to assess visual similarity of masses." Progress in Biomedical Optics and Imaging - Proceedings of SPIE 6917 (2008).
Source
scival
Published In
Proceedings of SPIE
Volume
6917
Publish Date
2008
DOI
10.1117/12.772125

Solving decentralized multi-agent control problems with genetic algorithms

In decentralized control of multi-agent systems each agent is making a decision regarding its action autonomously, based on its own observations. In the light of the formal models of decentralized environments presented in the last decade, finding an optimal solution to a decentralized control problem is computationally prohibitive, even for moderately complicated environments. The problem, however, is of great significance since many of the real world systems can be treated as multi-agent systems with decentralized control. In this article, the authors propose an approximate algorithm for the problem based on a genetic algorithm. First, the problem is formalized using Decentralized Partially Observable Markov Decision Processes. Then a way of representing a solution (joint policy) in a chromosome is introduced and a genetic algorithm is proposed as a search mechanism. Finally, a multi-agent tiger problem is used as an experimental framework to show the effectiveness of the algorithm. © 2007 IEEE.

Authors
Mazurowski, MA; Zurada, JM
MLA Citation
Mazurowski, MA, and Zurada, JM. "Solving decentralized multi-agent control problems with genetic algorithms." 2007 IEEE Congress on Evolutionary Computation, CEC 2007 (2007): 1029-1034.
Source
scival
Published In
2007 IEEE Congress on Evolutionary Computation, CEC 2007
Publish Date
2007
Start Page
1029
End Page
1034
DOI
10.1109/CEC.2007.4424583

Case-base reduction for a computer assisted breast cancer detection system using genetic algorithms

A knowledge-based computer assisted decision (KB-CAD) system is a case-based reasoning system previously proposed for breast cancer detection. Although it was demonstrated to be very effective for the diagnostic problem, it was also shown to be computationally expensive due to the use of mutual information between images as a similarity measure. Here, the authors propose to alleviate this drawback by reducing the case-base size. The problem is formalized and a genetic algorithm is utilized as an optimization tool. Appropriate for the problem representation and operators are presented and discussed. A clinically relevant index of the area under the receiver operator characteristic curve is used as a measure of the system performance during the optimization and testing stages. Experimental results show that application of the proposed method can significantly reduce the case-base size while the classification performance of the KB-CAD, in fact, increases. © 2007 IEEE.

Authors
Mazurowski, MA; Habas, PA; Zurada, JM; Tourassi, GD
MLA Citation
Mazurowski, MA, Habas, PA, Zurada, JM, and Tourassi, GD. "Case-base reduction for a computer assisted breast cancer detection system using genetic algorithms." 2007 IEEE Congress on Evolutionary Computation, CEC 2007 (2007): 600-605.
Source
scival
Published In
2007 IEEE Congress on Evolutionary Computation, CEC 2007
Publish Date
2007
Start Page
600
End Page
605
DOI
10.1109/CEC.2007.4424525

Impact of low class prevalence on the performance evaluation of neural network based classifiers: Experimental study in the context of computer-assisted medical diagnosis

This paper presents an experimental study on the impact of low class prevalence on the neural network based classifier performance as measured using Receiver Operator Characteristic (ROC) analysis. Two methods of dealing with the problem are investigated: oversampling and undersampling in the context of varying the class prevalence and the size of training datasets with uncorrelated and correlated features. The results show that the class imbalance can significantly decrease the classifier performance especially in the case of small training datasets. Furthermore, the oversampling method is shown to be more effective than the undersampling method in compensating the class imbalance. Statistically significant differences, however, are observed only in the cases with large total number of samples and very low prevalence. ©2007 IEEE.

Authors
Mazurowski, MA; Habas, PA; Zurada, JM; Tourassi, GD
MLA Citation
Mazurowski, MA, Habas, PA, Zurada, JM, and Tourassi, GD. "Impact of low class prevalence on the performance evaluation of neural network based classifiers: Experimental study in the context of computer-assisted medical diagnosis." IEEE International Conference on Neural Networks - Conference Proceedings (2007): 2005-2009.
Source
scival
Published In
IEEE International Conference on Neural Networks - Conference Proceedings
Publish Date
2007
Start Page
2005
End Page
2009
DOI
10.1109/IJCNN.2007.4371266

Stacked generalization in computer-assisted decision systems: Empirical comparison of data handling schemes

Computer-assisted decision (CAD) systems are becoming increasingly popular for the diagnostic interpretation of radiologic images. These CAD systems often involve the stacked generalization of several different decision models. Combining decision models is a common meta-analysis strategy to improve upon the diagnostic performance of each individual model. This study investigates how different data handling schemes may affect the performance evaluation of CAD systems that rely on stacked generalization. The study is based on a multistage CAD system for the detection of masses in screening mammograms. The CAD system consists of a series of knowledge-based modules that operate at Level 0 capturing morphological as well as multiscale textural information. Then, the knowledge-based predictions are combined with a Level 1 classifier. The study shows that a leave-one-out sampling scheme appears to be an effective and relatively unbiased strategy for the estimation of the overall performance of a CAD system that is based on stacked generalization. However, extra caution should be placed on the complexity of the Level 1 combiner. When the available dataset is relatively small, a relatively simple learning system such as a backpropagation neural network with very few hidden nodes is preferable to avoid optimistically biased estimates of diagnostic performance. ©2007 IEEE.

Authors
Tourassi, GD; Jesneck, JL; Mazurowski, MA; Habas, PA
MLA Citation
Tourassi, GD, Jesneck, JL, Mazurowski, MA, and Habas, PA. "Stacked generalization in computer-assisted decision systems: Empirical comparison of data handling schemes." IEEE International Conference on Neural Networks - Conference Proceedings (2007): 1343-1347.
Source
scival
Published In
IEEE International Conference on Neural Networks - Conference Proceedings
Publish Date
2007
Start Page
1343
End Page
1347
DOI
10.1109/IJCNN.2007.4371153

Solving multi-agent control problems using particle swarm optimization

This paper outlines an approximate algorithm for finding an optimal decentralized control in multi-agent systems. Decentralized Partially Observable Markov Decision Processes and their extension to infinite state, observation and action spaces are utilized as a theoretical framework. In the presented algorithm, policies of each agent are represented by a feedforward neural network. Then, a search is performed in a joint weight space of all networks. Particle Swarm Optimization is applied as a search algorithm. Experimental results are provided showing that the algorithm finds good solutions for the classical Tiger Problem extended to multi-agent systems, as well as for a multi-agent navigation task involving large state and action spaces. © 2007 IEEE.

Authors
Mazurowski, MA; Zurada, JM
MLA Citation
Mazurowski, MA, and Zurada, JM. "Solving multi-agent control problems using particle swarm optimization." Proceedings of the 2007 IEEE Swarm Intelligence Symposium, SIS 2007 (2007): 105-111.
Source
scival
Published In
Proceedings of the 2007 IEEE Swarm Intelligence Symposium, SIS 2007
Publish Date
2007
Start Page
105
End Page
111
DOI
10.1109/SIS.2007.368033

Emergence of communication in multi-agent systems using reinforcement learning

In this paper, the new approach to the emergence of communication between autonomous agents is introduced. The learning scheme is presented, which allows for emergence of efficient communication between agents in cooperative systems. Classical reinforcement learning framework extended to multi-agent systems is used. Language capabilities are modeled by modifying agents' policy. In order to do this, so called linguistic state and action variables are added to extend agents' state and action spaces. Linguistic state variables represent the signal received by an agent and linguistic action variables represent a signal sent by an agent. Set of agents is divided into receivers and senders on the basis of their ability to send and receive communication signals. The experiment with two-agent system is presented. It is shown how a simple communication evolves simultaneously with a non-linguistic behavior as a tool to coordinate agents actions in order to implement a task. At the end, conclusion is made that presented approach can be applied to ensure an efficient communication within real-world heterogenous, task-oriented multi-agent systems.

Authors
Mazurowski, MA; Zurada, JM
MLA Citation
Mazurowski, MA, and Zurada, JM. "Emergence of communication in multi-agent systems using reinforcement learning." 2006 IEEE International Conference on Computational Cybernetics, ICCC (2006).
Source
scival
Published In
2006 IEEE International Conference on Computational Cybernetics, ICCC
Publish Date
2006
DOI
10.1109/ICCCYB.2006.305696

Limitations of sensitivity analysis for neural networks in cases with dependent inputs

In this paper the limitations of the sensitivity analysis method for feedforward neural networks in the cases of dependent input variables are discussed. First, it is explained that in such cases there can be many functions implemented by neural networks that will accurately approximate training patterns. Then it is pointed out that many of these functions do not allow for proper estimation of the inputs importance using the sensitivity analysis method for neural networks. These two facts are demonstrated to be the reason why one can not completely rely upon the results of this method, when evaluating a real importance of inputs. Examples with graphs visualizing the discussed phenomena are presented. Finally, general conclusions about overall usefulness of the method are introduced.

Authors
Mazurowski, MA; Szecówka, PM
MLA Citation
Mazurowski, MA, and Szecówka, PM. "Limitations of sensitivity analysis for neural networks in cases with dependent inputs." 2006 IEEE International Conference on Computational Cybernetics, ICCC (2006).
Source
scival
Published In
2006 IEEE International Conference on Computational Cybernetics, ICCC
Publish Date
2006
DOI
10.1109/ICCCYB.2006.305714

Neural network sensitivity analysis applied for the reduction of the sensor matrix

The neural network sensitivity analysis, involving neural network training and the calculation of its outputs derivative on inputs, was applied to select the least significant sensor in the multicomponont gas mixtures annlysis system. The sensitivity analysis results, collected for various neural network structures wore compared with the real significances of the sensors, determined experimentally. The question of the influence of the correlation of the input vector elements on the analysis results was also illustrated and discussed. © Springer-Verlag Berlin Heidelberg 2005.

Authors
Szecówka, PM; Szczurek, A; Mazurowski, MA; Licznerski, BW; Pichler, F
MLA Citation
Szecówka, PM, Szczurek, A, Mazurowski, MA, Licznerski, BW, and Pichler, F. "Neural network sensitivity analysis applied for the reduction of the sensor matrix." Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 3643 LNCS (2005): 27-32.
Source
scival
Published In
Lecture notes in computer science
Volume
3643 LNCS
Publish Date
2005
Start Page
27
End Page
32
Show More