Abstract

Fully automatic vertebrae tumor diagnosis (FAVTD) means using an end-to-end network to directly perform vertebrae recognition and tumor diagnosis from MRI images. FAVTD is clinically crucial for tumor screening and treatment, which helps prevent further metastasis and save the patients’ lives. However, FAVTD has not yet been fully attempted due to the challenges raised by tumor appearance variability as well as MRI image field of view (FOV) and/or characteristics diversity. We propose a REasoning DiscriminativE diCtIonary-embeDded nEtwork (RE-DECIDE) to tackle the challenges in FAVTD. RE-DECIDE contains an elaborated enhanced-supervision recognition network (ERN) and a self-adaptive reasoning diagnosis network (SRDN). ERN is implemented in a feed-forward dictionary learning manner, which encodes each vertebra by the sparse codes and uses the sparse projections of the vertebrae coordinates onto multiple observation axes for supervision. ERN thus provides multiple sparse encodings of all vertebrae (and their ground truths) to enhance supervision, which reinforces the discrimination of different vertebrae and thus improves recognition performance. SRDN first highlights the most informative feature in the recognized vertebrae based on an attention mechanism. It then performs feature interaction, i.e., exchanges features of different vertebrae based on the graph reasoning mechanism. A reasoning controlling strategy is designed to prompt feature interaction in vertebrae with the same diagnosis labels and meanwhile reduces that in vertebrae with different labels, which avoids over-smoothing and improves diagnosis performance. RE-DECIDE is trained and evaluated using a challenging dataset consisting of 600 MRI images; the evaluation results show that RE-DECIDE achieves high performance in both recognition (accuracy: 0.940) and diagnosis (AUC: 0.947) tasks

Framework

Experiment

Conclusion

In this paper, we have designed a reasoning discriminative dictionary-embedded network (RE-DECIDE) as a novel computer aided diagnosis (CAD) tool for fully automatic vertebrae tumor diagnosis (FAVTD) from MRI images. RE-DECIDE contains two novel designs: (1) ERN uses feed-forward dictionary learning and projection-based enhanced supervision strategy to obtain discriminative representations for vertebrae recognition and tackle the FOV/characteristics challenges; (2) SRDN uses attentional module and self-adaptive graph reasoning strategy for tumor diagnosis and alleviates the tumor appearance variability challenges. The effectiveness of RE-DECIDE, as well as its advantage to the state-of-theart, have been demonstrated by extensive experiments. Readers are welcome to ask for the codes used in this work.