Nhu-Tai Do

Selected publications

2021

1. Diagnostics

   Multi-Level Seg-Unet Model with Global and Patch-Based X-ray Images for Knee Bone Tumor Detection

   Do, Nhu-Tai, Jung, Sung-Taek, Yang, Hyung-Jeong, and Kim, Soo-Hyung

   Diagnostics 2021

   Abs

   Tumor classification and segmentation problems have attracted interest in recent years. In contrast to the abundance of studies examining brain, lung, and liver cancers, there has been a lack of studies using deep learning to classify and segment knee bone tumors. In this study, our objective is to assist physicians in radiographic interpretation to detect and classify knee bone regions in terms of whether they are normal, begin-tumor, or malignant-tumor regions. We proposed the Seg-Unet model with global and patched-based approaches to deal with challenges involving the small size, appearance variety, and uncommon nature of bone lesions. Our model contains classification, tumor segmentation, and high-risk region segmentation branches to learn mutual benefits among the global context on the whole image and the local texture at every pixel. The patch-based model improves our performance in malignant-tumor detection. We built the knee bone tumor dataset supported by the physicians of Chonnam National University Hospital (CNUH). Experiments on the dataset demonstrate that our method achieves better performance than other methods with an accuracy of 99.05% for the classification and an average Mean IoU of 84.84% for segmentation. Our results showed a significant contribution to help the physicians in knee bone tumor detection.

2. Sensors

   Context-Aware Emotion Recognition in the Wild Using Spatio-Temporal and Temporal-Pyramid Models

   Do, Nhu-Tai, Kim, Soo-Hyung, Yang, Hyung-Jeong, Lee, Guee-Sang, and Yeom, Soonja

   Sensors 2021

   Abs

   Emotion recognition plays an important role in human–computer interactions. Recent studies have focused on video emotion recognition in the wild and have run into difficulties related to occlusion, illumination, complex behavior over time, and auditory cues. State-of-the-art methods use multiple modalities, such as frame-level, spatiotemporal, and audio approaches. However, such methods have difficulties in exploiting long-term dependencies in temporal information, capturing contextual information, and integrating multi-modal information. In this paper, we introduce a multi-modal flexible system for video-based emotion recognition in the wild. Our system tracks and votes on significant faces corresponding to persons of interest in a video to classify seven basic emotions. The key contribution of this study is that it proposes the use of face feature extraction with context-aware and statistical information for emotion recognition. We also build two model architectures to effectively exploit long-term dependencies in temporal information with a temporal-pyramid model and a spatiotemporal model with “Conv2D+LSTM+3DCNN+Classify” architecture. Finally, we propose the best selection ensemble to improve the accuracy of multi-modal fusion. The best selection ensemble selects the best combination from spatiotemporal and temporal-pyramid models to achieve the best accuracy for classifying the seven basic emotions. In our experiment, we take benchmark measurement on the AFEW dataset with high accuracy.

3. AACR 2021

   Abstract PO-024: Boosting up knee bone tumor detection from radiology and magnetic resonance imaging by using deep learning techniques

   Do, Nhu-Tai, Jung, Sung-Taek, and Kim, Soo-Hyung

   Clinical Cancer Research 2021

   Abs

   Abstracts: AACR Virtual Special Conference on Artificial Intelligence, Diagnosis, and Imaging; January 13-14, 2021Bone tumors can occur at all ages and also appear around skeletal positions of the human knee. Since the types and expression patterns are extremely diverse, some cases are non-specific symptoms or the onset of symptoms is delayed, and the diagnosis is delayed by low suspicion of physicians. However, timely diagnosis is very important for treatment to reduce extreme pain for patients. Therefore, detecting tumors from radiography and MRI images based on deep learning has recently emerged with the advantage of remarkably shortening time and a completely automatic diagnosis. In this study, we developed a radiology and MRI dataset for knee bone tumor and contributed a robust deep learning-based method to detect tumors. Through an association with Chonnam National University Hospital, we built the radiology knee bone tumor dataset with 1695 high-resolution images including normal, benign and malignant. It is the first dataset for knee bone tumor detection with five categories for classification, and tumor masks, bone regions for segmentation. We proposed multi-level distance features to exploit the tumors based on neighboring regions dealing with challenges from small size, appearance variety, and uncommon. It led to interpreting high-risk regions occurring tumors. Multi-task learning with classification and segmentation helped to learn the mutual benefit from the pixel-wise segmentation and image classification. Our model is fined-tuned in image patches to boost up the accuracy in malignant classification. In difficult cases, MRI has been proved to be a preeminent option to be capable of presenting bone marrow and soft tissue. Therefore, we built MRI knee bone tumor dataset with 104 study cases over 900 series. We used Slicer3D for making the ground-truth in T1-, T2-, and PD- weighted MRI. We generated tumor and bone segmentation masks and classified the study case into three groups: normal, benign, and malignant. We made the ablation study experiments to our radiology dataset. For multi-task learning, we increased Mean IoU from 69.5% to 77.3%. Multi-level distance features helped to increase to 78.8%. For classification, multi-task learning boosted the accuracy from 93% to 95.3%, and multi-level distance features helped the increase of 3%. Besides, the patching training improved malignant classification from 66.7% to 81.5%. The results demonstrated that our deep learning-based method made an important contribution to help doctors in the clinical diagnosis process using a radiology image. Furthermore, our method produced the high-risk neighbor tumor regions by multi-level distance features instead of extracting exactly tumor regions. To improve more accuracy in automatic detection, we hope that MRI images with high contrast resolution can describe clearly soft tissue components and help to recognize the attributes of the tumors clearly. Nonetheless, the dataset of knee bone tumor on MRI is raw and not standard with high complexity. Preprocessing these MRI images is also our initial success to the steppingstone of further research.Citation Format: Nhu-Tai Do, Sung-Taek Jung, Soo-Hyung Kim. Boosting up knee bone tumor detection from radiology and magnetic resonance imaging by using deep learning techniques [abstract]. In: Proceedings of the AACR Virtual Special Conference on Artificial Intelligence, Diagnosis, and Imaging; 2021 Jan 13-14. Philadelphia (PA): AACR; Clin Cancer Res 2021;27(5_Suppl):Abstract nr PO-024.

2020

1. FG 2020

   Affective Expression Analysis in-the-wild using Multi-Task Temporal Statistical Deep Learning Model

   Do, Nhu-Tai, Nguyen-Quynh, Tram-Tran, and Kim, Soo-Hyung

   In 2020 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020) 2020
2. IEEE Access

   Image Colorization Using the Global Scene-Context Style and Pixel-Wise Semantic Segmentation

   Nguyen-Quynh, Tram-Tran, Kim, Soo-Hyung, and Do, Nhu-Tai

   IEEE Access 2020