The week's pick
Random Articles
Reseach Article
A Hybrid Transformer-CNN Framework with Early and Late Fusion for Robust Skin Lesion Classification
| International Journal of Computer Applications |
| Foundation of Computer Science (FCS), NY, USA |
| Volume 187 - Number 51 |
| Year of Publication: 2025 |
| Authors: Raihan Tanvir |
10.5120/ijca2025925852
|
Raihan Tanvir . A Hybrid Transformer-CNN Framework with Early and Late Fusion for Robust Skin Lesion Classification. International Journal of Computer Applications. 187, 51 ( Oct 2025), 17-23. DOI=10.5120/ijca2025925852
Abstract
Skin lesion classification is a critical task in dermatological diagnosis, where early detection can significantly improve patient outcomes. The DermaMNIST dataset, a curated benchmark within the MedMNIST collection, provides a challenging testbed due to limited resolution, intraclass similarity, and class imbalance. In this work, we investigate the performance of advanced deep learning architectures, including Swin Transformer, ConvNeXt, and Vision Transformers, alongside fusion strategies that combine complementary representations. Specifically, we implement early fusion through feature concatenation and late fusion through ensemble averaging of logits. Our experiments on DermaMNIST with images of 224 × 224 resolution, demonstrate that Swin Transformer achieves an accuracy of 0.893, outperforming ConvNeXt (0.871), and Vision Transformer (0.873). Fusion strategies further improve robustness, with late fusion achieving the best accuracy of 0.895. Compared to the reported Google AutoML Vision baseline (0.768 accuracy), our models establish a new state-of-the-art on DermaMNIST. These results highlight the efficacy of hybrid deep learning strategies that integrate convolutional and transformerbased architectures for medical image classification.
References
- Titus Josef Brinker, Achim Hekler, Alexander H Enk, Joachim Klode, Axel Hauschild, Carola Berking, Sebastian Haferkamp, Dirk Schadendorf, Tim Holland-Letz, Jochen S Utikal, et al. Deep learning outperformed 136 of 157 dermatologists in a head-to-head dermoscopic melanoma image classification task. European Journal of Cancer, 113:47–54, 2019.
- Benjamin Cassidy, Connah Kendrick, Andrzej Brodzicki, Joanna Jaworek-Korjakowska, and Moi Hoon Yap. Analysis of the isic 2020 dataset using ensemble methods for skin lesion classification. Medical Image Analysis, 78:102412, 2022.
- Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations (ICLR), 2021.
- Andre Esteva, Brett Kuprel, Roberto A Novoa, Justin Ko, Susan M Swetter, Helen M Blau, and Sebastian Thrun. Dermatologist-level classification of skin cancer with deep neural networks. Nature, 542(7639):115–118, 2017.
- Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition, 2015.
- Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 10012–10022, 2021.
- Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. A convnet for the 2020s. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 11976–11986, 2022.
- Amirreza Mahbod, Georg Schaefer, Chunliang Wang, Rupert Ecker, and Isabella Ellinger. Transfer learning using vision transformers for skin lesion classification. International Symposium on Biomedical Imaging (ISBI), pages 1157–1160, 2021.
- Keiron O’Shea and Ryan Nash. An introduction to convolutional neural networks. CoRR, abs/1511.08458, 2015.
- Cees G. M. Snoek, Marcel Worring, and Arnold W. M. Smeulders. Early versus late fusion in semantic video analysis. ACM International Conference on Multimedia, pages 399–402, 2005.
- Mingxing Tan and Quoc Le. Efficientnetv2: Smaller models and faster training. In International Conference on Machine Learning (ICML), pages 10096–10106. PMLR, 2021.
- Philipp Tschandl, Cliff Rosendahl, and Harald Kittler. The ham10000 dataset, a large collection of multisource dermatoscopic images of common pigmented skin lesions. Scientific Data, 5:180161, 2018.
- Jiancheng Yang, Rui Shi, Donglai Wei, Ziming Liu, Lin Zhao, Bilian Ke, Hanspeter Pfister, and Bingbing Ni. Medmnist v2: A large-scale lightweight benchmark for 2d and 3d biomedical image classification. Scientific Data, 10(1):41, 2023.
- Jiancheng Yang, Rui Shi, DonglaiWei, Lin Zhao, Yunxiang Lei, Hao Li, Ziyan Xu, Dong Ni, Ali Hatamizadeh, Holger R Roth, et al. Medmnist v2: A large-scale lightweight benchmark for 2d and 3d biomedical image classification. Scientific Data, 10(1):41, 2023.
- Yu Zhang, Xiaohan Li, Haifeng Chen, and Ge Liu. Hybrid cnn-transformer architecture for skin lesion classification. Journal of Medical Imaging, 10(2):024502, 2023.
Index Terms
Keywords