Interactive Visualization Framework for Panoramic Super-Resolution Images Based on Localization Data

Wang Chenze; Shen Xuehao; Huang Zhenli; Wang Zhengxia

doi:10.7544/issn1000-1239.202330643

Journal of Computer Research and Development > 2024 > 61(7): 1741-1753. > DOI: 10.7544/issn1000-1239.202330643 CSTR: 32373.14.issn1000-1239.202330643

Wang Chenze, Shen Xuehao, Huang Zhenli, Wang Zhengxia. Interactive Visualization Framework for Panoramic Super-Resolution Images Based on Localization Data[J]. Journal of Computer Research and Development, 2024, 61(7): 1741-1753. DOI: 10.7544/issn1000-1239.202330643

Citation:

PDF (8042 KB)

Interactive Visualization Framework for Panoramic Super-Resolution Images Based on Localization Data

1.
School of Computer Science and Technology, Hainan University, Haikou 570228
2.
School of Information and Communication Engineering, Hainan University, Haikou 570228
3.
School of Biomedical Engineering, Hainan University, Haikou 570228

Funds: This work was supported by the National Natural Science Foundation of China (82160345), the Key Research and Development Program of Hainan Province (ZDYF2021GXJS017), and the Key Science and Technology Plan Project of Haikou (2021-016).

More Information

Author Bio:
Wang Chenze: born in 1996. Master candidate. His main research interest includes image processing

Shen Xuehao: born in 1995. PhD candidate. His main research interests include deep learning and image processing

Huang Zhenli: born in 1976. PhD, professor, PhD supervisor. His main research interests include digital diagnosis and optical microscopy technique

Wang Zhengxia: born in 1977. PhD, professor, PhD supervisor. Her main research interests include medical image processing and super resolution optical image processing
Received Date: July 31, 2023
Revised Date: December 18, 2023
Available Online: March 03, 2024

Graphical Abstract

Abstract

Abstract

Combining super-resolution localization microscopy with panoramic digital pathology provides a powerful tool for biomedical researchers to observe the subcellar structure of the entire sample, but also brings the challenge of visualization of panoramic super-resolution image. However, current super-resolution image visualization methods are unable to process large-scale localization data, provide high-resolution panoramic images, and interactively visualize panoramic images. In response to the above problems, an interactive visualization framework for panoramic super-resolution images based on localization data is proposed, called PNanoViewer, which aims to achieve rapid interactive visualization of large-scale localization data on ordinary computers. This framework constructs the multi-resolution hierarchical structure of localization data based on the random sampling strategy, and visualizes multi-scale panoramic super-resolution image in an interactive way. Meanwhile, the block strategy and multi-thread parallel strategy are used to process large-scale localization data in batches to prevent memory overflow and speed up processing. Experimental results on tens to hundreds of millions of localization datasets show that the proposed PNanoViewer framework can visualize localization data with any size. Compared this framework with three currently popular super-resolution image visualization methods: PALMsiever, ThunderSTORM, and QC-STORM, PNanoViewer has obvious advantages in terms of data volume, resolution, and speed. At the same time, PNanoViewer also provides a useful exploration for the visualization of large-scale localization data.

FullText(HTML)

References (32)

References

[1]	Lelek M, Gyparaki M T, Beliu G, et al. Single-molecule localization microscopy[J]. Nature Reviews Methods Primers, 2021, 1(1): 1−39 doi: 10.1038/s43586-020-00001-2
[2]	Fuhrmann M, Gockel N, Arizono M, et al. Super-resolution microscopy opens new doors to life at the nanoscale[J]. Journal of Neuroscience, 2022, 42(45): 8488−8497 doi: 10.1523/JNEUROSCI.1125-22.2022
[3]	Creech M K, Wang Jing, Nan Xiaolin, et al. Superresolution imaging of clinical formalin fixed paraffin embedded breast cancer with single molecule localization microscopy[J]. Scientific Reports, 2017, 7: 40766 doi: 10.1038/srep40766
[4]	Codron P, Letournel F, Marty S, et al. Stochastic optical reconstruction microscopy (STORM) reveals the nanoscale organization of pathological aggregates in human brain[J]. Neuropathology Applied Neurobiology, 2021, 47(1): 127−142 doi: 10.1111/nan.12646
[5]	Hanna M G, Ardon O, Reuter V E, et al. Integrating digital pathology into clinical practice[J]. Modern Pathology, 2022, 35(2): 152−164 doi: 10.1038/s41379-021-00929-0
[6]	Balzer C, Oktavian R, Zandi M, et al. Wiz: A web-based tool for interactive visualization of big data[J]. Patterns, 2020, 1(8): 100107 doi: 10.1016/j.patter.2020.100107
[7]	张新华,李才伟,张瑜,等. 多靶点全景数字病理:从原理到应用[J]. 中国光学,2022,15(6):1258−1274 doi: 10.37188/CO.2022-0091 Zhang Xinhua, Li Caiwei, Zhang Yu, et al. Multi-target panoramic digital pathology: From principle to application[J]. Chinese Optics, 2022, 15(6): 1258−1274 (in Chinese) doi: 10.37188/CO.2022-0091
[8]	Chalfoun J, Majurski M, Blattner T, et al. MIST: Accurate and scalable microscopy image stitching tool with stage modeling and error minimization[J]. Scientific Reports, 2017, 7(1): 4988−4998 doi: 10.1038/s41598-017-04567-y
[9]	Hoerl, D, Rusak F R, Preusser F, et al. BigStitcher: Reconstructing high-resolution image datasets of cleared and expanded samples[J]. Nature Methods, 2019, 16(9): 870−874 doi: 10.1038/s41592-019-0501-0
[10]	Cardona A, Saalfeld S, Schindelin J, et al. TrakEM2 software for neural circuit reconstruction[J]. Plos One, 2012, 7(6): 1−8
[11]	Du Yue, Wang Chenze, Zhang Chen, et al. Computational framework for generating large panoramic super-resolution images from localization microscopy[J]. Biomedical Optics Express, 2021, 12(8): 4759−4778 doi: 10.1364/BOE.433489
[12]	Hausen R, Robertson B E. FitsMap: A simple, lightweight tool for displaying interactive astronomical image and catalog data[J]. Astronomy and Computing, 2022, 39: 100586 doi: 10.1016/j.ascom.2022.100586
[13]	曹世翔,江洁,张广军,等. 边缘特征点的多分辨率图像拼接[J]. 计算机研究与发展,2011,48(9):1788−1793 Cao Shixiang, Jiang Jie, Zhang Guangjun, et al. Multi-resolution image stitching of edge feature points[J]. Journal of Computer Research and Development, 2011, 48(9): 1788−1793 (in Chinese)
[14]	Wu Yule, Tschanz A, Krupnik L, et al. Quantitative data analysis in single-molecule localization microscopy[J]. Trends in Cell Biology, 2020, 30(11): 837−851 doi: 10.1016/j.tcb.2020.07.005
[15]	Khater I M, Nabi I R, Hamarneh G. A review of super-resolution single-molecule localization microscopy cluster analysis and quantification methods[J]. Patterns, 2020, 1(3): 100038 doi: 10.1016/j.patter.2020.100038
[16]	Hagen G M, Kíek P, Ovesn M, et al. ThunderSTORM: A comprehensive ImageJ plugin for PALM and STORM data analysis and super-resolution imaging[J]. Bioinformatics, 2015, 30(16): 2389−2390
[17]	Pengo T, Holden S J, Manley S. PALMsiever: A tool to turn raw data into results for single-molecule localization microscopy[J]. Bioinformatics, 2015, 31(5): 797−798 doi: 10.1093/bioinformatics/btu720
[18]	Paul M W, de Gruiter H M, Lin Zhanmin, et al. SMoLR: Visualization and analysis of single-molecule localization microscopy data in R[J]. BMC Bioinformatics, 2019, 20(1): 1−7
[19]	Li Hongjia, Xu Fan, Gao Shan, et al. Live-SIMBA: An ImageJ plug-in for the universal and accelerated single molecule-guided Bayesian localization super resolution microscopy (SIMBA) method[J]. Biomedical Optics Express, 2020, 11(10): 5842−5859
[20]	Davis J L, Soetikno B, Song Kihee , et al. RainbowSTORM: An open-source ImageJ plug-in for spectroscopic single-molecule localization microscopy (sSMLM) data analysis and image reconstruction[J]. Bioinformatics, 2020, 36(19): 4972−4974
[21]	Henriques R, Lelek M, Fornasiero E F, et al. QuickPALM: 3D real-time photoactivation nanoscopy image processing in imageJ[J]. Nature methods, 2010, 7(5): 339−340 doi: 10.1038/nmeth0510-339
[22]	Li Luchang, Xin Bo, Kuang Weibing , et al. Divide and conquer: Real-time maximum likelihood fitting of multiple emitters for super-resolution localization microscopy[J]. Optics Express, 2019, 27(15): 21029−21049
[23]	Schütz M, Ohrhallinger S, Wimmer M. Fast out-of-core octree generation for massive point clouds[J]. Computer Graphics Forum, 2020, 39(7): 155−167
[24]	Schüffler P J, Stamelos E, Ahmed I, et al. Efficient visualization of whole slide images in web-based viewers for digital pathology[J]. Archives of Pathology Laboratory Medicine, 2022, 146(10): 1273−1280 doi: 10.5858/arpa.2021-0197-OA
[25]	Lajara N, Luis Espinosa-Aranda J, Deniz O, et al. Optimum web viewer application for DICOM whole slide image visualization in anatomical pathology[J]. Computer Methods and Programs in Biomedicine, 2019, 179: 104983
[26]	王弘堃,曹轶,肖丽. 基于图像的大规模数据集交互可视化[J]. 计算机研究与发展,2017,54(4):855−860 Wang Hongkun, Cao Yi, Xiao Li. Image-based interactive visualization of large-scale data sets[J]. Journal of Computer Research and Development, 2017, 54(4): 855−860 (in Chinese)
[27]	Yu Haijun, Li Shengyang, Zhang Tao. A space-time interactive visualization approach for managing remote sensing data[J]. Journal of Information Science Engineering, 2021, 37(2): 395−412
[28]	Stathonikos N, Nguyen T Q, Spoto C P, et al. Being fully digital: Perspective of a dutch academic pathology laboratory[J]. Histopathology, 2019, 75(5): 621−635 doi: 10.1111/his.13953
[29]	Pielawski N, Andersson A, Avenel C, et al. TissUUmaps 3: Interactive visualization and quality assessment of large-scale spatial omics data[J]. Biorxiv, 2022, 9(5): 1−14
[30]	Park S, Ju S, Yoon S, et al. An efficient data structure approach for bim-to-point-cloud change detection using modifiable nested octree[J]. Automation in Construction, 2021, 132: 103922 doi: 10.1016/j.autcon.2021.103922
[31]	Deibe D, Amor M, Doallo R. Supporting multi-resolution out-of-core rendering of massive LiDAR point clouds through non-redundant data structures[J]. International Journal of Geographical Information Science, 2018, 33(3): 1−25
[32]	van Oosterom P, van Oosterom S, Liu Haicheng, et al. Organizing and visualizing point clouds with continuous levels of detail[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2022, 194: 119−131