Panchromatic and multispectral image fusion for remote sensing and earth observation: Concepts, taxonomy, literature review, evaluation methodologies and challenges ahead

Simple item page
dc.contributor.authorZhang, Kai
dc.contributor.authorZhang, Feng
dc.contributor.authorWan, Wenbo
dc.contributor.authorYu, Hui
dc.contributor.authorSun, Jiande
dc.contributor.authorDel Ser, Javier
dc.contributor.authorElyan, Eyad
dc.contributor.authorHussain, Amir
dc.contributor.institutionIA
dc.date.issued2023-05
dc.descriptionPublisher Copyright: © 2022 The Author(s)
dc.description.abstractPanchromatic and multispectral image fusion, termed pan-sharpening, is to merge the spatial and spectral information of the source images into a fused one, which has a higher spatial and spectral resolution and is more reliable for downstream tasks compared with any of the source images. It has been widely applied to image interpretation and pre-processing of various applications. A large number of methods have been proposed to achieve better fusion results by considering the spatial and spectral relationships among panchromatic and multispectral images. In recent years, the fast development of artificial intelligence (AI) and deep learning (DL) has significantly enhanced the development of pan-sharpening techniques. However, this field lacks a comprehensive overview of recent advances boosted by the rise of AI and DL. This paper provides a comprehensive review of a variety of pan-sharpening methods that adopt four different paradigms, i.e., component substitution, multiresolution analysis, degradation model, and deep neural networks. As an important aspect of pan-sharpening, the evaluation of the fused image is also outlined to present various assessment methods in terms of reduced-resolution and full-resolution quality measurement. Then, we conclude this paper by discussing the existing limitations, difficulties, and challenges of pan-sharpening techniques, datasets, and quality assessment. In addition, the survey summarizes the development trends in these areas, which provide useful methodological practices for researchers and professionals. Finally, the developments in pan-sharpening are summarized in the conclusion part. The aim of the survey is to serve as a referential starting point for newcomers and a common point of agreement around the research directions to be followed in this exciting area.en
dc.description.sponsorshipThis work was supported in part by the Natural Science Foundation of China (61901246), the China Postdoctoral Science Foundation, China Grant (2019TQ0190, 2019M662432), the Scientific Research Leader Studio of Ji'nan (2021GXRC081), and Joint Project for Smart Computing of Shandong Natural Science Foundation, China (ZR2020LZH015). Amir Hussain acknowledges the support of the UK Engineering and Physical Sciences Research Council (EPSRC)-Grants Ref. EP/M026981/1, EP/T021063/1, EP/T024917/1. Hui Yu acknowledges the support of Royal Society, UK (NIF/R1/180909). J. Del Ser would like to thank the Spanish Centro para el Desarrollo Tecnologico Industrial (CDTI, Ministry of Science and Innovation) through the “Red Cervera” Programme (AI4ES project), as well as by the Basque Government, Spain through the ELKARTEK program and the Consolidated Research Group MATHMODE (Ref. IT1456-22). This work was supported in part by the Natural Science Foundation of China ( 61901246 ), the China Postdoctoral Science Foundation, China Grant ( 2019TQ0190 , 2019M662432 ), the Scientific Research Leader Studio of Ji’nan ( 2021GXRC081 ), and Joint Project for Smart Computing of Shandong Natural Science Foundation, China ( ZR2020LZH015 ). Amir Hussain acknowledges the support of the UK Engineering and Physical Sciences Research Council (EPSRC) -Grants Ref. EP/M026981/1 , EP/T021063/1 , EP/T024917/1 . Hui Yu acknowledges the support of Royal Society, UK ( NIF/R1/180909 ). J. Del Ser would like to thank the Spanish Centro para el Desarrollo Tecnologico Industrial (CDTI, Ministry of Science and Innovation) through the “Red Cervera” Programme (AI4ES project), as well as by the Basque Government, Spain through the ELKARTEK program and the Consolidated Research Group MATHMODE (Ref. IT1456-22 ).
dc.description.statusPeer reviewed
dc.format.extent16
dc.identifier.citationZhang , K , Zhang , F , Wan , W , Yu , H , Sun , J , Del Ser , J , Elyan , E & Hussain , A 2023 , ' Panchromatic and multispectral image fusion for remote sensing and earth observation : Concepts, taxonomy, literature review, evaluation methodologies and challenges ahead ' , Information Fusion , vol. 93 , pp. 227-242 . https://doi.org/10.1016/j.inffus.2022.12.026
dc.identifier.doi10.1016/j.inffus.2022.12.026
dc.identifier.issn1566-2535
dc.identifier.urlhttp://www.scopus.com/inward/record.url?scp=85145977442&partnerID=8YFLogxK
dc.language.isoeng
dc.relation.ispartofInformation Fusion
dc.relation.projectIDScientific Research Leader Studio of Ji'nan
dc.relation.projectIDScientific Research Leader Studio of Ji’nan, 2021GXRC081
dc.relation.projectIDEngineering and Physical Sciences Research Council, EPSRC, EP/T024917/1-EP/T021063/1-EP/M026981/1
dc.relation.projectIDRoyal Society, NIF/R1/180909
dc.relation.projectIDNational Natural Science Foundation of China, NSFC, 61901246
dc.relation.projectIDCentro para el Desarrollo Tecnológico Industrial, CDTI
dc.relation.projectIDChina Postdoctoral Science Foundation, 2019TQ0190-2019M662432
dc.relation.projectIDEusko Jaurlaritza, IT1456-22
dc.relation.projectIDMinisterio de Ciencia e Innovación, MICINN
dc.relation.projectIDNatural Science Foundation of Shandong Province, ZR2020LZH015
dc.rightsinfo:eu-repo/semantics/openAccess
dc.subject.keywordsImage fusion
dc.subject.keywordsImage quality evaluation
dc.subject.keywordsMultispectral image
dc.subject.keywordsPan-sharpening
dc.subject.keywordsPanchromatic image
dc.subject.keywordsSoftware
dc.subject.keywordsSignal Processing
dc.subject.keywordsInformation Systems
dc.subject.keywordsHardware and Architecture
dc.titlePanchromatic and multispectral image fusion for remote sensing and earth observation: Concepts, taxonomy, literature review, evaluation methodologies and challenges aheaden
dc.typejournal article
Files