-
软件方式又称间接方式[15],它包括基于多传感器图像融合的方法和基于颜色转换的方法,其中基于融合的方法是通过多个传感器采集分光谱图像进行融合得到彩色图像,而基于颜色转换的方法则是通过与参考图像作对比,使当前的灰度图像转换为彩色图像[6]。
-
多个传感器可采集多个分波段图像,例如微光、近红外、短波红外、中波红外和长波红外图像,它们携有不同的颜色信息,故经图像融合后可显示为彩色图像[25]。图像融合算法的选择则更具有灵活性,可根据不同的目的需求调整融合算法。大致分为以下4个步骤:(1)采集夜视分波段源图像;(2)源图像预处理;(3)源图像间组合得到处理后图像;(4)将处理后图像映射到彩色空间得到彩色夜视图像。其中第(3)步源图像间组合可分为线性组合法和非线性组合法[26],亦可跳过此步骤直接进行映射[10]。
最为典型的是美国麻省理工学院林肯实验室开发的MIT(Massachusetts Institute of Technology)法[9],它基于生物视觉特性,将微光与红外图像融合。人的视网膜分为on域和off域(统称为感受野),它们分别对光照具有兴奋和抑制的作用。on感受野增强中心,抑制周边,而off感受野正好相反,这就是拮抗处理过程,此过程可增强图像的对比度。中心-周边感受野神经动力学方程如下:
$ \begin{aligned} \frac{\mathrm{d} E_{i j}}{\mathrm{d} t} &=-A+\left(1-E_{i j}\right)\left[C I_{1}\right]_{i j}-\\ &\left(1+E_{i j}\right)\left[G(s) \times I_{2}\right] \end{aligned} $
(1) $ E_{i j} =\frac{\left[C I_{1}-G(s) I_{2}\right]_{i j}}{A+\left[C I_{1}+G(s) \times I_{2}\right]} $
(2) 式中,E是增强后的图像;I1是使感受野中心兴奋的输入图像;I2是使感受野周边抑制的输入图像;G(s)为高斯函数的缩写形式;i, j为像素位置;A, C为调节参量。
MIT法的基本思路为:(1)使用中心-周边神经分离网络分别使微光图像和红外图像对比度增强; (2)使用中心-周边神经分离网络,将微光图形送入中心,红外图像送入周边,形成单拮抗彩色对比度图像; (3)将以上得到的图像经过色彩空间映射成一幅彩色图像。图 2为MIT法基本流程图。而后在此基础上对MIT法做了许多改进,得到了不错的效果。
-
有学者于2001年提出一种颜色转换的思想[27],即将一幅图像赋予与参考图形颜色类似的彩色图像。TOET于2003年通过一幅具有自然感彩色的参考图像用色彩传递的方法使一幅经过直接映射得到的夜视图像“赋予了自然感彩色”。图 3是TOET得到的彩色夜视图像。
2008年,TOET通过建立2维查找表的方法使双通道微光夜视图像输出彩色图像,为彩色夜视的实时性处理带来便利,可实现视频处理[28-30]。他的方法流程如下:(1)将参考图像颜色传递给双通道夜视融合图像;(2)得到的彩色夜视融合图像与双通道的分波段图像的灰度信息建立色彩查找表;(3)通过查找表实现实时输出彩色夜视图像[31-32]。
自TOET提出基于色彩传递的彩色夜视方法提出后,许多学者热衷于如何将参考图像的自然感彩色信息“传递”给融合图像。2008年,北京理工大学开发了基于双数字信号处理的实时自然感彩色彩色融合处理板,它实现基于色彩传递的融合实时处理,将微光图像与红外热像仪图像融合后获得具有自然感彩色的彩色夜视图像[15]。
目前,基于色彩传递的方法已广泛应用于彩色夜视技术中,其获得的彩色夜视图像相比于多波段图像融合得到的假彩色夜视图像的色彩更符合人眼视觉特性,具有自然感彩色以及较好的实时性。
彩色夜视技术方法综述
Summary of color night vision technology
-
摘要: 为了得到具有自然彩色的夜视图像, 彩色夜视技术的研究成为夜视技术研究领域的一大热点。介绍了夜视技术的发展概况和获得具有自然感彩色夜视图像的彩色夜视技术手段, 主要可分为硬件方式和软件合成的方式, 其中软件方式主要有基于多传感器图像融合的方法和基于色彩传递的方法。详细介绍了两种方式的技术方案, 并对彩色夜视技术做出了技术展望。Abstract: In order to get night vision images with natural color, color night vision technology has become a hotspot in the field of night vision technology. The development of night vision technology and the technical means of obtaining natural color night vision images are introduced. It can be divided into hardware mode and software synthesis mode.The software methods are mainly based on multi-sensor image fusion and color transfer.The technical schemes of both the modes are introduced in detail.The technical prospect of color night vision technology is also given.
-
Key words:
- sensor technique /
- color night vision /
- hardware mode /
- software mode /
- image fusion /
- color transfer
-
[1] GUO H, XIANG Sh M, TIAN M Q, et al. A review of the development of low-light night vision technology[J]. Infrared Technology, 2013, 35(2): 63-68(in Chinese). [2] HU M, CAO W G, WANG J S, et al. Natural glimmer environmental simulation of glimmer night vision environment[J]. Journal of Changchun University of Science and Technology(Natural Science Edition), 2015, 38(1): 29-33(in Chinese). [3] CHEN Y L, WU B F, HUANG H Y, et al. A real-time vision system for nighttime vehicle detection and traffic surveillance. IEEE Transactions on Industrial Electronics, 2010, 58(5): 2030-2044. [4] CAI Y L. Multi-band color night vision technology research[D]. Nanjing: Nanjing University of Science and Technology, 2010: 2-5(in Chinese). [5] JIN W Q, WANG L X, WANG Sh X, et al. Color fusion technology of night vision image and its progress[J]. Infrared Technology, 2003, 25(1): 6-12(in Chinese). [6] ZHANG A M. Denoising and fusion method of night vision image based on wavelet transform[J]. Electronic Measurement Technology, 2015, 38(1): 38-40(in Chinese). [7] OTTNEY J C. Fusion night vision system: US Patent Application 12/952955[P]. 2011-04-21. [8] WROLSTAD R E, SMITH D E. Color analysis[M]. Boston, MA, USA: Springer, 2010: 573-586. [9] WANG L X, JIN W Q, LIU G R, et al. Overview of color fusion schemes for night viewing image[J]. Infrared Technology, 2002, 24(2): 9-13(in Chinese). [10] MA Sh, FANG J A, SUN Sh Y, et al. Colorizing algorithm of night-vision image based on clustering of false color fused image[J]. Acta Optica Sinica, 2009, 29(6):1502-1507(in Chinese). doi: 10.3788/AOS20092906.1502 [11] TEKE M, BASESKI E, OK A Ö, et al. Multi-spectral false color shadow detection[C]//ISPRS Conference on Photogrammetric Image Analysis. Berlin, Heidelberg: Springer, 2011: 109-119. [12] TOET A, HOGERVORST M A. Progress in color night vision[J]. Optical Engineering, 2012, 51(1): 010901. doi: 10.1117/1.OE.51.1.010901 [13] KEESLING M, FERREN B, KAEHLER A, et al. Color night vision cameras, systems, and methods: US 10362285[P]. 2019-07-23. [14] WAXMAN A M, GOVE A N, FAY D A, et al. Color night vision: opponent processing in the fusion of visible and IR imagery[J]. Neural Networks, 1997, 10(1): 1-6. [15] GAO W, ZHU M, HAO Zh Ch. Research progress of color night vision technology[J]. Chinese Journal of Liquid Crystals and Displays, 2016, 31(12):1168-1179 (in Chinese). doi: 10.3788/YJYXS20163112.1168 [16] KOYAMA S, INABA Y, KASANO M, et al. A day and night vision MOS imager with robust photonic-crystal-based RGB-and-IR[J]. IEEE Transactions on Electron Devices, 2008, 55(3): 754-759. doi: 10.1109/TED.2007.914828 [17] WU J W, SUN Sh Y, ZHANG Ch Q. A method for night vision vi-deo colorization[J]. Microcomputer Information, 2010, 26(30): 188-189 (in Chinese). [18] WU Y Y, WANG Y J, SHI X B, et al. The color night vision method combining NSST with color contrast enhancement[J]. Opto-Electronic Engineering, 2016, 43(11): 88-94(in Chinese). [19] JIN W Q, TAO Y, SHI F, et al. Progress of low level light video technology[J]. Infrared and Laser Engineering, 2015, 44(11): 3167-3176 (in Chinese). [20] LUO Y, WANG L X, JIN W Q, et al. Developments of image processing algorithms and systems for LLL(Vis.)/IR color night vision[J]. Infrared Technology, 2010, 32(6):337-344 (in Chinese). [21] XU T F, LI J T, ZHANG Y Zh, et al. True color transfer for dual band image fusion[J]. Chinese Optics, 2014, 7(3): 402-410(in Chinese). [22] JIN W Q, WANG L X, ZHAO Y M, et al. Developments of image processing algorithms for color night vision[J]. Infrared and Laser Engineering, 2008, 37(1): 147-150(in Chinese). [23] ZHU J, LI L, JIN W Q, et al. Natural-appearance colorization and enhancement for the low-light-level night vision imaging[J]. Acta Photonica Sinica, 2018, 47(4): 0410002(in Chinese). doi: 10.3788/gzxb20184704.0410002 [24] SHAMIR H, YAELI Y, EFRAT I. Synthetic colour night vision system: US 8212876[P]. 2012-07-03. [25] KRIESEL J, GAT N. True color night vision (TCNV) fusion: US 8836793[P]. 2014-09-16. [26] CHEN F, ZHANG W W, YU W J, et al. EMCCD low light image fusion algorithm based on wavelet transform[J]. Laser Technology, 2014, 38(2): 155-160(in Chinese). [27] DAI Zh H. Research on vehicle infrared video colorization algorithm[D]. Shanghai: Donghua University, 2011: 2-12(in Chinese). [28] SHI Sh M, WANG L X, JIN W Q, et al. A dual-band color imaging system for visible and thermal IR images based on color transfer in YUV color space[J]. Acta Armamentarii, 2009, 30(1): 30-35(in Chinese). [29] LI G X, XU Sh Y, ZHAO Y L, et al. Fast color image fusion based on color transfer technique[J]. Optics and Precision Engineering, 2010, 18(7): 1637-1647(in Chinese). [30] YANG J, CHEN Zh J. Analysis and research of globally matching color transfer algorithms in different color spaces[J]. Computer Engineering and Applications, 2007, 43(25): 80-82(in Chinese). [31] CHEN X E, CEHN Zh J. A comparative study of several application modes of color transfer[J]. Computer Application, 2007, 27(s1): 146-148(in Chinese). [32] LI G X, XU Sh Y. Fast color transfer method for image fusion[J]. Optics and Precision Engineering, 2009, 17(9): 2301-2310(in Chinese). [33] HAN Z, LIN S Zh, HUANG F Sh, et al. Multi-band fusion image colorization method based on convolutional neural network[J]. Journal of Test and Measurement Technology, 2018, 129(3): 23-28(in Chinese). [34] NI G Q. Research and new development of multi-band image fusion algorithms (Ⅱ)[J]. Journal of Atmospheric and Environmental Optics, 2001, 14(6): 11-17(in Chinese). [35] MA L M, YOU D D. Application development and countermeasure of night vision technology[J]. Journal of Atmospheric and Environmental Optics, 1996, 9(1): 34-38(in Chinese). [36] ZHI Y.The latest development of night vision technology in the united states[J]. Journal of Applied Optics, 2006, 27(4): 354(in Chinese).