Noise Reduction of Depth Cameras Images Based on Deep Neural Network

Majlesi Journal of Electrical Engineering

مقاله 12، دوره 14، شماره 3، آذر 2020، صفحه 95-100   

نوع مقاله: Review Article

شناسه دیجیتال (DOI): 10.29252/mjee.14.3.12

نویسندگان

Seyed Mehrdad Mahdavi¹؛ Mohsen Ashourian^{* 2}

¹Department of Computer engineering, Najafabad Branch, Islamic Azad University, Najafabad, Iran.

²Department of Electrical Engineering, Majlesi Branch, Islamic Azad University, Majlesi, Iran.

چکیده

Today, infrared sensors or depth sensors are widely used to control applications, games, information acquisition, dynamic and static 3D scenes. Despite the widespread use of these images, their quality is limited to low-quality images, as the infrared sensor does not have high resolution and the images produced by it have noise. Therefore, given the problems and the importance of using 3-D images, the quality of these images should be improved in order to provide accurate images from depth cameras. In this paper, the noise reduction of depth images using convolutional neural networks is considered. A convolutional neural network with a depth of 20 and three layers and a pre-trained neural network is used. We developed the system and tested its performance for two datasets of depth and color images, Middlebury and EURECOM Kinect Face. Results show that for EURECOM Kinect Face images, PSNR improvement is approximately 8 to 15 dB and for Middlebury images the PSNR improvement is about 5 to 14 dB.

کلیدواژه‌ها

Image Enhancement؛ Noise Reduction؛ convolution neural networks؛ Depth Camera Images

مراجع

[1] S. Foix, G. Alenya, and C. Torras, "Lock-in time-of-flight (ToF) cameras: A survey," IEEE Sensors Journal, Vol. 11, No. 9, pp. 1917-1926, 2011.

[2] D. Csetverikov, I. Eichhardt, and Z. Janko, "A brief survey of image-based depth upsampling," 2015.

[3] K. Zhang, W. Zuo, Y. Chen, D. Meng, and L. Zhang, "Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising," IEEE Transactions on Image Processing, Vol. 26, No. 7, pp. 3142-3155, 2017.

[4] A. Buades, B. Coll, and J.-M. Morel, "A non-local algorithm for image denoising," in Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on, 2005, Vol. 2, pp. 60-65: IEEE.

[5] K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, "Image denoising by sparse 3-D transform-domain collaborative filtering," IEEE Transactions on image processing, Vol. 16, No. 8, pp. 2080-2095, 2007.

[6] A. Buades, B. Coll, and J.-M. Morel, "Nonlocal image and movie denoising," International journal of computer vision, Vol. 76, No. 2, pp. 123-139, 2008.

[7] J. Mairal, F. Bach, J. Ponce, G. Sapiro, and A. Zisserman, "Non-local sparse models for image restoration," in Computer Vision, 2009 IEEE 12th International Conference on, 2009, pp. 2272-2279: IEEE.

[8] J. Xu, L. Zhang, W. Zuo, D. Zhang, and X. Feng, "Patch group based nonlocal self-similarity prior learning for image denoising," in Proceedings of the IEEE international conference on computer vision, pp. 244-252, 2015.

[9] U. Schmidt and S. Roth, "Shrinkage fields for effective image restoration," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2774-2781, 2014.

[10] K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," arXiv preprint arXiv:1409.1556 2014.

[11] A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks," in Advances in neural information processing systems, pp. 1097-1105, 2012.

[12] S. Ioffe and C. Szegedy, "Batch normalization: Accelerating deep network training by reducing internal covariate shift," arXiv preprint arXiv:1502.03167, 2015.

[13] K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.

[14] S. A. Gudmundsson, H. Aanaes, and R. Larsen, "Fusion of stereo vision and time-of-flight imaging for improved 3d estimation," International Journal of Intelligent Systems Technologies and Applications, Vol. 5, No. 3-4, pp. 425-433, 2008.

[15] J. Zhu, L. Wang, R. Yang, and J. E. Davis, "Reliability fusion of time-of-flight depth and stereo geometry for high quality depth maps," IEEE transactions on pattern analysis and machine intelligence, Vol. 33, No. 7, pp. 1400-1414, 2011.

[16] B.-S. Lin, W.-R. Chou, C. Yu, P.-H. Cheng, P.-J. Tseng, and S.-J. Chen, "An effective spatial-temporal denoising approach for depth images," in 2015 IEEE International Conference on Digital Signal Processing (DSP), 2015, pp. 64-67, IEEE.

[17] S. MJ, "Temporal and Spatial Denoising of Depth Maps," Sensors (Basel). 2015 Jul 29, No. 8, 2015.

[18] H. C. Burger, C. J. Schuler, and S. Harmeling, "Image denoising: Can plain neural networks compete with BM3D?," in Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on, 2012, pp. 2392-2399: IEEE.

[19] R. Min, N. Kose, and J.-L. Dugelay, "Kinectfacedb: A kinect database for face recognition," IEEE Transactions on Systems, Man, and Cybernetics: Systems, Vol. 44, No. 11, pp. 1534-1548, 2014.

[20] D. Scharstein and R. Szeliski, "A taxonomy and evaluation of dense two-frame stereo correspondence algorithms," International journal of computer vision, Vol. 47, No. 1-3, pp. 7-42, 2002.

[21] D. Scharstein and R. Szeliski, "High-accuracy stereo depth maps using structured light," in Computer Vision and Pattern Recognition, 2003. Proceedings. 2003 IEEE Computer Society Conference on, 2003, Vol. 1, pp. I-I: IEEE.

[22] D. Scharstein and C. Pal, "Learning conditional random fields for stereo," in Computer Vision and Pattern Recognition, 2007. CVPR'07. IEEE Conference on, 2007, pp. 1-8: IEEE.

[23] H. Hirschmuller and D. Scharstein, "Evaluation of cost functions for stereo matching," in Computer Vision and Pattern Recognition, 2007. CVPR'07. IEEE Conference on, 2007, pp. 1-8: IEEE.

[24] Y. Chen and T. Pock, "Trainable nonlinear reaction diffusion: A flexible framework for fast and effective image restoration," IEEE transactions on pattern analysis and machine intelligence, Vol. 39, No. 6, pp. 1256-1272, 2017.

[25] K. He, X. Zhang, S. Ren, and J. Sun, "Delving deep into rectifiers: Surpassing human-level performance on imagenet classification," in Proceedings of the IEEE international conference on computer vision, 2015, pp. 1026-1034.

[26] A. Vedaldi and K. Lenc, "Matconvnet: Convolutional neural networks for matlab," in Proceedings of the 23rd ACM international conference on Multimedia, 2015, pp. 689-692: ACM.