An Analog Photonic Computing Device Based on a Diffractive Neural Network for Video Stream Processing

Roman V. Skidanov; Leonid L. Doskolovich; Nikolay L. Kazanskiy; Alexandr E. Morozov; Alexey S. Pronin; Daniil M. Sorokin; Yuriy V. Khanenko; Victor A. Soifer

doi:10.14529/jsfi260208

Authors

Roman V. Skidanov S.P. Korolyov Samara National Research University, Samara, Russian Federation https://orcid.org/0000-0001-8514-7156
Leonid L. Doskolovich S.P. Korolyov Samara National Research University, Samara, Russian Federation https://orcid.org/0000-0001-8649-028X
Nikolay L. Kazanskiy S.P. Korolyov Samara National Research University, Samara, Russian Federation https://orcid.org/0000-0002-0180-7522
Alexandr E. Morozov S.P. Korolyov Samara National Research University, Samara, Russian Federation
Alexey S. Pronin S.P. Korolyov Samara National Research University, Samara, Russian Federation
Daniil M. Sorokin S.P. Korolyov Samara National Research University, Samara, Russian Federation https://orcid.org/0009-0008-8173-3359
Yuriy V. Khanenko S.P. Korolyov Samara National Research University, Samara, Russian Federation https://orcid.org/0009-0000-9741-1444
Victor A. Soifer S.P. Korolyov Samara National Research University, Samara, Russian Federation https://orcid.org/0000-0003-4239-4389

DOI:

https://doi.org/10.14529/jsfi260208

Keywords:

optical computing, diffractive neural network, image processing and classification, spatial light modulator, YOLO neural networks, video-streams

Abstract

This paper investigates the design principles of analog photonic computing devices for pattern recognition tasks. As a result of the conducted research, an analog photonic computing device (APCD) based on a diffractive neural network (DNN) is developed and implemented in two configurations. The DNN is realized using a phase spatial light modulator (SLM) placed in the Fourier plane of a two-lens optical system (4F system). Benefits of combining an optical DNN with a compact computer neural network for post-processing the optical recognition results are demonstrated. The presented results show the effectiveness of using the APCD for processing and recognition of high-dimensional imagery in real-time video streams. A comparative performance analysis of the APCD and a family of GPU-based YOLO neural networks is conducted for object image recognition in video streams. The results demonstrate that when processing high-dimensional frames, the power consumption of the APCD is an order of magnitude lower than that of modern GPU-class graphics platforms such as the RTX 4090.

References

Shen, Y., Harris, N., Skirlo, S., et al.: Deep learning with coherent nanophotonic circuits. Nature Photon. 11, 441–446 (2017). https://doi.org/10.1038/nphoton.2017.93

Harris, N.C., Carolan, J., Bunandar, D., et al.: Linear programmable nanophotonic processors. Optica 5(12), 1623–1631 (2018). https://doi.org/10.1364/OPTICA.5.001623

Zhu, H.H., Zou, J., Zhang, H., et al.: Space-efficient optical computing with an integrated chip diffractive neural network. Nat. Commun. 13(1), 1044 (2022). https://doi.org/10.1038/s41467-022-28702-0

Kazanskiy, N.L., Khorin, P.A., Golovastikov, N.V., Khonina, S.N.: Analog optical computing: principles, progress, and prospects. Opt. Laser Technol. 193(A), 114220 (2026). https://doi.org/10.1016/j.optlastec.2025.114220

Lin, X., Rivenson, Y., Yardimci, N.T., et al.: All-optical machine learning using diffractive deep neural networks. Science 361(6406), 1004–1008 (2018). https://doi.org/10.1126/science.aat8084

Chang, J., Sitzmann, V., Dun, X., et al.: Hybrid optical-electronic convolutional neural networks with optimized diffractive optics for image classification. Sci. Rep. 8, 12324 (2018). https://doi.org/10.1038/s41598-018-30619-y

Yan, T., Wu, J., Zhou, T., et al.: Fourier-space diffractive deep neural network. Phys. Rev. Lett. 123(2), 023901 (2019). https://doi.org/10.1103/physrevlett.123.023901

Luo, Y., Mengu, D., Yardimci, N.T., et al.: Design of task-specific optical systems using broadband diffractive neural networks. Light Sci. Appl. 8, 112 (2019). https://doi.org/10.1038/s41377-019-0223-1

Zhou, T., Fang, L., Yan, T., et al.: In situ optical backpropagation training of diffractive optical neural networks. Photon. Res. 8(6), 940–953 (2020). https://doi.org/10.1364/PRJ.389553

Mengu, D., Zhao, Y., Yardimci, N.T., et al.: Misalignment resilient diffractive optical networks. Nanophotonics 9(13), 4207–4219 (2020). https://doi.org/10.1515/nanoph-2020-0291

Zhou, T., Lin, X., Wu, J., et al.: Large-scale neuromorphic optoelectronic computing with a reconfigurable diffractive processing unit. Nat. Photonics 15(5), 367–373 (2021). https://doi.org/10.1038/s41566-021-00796-w

Chen, H., Feng, J., Jiang, M., et al.: Diffractive deep neural networks at visible wavelengths. Engineering 7(10), 1483–1491 (2021). https://doi.org/10.1016/j.eng.2020.07.032

Zheng, S., Xu, S., Fan, D.: Orthogonality of di ractive deep neural network. Opt. Lett. 47(7), 1798–1801 (2022). https://doi.org/10.1364/OL.449899

Motz, G.A., Doskolovich, L.L., Soshnikov, D.V., et al.: Design of diffractive neural networks for solving different classification problems at different wavelengths. Photonics 11(8), 780 (2024). https://doi.org/10.3390/photonics11080780

Soshnikov, D.V., Doskolovich, L.L., Motz, G.A., et al.: Designing robust diffractive neural networks with improved transverse shift tolerance. J. Opt. Soc. Am. A 42(5), 699–709 (2025). https://doi.org/10.1364/JOSAA.561906

Soshnikov, D.V., Doskolovich, L.L., Motz, G.A., et al.: Design of DOE robust to positioning errors for optical classification purposes. Computer Optics 50(1), 1693 (2026). https://doi.org/10.18287/COJ1693

Lohmann, A.W., Paris, D.P.: Binary Fraunhofer holograms, generated by computer. Appl. Opt. 6(10), 1739–1748 (1967). https://doi.org/10.1364/ao.6.001739

Fienup, J.R.: Phase retrieval algorithms: a comparison. Appl. Opt. 21(15), 2758–2769 (1982). https://doi.org/10.1364/ao.21.002758

Soifer, V.A., Kotlyar, V.V., Doskolovich L.L.: Iterative methods for diffractive optical elements computation. Taylor & Francis (1997). https://doi.org/10.1201/9781482272918

Ripoll, O., Kettunen V., Herzig, H.P.: Review of iterative Fourier transform algorithms for beam shaping applications. Opt. Eng. 43(11), 2549–2556 (2004). https://doi.org/10.1117/1.1804543

Glses A.A., Jenkins, B.K.: Cascaded diffractive optical elements for improved multiplane image reconstruction. Appl. Opt. 52(15), 3608–3616 (2013). https://doi.org/10.1364/AO.52.003608

Wang, H., Piestun, R.: Dynamic 2D implementation of 3D diffractive optics. Optica 5(10), 1220–1228 (2018). https://doi.org/10.1364/OPTICA.5.001220

Doskolovich, L.L., Mingazov, A.A., Byzov, E.V., et al.: Hybrid design of diffractive optical elements for optical beam shaping. Opt. Express 29(20), 31875–31890 (2021). https://doi.org/10.1364/oe.439641

Kazanskiy, N.L., Khonina, S.N., Butt, M.A.: Exploring the functional characteristics of diffractive optical element: A comprehensive review. Opt. Laser Technol. 183, 112383 (2025). https://doi.org/10.1016/j.optlastec.2024.112383

Zheng, M., Shi, L., Zi, J.: Optimize performance of a diffractive neural network by controlling the Fresnel number. Photonics Res. 10(11), 2667–2676 (2022). https://doi.org/10.1364/PRJ.474535

Lugt, A.V.: Signal detection by complex spatial filtering. IEEE Trans. Inf. Theory 10(2), 139–145 (1964). https://doi.org/10.1109/TIT.1964.1053650

Kussul, E., Baidyk, T.: Improved method of handwritten digit recognition tested on MNIST database. Image Vis. Comput. 22(12), 971–981 (2004). https://doi.org/10.1016/j.imavis.2004.03.008

LeCun, Y., Cortes, C., Burges, C.J.C.: The MNIST database of handwritten digits. https://web.archive.org/web/20210407152035/http://yann.lecun.com/exdb/mnist/, accessed: 2026-05-14

LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998). https://doi.org/10.1109/5.726791

Zhang, Y., et al.: Memory-less scattering imaging with ultrafast convolutional optical neural networks. Sci. Adv. 10(24), eadn2205 (2024). https://doi.org/10.1126/sciadv.adn2205

Zhang, Z., Feng, F., Gan, J., et al.: Space-time projection enabled ultrafast all-optical diffractive neural network. Laser Photonics Rev. 18(8), 2301367 (2024). https://doi.org/10.1002/lpor.202301367

Li, B., Zhu, Y., Fei, J., et al.: Multi-functional broadband diffractive neural network with a single spatial light modulator. APL Photonics 10(1), 016115 (2025). https://doi.org/10.1063/5.0245832

Goncharov, D.S., Starikov, R.S., Zlokazov, E.Y.: Pattern recognition system based on a coherent diffractive correlator with deep learned processing of downsampled correlation responses. Appl. Opt. 63(36), 9196–9204 (2024). https://doi.org/10.1364/AO.541305

Volkov, A.A., et al.: Optoelectronic spatial filtering system for neural network applications. Vestnik natsional'nogo issledovatel'skogo yadernogo universiteta 'MIFI' 15(2), 134–142 (2026). https://doi.org/10.26583/vestnik.2026.2.4

Wan, L., Zeiler, M., Zhang, S., et al.: Regularization of neural network using DropConnect. In: Dasgupta, S., McAllester, D. (eds.) Proceedings of the 30th International Conference on Machine Learning, ICML 2013, Atlanta, Georgia, USA, June 17-19, 2013. vol. 28, pp. 1058–1066 (2013).

Kowsari, K., Heidarysafa, M., Brown, D.E., et al.: RMDL: Random multimodel deep learning for classification. In: ACM International Conference Proceeding Series, ICISDM18, Lakeland, FL, USA, April 9-11, 2018. pp. 19–28 (2018). https://doi.org/10.1145/3206098.3206111

Wang, G., et al.: TridentYOLO: Improving the precision and speed of mobile device object detection. IET Image Process. 16(1), 145–157 (2022). https://doi.org/10.1049/ipr2.12340

Vychegzhanin, S.V., Tatarinova, A.G., Myshkin, R.E.: Comparative analysis of neural network models for detecting unmanned aerial vehicles. Computer Optics 50(2), 1728 (2026). https://doi.org/10.18287/COJ1728

https://technicalcity.b-cdn.net/ru/video/Tesla-V100-PCIe-32-GB-protiv-H100-PCIe-80-GB , accessed: 2025-03-04