Neuromorphic Computing Based on CMOS-Integrated Memristive Arrays: Current State and Perspectives
DOI:
https://doi.org/10.14529/jsfi230206Keywords:
memristor, CMOS integration, neuromorphic hardware, artificial intelligenceAbstract
The paper presents an analysis of current state and perspectives of high-performance computing based on the principles of information storage and processing in biological neural networks, which are enabled by the new micro- and nanoelectronics component base. Its key element is the memristor (associated with a nonlinear resistor with memory or Resistive Random Access Memory (RRAM) device), which can be implemented on the basis of different materials and nanostructures compatible with the complementary metal-oxide-semiconductor (CMOS) process and allows computing in memory. This computing paradigm is naturally implemented in neuromorphic systems using the crossbar architecture for vector-matrix multiplication, in which memristors act as synaptic weights – plastic connections between artificial neurons in fully connected neural network architectures. The general approaches to the development and creation of a new component base based on the CMOS-integrated RRAM technology, development of artificial neural networks and neuroprocessors using memristive crossbar arrays as computational cores and scalable multi-core architectures for implementing both formal and spiking neural network algorithms are discussed. Technical solutions are described that enable hardware implementation of memristive crossbars of sufficient size, as well as solutions that compensate for some of the deficiencies or fundamental limitations inherent in emerging memristor technology. The performance and energy efficiency are analyzed for the reported prototypes of such neuromorphic systems, and a significant (orders of magnitude) gain in these parameters is highlighted compared to the computing systems based on traditional component base (including neuromorphic ones). Technological maturation of a new component base and creation of memristor-based neuromorphic computing systems will not only provide timely diversification of hardware for the continuous development and mass implementation of artificial intelligence technologies but will also enable setting the tasks of a completely new level in creating hybrid intelligence based on the symbiosis of artificial and biological neural networks. Among these tasks are the primary ones of developing brain-like self-learning spiking neural networks and adaptive neurointerfaces based on memristors, which are also discussed in the paper.
References
Neurochip “Altai”. https://motivnt.ru/neurochip-altai/, accessed: 2023-05-15
NVIDIA Tesla V100 GPU architecture the world’s most advanced data center GPU. https://www.nvidia.cn/content/dam/en-zz/Solutions/Data-Center/tesla-product-literature/volta-architecture-whitepaper.pdf, accessed: 2023-05-15
Akopyan, F., Sawada, J., Cassidy, A., et al.: TrueNorth: Design and Tool Flow of a 65 mW 1 Million Neuron Programmable Neurosynaptic Chip. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 34(10), 1537–1557 (oct 2015). https://doi.org/10.1109/TCAD.2015.2474396
Amirsoleimani, A., Alibart, F., Yon, V., et al.: In-Memory Vector-Matrix Multiplication in Monolithic Complementary MetalOxideSemiconductor-Memristor Integrated Circuits: Design Choices, Challenges, and Perspectives. Advanced Intelligent Systems 2(11), 2000115 (nov 2020). https://doi.org/10.1002/AISY.202000115
Bianchi, S., Muñz-Martin, I., Covi, E., et al.: A self-adaptive hardware with resistive switching synapses for experience-based neurocomputing. Nature Communications 14(1), 1–14 (mar 2023). https://doi.org/10.1038/s41467-023-37097-5
Bocquet, M., Hirztlin, T., Klein, J.O., et al.: In-Memory and Error-Immune Differential RRAM Implementation of Binarized Deep Neural Networks. Technical Digest - International Electron Devices Meeting, IEDM 2018-December, 20.6.1–20.6.4 (jan 2019). https://doi.org/10.1109/IEDM.2018.8614639
Cai, F., Correll, J.M., Lee, S.H., et al.: A fully integrated reprogrammable memristor-CMOS system for efficient multiply-accumulate operations. Nature Electronics 2(7), 290–299 (jul 2019). https://doi.org/10.1038/s41928-019-0270-x
Cai, F., Yen, S.H., Uppala, A., et al.: A Fully Integrated System-on-Chip Design with Scalable Resistive Random-Access Memory Tile Design for Analog in-Memory Computing. Advanced Intelligent Systems 4(8), 2200014 (aug 2022). https://doi.org/10.1002/AISY.202200014
Chen, W.H., Dou, C., Li, K.X., et al.: CMOS-integrated memristive non-volatile computingin-memory for AI edge processors. Nature Electronics 2(9), 420–428 (aug 2019). https://doi.org/10.1038/s41928-019-0288-0
Chiaradia, I., Lancaster, M.A.: Brain organoids for the study of human neurobiology at the interface of in vitro and in vivo. Nature Neuroscience 23(12), 1496–1508 (nov 2020). https://doi.org/10.1038/s41593-020-00730-3
Chiolerio, A., Chiappalone, M., Ariano, P., Bocchini, S.: Coupling resistive switching devices with neurons: State of the art and perspectives. Frontiers in Neuroscience 11(FEB), 70 (feb 2017). https://doi.org/10.3389/FNINS.2017.00070/BIBTEX
Christensen, D.V., Dittmann, R., Linares-Barranco, B., et al.: 2022 roadmap on neuromorphic computing and engineering. Neuromorphic Computing and Engineering 2(2), 022501 (may 2022). https://doi.org/10.1088/2634-4386/AC4A83
Chua, L.O.: Memristor-The Missing Circuit Element. IEEE Transactions on Circuit Theory 18(5), 507–519 (1971). https://doi.org/10.1109/TCT.1971.1083337
Chua, L.O., Kang, S.M.: Memristive Devices and Systems. Proceedings of the IEEE 64(2), 209–223 (1976). https://doi.org/10.1109/PROC.1976.10092
Demin, V.A., Erokhin, V.V.: Hidden symmetry shows what a memristor is. International Journal of Unconventional Computing 12, 433–438 (2016)
Demin, V.A., Nekhaev, D.V., Surazhevsky, I.A., et al.: Necessary conditions for STDP-based pattern recognition learning in a memristive spiking neural network. Neural Networks 134, 64–75 (feb 2021). https://doi.org/10.1016/J.NEUNET.2020.11.005
George, R., Chiappalone, M., Giugliano, M., et al.: Plasticity and Adaptation in Neuromorphic Biohybrid Systems. iScience 23(10), 101589 (oct 2020). https://doi.org/10.1016/J.ISCI.2020.101589
Guggisberg, A.G., Koch, P.J., Hummel, F.C., Buetefisch, C.M.: Brain networks and their relevance for stroke rehabilitation. Clinical Neurophysiology 130(7), 1098–1124 (jul 2019). https://doi.org/10.1016/J.CLINPH.2019.04.004
Ham, D., Park, H., Hwang, S., Kim, K.: Neuromorphic electronics based on copying and pasting the brain. Nature Electronics 4(9), 635–644 (sep 2021). https://doi.org/10.1038/s41928-021-00646-1
Huang, Y., Kiani, F., Ye, F., Xia, Q.: From memristive devices to neuromorphic systems. Applied Physics Letters 122(11), 110501 (mar 2023). https://doi.org/10.1063/5.0133044/2880793
Hung, J.M., Xue, C.X., Kao, H.Y., et al.: A four-megabit compute-in-memory macro with eight-bit precision based on CMOS and resistive random-access memory for AI edge devices. Nature Electronics 4(12), 921–930 (dec 2021). https://doi.org/10.1038/s41928-021-00676-9
Kim, J., Pershin, Y.V., Yin, M., et al.: An Experimental Proof that Resistance-Switching Memory Cells are not Memristors. Advanced Electronic Materials 6(7), 2000010 (jul 2020). https://doi.org/10.1002/AELM.202000010
Krasnikov, G.Y.: The capabilities of microelectronic processes with 5 nm critical dimension and less. Nanoindustry Russia 13(S5-1(102)), 13–19 (2020)
Krishnan, G., Mandal, S.K., Pannala, M., et al.: SIAM: Chiplet-based Scalable In-Memory Acceleration with Mesh for Deep Neural Networks. ACM Transactions on Embedded Computing Systems (TECS) 20(5s) (sep 2021). https://doi.org/10.1145/3476999
LaPedus, M.: Chiplet Momentum rising. Semiconductor Engineering. https://semiengineering.com/chiplet-momentum-rising/ (2020), accessed: 2022-10-28
Li, C., Hu, M., Li, Y., et al.: Analogue signal and image processing with large memristor crossbars. Nature Electronics 1(1), 52–59 (dec 2017). https://doi.org/10.1038/s41928-017-0002-z
Liu, Q., Gao, B., Yao, P., et al.: A Fully Integrated Analog ReRAM Based 78.4TOPS/W Compute-In-Memory Chip with Fully Parallel MAC Computing. Digest of Technical Papers - IEEE International Solid-State Circuits Conference 2020-February, 500–502 (feb 2020). https://doi.org/10.1109/ISSCC19947.2020.9062953
Liu, X., Zeng, Z.: Memristor crossbar architectures for implementing deep neural networks. Complex and Intelligent Systems 8(2), 787–802 (apr 2022). https://doi.org/10.1007/S40747-021-00282-4/TABLES/7
Liu, Z., Tang, J., Gao, B., et al.: Multichannel parallel processing of neural signals in memristor arrays. Science Advances 6(41) (oct 2020). https://doi.org/10.1126/SCIADV.ABC4797/SUPPL_FILE/ABC4797_SM.PDF
Liu, Z., Tang, J., Gao, B., et al.: Neural signal analysis with memristor arrays towards high-efficiency brain-machine interfaces. Nature Communications 11(1), 1–9 (aug 2020). https://doi.org/10.1038/s41467-020-18105-4
Makarov, V.A., Lobov, S.A., Shchanikov, S., et al.: Toward Reflective Spiking Neural Networks Exploiting Memristive Devices. Frontiers in Computational Neuroscience 16, 62 (jun 2022). https://doi.org/10.3389/FNCOM.2022.859874/BIBTEX
Mehonic, A., Kenyon, A.J.: Brain-inspired computing needs a master plan. Nature 604(7905), 255–260 (apr 2022). https://doi.org/10.1038/s41586-021-04362-w
Mikhaylov, A., Pimashkin, A., Pigareva, Y., et al.: Neurohybrid memristive cmos-integrated systems for biosensors and neuroprosthetics. Frontiers in Neuroscience 14, 358 (apr 2020). https://doi.org/10.3389/FNINS.2020.00358/BIBTEX
Miranda, E., Suñé, J.: Memristors for Neuromorphic Circuits and Artificial Intelligence Applications. Materials 13(4), 938 (feb 2020). https://doi.org/10.3390/MA13040938
Mochida, R., Kouno, K., Hayata, Y., et al.: A 4M synapses integrated analog ReRAM based 66.5 TOPS/W neural-network processor with cell current controlled writing and flexible network architecture. Digest of Technical Papers - Symposium on VLSI Technology 2018-June, 175–176 (oct 2018). https://doi.org/10.1109/VLSIT.2018.8510676
Pei, J., Deng, L., Song, S., et al.: Towards artificial general intelligence with hybrid Tianjic chip architecture. Nature 572(7767), 106–111 (jul 2019). https://doi.org/10.1038/s41586-019-1424-8
Pfeiffer, P., Egusquiza, I.L., DI Ventra, M., et al.: Quantum memristors. Scientific Reports 6(1), 1–6 (jul 2016). https://doi.org/10.1038/srep29507
Pulvermüler, F., Tomasello, R., Henningsen-Schomers, M.R., Wennekers, T.: Biological constraints on neural network models of cognitive function. Nature Reviews Neuroscience 22(8), 488–502 (jun 2021). https://doi.org/10.1038/s41583-021-00473-5
Rao, M., Tang, H., Wu, J., et al.: Thousands of conductance levels in memristors integrated on CMOS. Nature 615(7954), 823–829 (mar 2023). https://doi.org/10.1038/s41586-023-05759-5
Roy, K., Jaiswal, A., Panda, P.: Towards spike-based machine intelligence with neuromorphic computing. Nature 575(7784), 607–617 (nov 2019). https://doi.org/10.1038/s41586-019-1677-2
Schegolev, A.E., Klenov, N.V., Soloviev, I.I., et al.: Superconducting Neural Networks: from an Idea to Fundamentals and, Further, to Application. Nanobiotechnology Reports 16(6), 811–820 (nov 2021). https://doi.org/10.1134/S2635167621060227/METRICS
Sebastian, A., Le Gallo, M., Khaddam-Aljameh, R., Eleftheriou, E.: Memory devices and applications for in-memory computing. Nature Nanotechnology 15(7), 529–544 (mar 2020). https://doi.org/10.1038/s41565-020-0655-z
Serb, A., Corna, A., George, R., et al.: Memristive synapses connect brain and silicon spiking neurons. Scientific Reports 10(1), 1–7 (feb 2020). https://doi.org/10.1038/s41598-020-58831-9
Shchanikov, S., Zuev, A., Bordanov, I., et al.: Designing a bidirectional, adaptive neural interface incorporating machine learning capabilities and memristor-enhanced hardware. Chaos, Solitons and Fractals 142, 110504 (jan 2021). https://doi.org/10.1016/J.CHAOS.2020.110504
Shen, J., Shang, D., Chai, Y., et al.: Nonvolatile Multilevel Memory and Boolean Logic Gates Based on a Single Ni/ [Pb (Mg1/3Nb2/3) O3] 0.7 [PbTiO3] 0.3 /Ni Heterostructure. Physical Review Applied 6(6), 064028 (dec 2016). https://doi.org/10.1103/PHYSREVAPPLIED.6.064028/FIGURES/5/MEDIUM
Spagnolo, M., Morris, J., Piacentini, S., et al.: Experimental photonic quantum memristor. Nature Photonics 16(4), 318–323 (mar 2022). https://doi.org/10.1038/s41566-022-00973-5
Su, F., Chen, W.H., Xia, L., et al.: A 462GOPs/J RRAM-based nonvolatile intelligent processor for energy harvesting IoE system featuring nonvolatile logics and processing-inmemory. Digest of Technical Papers - Symposium on VLSI Technology pp. C260–C261 (jul 2017). https://doi.org/10.23919/VLSIT.2017.7998149
Telminov, O., Gornev, E.: Possibilities and Limitations of Memristor Crossbars for Neuromorphic Computing. Proceedings - 6th Scientific School ”Dynamics of Complex Networks and their Applications”, DCNA 2022 pp. 278–281 (2022). https://doi.org/10.
/DCNA56428.2022.9923302
Vasileiadis, N., Ntinas, V., Sirakoulis, G.C., Dimitrakis, P.: In-Memory-Computing Realization with a Photodiode/Memristor Based Vision Sensor. Materials 14(18), 5223 (sep 2021). https://doi.org/10.3390/MA14185223
Vongehr, S., Meng, X.: The Missing Memristor has Not been Found. Scientific Reports 5(1), 1–7 (jun 2015). https://doi.org/10.1038/srep11657
Wan, W., Kubendran, R., Schaefer, C., et al.: A compute-in-memory chip based on resistive random-access memory. Nature 608(7923), 504–512 (aug 2022). https://doi.org/10.1038/s41586-022-04992-8
Wang, S., Li, Y., Wang, D., et al.: Echo state graph neural networks with analogue random resistive memory arrays. Nature Machine Intelligence 5(2), 104–113 (feb 2023). https://doi.org/10.1038/s42256-023-00609-5
Wang, Z., Wu, H., Burr, G.W., et al.: Resistive switching materials for information processing. Nature Reviews Materials 5(3), 173–195 (jan 2020). https://doi.org/10.1038/s41578-019-0159-3
Wu, T.F., Le, B.Q., Radway, R., et al.: 14.3 A 43pJ/Cycle Non-Volatile Microcontroller with 4.7μs Shutdown/Wake-up Integrating 2.3-bit/Cell Resistive RAM and Resilience Techniques. Digest of Technical Papers - IEEE International Solid-State Circuits Conference 2019-February, 226–228 (mar 2019). https://doi.org/10.1109/ISSCC.2019.8662402
Xia, Q., Yang, J.J.: Memristive crossbar arrays for brain-inspired computing. Nature Material 18(4), 309–323 (mar 2019). https://doi.org/10.1038/s41563-019-0291-x
Xue, C.X., Chen, W.H., Liu, J.S., et al.: 24.1 A 1Mb Multibit ReRAM Computing-In-Memory Macro with 14.6ns Parallel MAC Computing Time for CNN Based AI Edge Processors. Digest of Technical Papers - IEEE International Solid-State Circuits Conference 2019-February, 388–390 (mar 2019). https://doi.org/10.1109/ISSCC.2019.8662395
Xue, C.X., Chiu, Y.C., Liu, T.W., et al.: A CMOS-integrated compute-in-memory macro based on resistive random-access memory for AI edge devices. Nature Electronics 4(1), 81–90 (dec 2020). https://doi.org/10.1038/s41928-020-00505-5
Yao, P., Wu, H., Gao, B., et al.: Fully hardware-implemented memristor convolutional neural network. Nature 577(7792), 641–646 (jan 2020). https://doi.org/10.1038/s41586-020-1942-4
Yin, S., Sun, X., Yu, S., Seo, J.S.: High-Throughput In-Memory Computing for Binary Deep Neural Networks with Monolithically Integrated RRAM and 90-nm CMOS. IEEE Transactions on Electron Devices 67(10), 4185–4192 (oct 2020). https://doi.org/10.1109/TED.2020.3015178
Zhang, W., Gao, B., Tang, J., et al.: Neuro-inspired computing chips. Nature Electronics 3(7), 371–382 (jul 2020). https://doi.org/10.1038/s41928-020-0435-7
Zhao, H., Liu, Z., Tang, J., et al.: Energy-efficient high-fidelity image reconstruction with memristor arrays for medical diagnosis. Nature Communications 14(1), 1–10 (apr 2023). https://doi.org/10.1038/s41467-023-38021-7
Zhu, X., Wang, Q., Lu, W.D.: Memristor networks for real-time neural activity analysis. Nature Communications 11(1), 1–9 (may 2020). https://doi.org/10.1038/s41467-020-16261-1
Zhuk, M., Zarubin, S., Karateev, I., et al.: On-Chip TaOx-Based Non-volatile Resistive Memory for in vitro Neurointerfaces. Frontiers in Neuroscience 14, 94 (feb 2020). https://doi.org/10.3389/FNINS.2020.00094/BIBTEX
Zidan, M.A., Fahmy, H.A.H., Hussain, M.M., Salama, K.N.: Memristor-based memory: The sneak paths problem and solutions. Microelectronics Journal 44(2), 176–183 (feb 2013). https://doi.org/10.1016/J.MEJO.2012.10.001
Downloads
Published
How to Cite
License
Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution-Non Commercial 3.0 License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.