Are you interested in nanotechnology based AI processors?

Sign up for our newsletter for progress updates, product announcements and news.

Knowm Memristor Array

Neuromorphic Computing Symposium 2017 – Architectures, Models and Applications

In part a response to the Nanotechnology-Inspired Grand Challenge for Future Computing initiative, some of the key players in the neuromorphic and neuromemristive space gathered at the Crowne Plaza in Knoxville, Tennessee to present new research, develop collaborations, and provide a forum to publish work in this area. The symposium was heavily attended by University of Tennessee and Oakridge Laboratory researchers, as expected, but also drew participants from around the country. Knowm Inc was lucky to have the opportunity to sponsor the dinner with Duke university and provide a great ice-breaking atmosphere of free-flowing wine and great food to encourage idea exchange and future collaborations. If the USA is going to lead this race, it will require a great deal of teamwork.

Neuromorphic Symposium Dinner Sponsored by Knowm Inc and Duke University, Knoxville Tennessee, July 2017

Stan Williams – HP

Stan Williams gave a very strong presentation and made great contributions to the discussion over multiple days. Williams reviewed HP’s continued involvement in the development of memristive technology and gave a rallying cry for the Gen-Z Consortium, which is “an open systems interconnect designed to provide memory semantic access to data and devices via direct-attached, switched or fabric topologies.” Williams presented impressive results from HP collaborators at UMass Amherst (which we saw for the first time at the ECE Conference), which included (1) programmable memristive crossbars, which they are demonstrating for use in image processing applications as well as (2) what they call “competitive computing” where they have demonstrated simple learning circuits. While the similarity to AHaH Computing has not escaped our attention, we are excited that others are converging on the same ideas.

Catherine Schuman

Catherine Schuman was out in force, presenting multiple times and leading a session. Of particular interest was (1) her absolutely massive literature review of the neuromorphic computing field, which will be arriving as a review paper shortly and (2) her group’s results showing their genetic algorithms actually beat convolutional neural networks on a non-trivial benchmark task.

Dhireesha Kudithipudi

Dhireesha Kudithipudi presented results from a systems-level integration of deep convolutional networks and reservoirs for transfer learning, showing results that bested more traditional ML techniques and provides a path toward efficient hardware implementations for hand-held devices.

Jack Kendall

Jack Kendall presented some refreshing ‘out of the crossbar’ thinking from his start-up company Rain Neuromorphics, an offshoot of work at University of Florida. The core idea is to take advantage of the oxides that form over nanowires laid down in a ‘rats nest’ to form long-range memristive connections. By taping this tangle of nanowires with electrode ‘pilars’, we have a new type of all-to-all connective not conditioned on the typical cross-bar arrangement and potentially offering new possibilities. The group has shown (theoretically) how such a network could implement back-propagation of error.

Alex Nugent

Knowm CEO Alex Nugent presented recent work on a new form of compositional (deep) learning not based on the backprop algorithm that can more easily be mapped to neuromemristive processors such as kT-RAM as well as presented a poster on the open-source Memristor Discovery platform.

Beyond CMOS: From Devices to Systems

The Beyond CMOS: From Devices to Systems conference on June 5-6, 2017 took place in Meyer Building (EE) Technion, Haifa, Israel. Knowm’s Tim Molter attended all the memristor talks and presented a poster and demonstration of Knowm’s Memristor Discovery to interested parties. All in all, there was a lot of excitement about using memristors to go “beyond Moore’s law and CMOS”, and from Knowm’s point of view the industry is continuing to converge on the idea that using memristor pairs (instead of single memristors) for synapses is the way forward. Many new device types were presented and well as some rudimentary machine learning demos and STDP behavior.

Tim’s poster can be downloaded here: Beyond_CMOS_2017_Knowm_Tim_Molter_Poster.pdf.

Giovanni De​ Micheli, EPFL – Emerging Technologies for Computing

Giovanni De Micheli is Professor and Director of the Institute of Electrical Engineering and of the Integrated Systems Centre at EPF Lausanne, Switzerland. He is program leader of the Nano-Tera.ch program.

1. Tri-gate transistors – lower leakage, faster
2. Box (buried oxide) transistors
3. 7 to 5 nm nodes
4. silicon nanowire transistor
5. Carbon nanotube transistor
6. Oxide resistive RAMs TaOx
7. Switch from von Neumann arch.

Onur Mutlu, ETH Zurich/CMU – Rethinking Memory System Design (and the Computing Platforms We Design Around It)

1. DRAM “row hammer” security problem in dense arrays – flipping bits really fast before a DRAM cap refresh occurs affects adjacent rows.
2. Manufacturers had to release patches to fix this security hole
3. Need a paradigm shift to eleiminate data movement (von Neumann architecture) – more memory-centric
4. In-memory page copying – eliminate round trip to CPU.
5. Orders of magnitude improvements for new memory-centric architectures

Engin Ipek, University of Rochester – Memristive Accelerators for Data Intensive Computing: From Machine Learning to High-Performance Linear Algebra

1. In-Situ Memristive Accelerators
2. Combinatorial optimization
3. Analog dot product computation with memristors
4. First time we saw Htree-Controller-RAM on a slide

Michael Kozicki, Arizona State University – Applications of Cu-SiO2 Devices in Electronic Systems

1. Cu-silica-Pt devices – low temp process
2. Schottky diode produced by Cu electrode for eliminating cross talk
3. 10 pA programming current – took a week to program though.
4. nice incremental response – relatively linear and symmetric – for neuromorphics
5. CBRAM by Adesto
6. 3d cross point – coming soon

Thomas Mikolajick, Namlab and TU Dresden – Nonvolatile Memories based on Ferroelectricity in Hafnium Oxide and Zirconium Oxide

1. Ferroelectric memory – for niche applications
2. hysteresis observed in HfOx doped ferroelectrics
3. HfOx Ferroelectric memory potentially optimal for new memory
4. Fe-FET for STDP operated as synapse

Eli Yablonovitch, UC Berkeley – The Scientific Challenge of Replacing the Transistor with a Lower Voltage Device

1. Landauer’s limit says all electronics are operating 10,000 times more energy budgets than they need to.
2. Canter for Energy Efficient Electronics Science
3. At the nanoscale we need atomic scale precision

Michal Lipson, Columbia University – Next Generation Silicon Photonics

1. Data Center Crisis – Microsoft dropping their datacenters in the ocean
2. Power dominated by the communication from RAM
3. CMOS ring waveguides of different diameters capture different wavelengths, all other wavelengths pass by.
4. Inject a voltage signal in the waveguide and the index of refraction changes

Ioannis Vourkas, PCUC – Studying the Resistive Switching Behavior from the Energy Point of View

1. Stochastic Nature of Resistive Switch
2. Energy require to switch state not dependent on applied signal rise time, but the energy required is different
3. Knowm memristor showing hysteresis response

Mircea Stan, University of Virginia – Back to the Future: Digital Circuit Design in the FinFET Era

1. FinFETs make CMOS doable at 0.2V Vdd
2. Independent-gated finFET can act as a logic gate
3. FinFETs allowing continuation of Moore’s Law
4. The fins are getting closer together AND higher
5. Intel calls this hyperscaling
6. Aspect ratio of 25 demoed in lab

Pierre-Emmanuel Gaillardon, University of Utah – Towards Lower-Power High-Performance FPGAs with RRAM-Based Routing

1. RRAM – non-volatile storage, radiation tolerant, engineered properties
2. FPGAs inefficient architectures – routing and routing multiplexors
3. Replace switches (transmission gates) in MUX with RRAM
4. In RAM 4T1R can actually give you more density than 2T1R when comparing power density

Edwin Kan, Cornell – Hybrid Ferroelectrics-CMOS: Devices and Circuits

Stuart Parkin, Max Planck Institute – Chiral Spin-Orbitronics

1. MTJ – magnetic tunneling junction
2. 2016 MRAM now commercially available
3. $1B in funds to make it possible
4. 3d racetrack memory increases memory density by 100x

Leon Chua, UC Berkeley – 10 Things You Didn’t Know About Memristors

1. The only valid DC-cureve for a memristor is a single point at the (0,0) origin
2. egg white can be used as a memritor
3. Not all memristors are non-volatile
4. Synapses are memristors
5. Axons are made of memristors
6. Some memristor circuits have Hamiltonians

Damien Querlioz, Paris-Sud University – Neuromorphic Computing with Spintronics

1. ST-MRAM is reaching maturity
2. using magnetic tunnel junctions as synapses – The MJTs are binary as far as I can tell, so not like synapses at all.
3. cross bar architecture where MTJs are at each cross. Simplified STDP for signals.
4. Detecting cars in multiple lanes, but this is such a trivial demo, I don’t believe M.L is even necessary – completely linearly seperable.

Jennifer Rupp, MIT – Defect Engineering for Oxide-Memristors

1. Adesto maviq 512 kb memristor RAM
2. Memristor pairs as synapses – addressing each memristor individually and as a pair

Sabina Spiga, CNR-IMM – Tailoring RRAM for Neuromorphic Computing

1. HfOxRRAM – Hafium
2. Analog synapses – incremental response to pulses.
3. Their memristors look like good condidates for kT-RAM
4. MNIST benchmark on a real chip with memritsors connected to separate CMOS chip via pads 65% performance

Giacomo Indiveri, ETH Zurich – Real-Time On-Line Learning and Computing in Event-Based Neuromorphic Memristive Architectures

1. Speed not a requirement for brains
2. Bees – 10^-15 J/spike, 960,000 neurons, 1 mm^3 volume, 1 mg weight
3. asynchronous self-clocked circuits, non von Neumann acrchs., co-localized memory and computing elements, new programming pardigms.
4. 28 nm FD_SOI multi-core Neuromorphic processor
5. Shows many simulated M.L capabilities: robotic control, Comb. Opt.
6. Attractor networks to do computation

Abu Sebastian, IBM – Phase-Change Memory: Applications in Neuromorphic Computing

1. Watson – 2880 core, 80 kW of power, 20 tons air conditioning cooling capacity
2. NOt interested in exact results, in place computing provide approximations that are good enough
3. PCM core for MNIST solver neural network
4. spiking neural networks – what re the killer apps is the #1 challenge
5. IBM TrueNorth – no in-situ learning – PCM neuron can help here
6. 64k PCM core built at IBM

Hai Li, Duke University – Emerging RRAM Technology Enabled Cognitive Computing Systems: Challenges and Opportunities

1. Memristor crossbar to mimic neural network
2. “Brain-Ste-in-a-box”
3. Using two cross bar arrays –
4. level based design use digital to analog voltage applied to synapse inputs
5. Working with HP labs

Massimiliano Di Ventra, UCSD – Memcomputing: a Brain-Inspired Topological Computing Paradigm

1. Memcomputing Inc. Fabio Travera, John Beane – brain inspired topological computing
2. NP-complete problems – standard alg.=2^N explosion, memcomputing=N^4
3. NP hard problems in polinomial time – 2 orders of magnitude better than competitors in competitions
4. random initial conditions evolve over time and converge on the solution
5. Like quantum computers but ready to be built today – “classical objects”
6. Seems like smoke and mirrors to me in the way he presented it. No attempt to explain it to a layman.

EMN Europe Meetings

EMN Memristive Switching & Networks

The EMN Memristive Switching & Networks on August 14-18, 2017 took place in Milan Italy. The memristor session was relatively small, but a lot of exciting research and advancements was presented.

Martin Klimo – University of Zilina, Slovakia – Memristive Implementation of Fuzzy Logic and Neural Networks

1. Von Neumann architecture good for arithmetic/algorithmic computing.
2. Cognitive computing requires a different kind of hardware.
3. Both types of architecture will cooperate.
4. Fuzzy logic system uses nearest neighbor pattern matching calculating distances from the input data to the feature library.
5. The next challenge is to reduce the number of memristsors needed to hold all the patterns.

Masayoshi Higuchi – National Institute for Materials Science, Japan – Metallo supramolecule polymers as memristor

1. Metallo supramolecule polymers as memristor.
2. The color of the material can be changed by applying a 1V signal to it (electrochromism).
3. Using different metals creates different color ranges.
4. Color state remains after removing voltage signal.
5. The material exhibits a hysteresis loop like a memristor.

Mehdi Anwar – University of Connecticut, USA – Memristors: Material, Device, and Systems

1. Creation of Zinc Oxide memristors as nanowires.
2. CMOS compatible. MOCVD growth. ZnO memristor (horizontal nano-cone) LRS=189 Ohm, HRS = 129 kOhm.
3. Filamentary dictated transport.
4. Most of the work they do is on secure electronics: physical unclonable functions (PUF).

Gennady N. Panin – Russian Academy of Sciences, Russia – Photomemristive Systems Based on Two-Dimensional Crystals

1. Memory based on graphene is bio-compatible and can self assemble.
2. Current < 1 nA, switching voltage < 0.7V. I-V characteristics dependent on light excitation.

Ken Saito – Nihon University, Japan – Artificial Insect Equipped with Hardware Neural Networks

1. Miniature nano robots will be used for medical applications inside human bodies.
2. Body parts are all made of silicon wafer material. Locomotion is like an ant walks: alternating tripod gait.
3. Temperature sensitive wire coils expand and shrink.
4. Future work will integrate the power source, sensory and controlling system on the robot as well.
5. Electrostatic inchworm motors.

Kouichi Takase – Nihon University, Japan – Switching property of nanowire based ReRAM

1. Trying to figure out why the switching voltage of oxide memristors as such a large distribution.
2. He thinks is the many possible filamentary connections that form during each cycle.
3. His solution was to first coat the wafer surface with vertically-standing 500 nm nano wires densely packed.
4. This produced good switching windows (no overlap between Von and Voff).
5. 5 minutes of oxidation additionally improves the switching voltage distribution (narrower).

Josep L. Rosselló – Balearic Islands University, Spain – Hardware Neural Network: Theory and Applications

1. General talk on STDP-based memristor crossbar neural networks.
2. Memristors provides the highest performance but has the lowest flexibility and endurance.
3. Stochastic spiking neuron (SSN) was implemented in FPGA.
4. Successful results on handwritten digit benchmarks.

Nan Du – University of Technology Chemnitz, Germany – Resistive Switching dynamics in memristive BiFeO3 (BFO) Device

1. Analog bipolar resistive switching device.
2. Electro-forming free.
3. 10000+ cycle endurance perfect.
4. Set/reset is +/- 8 V.
5. All 16 logic functions realized with bilayer memristor.
6. STDP functions demonstrated.
   vReally good looking bidirectional incremental behavior.

Alex Nugent – Knowm Inc.,USA – Memristors to Compositional Learning

1. Introduction to Knowm’s Memristor symbol
2. Define life as self-organizing mechanism to harness free energy and intelligence as the ability to learn
3. The future of computing is asking for synapses – lot of synapses
4. Introduced the kT-Synapse – in nature and as a memristor differential pair and showed many examples of other researchers converging on the idea as well.
5. Introduced compositional learning with kT-RAM
6. Talked about Knowm’s full stack AI framework from memristors to deep learning
7. Introduce Knowm memristors and Memristor Discovery

2017 USENIX Vail Computer Elements Workshop

The USENIX Vail Computer Elements Workshop is a unique four day workshop that has been around for 47 years serving leading architects of the computer industry. This intentionally small workshop is intended to allow a lively interaction between the participants and the speakers.

The venue was informal, lively and engaging and certainly unlike most conference we have been to.

Alex Nugent and Tim Molter enjoying some downhill mountain biking at the Vale Colorado 2017 USENIX Computer Elements Workshop