Lattice CrossLink-NX: Focus on Embedded Vision Processing at the Network Edge

The network edge has quickly become a vital part of the future of AI processing. As 5G connects billions of devices, interconnection will be everywhere. Market interest in applied artificial intelligence and machine learning (AI/ML) is growing, and there is a huge demand for low-power, high-tech devices with AI/ML processing capabilities running at the network edge. Processing at the edge of the network can greatly reduce latency for terminals and end users, and better protect user privacy. In addition, smart devices at the network edge can filter data sent to the cloud, reducing network costs and bandwidth requirements.

Most of these network edge smart devices use image sensors to enable a variety of embedded vision applications, including AI/ML-based applications such as object counting and presence detection. However, supporting embedded vision applications at the network edge requires devices with certain design and performance characteristics: such as low power consumption, high performance, high reliability, and small size. For such applications, Lattice has introduced the new CrossLink-NX family of FPGAs. The new chips are designed to meet trends in video processing: mixing multiple sensors and Displays, higher video resolution, use of multiple interfaces, and AI processing at the network edge.

CrossLink meets Nexus

To help developers support new and existing embedded vision systems, Lattice has introduced CrossLinkPlus, a family of reduced, small form factor, low power FPGA products.

Figure 1: CrossLinkPlus vs CrossLink-NX

The programmable logic of CrossLinkPlus can meet the processing needs and also supports various interface standards, making it the choice for video signal aggregation and image co-processing in network edge applications. CrossLinkPlus FPGAs are fabricated on a 40 nm bulk CMOS process.

For embedded vision systems that require higher performance, Lattice has released the CrossLink-NX series of FPGAs. This is the first FPGA based on Lattice Nexus, Lattice’s new FPGA technology platform, and the industry’s first mainstream low-power FPGA platform using a 28 nm fully depleted silicon-on-insulator (FD-SOI) process.Benefit from 28nm FD-SOI process and optimized for low power consumption and small package

Based on a new FPGA architecture, CrossLink-NX greatly expands the capabilities of the CrossLink FPGA family. CrossLink-NX provides lower power consumption,

Smaller packages, higher performance and reliability, and new, easy-to-use tools provide developers with innovative options for video and sensor processing that significantly outperform competing FPGAs.

Application Market of CrossLink-NX: Automotive, Mobile, Industrial, etc.

The CrossLink-NX family of products is flexible enough to meet the needs of multiple market segments in video switching and image processing.

Most applications in these areas need to support bridging and/or aggregating data streams between multiple displays, cameras, and sensors (image sensors, etc.). Additionally, these components can be used for instant video processing and AI/ML inference. Because of their high energy efficiency, FPGAs can be used in battery-powered mobile devices and can provide enough performance to shine in computing and industrial control applications.

Figure 2: Trends in Embedded Vision Source: Lattice

CrossLink-NX enables AI inference to preprocess sensor data in video surveillance and security applications, including face recognition and presence detection. Additionally, it can be used for sensor aggregation and bridging.

The chip can be used as an embedded vision co-processor, incorporating up to 14 sensors. It can also be used for video scaling, rotation and color space conversion. Designers can use the chip to send a stream of sensor data to multiple locations, which is useful in automotive applications where cameras often need to feed data to multiple processing units.

In the following sections of this white paper, we will compare CrossLink-NX to FPGAs with similar logic cell density and I/O support. Below are performance and power comparison tests provided by Lattice.

High-performance CrossLink-NX enables AI at the network edge

It is often assumed that AI processing needs to be done in the cloud, however it is increasingly important to implement on-device AI capabilities at the edge of the network because it can better protect user privacy, reduce the amount of data uploaded to the cloud, and reduce data latency to shorten devices Response time.

For hardware developers, several factors make implementing AI/ML at the network edge challenging. For example, to improve the accuracy of AI/ML results or enable new applications, embedded vision developers need to add more sensors and/or higher resolution/higher frame rate cameras to their systems.At the same time, embedded vision designers want to use

Components of the MIPI standard. MIPI was originally developed for the mobile market, and today designers of various applications are looking for ways to take advantage of the high performance and economies of scale that MIPI components such as application processors bring.

Figure 3: CrossLink-NX block diagram Source: Lattice

The CrossLink-NX FPGA has enough resources to process multiple video data streams and perform AI computing functions on the fly, and it also supports a variety of existing interfaces.

This product family has multiple MIPI D-PHY interfaces and CSI-2 camera interfaces and is designed to serve video and Display applications. Two hard D-PHY interfaces, each supporting four lanes at up to 2.5 Gbps (10 Gbps total), and an additional soft MIPI D-PHY configuration supporting up to 12 MIPI streams at 1.5 Gbps. Other I/Os include hard 5 Gbps PCIe Gen2 and 1066 Mbps LPDDR3 DRAM.

CrossLink-NX’s hard-core DSP blocks can be used to process Neural Networks (NNs) locally. This series has an average of 170 bits of storage space per logic unit, which is the highest storage-to-logic ratio in its class. The FPGA’s large on-chip SRAM, ranging from 1 to 2.5 MB, stores neural network activations and weights.

FD-SOI process enables innovative power modes and improved stability

Lattice’s FD-SOI process offers developers an innovative way to manage power consumption. This process enables the application of a voltage (negative bias) to the backside of the die, thereby changing the threshold voltage. The chip can therefore provide two operating modes – low-power mode and high-performance mode. Through power consumption optimization, the operating power at higher frequencies can be around 200mW, while the static leakage power consumption is only a dozen milliwatts. CrossLink-NX FPGAs consume up to 75% less power than other competing products.

Another advantage that the FD-SOI process brings is a dramatic reduction in soft errors (the phenomenon in which high-energy particles hit transistors in an FPGA and damage performance). Using the FD-SOI process can significantly reduce the physical area on the chip that is susceptible to soft errors. To ensure stable operation of the SRAM-based LUT, software error checking is also performed using memory block error correction code. All in all, the Soft Error Rate (SER) of CrossLink-NX based on FD-SOI process is more than 100 times lower than that of competing FPGA solutions fabricated on bulk CMOS process.

CrossLink-NX is smaller

Mobile apps tend to have very strict space constraints. Lattice will introduce two products in this family, each supporting different levels of logic size (17K or 40K logic cells). A great advantage of the CrossLink-NX family is that it occupies a very small footprint, measuring only 3.7 mm x 4.1 mm (17K logic cell device). The devices in the CrossLink-NX family are pin-compatible, so the same board design can be used for different applications, as well as performance upgrades for future versions of designs using these devices.

Instant start

CrossLink-NX FPGAs use quad-SPI flash to store configuration files. The I/O pins can be configured first in 3ms, and the logic can be configured in 14ms. These features enable instant start-up, reducing operational problems that can occur in many applications.

CrossLink-NX provides an easy-to-use experience

CrossLink-NX FPGAs have extensive and continuously updated software support, including Lattice Radiant design tools (nearly

Updates include on-chip debug, signal integrity analysis, and engineering change order editor capabilities) and various IP cores (MIPI D-PHY, format conversion, PCIe, SGMII, and OpenLDI) for embedded vision applications. Lattice also offers development boards and reference designs for common applications such as camera aggregation, with more boards and reference designs to be released in the future.

in conclusion

Lattice combines programmability with high-performance processing and high-speed I/O interconnect with the CrossLink-NX FPGA family. These devices will provide developers with multiple options for implementing embedded smart vision applications with a high degree of flexibility, low power consumption, programmability, small size and high-speed video interface. This series is only the first product based on the Lattice Nexus FPGA technology platform, and new products in the future are worth looking forward to!