Hyper-Data: Push the Boundaries of the Five Human Senses

Technology is advancing at a blistering pace, and before we know it, digital products will be able to sense, process, and react to data in a way that replicates human senses. In 2023, Samsung has taken major steps toward realizing the System LSI Humanoid by increasing its focus on Hyper-Data solutions, which include CIS, Auto CIS, and Display IC. At Tech Day 2023, Hyper-Data was one of the major focuses, as presentations throughout the day focused on the different ways in which it will play a role in the System LSI Humanoid future.

Innovating Image Sensors

Joonseo Yim, EVP and Head of the Sensor Business, delivered the first Hyper-Data keynote speech of the day, titled, “Beyond the Human Eye: The Evolution of Image Sensors.” To start things off, Yim grabbed the audience’s attention by showing them an impressive photo of the Milky Way in the night sky before revealing that it had in fact been taken by a smartphone.

Thanks to Samsung System LSI’s innovations to ISOCELL sensor technology, image quality like this has become a reality. The innovations that have made this possible can be broken into seven main categories: pixels, dark noise, dynamic range, quantum efficiency, optics, AI, and beyond-visible imaging.

Increased Resolution and Reduced Pixel Size

Back Side Illumination (BSI) and Full-Depth Deep Trench Isolation (FDTI) technology has reduced the size of pixels and increased resolution from 300,000 to 200 million pixels — all while keeping camera size compact. BSI contributed to this feat by enhancing quantum efficiency. At the same time, FDTI creates complete separation of pixels and enables a higher full well capacity.

Improvements in both pixel and micro-optics technologies have brought about Samsung System LSI’s 200MP image sensor, which allows smartphones to capture highly detailed pictures of celestial bodies, like in the image Yim showed the audience at the beginning of his presentation.

Clarity in Both Bright and Dark Areas

It is particularly challenging to distinguish stars from noise in astrophotography. Dark noise is generated by silicon, and if stars are to be distinguished from it, it needs to be kept extremely low. Samsung System LSI has accomplished this by focusing on higher conversion gain and source follower engineering. A new DTI center cut (DCC) structure reduces the capacitor of the floating diffusion (FD) node, which therefore minimizes dark noise. The result is night photos that come out cleaner and with better gradation.

Inside ISOCELL pixels equipped with multiple conversion gains, a low conversion gain in bright areas allows for signal saturation to be avoided, while a higher conversion gain minimizes noise in dark areas. This combination allows for HDR images with only a single exposure. In videos, it reduces motion artifacts.

Samsung System LSI is also developing nano-structured technology to maximize quantum efficiency, which is very important for the sensitivity of mobile cameras. This technology can be applied especially to telephoto sensor to maximize its potential.

A Full Camera System Defined by Optics and AI

Long focal length is a challenge in telephoto lens design, so in addition to sensor solutions, Samsung System LSI is offering alternative optic technologies like Pop-out and Lens on Prism. These technologies help overcome lens limitations and enhance the telephoto user experience.

An additional advancement in this area is Foveated Rendering, which processes only the region of interest in high resolution while handling the rest of the image in low resolution — just like the human eye. This technology serves as a basis for the ISOCELL Zoom Anyplace solution, which records an area of interest with an AI tracker and 200MP high-resolution in-sensor-zoom while the entire scene is captured with a wide-angle lens.

Furthermore, Samsung System LSI’s sensor portfolio is expanding to multi-spectrum sensors. ISOCELL’s vision solutions, which include global shutters and time of flight sensors, can capture motion and depth extremely well, using not only the visible, but also infrared spectrum. Regarding this technology and its practical applications, Yim said, “We shall not be confined within the visible spectrum. Ultimately, our image sensors are aiming to serve as the eyes of the System LSI Humanoid and there is no limit to what image sensors can do.”

Automotive Image Sensors for a Safe, Relaxed Driving Experience

Later in the day, Haechang Lee, EVP and Head of Next Generation Sensor Development, discussed Hyper-Data as it applies to automotive imaging solutions in his presentation titled, “Sensor Technology for Vehicle Safety.” The stresses and demands of modern driving conditions seem to grow each day, but next-generation sensors can help solve this problem by enhancing advanced driver assist systems (ADAS) and autonomous driving (AD).

ADAS and AD utilize various sensors with different strengths, but cameras are the core sensing device that enable not just detection but also recognition, since they provide high resolution and detailed color information. Starting next year, some vehicles will utilize as many as 12 cameras with resolutions as high as 8 megapixels — and the demand for more cameras and more pixels is only set to increase. However, camera count and resolution are not the only considerations for automotive image sensors; it is also essential that they provide HDR, LED flicker mitigation (LFM), and motion artifact reduction to reach necessary safety levels.

Samsung System LSI’s sensors provide these features and a number of benefits, which include true HDR technology, low power consumption, and in-house manufacturing capabilities.

Paving the Way Toward 140dB True HDR

Thanks to CornerPixel™, which is an advanced pixel structure formed as pairs of large and small pixels, ISOCELL Deep Trench Isolation, and In-Pixel High Density Capacitance technologies, the ISOCELL Auto 1H1 sensor achieves a 120dB HDR using a single exposure. Further advancements of this technology has demonstrated up to 140dB HDR using a single exposure.

True HDR based on a single exposure makes it less likely for drivers to experience white-out or black-out conditions, while minimizing LED flicker and motion artifacts across the full range of brightness. This is in contrast to HDR needing multiple exposures, employed by the majority of the industry today.

Industry-Leading Automotive Sensors Made In-House
Samsung is the only sensor company to have full in-house manufacturing of pixel and logic wafers. And since the company has recently won multiple OEM awards due to its strong partnerships with key ADAS solution providers, quality and smooth implementation is to be expected. Samsung System LSI is also currently working on expanding partnerships within the automotive industry.
Plus, thanks to having the only 28nm automotive sensor in production — with a roadmap to FinFET¹, as well — Samsung System LSI’s automotive sensor solutions consume a low amount of power.

User Experience in Display and Touch

The System LSI Humanoid future is dependent on the ability to communicate with human users. The third Hyper-Data session presentation, delivered by JaeYoul Lee, EVP and Head of LSI Product & Technology, covered this topic in detail. Lee’s presentation was titled, “Extended User Experience in Display and Touch: OLED Touch-Embedded DDI,” and centered display and touch technologies as modes of communication between the System LSI Humanoid and users.

The first requirement for System LSI Humanoid communication is noise-free touch, which allows users to touch without interruption in any environment despite sources of interference like fluorescent lights. To reach the type of environmental noise immunity needed for noise-free touch, devices need to achieve a high SNR of over 45dB, and the touch response needs a report rate exceeding 240Hz. These aren’t the only challenges: Companies need to figure out how to manage displays that are getting increasingly thinner and larger since this display trends make the touch load heavier by increasing resistance and capacitance, which will make touch sensing difficult.

To overcome these hurdles, Samsung System LSI is introducing the OLED touch-embedded DDI (TDDI). Combining the previously separate DDI and Touch controller ICs into TDDI maximizes touch performance, which can be achieved by analyzing image data to predict touch data. Additional benefits are reduced power consumption and simplified design of the display module.

Furthermore, the OLED TDDI can achieve lower power consumption by reducing the touch drive voltage with shrinking of the touch process from 80nm to 28nm. System LSI’s TDDI can support a line sensor that reflects the latest mobile-oriented touch performance, 480Hz report rate, improve SNR by 6dB, and reduce power consumption by up to 25% by utilizing its Low Power Library. System LSI has also developed an AI-based algorithm that combines its own touch competitiveness with the strengths of the TDDI to simultaneously remove external noise for improved touch sensing and create a proximity sensing solution. Because of the high level of quality of these touch sensing solutions, touch sensors can replace proximity sensors altogether. A change such as this will increase set design flexibility.

Near the end of his presentation, Lee made it clear that Samsung will continue to innovate beyond a flexible display-oriented TDDI. In describing its unwavering commitment to innovation, he said, “Through relentless innovation, Samsung System LSI will enable communication between humans and humanoids. Our cutting-edge display and touch solutions not only make this communication possible, but also effortless.”

Empowering System LSI Humanoid with Smart Healthcare and its Ecosystem

Hyuck Choo, Executive Vice President and Head of the New Business Development Team, covered the implications of System LSI’s new effort in smart healthcare. He opened his talk by illustrating how the five senses played important roles in ancient medicine, including pulse reading, and how modern sensor technologies improved productivity of medical personnel and the quality of healthcare services over the years.

After identifying the top three causes of death around the world, Choo stated that developing countries could greatly benefit from smart healthcare technologies by saving costs and improving the quality of healthcare services provided to the growing and aging population. Given the clear need for these technologies, the smart healthcare market is expected to grow rapidly in the near future.

Then Choo introduced a number of hardware and algorithm solutions that Samsung System LSI has been preparing for the new healthcare era, including the second-generation Samsung smart health processor S1SBP6A that handles various biometric signals from wearable devices such as an ECG patch and revolutionary temperature-sensing algorithms that could potentially battle the rising health issues due to the global warming. All the solutions address the major causes of mortality globally and work to improve the productivity of doctors and nurses by allowing remote and more accurate diagnoses and monitoring.

Choo also discussed System LSI’s vision for the Samsung Smart Health Alliance, an ecosystem that seamlessly connects research organizations, hardware & software developers, and hospitals & service providers. In this ecosystem, developing a new smart healthcare technology, starting from inception to the final approval can occur in a seamless, time-and-resource-efficient manner because final customers such as hospitals and service providers are actively involved in the development and testing processes from the very beginning. Choo emphasized, “Samsung Smart Health Alliance is the fellowship of smart healthcare through which all of us can cooperate from the original concept to the final service and save lives.”

After sharing that the vision that combines healthcare technology and the System LSI Humanoid concept would create another breakthrough in smart healthcare and maximize the System LSI’s contribution to mankind, Choo ended his presentation with a ringing appeal, “Join our fellowship! Together we can empower the future of smart healthcare.”

Smart health is a growing industry, and System LSI is right at the forefront of it.

Hyper-Data and the System LSI Humanoid Future

Whether it be through clearer image quality, enhanced automotive safety, or touch and display efficiency, all of Samsung System LSI’s sensor solutions rely on massive amounts of data input and response. The company is demonstrating that it is up to the task of utilizing this data for new developments and is on the verge of technological breakthroughs that will change day-to-day life as we know it. Machines that sense and react just as well as — or better than — humans will make everything easier for all of us. With its innovative technological solutions, Samsung is making System LSI Humanoid future possible.