Hiroo Takahashi, MIPI Camera Service Extensions Subgroup Lead
A revolution in automotive electronics is accelerating the demand for more onboard sensors (camera/radar/lidar) and displays with higher performance. Advanced driver-assistance systems (ADAS) and autonomous driving systems (ADS) are generating ever richer streams of image sensor data — and these data streams require interfaces with high bandwidth, low latency and importantly, functional safety.
To help solve this need, MIPI Alliance recently released the Camera Service Extensions (MIPI CSESM) v1.0 specification, which enhances the MIPI Camera Serial Interface 2 (MIPI CSI-2®) image sensor interface with end-to-end functional safety and other features for automotive applications.
In this post, I'll describe how CSE enables functionally safe end-to-end connectivity between image sensors and their associated electronic control units (ECUs) to overcome some of the challenges presented by today's need to integrate ever greater numbers of advanced safety-critical cameras and other sensors into vehicles.
About MIPI CSE v1.0
MIPI CSE provides a set of functional safety enablers for both image sensor data and any associated sensor control channels. Like its related specification, MIPI Display Service Extensions (MIPI DSESM), which enables functional safety and support for High-bandwidth Digital Content Protection (HDCP) for displays, it is a key component of MIPI Automotive SerDes Solutions (MASS), a framework that provides end-to-end connectivity solutions for automotive sensors and displays using the long-reach, ultra-reliable MIPI A-PHYSM SerDes interface as its foundation.
In protocol stacks for automotive applications based on MASS, CSE v1.0 is an upper-layer protocol, adding functional safety features to the CSI-2 interface. Security features will be enabled with the CSE v2.0 specification currently in development.
Solving industry challenges
Until the introduction of A-PHY, automotive integrators had to rely on proprietary SerDes interfaces, which required considerable resources to build in functional safety for a specific vendor solution. In turn, the lack of standardization has limited vendor choice and prevented economies of scale, as well as hindered ecosystem growth and slowed the development of new architectures.
MIPI CSE can be implemented in sensor solutions that use MIPI A-PHY bridges and sensor solutions that use integrated A-PHY.
Today, leveraging CSE as part of the MASS framework, OEMs and suppliers can satisfy functional safety requirements using standard CSI-2 interfaces over A-PHY. CSE can be implemented in sensor components with integrated A-PHY connectivity, as well as in components that use shorter-length MIPI C-PHYSM and MIPI D-PHYSM interfaces in combination with long-reach A-PHY bridges. Use of CSE is also compatible with legacy sensors and ECUs, allowing manufacturers to retain those components while gaining the benefits of CSE elsewhere in the A-PHY network.
Functional safety in CSE
CSE includes features that satisfy the functional safety requirements defined in the ISO 26262 standard, enabling automotive applications to meet Automotive Safety Integrity Levels (ASIL) B through D for transmission of image sensor data and control data using CSI-2. These features ensure that both the image data transferred from sensors to ECUs, and the data that control those sensors, can be monitored at the protocol level to ensure any errors and failures are detected at the application layer.
Functional safety for image data transmission is supported by the CSE_i protocol, which utilizes three main features:
- A cyclic redundancy check (CRC-32) to detect data transmission errors, ensuring the image data captured by the sensor is accurate when received at the ECU
- A frame counter to detect frame loss or duplication, with its accuracy verified by the CRC, ensuring the continuity of video streams from cameras to ECU
- A message counter that can be used as a timeout checker to detect any loss of data caused by a stopped or stuck transmission between an image sensor and ECU
Service Extension Packet (SEP)
CSE functionality is implemented over the entire PHY link using the Service Extension Packet (SEP) format, which provides packetization and uniform delivery of image data. This meets ISO 26262 requirements for error detection in data packets transmitted from source to destination. Because the SEP format is MIPI PHY-agnostic, it allows for data transfers over heterogeneous combinations of physical-layer interfaces, lane configurations and symbol rates, maintaining functional safety and other features end to end.
The image sensor generates packets in the SEP Data Type (SEP-DT) format. SEP-DT is converted to the A-PHY A-Packet format, either in a sensor with integrated A-PHY or in a SerDes bridge to the A-PHY network. In sensors that support CSE_i and use short-reach C-PHY or D-PHY interfaces, the SEP-DT format is encapsulated for transfer to the SerDes bridge. In both cases, the SEP-DT fields for functional safety and other CSE functions are preserved. Also, for backward compatibility, SEP-DT packets may be tunneled through older CSI-2 components that do not support CSE.
Enhanced Safety and Security Camera Control Interface (ESS CCI)
The CSE_c protocol provides services for CSI-2 control data. CSE_c uses either the I2C or A-PHY physical-layer interface. The A-PHY protocol adaptation layer for I2C (MIPI PAL/I2CSM) defined in MASS allows for transmission over A-PHY.
CSE_c extended functions are supported through the Enhanced Safety and Security Camera Control Interface (ESS CCI). CSE specifies one service for this protocol: ESS CCI Functional Safety Service. It's designed to detect errors in CCI camera control data, such as lost messages and messages accidentally or maliciously altered by a small number of bits. This service provides CRC and message counter tags that are used to verify the integrity of CCI control messages sent between CSE image sensors and ECUs.
There are two verification modes available for ESS CCI: Mode 1 sends CRC and message counter tags immediately after each CCI message, with the receiver verifying the tags as soon as the message is received. Mode 2 sends CRC and message counter tags after a group of messages, as CSI-2 embedded data or explicit read messages, to reduce overhead on the low-bandwidth CCI interface.
Other features of CSE
CSE also adds two other extended functions to CSI-2 for next-generation image sensor applications:
Extended Virtual Channel (eVC): The extended Virtual Channel (eVC) feature of CSE increases the number of virtual channels available in CSI-2 to 64. This allows manufacturers to assign more logical channels for different data flows, such as autofocus and phase detection, that can be interleaved along with image data in a single data stream.
Extended Data Type (eDT): CSE also introduces extended Data Type (eDT), which allows for transmission of higher-resolution image data over CSI-2 interfaces. It expands the number of data types from the 64 provided for in CSI-2 up to 256. This will enable manufacturers to integrate emerging and future generations of image sensors, such as cameras capable of RAW28 resolution (and beyond), in new vehicle platforms. Furthermore, it expands the number of user-defined data types from 8 in CSI-2 to 56.
Additional applications and what's next
In addition to automotive applications, the features in CSE make it well-suited to be leveraged in other safety-critical applications that use cameras and other types of image sensors for machine vision, such as robotics and factory automation.
While the initial release of CSE provides the functional safety enabling features described above, the next version is already under development. CSE v2.0 will add security features to CSE_i and CSE_c, and will work with the upcoming MIPI Security specification to bring end-to-end security to the MASS framework. Version 2.0 is expected to be completed in mid-2022.
Additional resources
Read more about the CSE specification and the MASS framework in the recently published white paper, "An Introductory Guide to MIPI Automotive SerDes Solutions (MASS)."
I also co-presented a session at the MIPI Automotive Workshop on 17 November, joining a comprehensive program of eight sessions covering MASS specifications, functional safety and security. That presentation, "How MASS Simplifies the Integration of Cameras and Displays in Automotive Architectures," is available here.
To gain access to all MIPI specifications and contribute to the development of future standards, consider joining MIPI Alliance.