Ambarella, a semiconductor design company that focuses mainly on imaging hardware, has announced the released of its new CV5 system on a chip (SoC). This new 5nm SoC can process one 8K or four 4K streams while using minimal power consumption and provides plenty of overhead for AI-powered features and capture modes.
Ambarella might not be as familiar a name as other semiconductor manufacturers, but it’s played a major role in the imaging industry. Since 2008, the Santa Clara-based company has been manufacturing chipsets for popular consumer-facing products such as GoPro action cams and DJI drones.
The new CV5 SoC is powered by Ambarella’s CVflow AI engine with dual Arm Cortex-A76 1.6GHz CPUs. This setup works alongside the image signal processor to encode 8K video at up to 60 frames per second (fps) in both H.264 (AVC) and H.265 (HEVC) while using just 5W of power (2W if dropped to 8K30p). Alternatively, the SoC can handle up to four 4K streams, making it an ideal solution for 360-degree cameras that require multiple camera modules to record in-sync.
|A diagram showing an abstracted, macro-level view of the SoC’s architecture. Click to enlarge.
In addition to images for human viewing, the CV5 can simultaneously process images for machine-based processing, such as AI-powered features and capture modes. Specifically, Ambarella notes HDR and timelapse capture modes in action cameras as well as face tracking. For drones, Ambarella notes data from additional sensors, such as LiDAR and time-of-flight (ToF), can also be processed alongside the standard footage to provide object detection, path planning and more.
Ambarella hasn’t provided details on what device manufacturers will use this new SoC in upcoming products. But if it’s the usual suspects, we can likely expect DJI, GoPro, Parrot and others to use this SoC inside future devices.
Ambarella introduces CV5 high performance AI vision processor for single 8K and multi-imager AI cameras
Ambarella’s new 5 nm CV5 AI vision CVflow® processor sets a new standard for power efficiency with 8K video recording in under 2 watts
SANTA CLARA, Calif.,—January 11, 2021—Ambarella Inc. (Nasdaq: AMBA), an AI vision silicon company, today unveiled CV5, an artificial intelligence (AI) vision processor capable of recording 8K video or four 4K video streams. The new system on chip (SoC) will enable the development of intelligent automotive camera systems, consumer cameras (drone, action, and 360°), and robotic cameras. It combines Ambarella’s powerful CVflow AI engine with dual Arm® A76 CPUs to provide the performance necessary for a wide range of AI-based algorithms. Its advanced image signal processor (ISP) can simultaneously process images for both human viewing and machine processing. Fabricated in advanced 5 nm process technology, CV5 consumes under 2 watts of power while encoding full 8K video at 30 frames per second.
“With the introduction of CV5, Ambarella is defining the next generation of automotive, consumer, and robotic cameras,” said Fermi Wang, CEO of Ambarella. “By combining 8K single-channel and 4K multi- channel recording with the high performance of our CVflow AI engine, we are enabling cameras with the highest-quality imaging and innovative new AI features.”
In automotive video telematics applications, CV5 provides the performance necessary to encode multiple video streams from front ADAS, driver monitoring, cabin monitoring, and side-view cameras. Its CVflow AI engine can simultaneously run advanced driver assistance systems (ADAS) algorithms such as lane departure, and forward collision warning as well as driver monitoring algorithms such as drowsy driver detection. The combination of high-resolution video capture with advanced AI processing enables ADAS cameras to recognize images over long distances and with high accuracy.
The exceptionally low power consumption of the CV5 SoC also makes it an ideal solution for the next generation of high resolution and high frame-rate action, 360°, and virtual reality (VR) cameras, offering 8K recording and playback in very small form factor designs.
In robotic and drone applications, CV5’s CVflow AI engine can accelerate simultaneous localization and mapping (SLAM), path planning, and obstacle detection, and avoidance algorithms for navigation and autonomous operation. For drone-based aerial videography and cinematography, CV5 can simultaneously perform flight control and navigation functions while recording up to 8K resolution video recording at 60 frames per second.
The CV5 SoC shares common SDK and computer vision (CV) tools with Ambarella’s other CVflow SoC families, simplifying development of cameras with multiple price and performance options. A complete
set of CV tools helps customers port their own neural networks onto CV5, including a compiler, debugger, and support for industry-standard machine learning frameworks, such as PyTorchTM, ONNXTM, CaffeTM, and TensorFlowTM, as well as extensive guidelines for convolutional neural network (CNN) performance optimization.
CV5 SoC key features:
CVflow architecture with DNN support
Dual-core Arm® Cortex®-A76 up to 1.6 GHz with NEONTM DSP extensions and FPU
High-speed SLVS-EC, MIPI-CSI (C/D PHY) interfaces, connecting up to 14 cameras
Multi-channel ISP with up to 8KP60 throughput
Native support for RGGB, RCCB, RCCC, RGB-IR, and monochrome sensor formats
Multi-exposure high dynamic range (HDR) processing
Real-time, hardware-accelerated fisheye dewarping and lens distortion correction (LDC)
AVC and HEVC encoding with up to 8KP60 throughput and multiple stream support
Rich set of interfaces includes 4-lane PCIe, CAN FD, Gigabit Ethernet, USB 3.1 host and device,
three SD card controllers, MIPI DSI/CSI-2 and HDMI video outputs
LPDDR4x / LPDDR5 / LPDDR5x support, 64-bit data bus and up to 32 GB capacity
Advanced security features, including secure boot with TrustZone® and secure memory, TRNG,
OTP, DRAM scrambling and virtualization
5 nm process technology
16×16 FBGA package with 0.5 mm ball pitch