Smart Embedded Vision

Overview

As compute workloads move to the edge, PolarFire FPGAs offer 30-50% lower total power than competing mid-range FPGAs, with 5-10x lower static power, making them ideal for a new range of compute-intensive edge devices, including those deployed in thermally and power-constrained environments. PolarFire FPGA Smart Embedded Vision solutions include video, imaging, and machine learning IP and tools for accelerating designs that require high performance in low-power, small form-factors across the industrial, medical, broadcast, automotive, aerospace and defense markets.

Key PolarFire Benefits in Smart Embedded Vision

1. Lowest Total Power Consumption among competitive mid-density FPGAs

• 30% to 50% lower power consumption over competitive mid-density FPGAs
• Extended battery life in portable devices like drones, AR/ VR headset, HUDs, portable Ultrasound etc.
• Reduce system cost by eliminating thermal fans and heat sinks
• Realize lower power in 4kp60 HDMI 2.0, 12G SDI, 10G Ethernet, 12.5G CoaXPress etc. using SERDES lanes rated for 90 mW at 10G 

2. Small form-factor packages starting at 11x11 mm for 100K LE

• Create compact modules with 100K (11x11 mm) and 200K (11x14.5 mm) LE devices
• Design your next-gen vision platform with small form-factor PolarFire FPGAs with unmatched features like DDR4, LPDDR3, 12.7G SERDES, 1.6 Gbps LVDS I/Os and PCIe Gen2x4

Power Consumption Competitive Benchmark with 16 Channel Audio-Video Bridge Design Example
The example design demonstrates an Audio-to-Video bridge implementation. It uses 56% LUTs, 35% Mathblocks, DDR3 and 16 channel transceivers in a PolarFire MPF500T device. Benchmarks demonstrate up to 52% lower power consumption when compared to similar density FPGAs.

16 Channel AVB Switcher Power Estimator Results

*I/O and Transceiver power estimates include static and dynamic components. **Power Optimized Device

16 Channel AVB Switcher Resource Utilization

Learn More

Getting Started

PolarFire Video Kit (MPF300-VIDEO-KIT-NS)  Kit Features 4k30 MIPI CSI -2 cameras 3 HDMI ports FMC connection 2GB DDR4 PolarFire MPF300 Device

PolarFire Video FMC Portfolio 

 Out of Box Demo Design for the PolarFire Video Kit

Demo Guide & Programming Files

Imaging and Video IPs

Microchip's FPGA product portfolio offers scalable production-ready IPs through in-house development and using our third-party partner ecosystem. Our in-house IP portfolio covers the latest embedded vision IPs like MIPI, SLVS-EC, CoaXPress, Serial Digital Interface (SDI), HDMI, etc, while our partners cover offerings in Machine Learning, H.264, DisplayPort and HDCP IPs, with many more IPs being added continuously.

Click here to see our Partner IPs.

In-house DirectCore Smart Embedded Vision IPs for PolarFire FPGA

For evaluation/purchase of any of the IPs, please fill in the Request Information form, or contact Local Sales in your area.

CompanionCore Smart Embedded Vision Partner IPs

Partners	Description	Supported FPGAs
	HDMI 2.0b/ 1.4, DisplayPort 1.4a HDCP 1.x/ 2.x	PolarFire FPGAs
	CoaXPress 12.5Gbps Host and Device Turnkey Solutions	PolarFire FPGAs
	H.264 Encoder JPEG 2000 Compression	PolarFire FPGAs
	DDR4/3, LPDDR4 CSI-2, DSI-2	PolarFire FPGAs SmartFusion2 SoC FPGAs IGLOO2 FPGAs
	Core Deep Learning (CDL) Caffe CNN Compiler	PolarFire FPGAs
	Alpha Blender, Chroma Resample, Image Rotation SD, SDIO and MMC Controllers	SmartFusion2 SoC FPGAs IGLOO2 FPGAs
	H.264 Encoder	SmartFusion2 SoC FPGAs IGLOO2 FPGAs
	USB 3.1 Gen1 Device (5Gbps) USB 3.1 Gen2 Device (10Gbps)	PolarFire FPGAs

Request Information

VectorBlox Accelerator Software Development Kit

CoreVectorBlox IP

The CoreVectorBlox IP consists of a Matrix processor (MXP) and a MXP CNN IP. It can be instantiated as a single core accelerator or in multi-cores when neural network workloads need to be shared. The MXP consists of eight 32-bit ALUs and is responsible for elementwise tensor operations like add, sub, xor, shift, mul, dotprod etc. The MXP CNN IP consists of a 2D array of multiply and accumulate functions implemented using math blocks. As the name suggests, the MXP CNN IP is responsible for executing the convolutional layers of CNNs. Multiple networks can be overlayed during the runtime and switched dynamically. It is also possible to run simultaneous networks on a single CoreVectorBlox instantiation through time-slicing.

Development Flow

Step 1: Prepare your trained model

Use Python scripts provided in the SDK to convert your trained model into an optimized INT8 representation called a Binary Large Object (BLOB). Run the BLOB through the VectorBlox Simulator to verify accuracy of the network and to ensure successful conversion of the network.

Step 2: Prepare your hardware

The PolarFire Video Kit is configured to run as an AI enabled Smart Camera. The SDK includes a pre-compiled kit image bitstream.  Write the bistream into the PolarFire FPGA using the FlashPro programmer included with the kit. Write the BLOB, generated from Step 1, into the SPI flash of the kit.

Step 3: Write your embedded code

Use the provided embedded code in C/C++ based SoftConsole IDE and generate and program the hex file. Connect the video kit to a HDMI monitor and turn it on. Modify the embedded code to load and run many CNN BLOBs, switch CNNs dynamically on the fly or load CNNs sequentially for simultaneous inferencing.

Documentation

Getting Started

Deployment Options

PolarFire Video Kit
Smart Camera Reference Design using the PolarFire Video Kit
1. Video frame is received via MIPI CSI-2 2. And stored in the DDR via AXI-4 interconnect 3. Before inference – the frame is read back from the DDR 4. Converted from RAW to RGB, RGB to planer R, G, B and written back into DDR 5. CoreVectorBlox engine runs inference on R, G, B arrays and writes results back into DDR 6. Mi-V sorts probabilities, creates an overlay frame with bounding boxes, classification results, fps etc. and stores the frame in DDR 7. The original video frame is read and blended with the overlay frame and sent out to an HDMI display

Request More Information