Smart Embedded Vision
Overview
Smart Embedded Vision systems are increasingly being deployed in low-power, small-form factors across a broad spectrum of applications.
Current generation FPGAs must support high-bandwidth video streams, advanced image processing techniques, real-time analysis, including deep learning inference, compression, security, reliability and high-speed interfaces. Microchip’s FPGAs enable Smart Embedded Vision by providing:
- Most performance/ watts (vs. other FPGAs)
- In-built security and reliability
- Immunity against Single Event Upsets (SEU)
- Most integrated Smart Embedded Vision platform (Video datapath + ISP + Deep Learning + Custom Logic)
- Most flexible and scalable architecture across PolarFire, SmartFusion2 and IGLOO2 FPGAs
| Featured | Description | Link |
PolarFire FPGA |
|
PolarFire FPGAs |
PolarFire Video Kit |
|
MFP300-VIDEO-KIT |
| PolarFire Deep Learning Inference |
|
Click on the Machine Learning Inference IP tab above to learn more |
| Smart Embedded Vision Demo |
|
MFP300-VIDEO-KIT Resources |
| Imaging and Video Solutions for Smart Embedded Vision | 1.64 MB | 2/2019 |
Imaging Software and IP
Microsemi offers various IPs to reduce the development time of the video application using the PolarFire, SmartFusion2 and IGLOO2 devices. Block diagram shown here highlights some of the available IPs for PolarFire that can be useful for image processing and display applications.
Intuitive Software GUI Application (Enables Video and Audio Configurations)
- Source selectable picture-in-picture menu
- Alpha blending and overlay
- Image edge detection and enhancements - color balance, brightness and contrast
 |
 |
Software Tools
- Libero SoC PolarFire Design Suite offers high productivity with its comprehensive, easy-to-learn, easy-to-adopt development tools for designing with Microsemi's PolarFire FPGA Family
- Libero SoC Design Software integrates industry leading synthesis, debug and DSP support from Synopsys, and simulation from Mentor Graphics with power analysis, timing analysis and push button design flow.
Solution User Guides
| UG0850: PolarFire FPGA Video Solution User Guide | 2/2019 |
IP User Guides
| UG0651: Scaler User Guide | Video scaler based on the nearest neighbor algorithm | 3/2018 |
| UG0643: Image De-Noising Filter User Guide for SmartFusion2 |
Removes impulse noise. Supports 3x3 2D median filtering |
3/2017 |
| UG0642: Image Sharpening Filter User Guide | Unsharp Mask algorithm. A programmable gain for edge directions |
3/2018 |
| UG0646: Display Enhancement User Guide | Enables a user to adjust brightness, contrast, saturation, and hue of the final video display |
2/2019 |
| UG0640: Bayer Conversion User Guide | Combines bilinear interpolation and median filtering | 2/2019 |
| UG0649: Display Controller User Guide | Reads video data from display frame buffers and generates the required timing |
5/2019 |
| UG0641: Alpha Blending User Guide | Supports per-pixel alpha blending and global alpha | 11/2018 |
| UG0639: Color Space Conversion User Guide | RGB video data to YCrCb color space and vice versa | 3/2018 |
| UG0645: Low Voltage Differential Signaling 7:1 User Guide for SmartFusion2 |
Supports Digital Display Interface | 8/2018 |
| UG0644: DDR AXI Arbiter User Guide | Video DMA accesses DDR memory from the fabric through the AXI interface |
3/2018 |
| UG0682: Pattern Generator User Guide | Generates a test color pattern for different colors | 3/2018 |
| UG0693: Image Edge Detection User Guide | Based on Sobel Operator | 2/2019 |
| UG0763: MIPI CSI-2 Receiver Decoder for SmartFusion2 |
IP supports interfacing with image sensors using MIPI CSI-2 receive interface for SmartFusion2 |
6/2017 |
| UG0806: MIPI CSI-2 Receiver User Guide for PolarFire | IP supports interfacing with image sensors using MIPI CSI-2 receive interface for PolarFire |
2/2019 |
| UG0826: MIPI CSI-2 Transmitter User Guide for PolarFire | IP supports interfacing with image sensors using MIPI CSI-2 transmit interface for PolarFire |
10/2018 |
| UG0830: Low Voltage Differential Signaling 7:1 for PolarFire | IP supports LVDS 7:1 Transmit and Receive | 7/2018 |
Demo Guide & Programming Files
Application Notes
| AC469: PolarFire FPGA Video and Imaging Solution Application Note | 906 KB | 3/2018 |
Click here for more Application Notes on PolarFire
Click here for more application notes on SmartFusion2
Tutorials
Click here for tutorials on PolarFire
Click here for tutorials on SmartFusion2
To get the more details on the IPs or any further information, please contact imaging@microsemi.com
Machine Learning Inference IP
Deep Learning Inference Acceleration for sensor / edge inference
It is now possible to make your sensors (e.g. camera for industrial, drone, robotics or UAV) very smart with Microchip Deep Learning Inference toolkit – within your power budget. Highly optimized CNN implementation with 8bit quantization enables Microchip PolarFire FPGAs deliver 100 Giga ops per Watt – 2x to 4x of any competitive FPGA devices in the market today.
 Traffic Monitoring |
 Machine Vision |
 ADAS |
 Surveillance |
 PCB Inspection |
 Drone Vision |
| Inference in edge driven by the need for low-latency | FPGAs offer DSP compute capability vs. MCU/ MPU and CPU |
 |
 |
| Edge devices requiring real-time decision making cannot use Cloud AI | 1480 math blocks = 1.5 Tera Ops/sec performance |
| High efficiency INT8 Dot Product Mode (DOTP) | Lowest Power mid-range FPGAs |
 |
 |
| 25% higher efficiency math blocks vs. other FPGAs | PolarFire offers up to 50% lower power consumption than SRAM FPGAs |
| Integrate deep learning inference with video-imaging pipeline | Upgrading deep learning and video firmware |
|
 |
| PolarFire = Single-chip Machine Learning Camera | Upgrading firmware in PolarFire is easy |
Check out our demos, developed by our partner, ASIC Design Services. Mail us at imaging@microsemi.com or fill the form located in the Request Imaging and Video IP tab for more information.
| Inference on Video Stream - Tiny YOLO v2 (Caffe), PolarFire MPF300, 3W | Inference on Images - Tiny YOLO v2 (Caffe), PolarFire MPF300, 3W |
Hardware
PolarFire Imaging and Video Block Diagram
|
PolarFire Video and Imaging Kit
|
|
|
SmartFusion2 Advanced Development Kit
|
  |
Board / Schematics
| PolarFire Video Kit Schematics - Rev 1 | 2/2019 |
| PolarFire Video and Imaging Kit Quickstart Guide | 2/2019 |
| QG0695: Imaging and Video Kit Quickstart Guide- Parallel | 3/2017 |
| QG0719: Imaging and Video Kit Quickstart Guide- MIPI CSI-2 | 6/2017 |
| VIDEO-DC-PRL Board Schematics (Assembly, PCB, Schematics, BOM) | 8/2016 |
| VIDEO-DC-MIPI Board Schematic (Assembly, PCB, Schematics, BOM) | 8/2016 |
Thermal Imaging
Thermal imaging infrared cameras are used in an increasing number of applications such as industrial, security, defense, and medical applications. It is important for a thermal imaging system to dissipate a very low power so that its heat dissipation doesn't interfere with the working of a thermal/Infrared image sensor. Microsemi's FPGAs and SoC FPGAs are best suited for thermal imaging applications as they have a low total power owing to its flash-based FPGA architecture.
Thermal Imaging Demo
Future Electronics has developed a thermal imaging demo using FLIR thermal image sensor and IGLOO2-based Future Creative Board.
The design for the demo is completely implemented in the FPGA fabric and is optimized for both IGLOO2 FPGA or SmartFusion2 SoC FPGA devices.
Once the thermal imaging design is programmed in the FPGA it can be configured and controlled real-time from any host PC using the demo GUI software. 
Thermal Imaging demo GUI
The Future-designed Creative Development Board features Microsemi’s IGLOO2 FPGA or SmartFusion2 SoC FPGA and Microsemi’s LX series power devices. The Future - Microsemi Creative Development Board boasts a 25k logic element (LE) FPGA, offering the lowest cost of entry for both software and hardware engineers who want to evaluate and implement their own unique designs.
For more information on this solution visit: http://www.futureelectronics.com/en/campaign/microsemi/Pages/CreativeDevelopmentBoard.aspx
Future Creative Development Board
