Ultra fast cnn based Hardware Computing Platform Concepts for adas visual Sensors and Evolutionary Mobile Robots

 
2 
ABSTRACT 
Vehicle driving and traveling with car are part of our daily life. More than 80% of all 
personal travels are done by car and only 20% are done by public transportation. Alone in 
Europe every year we have more than 3 million additional private cars on roads and 
highways. Due to the congestion and capacity limitations of both highways and roads every 
year we have more than 7500 kilometers of blocked roads in Europe. This congestion and 
traffic has a negative direct impact on the economy and on total social costs. The 
estimation of traffic congestion and traffic safety on social costs for European people is 
more than 130 billion euro per year; see Ref. [1-3] .
Monitoring and controlling traffic and 
improving road safety can reduce this cost; but still every year 40,000 people die because 
of car accidents in Europe [1], [4-6] . There are two solutions for overcoming the critical 
issue of road safety: improving the driver safety education programs and improving the 
vehicle safety using advance technology like ADAS (Advanced Driver Assistance System). 
One of the main factors in car accidents and traffic safety is the human factor. If the driver 
is tired or asleep the probability of accident will dramatically increase. A convenient way to 
avoid these types of accidents is using an assistance system.
This thesis answers the following eight research questions which are related to the 
potential performance improvement of ADAS technology with respect to the involved real-
time image processing: 
Research question 1: What are the hard requirements of ADAS concerning real-
time image processing and design flexibility? How far do traditional approaches fail 
to satisfy these requirements? 
Research question 2: What are the major limitations of traditional high 
performance computing ap
proaches if used to ensure “real
-
time” image
processing?
Research question 3: What is the huge potential of neurocomputing involving 
either traditional neural networks (NN) or cellular neural networks (CNN) for high-
speed and flexible image processing for ADAS? Are there any limitations and how 
can these eventually be addressed? 
Research question 4: What are the major template calculation schemes of relevance 
for CNN based image processing? How can these calculations be performed in a 
real-time high performance computing context? 

 
3 
Research question 5: How far can the advantages of "analog computing" be 
used/gained through an emulation of analog computing on digital hardware 
platforms like FPGA (for the benefit of an ultrafast image processing)? 
Research question 6: How far can an efficient implementation of CNN on FPGA and 
GPU be designed and implemented?
Research question 7: How far can CNN be used/involved in an evolutionary 
computing/control context example (for illustration)? 
To cover the research question-1, we have conducted a survey concerning different ADAS 
concepts. High definition cameras are playing a very important role in ADAS concept and 
almost every ADAS concept includes one camera or some form of visual radar. We could 
formulate the overall requirements that ADAS systems set to the image processing based 
sensor functionality. And to finish we have shown as an example that there are many 
common modules for different ADAS systems. 
Concerning research question-2, we have shown the limitations of traditional/sequential 
computing concepts for processing high quality images in the ADAS context and did 
propose a parallel processing model based on CNN. 
Concerning research question-3 we have shown the advantages of neuro-computing 
especially of CNN as a high performance computing system in terms of flexibility in design 
and robustness.
Research question-4 is considering different methods for CNN template calculation. We did 
pass a review of the related state-of-the-art before proposing a genetic algorithm based 
approach for the calculation of complex templates. This concept can be implemented in 
hardware, for example of FPGA along with the CNN processor system. This should ensure a 
performance in real-time. 
Concerning research question-5 we have demonstrated that the advantage of analog 
computing can be used in a real time solution for solving complex dynamic systems. 
Further we have shown the implementation an “analog computing” emulation on a digital
platform (FPGA); the implementation was based on the 
Digital Differential Analyzer
(DDA) 
method which has shown to be thousand time faster than a CPU.

 
4 
The research question 6 does concern the implementation of CNN in a discrete-time 
version on both FPGA and GPU. We have shown the advantages and drawbacks of each of 
the implementations. Overall we could realize a CNN implementation on both platforms 
and the performance was very good. 
Concerning the last research question 7, we have shown the efficient use of CNN and a 
genetic algorithm for controlling a legged-robot in the form of a neuro-evolutionary 
technique. The results did clearly demonstrate the effectiveness of the approach. 
The evaluation of image sequences with the purpose of extracting useful information 
(about the environment, vehicle situation and traffic) is the main and challenging issue in 
ADAS visual sensors. This process does however involve a huge computational effort. 
Therefore, to have a real time processing platform we need appropriate hardware and 
software architectures. Today, there are many platform options for machine vision: DSP, 
FPGA and GPU. The requirements related to image quality, image size and frame rate per 
second for a processing in hardware are increasing. Overall we do face a tradeoff between 
processing time, power consumption on one hand, and video quality and precision on the 
other hand. This challenge does motivate research related to new architectures and 
software algorithms for video and signal processing. Designing and implementing a 
“single
task” image processing functionality in hardware in not a big issue. However, the “multi
tasking” case is much challenging. Having a good model for image processing will reduce
hardware resources. A CNN model has this potential as by dynamically changing the 
related templates values we can change the functionality of the system and thereby 
without any further hardware or software explicit reconfiguration. Taking advantage of the 
inherent parallel processing nature of CNN processors a strong integration of both 
hardware and software can be ensured. This thesis does address two main challenging 
issues in ADAS technology: a) a universal model and architecture for a real time visual 
processing; and b) the implementation of a prototype system on both GPU and FPGA.

 
5 
Chapter 1 

Download 3,22 Mb.