A real-time system for vehicle detection with shadow removal and vehicle classification based on vehicle features at urban roads

Monitoring traffic in urban areas is an important task for intelligent transport applications to alleviate the traffic problems like traffic jams and long trip times. The traffic flow in urban areas is more complicated than the traffic flow in highway, due to the slow movement of vehicles and crowded traffic flows in urban areas. In this paper, a vehicle detection and classification system at intersections is proposed. The system consists of three main phases: vehicle detection, vehicle tracking and vehicle classification. In the vehicle detection, the background subtraction is utilized to detect the moving vehicles by employing mixture of Gaussians (MoGs) algorithm, and then the removal shadow algorithm is developed to improve the detection phase and eliminate the undesired detected region (shadows). After the vehicle detection phase, the vehicles are tracked until they reach the classification line. Then the vehicle dimensions are utilized to classify the vehicles into three classes (cars, bikes, and trucks). In this system, there are three counters; one counter for each class. When the vehicle is classified to a specific class, the class counter is incremented by one. The counting results can be used to estimate the traffic density at intersections, and adjust the timing of traffic light for the next light cycle. The system is applied to videos obtained by stationary cameras. The results obtained demonstrate the robustness and accuracy of the proposed system.

INTRODUCTION Traffic problems are one of the most problems encountered by the residents of large cities. Therefore, the traffic management companies have paid great attention to solve these problems. The first step in traffic analysis is the collection of traffic information. Several techniques have been developed for traffic data collection; many detectors (such as loop, radar, infrared and microwave) are utilized to do this task. These detectors help traffic flow management by providing information about the level of the traffic density on the roads. However, they have many drawbacks that lead to reduce their use, as their installation requires pavements cuts; in addition, their detection zone is small. In recent years, the vision-based systems have been widely used in traffic management, due to their advantages compared to electronic sensors. The vision-based systems extract useful data by covering wide-area detection with ability in determining the shape of the detection zone. The Ì ISSN: 2088-8694 first phase to analyze the traffic parameters is the vehicle detection. Several methods have been developed for the vehicles detection, and these methods can be grouped into two main approaches: texture-based approach and motion-based approach. The first one utilizes vehicle features like edges, corners, colors and so on; this approach is good to detect the stopping vehicles. While the other one depends on the movement of vehicles, it is widely used in intelligent transportation systems. There are two main methods to detect the moving objects: optical flow and background subtraction. The optical flow is accurate in the detection of object motion and gives more information about the motion like the velocity and the motion direction. However, it has high computational time and is not suitable for real time applications [1]. The background subtraction is the most common method used in literature for detecting moving vehicles. In this work, the background subtraction is utilized to separate the moving vehicles from the background model. However, the shadows of moving vehicles are also detected; thus they are considered as a part of vehicle dimensions, and this leads to misclassify the different vehicles. To cope with this issue, we proposed an algorithm to remove the shadow region based on the edges of detected regions. This algorithm outperforms the other shadow removal algorithms. When the vehicles are detected without shadows, they will be tracked until they reach the classification line. Then the classification is performed to category them into a number of predefined types. The classification methods can be grouped into two groups, one group classifies the vehicles by measuring the vehicle dimensions [2], while the other utilizes machine learning techniques [3]. Various systems have been developed to detect and classify the vehicles. A real time system for vehicles detection and classification is described in [4]. The first phase of their work is the vehicle detection, they used the background subtraction to perform this task by employing the adaptive background update method, then the vehicles were tracked by using the association graph between the consecutive frames, finally they used the vehicle dimensions to classify the vehicles into two categories, small vehicles (cars) and big vehicles(vans, trucks and buses). In [3], they developed a system for vehicle classification. The moving objects(vehicles)are separated from the static objects(background objects) by using GMM method, and then they used an approach based on two level of support vector machine (SVMs) to classify the detected vehicles. The second level was employed for solving the occlusion problem. The vehicles were classified into four classes (bus, cars, minibus and trucks). In [5], an image processing system is proposed to detect and classify the vehicles on rear view vehicle. This system utilized the temporal median filter to establish the background model, and then the scene frame was subtracted from the background model to detect the moving vehicles. The classification was performed by using SVM method after extracting vehicle features by using deep convolutional neural network. They classified the vehicles into two types, the passenger vehicles and other vehicle class. In [2], a real-time traffic surveillance system has been proposed to measure traffic flow by detecting and counting the vehicles. The background model was performed by using the temporal information of the mean and standard deviation of gray level distribution in consecutive frames for each point. After the segmentation process, each detected object was bounded by a rectangle box, and then the calculation of object features (height, width and aspect ratio) was established to achieve a robust and accurate classification. Then the vehicles were classified into two classes (cars and bikes). In this paper, a vehicle detection and classification system at intersections is proposed. The system consists of three main phases: vehicle detection, vehicle tracking and vehicle classification. In the vehicle detection, the background subtraction is utilized to detect the moving vehicles by employing mixture of Gaussians (MoGs) algorithm, and then the removal shadow algorithm is developed to improve the detection phase and eliminate the undesired detected region (shadows). After the vehicle detection phase, the vehicles are tracked until they reach the classification line. Then the vehicle dimensions are utilized to classify the vehicles into three classes (cars, bikes, trucks). After the classification phase, the vehicles in each class will be counted. The counting results can be used to estimate the traffic density at intersections, and adjust the timing of traffic light for the next light cycle. The system is applied on videos obtained by stationary cameras. The rest of paper is organized as follows: section 2 describes the vehicle detection and shadow removal algorithm. The vehicle classification method is presented in section 3. Experimental result is presented in section 4. Finally, conclusion is given in sections 5.

VEHICLE DETECTION
Moving object detection methods including human [6,7], vehicles [8,9], have been developed by several researchers. The most common method used in literature is the background subtraction method. In the past decade, numerous background subtraction algorithms have been proposed to extract and update the background model. These algorithms can be a non-recursive or a recursive, the non recursive algorithm utilizes a buffer of video frames to get the background, while the recursive algorithm updates the background model recursively based on each input frame. Aljammal et al [10] proposed a non-recursive method called non-parametric model. In their method, they used the entire history of pixels to estimate the background by calculating the pixel density function for each pixel. The pixel is considered as background if this function is greater than a predefined threshold. The drawbacks of this method are: it is time consuming and requires high memory storage .Karmann and von [11] developed a recursive technique for estimating the background model based on Kalman filter, they utilized the intensity and its temporal derivative model. The background is recursively updated by using three matrices: the background dynamics matrix, the measurement matrix and the Kalman gain matrix. However, this method is affected by the foreground pixels even if it works in slow rate adaptation. Kim et al [12] proposed a multimodal backgrounds method, it is called code book method. In this method, each pixel is summarized by a number of codewords stored in a coodbook; each codeword contains of a set of parameters.
The input pixel is considered as a background pixel if its brightness falls within the brightness range of some codeword. In addition, the color distortion of that codeword is smaller than the detection threshold. However, this method cannot correctly detect the darkgrey and white moving objects, because it considers the darkgrey objects as a shadow and the white object as a sudden increase of illumination [13]. In this paper, MoGs method [14] is used to perform the background subtraction process. It is the most method used in literature due to its robustness against the environmental changes and its capability to handle multimodal background distributions. The idea of this method is as such: a number of Gaussian distributions (components) represent each pixel. The number of Gaussian components falls between three and five depending on the storage limitation and the possibility of system realization in real time, three components are sufficient for our system. The component is considered as a matched component if the difference between the component mean and the pixel value is less than a predefined threshold, and then its parameters are updated as follows: the weight increases, the standard deviation decreases, and the mean moves to be close to the pixel value. If the component is none matched; the only parameter which is updated, is the weight, it decreases exponentially. If the pixel does not have any matched component; the component that has least weight is replaced by a new component with mean equals the pixel value, a large initial variance, and a small weight. After that the components are ranked according to a confidence metric (weight/standard deviation), and then a predefined threshold is applied to the components weights. The background model is the first components, whose weights are higher than the threshold. While the foreground pixels (moving objects pixels) are those that do not have any component in the background model. Figure 1 shows the result of applying MoGs method to two different traffic scenes.

Shadow removal
In this section, an algorithm that removes vehicle shadows will be developed to improve the performance of the system. The system performance will be improved from two respective views: It reduces the cases of occlusion problems between the vehicles. In addition, if the shadow region is not removed, it will be considered as a part of vehicle dimensions, and the vehicle will be classified into the wrong class. Thus, the shadow removal algorithm leads to improve the classification process and increases the system performances. Many methods have been developed to detect and remove vehicle shadows [15][16][17][18][19], some methods utilize the color information to distinguish between vehicle pixels and shadow pixels because of the difference of the chromatic and luminance between the shadow and vehicle [20,21]. However, these methods fail to identify shadow pixels from dark vehicle pixels. The other methods employ the texture information to identify the vehicle region [22]. Since the shadow has a little texture, it is simple to separate the vehicle region from the shadow region. In this paper, we proposed an algorithm to remove the shadow region based on the edges of detected regions. The proposed algorithm involves the following steps: − When the background subtraction method is applied, the moving objects (vehicles) are detected with their shadows (detected regions). Then the shadow removal algorithm is applied on these detected regions in the resulted image from background subtraction method and the gray scale source image. Figure 2  and what will remain are vehicle edges. Then we apply close operation on the resulted image to fill in spaces between the edges to preserve the vehicle details.
The resulted image obtained from this step is shown in Figure 2

Vehicle tracking
There are several methods to track moving objects through image sequences. These methods can be grouped into two categories: feature-based tracking methods, and model-based tracking methods. The featurebased methods are most widely used in literature because of its robustness [2]. In this work, after the detection of vehicles without shadows, a feature-based method is employed to track the moving vehicles through the image sequences in the detection zone (that is designated by two blue horizontal lines as shown in Figure 3).
When the vehicle appears in the detection area, the centroid of vehicle is calculated, and then this feature is used to track the detected vehicle between the consecutive frames. First, an empty vector is initialized to maintain the position of vehicles centroid. If moving vehicles are detected in the current frame, their centroid position are recorded in this vector. In adjacent frames, the moving objects that are spatially closest are correlated. Therefore, the measurement of the distance between the consecutive frames is sufficient to track the moving objects. The euclidean distance (ED) is utilized to measure the distance between the position of vehicle centroid in the current frame and in the previous frame. For each moving vehicle in the current frame, a vehicle with the minimum distance is searched for the previous frame to change its record to the new position. When the vehicle exceeds the detection area, the record of this vehicle is removed from the vector.

VEHICLE CLASSIFICATION
After the detection of vehicles without shadows, the tracking process is implemented. When the vehicle reaches the classification line (that is designated by a red horizontal line as shown in Figure 3), the classification process is implemented based on vehicle dimensions. In this work, three parameters (aspect ratio (AR) = height/width, height, and width) of vehicles are utilized to classify them into three classes (cars, bikes, and trucks). The aspect ratio is calculated by using the dimensions of different types of vehicles; vehicle dimensions are taken from [23]. We took into our consideration the transformation from 3D to 2D. The aspect ratio of different vehicles types are as follows: − The classification process involves the following steps: − When the vehicle attains the classification line, its height and width are calculated. − If the vehicle width is greater than a specific threshold; then there is a horizontal occlusion (when the distance between the two adjacent vehicles is very small; they are detected as one object. This is called a horizontal occlusion). The detected region is considered as two adjacent vehicles, and it is separated into two regions by dividing the width by two. − If the vehicle height is greater than a specific threshold; then there is a vertical occlusion (when the distance between the two consecutive vehicles is very small; they are detected as one object. This is called a vertical occlusion). The detected region is considered as two consecutive vehicles, and it is separated into two regions by dividing the height by two. − The aspect ratio of each vehicle is calculated. If AR is between 1.17 and 1.3, then the vehicle is a car. If AR is between 1.41 and 1.79, then the vehicle is a truck. If AR is between 1.91 and 2.4, then the vehicle is a bike. If AR falls in the overlapping interval [1.3-1.4], the vehicle height is used to distinguish if the vehicle is a car or a truck, if the height is higher than a predefined threshold, then the vehicle is a truck, else the vehicle is considered as a car. If AR falls in the overlapping interval [1.8-1.9], the vehicle width is used to distinguish if the vehicle is a bike or a truck, if the width is higher than a predefined threshold then the vehicle is a truck, else the vehicle is considered as a bike. − Three counters have been proposed to count the number of vehicles in each class. These counters are called C-car, C-bike, and C-truck. They are initialized to zero, and when the detected vehicle is classified into one class from the existing three classes, the counter of this class will be incremented by one.

EXPERIMNTAL RESULT
In this paper, a surveillance traffic system is proposed to detect and classify the different types of vehicles. In order to confirm that our proposed method is effective to perform this task, we utilized database that is taken from stationary traffic cameras of Casablanca city. This database contains of two different traffic videos; the first contains of 3580 frames with a resolution 240x320, the other contains of 2030 frames with the same resolution. Two different scenes of traffic videos are utilized. One video is taken in the area just after a traffic light (Scene 1) and the other is taken in an urban road (Scene 2). The results obtained during realized system operation contains of vehicle type and vehicle number. Vehicle number is displayed above the boundary box. Each vehicle is labelled by a black rectangular box until they reach the classification line, then the box color changes according to classification result. If the vehicle type is car, then the box color changes to blue. If the vehicle type is bike, then the box color changes to yellow. If the vehicle type is truck, then the box color changes to red. Figure 3 (a) shows the result of the 438-th frame in which 4 cars in the detection area, three of them crossed the classification line. They are classified as cars and have been labelled by blue rectangular box. Figure 3 (b) shows the result of the 350-th frame in which one truck has been counted and labelled by red rectangular box. Figure 3 (c) shows the result of the 1005-th frame in which one bike has been counted and labelled by yellow rectangular box and one car has been counted and labelled by blue rectangular box. Figure 3 (d) shows the result of the 1572-th frame from video sequences 2 in which two cars have been detected. The first one crossed the classification line, so it has been counted and labelled by blue rectangular box. According to these results, the different types of vehicles can be classified and counted correctly in this proposed system. To evaluate the efficiency of the proposed system, a comparative study is established on different surveillance systems. There are difficulty to make a fair comparative study due to the utilization of different database. Therefore, we conducted a qualitative comparative study instead of quantitative study. Table 2  the results of the comparative study. As noted in this table, the counting accuracy of the proposed algorithm on an average of 96.78% denotes that the proposed algorithm is more efficient than the other compared algorithms.

CONCLUSION
A system for vehicles detection and classification has been introduced in this paper. The system consists of three main phases: vehicle detection, vehicle tracking and vehicle classification. In the vehicle detection, the background subtraction is utilized to detect the moving vehicles by employing mixture of Gaussians (MoGs) algorithm, and then the removal shadow algorithm is developed to improve the detection phase and eliminate the undesired detected region. Then the vehicles are tracked until they reach the classification line. After that, the vehicle dimensions are utilized to classify the vehicles into three classes (cars, bikes, and trucks). After the classification phase, the vehicles in each class will be counted. The system has been applied on various traffic scenes under different weather and lighting conditions. The experimental results confirm that the proposed system has the ability in detecting and classifying the vehicles accurately and efficiently in real time.