In recent years, the advancements in deep learning and high-performance edge-computing systems have increased tremendously and have become the center of attention when it comes to the analyzing of video-based systems on the edge by making use of computer vision techniques. Intelligent Transportation Systems (ITS) is one area where deep learning can be used for several tasks including highway-based vehicle counting systems where by making use of computer vision techniques, an edge computing device and cameras installed in specific locations on the road, we are able to obtain very accurate vehicle counting results and replace the use of traditional and laborious hardware devices with modern low-cost solutions. This thesis proposes and implements a modern, compact and reliable vehicle counting system which is based on the most recent and popular object detection algorithm as of writing this thesis known as the YOLOv5, combined with a state-of-the-art object tracking algorithm known as DeepSORT. The YOLOv5 will be used in the following system for the detection and classification of four different classes of vehicles whereas DeepSORT will be used for the tracking of those vehicles across different frames in the video sequence. Finally, a unique and efficient vehicle counting method will be implemented and used for the counting of tracked vehicles across the highway scenes. A new highway vehicle dataset consisting of four vehicle classes namely: car, motorcycle, bus and truck were collected, cleaned and annotated with a total of 11,982 images which will be published in the following study and used for the training of our robust vehicle detection model. From the results observed over real-world highway surveillance data, the following system was able to obtain an average vehicle detection mAP score of 96.1% and a vehicle counting accuracy of 95.39%, all while being able to be deployed on a compact Nvidia Jetson Nano edge computing device with an average speed of 15 FPS which outperforms other previously proposed tools in terms of both accuracy and speed.

Automatic License Plate Recognition (ALPR) has become a common study area because of its many practical applications, such as automatic toll collection and traffic law enforcement. However, most existing methods of Malaysian ALPR are not robust enough to be used in everyday situations. The lack of high-quality benchmarked datasets that accurately represent real-world complexities in Malaysian license plates (LP) and the absence of a comprehensive dataset to demonstrate system robustness is a significant limitation. In addition, the reliance on shallow techniques in the studies on Malaysian ALPR causes inefficiency of the systems, particularly in handling complicated scenarios involving different image backgrounds and variations in LP size or shape and also the non-standard LPs. This dissertation presents a robust Malaysian ALPR system based on the single-shot detector You Only Look Once (YOLO) in two stages; license plate detection (LPD) and license plate recognition (LPR). The system is designed by evaluating and optimizing different models with different dataset optimization to achieve the best speed versus accuracy trade-off in ALPR system. The models are trained using a large-scale dataset containing images from several places around Malaysia, with the addition of data augmentation techniques to make them robust under various circumstances (e.g., with variations in lighting, camera position and settings, and license plate types). A dataset augmentation has also been accomplished by systematically generating a large, controlled synthetic dataset. The purpose is to achieve a balanced dataset and ensure the robustness of the dataset in terms of variations that exist in Malaysian license plates in the form of non-standardized license plates and special license plates. Thus, this work introduces a dataset for Malaysian ALPR with more than 176,000 images from real-world scenarios and synthetically produced images covering various aspects. This dataset will be public to the research community. The name of this dataset is Malaysian Number Plate and in short (MYNO). This dataset can be used for further training and evaluation of ALPR models. A separate challenging dataset is created for testing the models. Many experiments are carried out in detail with different models, data size, number of epochs, and real and synthetic datasets. When adding the synthetic dataset, the system performed better with 97.6% mAP compared to 85.5% mAP for the only real-world dataset at the same number of epochs. The proposed system achieved a recognition rate of 98.1% mAP on a real-world dataset collected from different toll plazas around Malaysia containing comprehensive environment distinctions with over 50 thousand labeled images. The system was tested on a challenging test dataset with low visibility and an unconstrained environment, resulting in 95.96% end-to-end accuracy. The results demonstrate the significance of incorporating synthetic datasets into the training process for improved performance in ALPR systems. The inclusion of a synthetic dataset led to a substantial increase in mean average precision (mAP), with a notable improvement of 12.1% when combined with the synthetic dataset. The system showcases its effectiveness in handling diverse environmental conditions by achieving 98.1% mAP on a real-world dataset collected from various toll plazas in Malaysia. In addition, achieving an impressive end-to-end accuracy of 95.96% despite low visibility further validates the system’s performance on challenging dataset.