10.14489/vkit.2024.12.pp.003-015

DOI: 10.14489/vkit.2024.12.pp.003-015

Воробьев Т. К., Ермаков П. Г., Зотов А. В., Комаров Д. В., Лебедев М. А.
ИНТЕЛЛЕКТУАЛЬНАЯ ОБРАБОТКА ИЗОБРАЖЕНИЙ ОТ СИСТЕМ ТЕХНИЧЕСКОГО ЗРЕНИЯ НА ЭТАПАХ ПОСАДКИ ВОЗДУШНЫХ СУДОВ
(c. 3-15)

Аннотация. Предложено решение задачи определения местоположения взлетно-посадочной полосы (ВПП) и ее элементов на изображениях подстилающей поверхности от системы технического зрения на этапах захода на посадку и посадки. Данный метод включает отдельные нейросетевые архитектуры, которые осуществляют решение следующих подзадач: обнаружение ВПП, обнаружение порога ВПП, обнаружение и распознавание символьных обозначений (номер ВПП) и уточнение дополнительных параметров ВПП, таких как положение ее угловых точек, положение порога ВПП, определение степени достоверности и углов наклона боковых линий ВПП относительно горизонта.

Ключевые слова: обнаружение взлетно-посадочной полосы; YOLO; MnasNet; RegNet, трекинг; фильтр Калмана.

Vorobyov T. K., Ermakov P. G., Zotov A. V., Komarov D. V., Lebedev M. A.
INTELLIGENT PROCESSING OF IMAGES FROM VISION SYSTEMS AT STAGES OF AIRCRAFT LANDING
(pp. 3-15)

Abstract. Since 2016, the appearance of the 4th generation intelligent technical vision system has begun to take shape based on the use of modern highly sensitive sensors of different spectral ranges and physical nature. This research solves the problem of determining the location of the runway and its elements on images of an underlying surface from the technical vision system at the stages of approach and landing. The proposed method includes separate neural network architectures that solve the following subtasks: the runway detection, the runway’s threshold detection, the symbol detection (the runway number) and clarification of additional the runway parameters such as: a position of its corner points, a position of the runway’s threshold, a determination of the degree of reliability and a determination of an inclination angles of the runway sidelines relative to the horizon. The developed method is based on such neural network architectures as YOLO8n-p6 for the task of object detection, MnasNet and RegNet for the task of regression of 4 corner points of an object, as well as on the architecture of multivalued classification for runway number recognition. To neutralize effect of possible difficult weather conditions during the landing process it’s proposed to track the runway corner points using Kalman filter. The results of a simulation modelling of a determination of the runway location and its elements based on processing public videos of aircraft’s landing stage are presented.

Keywords: Runway detection; YOLO; MnasNet; RegNet; Tracking; Kalman filter.

+ - Информация об авторах (About the Authors) Click to collapse

Рус

Т. K. Воробьев, П. Г. Ермаков, А. В. Зотов, Д. В. Комаров, М. А. Лебедев (ФАУ «Государственный научно-исследовательский институт авиационных систем», Москва, Россия) E-mail: Этот e-mail адрес защищен от спам-ботов, для его просмотра у Вас должен быть включен Javascript

Eng

T. K. Vorobyov, P. G. Ermakov, A. V. Zotov, D. V. Komarov, M. A. Lebedev (FAI “State Research Institute of Aviation Systems”, Moscow, Russia) E-mail: Этот e-mail адрес защищен от спам-ботов, для его просмотра у Вас должен быть включен Javascript

+ - Библиографический список (References) Click to collapse

Рус

1. Ajay P., Raja M., Rizwanbasha A. Real Time Object Detection based on RCNN Technique // 3rd International Conference on Pervasive Computing and Social Networking (ICPCSN). Salem, India. 19–20 June 2023. IEEE, 2023. P. 698–702.
2. Ren S., He K., Girshick R., Sun J. Faster R-CNN: Towards real-time object detection with region proposal networks // IEEE transactions on pattern analysis and machine intelligence. 2016. V. 39, No 6. P. 1137–1149.
3. Girshick R. Fast R-CNN //Proceedings of the IEEE international conference on computer vision. Santiago, Chile. 07 – 13 December 2015. P. 1440–1448.
4. Erhan D., Szegedy C., Toshev A., Angualov D. Scalable object detection using deep neural networks // Proceedings of the IEEE conference on computer vision and pattern recognition. Columbus, OH, USA. 23 – 28 June 2014. P. 2147–2154.
5. Glenn J. YOLOv8 Ultralytics [Электронный ресурс]. URL: https://github.com/ultralytics/ultralytics (дата обращения: 01.11.2024).
6. Mnasnet: Platform-aware neural architecture search for mobile / M. Tan, Bo Chen, R. Pang et al. // Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. Long Beach, CA, USA. 15 – 20 June 2019. P. 2820–2828.
7. RegNet: Self-regulated network for image classification / J. Xu, Y. Pan, X. Pan et al. // IEEE Transactions on Neural Networks and Learning Systems. 2022. V. 34, No 11. P. 9562–9567.
8. Multi-digit number recognition from street view imagery using deep convolutional neural networks / Ian J. Goodfellow, Y. Bulatov, J. Ibarz et al. // arXiv preprint arXiv:1312.6082. 2013.
9. Mohammed H. R., Hussain Z. M. Detection and recognition of moving video objects: kalman filtering with deep learning // International Journal of Advanced Computer Science and Applications. 2021. V. 12, No 1.
10. Redmon J. YOLOv3: An incremental improvement //arXiv preprint arXiv:1804.02767. 2018.
11. Glenn J. YOLOv5 Ultralytics [Электронный ресурс]. URL: https://docs.ultralytics.com/ru/yolov5/ (дата обращения: 01.11.2024).
12. YOLOv6: A single-stage object detection framework for industrial applications / C. Li, L. Li, H. Jiang et al. // arXiv preprint arXiv:2209.02976. 2022.
13. Tan M. Efficientnet: Rethinking model scaling for convolutional neural networks // arXiv preprint arXiv:1905.11946. 2019.
14. Howard A. G. MobileNets: Efficient convolutional neural networks for mobile vision applications // arXiv preprint arXiv:1704.04861. 2017.

Eng

1. Ajay P., Raja M., Rizwanbasha A. (2023). Real Time Object Detection based on RCNN Technique. 3rd International Conference on Pervasive Computing and Social Networking (ICPCSN), 698 – 702. IEEE. Salem.
2. Ren S., He K., Girshick R., Sun J. (2016). Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE transactions on pattern analysis and machine intelligence, 39(6), 1137 – 1149.
3. Girshick R. (2015). Fast R-CNN. Proceedings of the IEEE international conference on computer vision, 1440 – 1448. Santiago.
4. Erhan D., Szegedy C., Toshev A., Angualov D. (2014). Scalable object detection using deep neural networks. Proceedings of the IEEE conference on computer vision and pattern recognition, 2147 – 2154. Columbus.
5. Glenn J. YOLOv8 Ultralytics. Retrieved from https://github.com/ultralytics/ultralytics (Accessed: 01.11.2024).
6. Tan M., Chen Bo, Pang R. et al. (2019). Mnasnet: Platform-aware neural architecture search for mobile. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2820 – 2828. Long Beach.
7. Xu J., Pan Y., Pan X. et al. (2022). RegNet: Self-regulated network for image classification. IEEE Transactions on Neural Networks and Learning Systems, 34(11), 9562 – 9567.
8. Goodfellow Ian J., Bulatov Y., Ibarz J. et al. (2013). Multidigit number recognition from street view imagery using deep convolutional neural networks. arXiv preprint arXiv:1312.6082.
9. Mohammed H. R., Hussain Z. M. (2021). Detection and recognition of moving video objects: kalman filtering with deep learning. International Journal of Advanced Computer Science and Applications, 12(1).
10. Redmon J. (2018). YOLOv3: An incremental improvement. arXiv preprint arXiv:1804.02767.
11. Glenn J. YOLOv5 Ultralytics. Retrieved from https://docs.ultralytics.com/ru/yolov5/ (Accessed: 01.11.2024) [in Russian language]
12. Li C., Li L., Jiang H. et al. (2022). YOLOv6: A singlestage object detection framework for industrial applications. arXiv preprint arXiv:2209.02976.
13. Tan M. (2019). Efficientnet: Rethinking model scaling for convolutional neural networks. arXiv preprint arXiv:1905.11946.
14. Howard A. G. (2017). MobileNets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861.

+ - Заказать электронную версию статьи (Purchase digital version of a single article) Click to collapse

Рус

Статью можно приобрести в электронном виде (PDF формат).

Стоимость статьи 500 руб. (в том числе НДС 20%). После оформления заказа, в течение нескольких дней, на указанный вами e-mail придут счет и квитанция для оплаты в банке.

После поступления денег на счет издательства, вам будет выслан электронный вариант статьи.

Для заказа скопируйте doi статьи:

10.14489/vkit.2024.12.pp.003-015

и заполните форму

Отправляя форму вы даете согласие на обработку персональных данных.

Eng

This article is available in electronic format (PDF).

The cost of a single article is 500 rubles. (including VAT 20%). After you place an order within a few days, you will receive following documents to your specified e-mail: account on payment and receipt to pay in the bank.

After depositing your payment on our bank account we send you file of the article by e-mail.

To order articles please copy the article doi:

10.14489/vkit.2024.12.pp.003-015

and fill out the form