10.14489/vkit.2020.08.pp.012-018

DOI: 10.14489/vkit.2020.08.pp.012-018

Бойцова Э. П., Лебедев И. С.
МЕТОД ВЫЧИСЛЕНИЯ ПЛОТНОГО ОПТИЧЕСКОГО ПОТОКА НА ОСНОВЕ АУГМЕНТАЦИИ ХАРАКТЕРИСТИЧЕСКИХ ТОЧЕК
(с. 12-18)

Аннотация. Предложен метод вычисления плотного оптического потока на основе устойчивого оптического потока RLOF (Robust Local Optical Flow). Для улучшения качественных показателей использован AGAST-детектор (Adaptive and Generic Accelerated Segment Test) с аугментацией характеристических точек. Его применение позволило сократить время вычисления оптического потока без потери качества результата. Проведена денсификация потока методом растровой интерполяции, где геодезические веса заменены весами на основе вторых частных производных. Применено вариационное уточнение для финальной обработки оптического потока.

Ключевые слова: оптический поток; детекторы ключевых точек; устойчивый локальный оптический поток; денсификация оптического потока; оценка больших смещений; инициализация по глобальному движению.

Boytsova E. P., Lebedev I. S.
METHOD OF OPTICAL FLOW CALCULATION BASED ON FEATURE AUGMENTATION
(pp. 12-18)

Abstract. The paper addresses the problem of robust optical flow computation for real-time applications. We present an efficient, in presence of varying illumination and large displacements, method of dense optical flow calculation based on Robust Local Optical Flow (RLOF). Introduction of Adaptive and Generic Accelerated Segment Test (AGAST) feature detector with additional augmentation by simple grid points provides a stable and reliable under difficult conditions set of features and uniform points that produces dense and accurate optical flow and leads to better performance. The usage of feature points for prior initialization leads for robustness for large displacements as well. The other essential step of optical flow calculation – densification of optical flow – was strengthen by modification of weights. The raster interpolation was applied with weights on second derivatives to make more edgesensitive optical flow. To enhance the overall result, a post-processing step of variational refinement process were added to optical flow calculation framework. The proposed method was evaluated on Max Planck Institute (MPI) Sintel dataset – a challenging set of 3D-animated movies with large range of motions and illuminations, which has ground truth optical flow available. The comparison with initial RLOF method were measured by running time and quality parameters, measured by End Point Error (EPE) and Angular Error (AE). The evaluation shows the decrease of running time by approximately 40 % and improvement of EPE and AE, that demonstrates the effectiveness of the method.

Keywords: Optical flow; Feature detectors; Robust local optical flow; Optical flow densification; Large motion estimation; Global motion prior initialization.

+ - Информация об авторах (About the Authors) Click to collapse

Рус

Э. П. Бойцова, И. С. Лебедев (Санкт-Петербургский институт информатики и автоматизации Российской академии наук, Санкт-Петербург, Россия) E-mail: Этот e-mail адрес защищен от спам-ботов, для его просмотра у Вас должен быть включен Javascript

Eng

E. P. Boytsova, I. S. Lebedev (St. Petersburg Institute for Informatics and Automation of Russian Academy of Sciences, St. Petersburg, Russia) E-mail: Этот e-mail адрес защищен от спам-ботов, для его просмотра у Вас должен быть включен Javascript

+ - Библиографический список (References) Click to collapse

Рус

1. Lucas B., Kanade T. An Iterative Image Registration Technique with an Application to Stereo Vision // Intern. Joint Conf. on Artificial Intelligence (IJCAI). 1981. V. 81. P. 674 – 679.
2. Tomasi C., Kanade T. Detection and Tracking of Point Features // Intern. Journal of Computer Vision. 1991. V. 9. P. 185 – 203.
3. Bouguet J.-Y. Pyramidal Implementation of the Lucas-Kanade Feature Tracker Description of the Algorithm. Intel Corporation, Microprocessor Research Labs, 2000. 10 p.
4. Horn B., Schunck B. Determining Optical Flow // Intern. Joint Conf. on Artiﬁcial Intelligence. 1981. V. 17. P. 185 – 203.
5. Fast Optical Flow Using Dense Inverse Search / T. Kroeger et al. // ECCV 2016. Lecture Notes in Computer Science. 2016. V. 9908. P. 471 – 488.
6. High Accuracy Optical Flow Estimation Based on a Theory for Warping / T. Brox et al. // Proc. of the European Conf. on Computer Vision (ECCV). 2004. V. 3024. P. 25 – 36.
7. EpicFlow: Edge-Preserving Interpolation of Correspondences for Optical Flow / J. Revaud et al. // Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition. 2015. P. 1164 – 1172.
8. Deep Matching: Hierarchical Deformable Dense Matching / J. Revaud et al. // Intern. Journal of Computer Vision. 2016. V. 120, Is. 3. P. 300 – 323.
9. Geistert J., Senst T., Sikora T. Robust Local Optical Flow: Dense Motion Vector Feld Interpolation // Picture Coding Symposium (PCS). 2016. P. 1 – 5.
10. On Benchmarking Optical Flow / B. McCane et al. // Computer Vision and Image Understanding. 2001. V. 84. P. 126 – 143.
11. A Naturalistic Open Source Movie for Optical Flow Evaluation / D. J. Butler et al. // ECCV 2012. Lecture Notes in Computer Science. 2012. V. 7577. P. 611 – 625.
12. Senst T., Eiselein V., Sikora T. Robust Local Optical Flow for Feature Tracking // IEEE Transactions on Circuits and Systems for Video Technology. 2012. V. 22. P. 1377 – 1387.
13. Cross Based Robust Local Optical Flow / T. Senst et al. // 2014 IEEE Intern. Conf. on Image Processing (ICIP’2014). 2014. P. 1967 – 1971.
14. Senst T., Geistert J., Sikora T. Robust Local Optical Flow: Long-Range Motions and Varying Illuminations // 2016 IEEE Intern. Conf. on Image Processing (ICIP). 2016. P. 4478 – 4482.
15. Adaptive and Generic Corner Detection Based on the Accelerated Segment Test / E. Mair et al. // ECCV 2010. Lecture Notes in Computer Science. 2010. V. 6312. P. 183 – 196.
16. Geodesic Image and Video Editing / A. Criminisi et al. // ACM Transactions on Graphics. 2010. V. 29, Is. 5. P. 134:1 – 134:15.
17. Fast Global Image Smoothing Based on Weighted Least Squares / D. Min et al. // IEEE Transactions on Image Processing. 2014. V. 23, Is. 12. P. 5638 – 5653.

Eng

1. Lucas B., Kanade T. (1981). An Iterative Image Registration Technique with an Application to Stereo Vision. International Joint Conference on Artificial Intelligence (IJCAI), Vol. 81, pp. 674 – 679.
2. Tomasi C., Kanade T. (1991). Detection and Tracking of Point Features. International Journal of Computer Vision, Vol. 9, pp. 185 – 203.
3. Bouguet J.-Y. (2000). Pyramidal Implementation of the Lucas-Kanade Feature Tracker Description of the Algorithm. Intel Corporation, Microprocessor Research Labs.
4. Horn B., Schunck B. (1981). Determining Optical Flow. International Joint Conference on Artiﬁcial Intelligence, Vol. 17, pp. 185 – 203.
5. Kroeger T. et al. (2016). Fast Optical Flow Using Dense Inverse Search. ECCV 2016. Lecture Notes in Computer Science, Vol. 9908, pp. 471 – 488.
6. Brox T. et al. (2004). High Accuracy Optical Flow Estimation Based on a Theory for Warping. Proceedings of the European Conference on Computer Vision (ECCV), Vol. 3024, pp. 25 – 36.
7. Revaud J. et al. (2015). EpicFlow: Edge-Preserving Interpolation of Correspondences for Optical Flow. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1164 – 1172.
8. Revaud J. et al. (2016). Deep Matching: Hierarchical Deformable Dense Matching. International Journal of Computer Vision, Vol. 120, (3), pp. 300 – 323.
9. Geistert J., Senst T., Sikora T. (2016). Robust Local Optical Flow: Dense Motion Vector Feld Interpolation. Picture Coding Symposium (PCS), pp. 1 – 5.
10. McCane B. et al. (2001). On Benchmarking Optical Flow. Computer Vision and Image Understanding, Vol. 84, pp. 126 – 143.
11. Butler D. J. et al. (2012). A Naturalistic Open Source Movie for Optical Flow Evaluation. ECCV. Lecture Notes in Computer Science, Vol. 7577, pp. 611 – 625.
12. Senst T., Eiselein V., Sikora T. (2012). Robust Local Optical Flow for Feature Tracking. IEEE Transactions on Circuits and Systems for Video Technology, Vol. 22, pp. 1377 – 1387.
13. Senst T. et al. (2014). Cross Based Robust Local Optical Flow. IEEE International Conference on Image Processing (ICIP’2014), pp. 1967 – 1971.
14. Senst T., Geistert J., Sikora T. (2016). Robust Local Optical Flow: Long-Range Motions and Varying Illuminations. IEEE International Conference on Image Processing (ICIP), pp. 4478 – 4482.
15. Mair E. et al. (2010). Adaptive and Generic Corner Detection Based on the Accelerated Segment Test. ECCV 2010. Lecture Notes in Computer Science, Vol. 6312, pp. 183 – 196.
16. Criminisi A. et al. (2010). Geodesic Image and Video Editing. ACM Transactions on Graphics, Vol. 29, (5), pp. 134:1 – 134:15.
17. Min D. et al. (2014). Fast Global Image Smoothing Based on Weighted Least Squares. IEEE Transactions on Image Processing, Vol. 23, (12), pp. 5638 – 5653.

+ - Заказать электронную версию статьи (Purchase digital version of a single article) Click to collapse

Рус

Статью можно приобрести в электронном виде (PDF формат).

Стоимость статьи 350 руб. (в том числе НДС 18%). После оформления заказа, в течение нескольких дней, на указанный вами e-mail придут счет и квитанция для оплаты в банке.

После поступления денег на счет издательства, вам будет выслан электронный вариант статьи.

Для заказа скопируйте doi статьи:

10.14489/vkit.2020.08.pp.012-018

и заполните форму

Отправляя форму вы даете согласие на обработку персональных данных.

Eng

This article is available in electronic format (PDF).

The cost of a single article is 350 rubles. (including VAT 18%). After you place an order within a few days, you will receive following documents to your specified e-mail: account on payment and receipt to pay in the bank.

After depositing your payment on our bank account we send you file of the article by e-mail.

To order articles please copy the article doi:

10.14489/vkit.2020.08.pp.012-018

and fill out the form