Motion estimation

Motion Estimation(ME) 이란, 동영상 비디오에서 각 객체(Object) 혹은 처리 단위 블록(Macroblock)이 시간상 앞뒤 프레임에서 어느 위치로 움직였는지를 추정(Estimation)하는 것을 말하는 것으로 MPEG 등의 동영상 압축과 화소 디모자이킹(Pixel Demosaicking), 프레임 보간(Frame Interpolation) 등의 비디오 영상처리에서 광범위하게 이용된다.

Category

• 물체인식 (Object Recognition)
• OpenCV:tracking - OpenCV의 Tracking 모듈 및 API에 대한 설명
• Particle filter
• Mean shift
• Camshift
• MedianFlow
• Tracking Learning Detection (TLD)
• Kernelized Correlation Filters (KCF)
• Fast Tracking via Spatio-Temporal Context Learning (STC) - 이 곳에서 FPS가 KCF보다 높게 나타남. 확인 필요.
• BOOSTING
• KLT (비교적 빠르다는 평가가 있음; 조사 필요.)
• High-Speed Tracking-by-Detection Without Using Image Information (IOU Tracker)
• Multiple Hypothesis Tracking Revisited
• CREST: Convolutional Residual Learning for Visual Tracking
• Faster Spatially Regularized Correlation Filters for Visual Tracking
• Integrating Boundary and Center Correlation Filters for Visual Tracking with Aspect Ratio Variation
• Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields
• Attentional Correlation Filter Network for Adaptive Visual Tracking
• Action-Decision Networks for Visual Tracking with Deep Reinforcement Learning
• ROLO
• STRUCK
• VOTR: Visual Object Tracking Repository.
• Simple Online and Realtime Tracking (SORT)
  • Simple Online and Realtime Tracking with a Deep Association Metric (DeepSORT)
• Co-Fusion
• Re-Identification
• MediaPipe
• Hungarian algorithm (헝가리안 알고리즘) - 배정(Assignment) 문제 해결이 필요할 경우 사용.
• Multi-Object Tracking (MOT)
  • 3D Multi-Object Tracking

Human Pose Estimation

• YouTube - Pose-driven Human Action Recognition and Anomaly Detection
• Human Pose Detection

사람의 움직임 추정.

• DensePose
• OpenPose
• AlphaPose
• PoseFlow
• PoseNet
• tf-pose-estimation
• PoseTrack

ETC

Convolutional Pose Machines - Tensorflow

https://github.com/timctho/convolutional-pose-machines-tensorflow

PoseTrack Dataset and Benchmark

https://posetrack.net/

Detect And Track - Efficient Pose Estimation in Videos

https://github.com/facebookresearch/DetectAndTrack

Pose Animator

https://github.com/yemount/pose-animator/

Pose Animator takes a 2D vector illustration and animates its containing curves in real-time based on the recognition result from PoseNet and FaceMesh.

Critter Camera

폰/랩탑을 동작감지 카메라로

https://critter.camera/

안쓰는 폰/태블릿/랩탑을 이용해서 특정 위치의 동작 감지 카메라로 활용

브라우저상에서만 실행되는 로컬 앱 : 모든 이미지는 로컬에만 저장되어 안전

사람부터 아주 작은 물체까지 모든 것을 인식하는 고감도 모션 감지 동작

켜두면 나중에 움직임이 있었던 것들만 캡쳐이미지로 다운로드 가능

unknown

• https://github.com/hafizas101/Real-time-human-pose-estimation-and-classification
• https://gkioxari.github.io/PersonNet/index.html
• https://github.com/LZQthePlane/Online-Realtime-Action-Recognition-based-on-OpenPose
• https://github.com/felixchenfy/Realtime-Action-Recognition
• https://github.com/dakenan1/Realtime-Action-Recognition-Openpose
• https://dronefreak.github.io/human-action-classification/

Documentation

Discrete-Continuous Energy Minimization for Multi-Target Tracking

http://www.milanton.de/dctracking/

https://bitbucket.org/amilan/dctracking

MatLab Source code: Dctracking-v1.0.zip

Automatic Tracker Selection w.r.t Object Detection Performance

https://arxiv.org/abs/1404.2005v1

Globally Optimal Solution to Multi-object Tracking with Merged Measurements

http://www.robots.ox.ac.uk/~joao/merged/

Stable Multi-Target Tracking in Real-Time Surveillance Video (CVPR 2011)

http://www.robots.ox.ac.uk/ActiveVision/Publications/benfold_reid_cvpr2011/benfold_reid_cvpr2011.html

http://www.robots.ox.ac.uk/ActiveVision/Publications/benfold_reid_cvpr2011/benfold_reid_cvpr2011.pdf

Video Tracking with an Adaptive Particle Filter (Person)

Youtube video

Discrete-Continuous Optimization for Multi-Target Tracking (CVPR 2012)

Youtube video

Tracking The Untrackable: Learning To Track Multiple Cues with Long-Term Dependencies

https://arxiv.org/abs/1701.01909v2

Tracking the Trackers: An Analysis of the State of the Art in Multiple Object Tracking

https://arxiv.org/abs/1704.02781v1

[추천] Comparison of tracking algorithms implemented in OpenCV

OpenCV의 Tracking 알고리즘 비교

https://www.matec-conferences.org/articles/matecconf/pdf/2016/39/matecconf_cscc2016_04031.pdf

Video Segmentation via Object Flow

http://files.is.tue.mpg.de/black/papers/TsaiCVPR2016.pdf

Real-Time Multiple Object Tracking - A Study on the Importance of Speed

https://arxiv.org/abs/1709.03572

Github - Code for my master thesis, titled "Real-Time Multiple Object Tracking: A Study on the Importance of Speed".

Deep Learning based Tracker

Learning A Deep Compact Image Representation for Visual Tracking (NIPS2013')

http://winsty.net/dlt.html

Robust Visual Tracking via Convolutional Networks

https://arxiv.org/abs/1501.04505

Deep Tracking - Seeing Beyond Seeing Using Recurrent Neural Networks

https://arxiv.org/abs/1602.00991

Fully-Convolutional Siamese Networks for Object Tracking (SiamFC)

http://www.robots.ox.ac.uk/~luca/siamese-fc.html

https://github.com/bertinetto/siamese-fc

Tensorflow Object Tracking Video

https://github.com/DrewNF/Tensorflow_Object_Tracking_Video

Object Tracking in Tensorflow ( Localization Detection Classification ) developed to partecipate to ImageNET VID competition

Deep Reinforcement Learning for Visual Object Tracking in Videos

https://arxiv.org/abs/1701.08936v1

Online Multi-Target Tracking Using Recurrent Neural Networks

https://arxiv.org/abs/1604.03635v2

GOTURN (Learning to Track at 100 FPS with Deep Regression Networks)

https://github.com/davheld/GOTURN

http://davheld.github.io/GOTURN/GOTURN.html

http://davheld.github.io/GOTURN/GOTURN.pdf

Adaptive Correlation Filters with Long-Term and Short-Term Memory for Object Tracking

https://arxiv.org/abs/1707.02309

Tracking in Aerial Hyperspectral Videos using Deep Kernelized Correlation Filters

https://arxiv.org/abs/1711.07235v1

KCF를 사용하는듯 하다. 정확한 확인이 필요하다.

Efficient Video Object Segmentation via Network Modulation

One-Shot Modulation Network for Semi-supervised Video Segmentation 참조.

Action-Decision Networks for Visual Tracking with Deep Reinforcement Learning (ADNet)

https://sites.google.com/view/cvpr2017-adnet

Deep Learning-Based Multiple Object Visual Tracking on Embedded System for IoT and Mobile Edge Computing Applications

https://arxiv.org/abs/1808.01356

Learning Hierarchical Features for Visual Object Tracking with Recursive Neural Networks

https://arxiv.org/abs/1801.02021v1

Spatially Supervised Recurrent Convolutional Neural Networks for Visual Object Tracking

https://arxiv.org/abs/1607.05781

Multi-Class Multi-Object Tracking using Changing Point Detection

https://arxiv.org/abs/1608.08434

https://www.youtube.com/watch?v=xOGQ1lP7pt8

Detect to Track and Track to Detect

https://arxiv.org/abs/1710.03958

Deep Network Flow for Multi-Object Tracking

http://www.nec-labs.com/uploads/images/Department-Images/MediaAnalytics/papers/cvpr17_deepnetworkflow.pdf

Joint detection and online multi-object tracking

http://openaccess.thecvf.com/content_cvpr_2018_workshops/papers/w29/Kieritz_Joint_Detection_and_CVPR_2018_paper.pdf

Learning Dynamic Memory Networks for Object Tracking (MemTrack)

https://arxiv.org/abs/1803.07268

http://openresearch.ai/t/memtrack-learning-dynamic-memory-networks-for-object-tracking/235

Learning Policies for Adaptive Tracking with Deep Feature Cascades (EAST)

https://arxiv.org/abs/1708.02973

Deep Occlusion Reasoning for Multi-Camera Multi-Target Detection

https://arxiv.org/abs/1704.05775

Parallel Tracking and Verifying - A Framework for Real-Time and High Accuracy Visual Tracking (Parallel Tracking and Verifying; PTAV)

https://arxiv.org/abs/1708.00153

Robust and Real-time Deep Tracking Via Multi-Scale Domain Adaptation

https://arxiv.org/abs/1701.00561

Long-term face tracking in the wild using deep learning

https://arxiv.org/abs/1805.07646

Deep SORT

Simple Online Realtime Tracking with a Deep Association Metric

https://github.com/nwojke/deep_sort

TrackNet

Simultaneous Object Detection and Tracking and Its Application in Traffic Video Analysis

https://arxiv.org/abs/1902.01466

Multi Object Tracking

• adipandas/multi-object-tracker - Multi-object trackers in Python

Deep Learning Based

• Girish Varma - Multi Object Tracking using Deep Learning

Multi Object Tracking is the task tracking objects in video frames. We survey the various types of MOT methods, with special focus on the latest methods that use Deep Learning.

Visual tracking library based on PyTorch

https://github.com/visionml/pytracking

Learning to Track: Online Multi-object Tracking by Decision Making

http://cvgl.stanford.edu/papers/xiang_iccv15.pdf

Online Multi-Target Tracking Using Recurrent Neural Networks

https://arxiv.org/abs/1604.03635

Deep Network Flow for Multi-Object Tracking

https://arxiv.org/abs/1706.08482

Learning to Track at 100 FPS with Deep Regression Networks

https://link.springer.com/chapter/10.1007/978-3-319-46448-0_45

Deep tracking in the wild: End-to-end tracking using recurrent neural networks

http://www.robots.ox.ac.uk/~mobile/Papers/2017_IJRR_Dequaire.pdf

Survey Slides by Merc Benz

https://32e624bb-a-62cb3a1a-s-sites.googlegroups.com/site/dlitsc17/A%20Survey%20on%20Leveraging%20Deep%20Neural%20Networks%20for.pdf

Multiple Object Tracking - A Literature Review

https://arxiv.org/abs/1409.7618

https://pdfs.semanticscholar.org/8b42/8625c8a496c7e56b419197df55c751c22bc3.pdf

A survey on multiple object tracking algorithm

http://ieeexplore.ieee.org/document/7832121/

Target Re-identification and Multi-Target Multi-Camera Tracking (DeepCC)

https://reid-mct.github.io/2019/

Deep Affinity Network for Multiple Object Tracking (Deep Affinity Network; DAN)

https://github.com/shijieS/SST

bendidi/Tracking-with-darkflow

Real-time people Multitracker using YOLO v2 and deep_sort with tensorflow

https://github.com/bendidi/Tracking-with-darkflow

pan1028/deep-multi-tracking

Multiple Object Tracking Using Deep Learning and Kalman Filter

https://github.com/pan1028/deep-multi-tracking

ReIdentificationNet

Enhance Multi-Camera Tracking Accuracy by Fine-Tuning AI Models with Synthetic Data

Benchmark

Generic Object Tracking Benchmark (GOT-10k; got10k)

http://got-10k.aitestunion.com/index

Visual Tracker Benchmark

http://www.visual-tracking.net

[추천] Visual Tracker Benchmark - result

foolwood/benchmark_results - visual tracker benchmark results

https://github.com/foolwood/benchmark_results

Image Matching Benchmark

https://arxiv.org/abs/1709.03917

Long-Term Visual Object Tracking Benchmark

https://arxiv.org/abs/1712.01358v1

A Comparison of Feature Detectors and Descriptors forObject Class Matching

http://vision.cs.tut.fi/data/publications/neurocomputing2015_accepted.pdf

PoseTrack Dataset and Benchmark

https://posetrack.net/

Challenge

Welcome to MOTChallenge - The Multiple Object Tracking Benchmark!

https://motchallenge.net/

MOT17 Results

VOT challenges

http://votchallenge.net/

[추천] The sixth Visual Object Tracking VOT2018 challenge results ¹

VideoNet - A benchmark initiative for all things video

http://videonet.team/

VOT2018 Result

Evolution of Siamese Visual Tracking with Very Deep Networks (SiamRPN++)

https://arxiv.org/abs/1812.11703

High Performance Visual Tracking with Siamese Region Proposal Network (SiamRPN)

http://openaccess.thecvf.com/content_cvpr_2018/papers/Li_High_Performance_Visual_CVPR_2018_paper.pdf

Distractor-aware Siamese Networks for Visual Object Tracking (DaSiamRPN)

https://arxiv.org/abs/1808.06048

LADCF - No 1 Algorithm on the public dataset of VOT2018

Demo for Learning Adaptive Discriminative Correlation Filters (LADCF) via Temporal Consistency preserving Spatial Feature Selection for Robust Visual Tracking

https://github.com/XU-TIANYANG/LADCF

MBMD (MobileNet-based tracking by detection algorithm) for VOT2018 Long-Term Challenge.

https://github.com/xiaobai1217/MBMD

Learning regression and verification networks for long-term visual tracking

Multi-solution Fusion for Visual Tracking (MFT)

https://github.com/ShuaiBai623/MFT

Multi-hierarchical Independent Correlation Filters for Visual Tracking

SiamFC series

See also

• Histogram
• Optical Flow
• 특징 추출 (Feature detection)
• Mask R-CNN
• Object Recognition

Favorite site

• Wikipedia (en) 움직임 추정에 대한 설명
• Naver 오픈백과: 동영상 압축에 있어서의 Motion Estimation
• OpenCV 3.0 Global Motion Estimation
• Github - Deep-Learning-for-Tracking-and-Detection (Collection of papers and other resources for object tracking and detection using deep learning)
• Deep Learning for Video: Object Tracking (UPC 2018)

Site collection

• Tracking - handong1587
  • Github - handong1587.github.io - 2015-10-09-tracking.md ²
• Github - Collection of papers and other resources for object tracking and detection using deep learning ³
• Github - A collection of Multiple Object Tracking (MOT) papers in recent years, with notes ⁴
• Github - Multi-Object-Tracking-Paper-List ⁵

Object Tracking

• 다크 프로그래머 - 영상추적 프로그램(visual tracker) (Mean shift, Camshift, Histogram, Optical Flow)
• 다크 프로그래머 - 영상추적#2: TLD - 추적하면서 학습한다
• Tracker Benchmark v1.0
• OpenCV 동작 감지, 개체 추적
• Condensation 알고리즘
• camshift를 이용한 tracking (opencv 제공 함수 이용)
• [추천] Object Tracking using OpenCV (C++/Python) ⁶ (OpenCV의 Tracking 알고리즘의 장단점 비교)
• Object Tracking - 객체 탐지 및 추적 방법 (1) ⁷
• Object Tracking - 객체 탐지 및 추적 방법 (2) ⁸
• Object Tracking - SVM을 이용한 HOG 기반 객체 추적 ⁹
• Object Tracking - Optical Flow 비교 ¹⁰
• Fast Object Tracking – Robot Computer Vision (Color based tracking, Shape based tracking)
• Multi-object tracking with dlib (dlib)
• [추천] Vehicle Detection and Tracking using Machine Learning and HOG ¹¹
  • Github - Term 1, Project 5 - Udacity Self Driving Car Nanodegree
• Tracking multiple objects with OpenCV - PyImageSearch
• Visual Tracking Decomposition
• Adaptive Color Attributes for Real-Time Visual Tracking (KCF)
  • Accurate Scale Estimation for Robust Visual Tracking
• tracking. OpenCV 3.0.0-dev documentation » OpenCV API Reference » Tracking API
  • Tracker Algorithms
• Tracking API Classes
• 영상추적#2: TLD - 추적하면서 학습한다
• 영상추적#1: Mean Shift 추적
• Kernelized Correlation Filters
• KCF(Kernelized Correlation Filters) (OpenCV 샘플: opencv_contrib/modules/tracking/samples/kcf.cpp)
• OpenCV - 동작 감지, 개체 추적 (Mean-Shift, CamShift, 칼만 필터 (Kalman Filter), 배경 제거, 옵티컬 플로우)
  • Flow accuracy and interpolation evaluation (Optical Flow 알고리즘별 성능 측정 사이트)
• OpenCV 객체추적 프로그램 접근법 (MeanShift and CAMShift)
• 45편 : 비디오에서 객체 추적하기1-MeanShift
• 옵티컬 플로우 - 피라미드 루카스-카나데(객체추적)
• 물체인식에 관한 이야기 - (3) SURF 특징추출하기
• OpenCV - 움직임검출 참고자료
• OpenCV - 화면 이동 검출. Optical flow인 루카스 카나데(cvCalcOpticalFlowPyrLK)예제
• (Object Tracking) 객체 탐지 및 추적 방법 (1)

Multiple Object Tracking

• Youtube: Multiple Object Tracker (OpenCV) (Demo)
• Github - Smorodov/Multitarget-tracker - Multitarget tracker implementation, Hungarian algorithm + Kalman filter
• OpenCV: Using MultiTracker

References