Skip to content

Image stitching

Image stitching or photo stitching is the process of combining multiple photographic images with overlapping fields of view to produce a segmented panorama or high-resolution image. Commonly performed through the use of computer software, most approaches to image stitching require nearly exact overlaps between images and identical exposures to produce seamless results, although some stitching algorithms actually benefit from differently exposed images by doing high-dynamic-range imaging in regions of overlap. Some digital cameras can stitch their photos internally.

OpenCV Example

cv2.Stitcher_create를 사용한다.

대략적인 방법 (C++)

SURF이용해서 특징점 추출 

int nMinHessian = 300; // threshold (한계점)
Ptr<SurfFeatureDetector> Detector = SURF::create(nMinHessian);

vector <KeyPoint> vtKeypointsObject, vtKeypointsScene;

Detector->detect(matGrayLImage, vtKeypointsObject);
Detector->detect(matGrayRImage, vtKeypointsScene);

Dscriptor(기술자)들 사이의 매칭 결과를 matches에 저장한다. 

Ptr<SurfDescriptorExtractor> Extractor = SURF::create();

Mat matDescriptorsObject, matDescriptorsScene;

//descriptor 생성 
Extractor->compute(matGrayLImage, vtKeypointsObject, matDescriptorsObject);
Extractor->compute(matGrayRImage, vtKeypointsScene, matDescriptorsScene);

FlannBasedMatcher Matcher; //kd트리를 사용하여 매칭을 빠르게 수행
vector <DMatch> matches;
Matcher.match(matDescriptorsObject, matDescriptorsScene, matches);

Good matching 고르기 

// 두 개의 keypoint 사이에서 min distance를 계산
    for (int i = 0; i < matDescriptorsObject.rows; i++) {
        dDistance = matches[i].distance;
        if (dDistance < dMinDist) dMinDist = dDistance;
    }

//match의 distance 값이 작을수록 matching이 잘 된 것
//min의 값의 3배 또는 good_matches.size() > 60 까지만 goodmatch로 인정해준다.
    vector<DMatch>good_matches;
    int distance = 10;
    do {
        vector<DMatch>good_matches2;
        for (int i = 0; i < matDescriptorsObject.rows; i++) {
            if (matches[i].distance < distance * dMinDist)
                good_matches2.push_back(matches[i]);
        }
        good_matches = good_matches2;
        distance -= 1;
    } while (distance != 2 && good_matches.size() > 60);

Goodmatch에서의 keypoint를 저장 

    for (int i = 0; i < good_matches.size();i++) {
        obj.push_back(vtKeypointsObject[good_matches[i].queryIdx].pt);
        scene.push_back(vtKeypointsScene[good_matches[i].trainIdx].pt);
    }

 // Homography 
 ////RANSAC기법을 이용하여 첫 번째 매개변수와 두번째 매개변수 사이의 3*3 크기의 투영행렬변환 H를 구한다
    Mat HomoMatrix = findHomography(scene, obj, CV_RANSAC);

//Homograpy matrix를 사용하여 이미지를 삐뚤게
    Mat matResult;
    warpPerspective(matRightImage, matResult, HomoMatrix, Size(matLeftImage.cols*2, matLeftImage.rows*1.2), INTER_CUBIC);

Deep learning based

Unsupervised Deep Image Stitching: Reconstructing Stitched Features to Images
https://github.com/nie-lang/UnsupervisedDeepImageStitching
https://arxiv.org/pdf/2106.12859.pdf
Deep Rectangling for Image Stitching - A Learning Baseline
https://arxiv.org/abs/2203.03831
https://github.com/nie-lang/DeepRectangling

See also

Favorite site