Perspective projection with Homography – OpenCV

Posted: June 14, 2014 in General

Tools:
It is assumed that OpenCV is already installed.

The goal:
To make a perspective transformation on an image using homography and overlay it onto the other image.

Input :
Two image files – “main image” and “logo image”.
main

logo

Output:
Overlayed images

out1

out2

 

Algorithm
The “logo image” is overlayed onto the main image. We need a homography matrix to transform the image points of “logo image” before it is overlayed. To calculate a homography matrix we need 4 correspomding pair of points from “logo image” and “main image”. The 4 points for “logo image” are taken as the four corners of the image where as the 4 points for “main image” are chosen by the user. Remember each of the 4 points in each image is of the form (x,y).

Once homography matrix is calculated the “logo image” is perspectively projected onto the “main image”. In this implementation the pixels of “logo image” replace the pixels of “main image”. Users can change this to do any other kind of blending.

Running the code:

Github code
Assume that the executable generated is “homography”. The images files(main.jpg,logo.jpg) are passed as two arguments. Note that the image files are present in the Github repository.

$ ./homography main.jpg logo.jpg
The video link with demo :

 Code:

#include <opencv2/core/core.hpp>
#include <opencv2/highgui/highgui.hpp>
#include "opencv2/imgproc/imgproc.hpp"
#include "opencv2/calib3d/calib3d.hpp"
#include <iostream>
#include <limits>
#include <numeric>
using namespace cv;
using namespace std;

// We need 4 corresponding 2D points(x,y) to calculate homography.
vector<Point2f> left_image;      // Stores 4 points(x,y) of the logo image. Here the four points are 4 corners of image.
vector<Point2f> right_image;    // stores 4 points that the user clicks(mouse left click) in the main image.

// Image containers for main and logo image
Mat imageMain;
Mat imageLogo;

// Function to add main image and transformed logo image and show final output.
// Icon image replaces the pixels of main image in this implementation.
void showFinal(Mat src1,Mat src2)
{

    Mat gray,gray_inv,src1final,src2final;
    cvtColor(src2,gray,CV_BGR2GRAY);
    threshold(gray,gray,0,255,CV_THRESH_BINARY);
    //adaptiveThreshold(gray,gray,255,ADAPTIVE_THRESH_MEAN_C,THRESH_BINARY,5,4);
    bitwise_not ( gray, gray_inv );
    src1.copyTo(src1final,gray_inv);
    src2.copyTo(src2final,gray);
    Mat finalImage = src1final+src2final;
    namedWindow( "output", WINDOW_AUTOSIZE );
    imshow("output",finalImage);
    cvWaitKey(0);

}

// Here we get four points from the user with left mouse clicks.
// On 5th click we output the overlayed image.
void on_mouse( int e, int x, int y, int d, void *ptr )
{
    if (e == EVENT_LBUTTONDOWN )
    {
        if(right_image.size() < 4 )
        {

            right_image.push_back(Point2f(float(x),float(y)));
            cout << x << " "<< y <<endl;
        }
        else
        {
            cout << " Calculating Homography " <<endl;
            // Deactivate callback
            cv::setMouseCallback("Display window", NULL, NULL);
            // once we get 4 corresponding points in both images calculate homography matrix
            Mat H = findHomography(  left_image,right_image,0 );
            Mat logoWarped;
            // Warp the logo image to change its perspective
            warpPerspective(imageLogo,logoWarped,H,imageMain.size() );
            showFinal(imageMain,logoWarped);

        }

    }
}


int main( int argc, char** argv )
{
//  We need tow argumemts. "Main image" and "logo image"
    if( argc != 3)
    {
        cout <<" Usage: error" << endl;
        return -1;
    }


// Load images from arguments passed.
    imageMain = imread(argv[1], CV_LOAD_IMAGE_COLOR);
    imageLogo = imread(argv[2], CV_LOAD_IMAGE_COLOR);
// Push the 4 corners of the logo image as the 4 points for correspondence to calculate homography.
    left_image.push_back(Point2f(float(0),float(0)));
    left_image.push_back(Point2f(float(0),float(imageLogo.rows)));
    left_image.push_back(Point2f(float(imageLogo.cols),float(imageLogo.rows)));
    left_image.push_back(Point2f(float(imageLogo.cols),float(0)));



    namedWindow( "Display window", WINDOW_AUTOSIZE );// Create a window for display.
    imshow( "Display window", imageMain );


    setMouseCallback("Display window",on_mouse, NULL );


//  Press "Escape button" to exit
    while(1)
    {
        int key=cvWaitKey(10);
        if(key==27) break;
    }


    return 0;
}
Advertisements
Comments
  1. Przemek says:

    Is any way to move this code to android?

    • What do you mean “move this code to android” ? It’s in C++, ofcourse you can simple use it directly if you’ve setup Android NDK. Also, you’d need to add opencv libs in your build path. Lots of examples on the web on how to do that, infact, it’s right there in OpenCV’s docs.

  2. Hi, thanks for such a great article.

    I converted it to Python, and it ran beautifully. I’ll open source my script on Github soon.

    But I ran into a problem – if the logo image has black parts inside it, than those parts will become transparent in the final image. And instead of seeing the black parts, we see the main image’s pixels.

    How can I fix this? Any ideas? Thanks.

    • I see your problem. Good catch.
      Do an imshow(gray) and also imshow(gray_inv) images to get a clear understanding of the problem.

      1) Once you get gray image, find the bounding box of the logo region and make everything inside it white. Basically we want to create a mask for the outline of the gray image(i.e logo image). Then generate gray_inv from gray. Contours function in opencv should help you in achieving this.

      Best,
      Ramsri

  3. Tamara says:

    Hello,

    I am sorry in advance for asking but i am pretty new to using opencv. When trying to do this tutorial first thing to do is to run the code? and i use visual studio 2015 as my IDE so do i run the code there as a .cpp file?
    and then use the command you provided $ ./homography main.jpg logo.jpg ?

    Thank you,

    Tamara

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s