Computer Vision is so fun!

I work as a Software Engineer at Endtest.

In this article, I will show you how easy it is to use Computer Vision.

The term itself sounds intimidating, it might make you think that you need a PhD in Machine Learning.

Why is it useful?

Computer Vision can be used to detect objects in images.

If you have an Arduino, you can build your own self-driving toy car that detects the road signs.

It can also be used to detect differences between images.

This second use case can be extremely useful for developers who want to do automated visual testing.

Let's focus on that one.

Detecting differences between images

The entire concept is dead simple, we just compare each pixel and we calculate the percentage of different pixels.

Well just need to make sure our system has Python, OpenCV, scikit-image, Pillow and imutils.

You can find instructions for installing OpenCV here.

As for the rest, you can just use pip:

pip install scikit-image

pip install imutils

pip install pillow

First, we need to make sure that the images have the same size.

The quickest way to achieve that is just to resize the bigger one.

import argparseimport imutilsimport cv2from PIL import Imagefrom PIL import ImageFilterfrom PIL import ImageDrawimage1 = Image.open("/path/to/image1.png")image2 = Image.open("/path/to/image2.png")image1_width, image1_height = image1.sizeimage2_width, image2_height = image2.sizeimage1_surface = image1_width * image1_heightimage2_surface = image2_width * image2_heightif image1_surface != image2_surface:   if image1_surface > image2_surface:      image1 = image1.resize((image2_width, image2_height), Image.ANTIALIAS)      image1.save("/path/to/image1.png")   if image2_surface > image1_surface:      image2_surface = image2_surface.resize((image1_width, image1_height), Image.ANTIALIAS)      image2.save("/path/to/image2.png")

Now our images have the same size.

Time to compare them:

image1 = cv2.imread("/path/to/image1.png")image2 = cv2.imread("/path/to/image2.png")i1 = image1i2 = image2assert i1.mode == i2.mode, "Different kinds of images."pairs = izip(i1.getdata(), i2.getdata())   if len(i1.getbands()) == 1:      dif = sum(abs(p1 - p2) for p1, p2 in pairs)   else:      dif = sum(abs(c1 - c2) for p1, p2 in pairs for c1, c2 in zip(p1, p2))ncomponents = i1.size[0] * i1.size[1] * 3diff = (dif / 255.0 * 100) / ncomponentsdiff = Decimal(diff)diff = round(diff, 2)print "Difference between images is " + str(diff) + "%." 

Difference between images is 7.11%.

We can also generate a third image that shows us the differences.

This can be done by generating a mask and adding it on top of the first image.

difference = cv2.subtract(image1, image2)Conv_hsv_Gray = cv2.cvtColor(difference, cv2.COLOR_BGR2GRAY)ret, mask = cv2.threshold(Conv_hsv_Gray, 0, 255,cv2.THRESH_BINARY_INV |cv2.THRESH_OTSU)difference[mask != 255] = [0, 0, 255]image1[mask != 255] = [0, 0, 255]image2[mask != 255] = [0, 0, 255]cv2.imwrite("/path/to/image3.png", image1)

Benefits

There have been documented cases where elements have disappeared from pages and no one noticed it for weeks.

Implementing a visual check for your site can't hurt.

And think of how much fun you can have by testing your implementation on all of those Spot the differences challenges.

POC vs Reality

What I showed you was a basic example.

If you would apply this on an actual system, you would also need to make small tweaks.

For example, let's say that your images are identical, but one of them is 2px higher than the other.

You would need a function to focus the 2 images before comparing them.

Almost forgot

If you want to perform Automated Testing without reinventing the wheel, you can use Endtest.