I started a project recently that utilizes several pieces of code to analyze imagery. The project takes imagery captured by a camera lens and then proceeds to extract information from the image. This process is used in "Machine Learning" and "Face Detection".
In order to process an image accurately , we need to traverse the entire image pixel by pixel. For each pixel we then need to extract the color information and decide if the brightness or color fits in with conditional statements we have set, in other words, is the pixel good or bad, yes or no, 0 or 1.
The issue is speed. The number of pixels in a 800 x 600 image matrix equals 480,000. This can take several seconds, all depending on the language you using. In this particular case C# was used.
At first the standard method of GetPixel() and SetPixel() was tested. This can be found in many languages including Java and C#. The time taken to run these functions is incredibly slow, several seconds.... by todays computing standards this is unacceptable.
If we use a for loop to traverse a pixel matrix, we start by looping through the number of pixels in the height (600) and proceed to use a nested loop to loop through the width(800).
for(int i = 0 ; i<image.Height(); i++){
for(int j = 0 ; j<image.Width(); j++){
image.GetPixel(i,j);
}
}
Now you might think that looping through 480,000 pixels is what slows down the entire process..... its does but it is not the main culprit. It is the GetPixel() function/method that causes the bottleneck. Here is a list of different languages that I employ and their equivalent of this function:
- Java: getRGB() setRGB()
- C#: GetPixel() SetPixel()
- Objective-C (macOS): NSReadPixel()....As far as I know , iOS does not contain this function, Apple obviously think its a waste of time.... and rightfully so:)
Instead of using these methods we can employ a slightly more complicated (under exaggeration) method to retrieve the values of each individual pixel. The technique involves accessing the raw bits/bytes of an image.
In order to to do this we need to create a byte buffer , copy the pixel data from memory into it and loop through this new buffer. I know this might seem like it would take longer , but the trick is that the bytes are in memory and therefor can now be accessed far quicker than trying to read a file from your hard drive.
The really impressive techniques actually lock the image data in your computers memory (RAM) and allow you to access the bytes directly without copying the data back and forth from your hard drive, its very quick! This basically operates on the same principles as software such as OpenGL which deals with GPU (graphic memory). Graphics card increase their pixel read and write performance by keeping most of the pixel data in the graphics card.
- Java:ByteBuffer & imageData.copyPixelsToBuffer()
- C#: image.LockBits()
- Objective-C: CFDataRef pixelData = CGDataProviderCopyData(CGImageGetDataProvider())
