Archive for January, 2017

Spatial convolution and bokeh

Tuesday, January 17th, 2017

On my path to gaining proficiency in ecmascript6, I am exercising with a windows bitmap decoder, spatial convolution and bokeh effect filter.

49

Windows Bitmap decoder

This decoder is capable only of loading 24bpp uncompressed RGB image from your local drive at current.  The source code is based on a comprehensive version written in actionscript back in August 1, 2009.  The Windows 3.x Bitmap format (BMP) is defined by four sections below.  Please consider reference text below for detail.

  1. BITMAP File-Header
  2. Bitmap Info (Bitmap Info-header)
  3. Bitmap palette (if exists)
  4. BITMAP-Picture data

Spatial Convolution

Filters, blur, laplacian and sharpen have been implemented for testing.  For blur, a RECT function of 5×5 kernel is set for default and configurable.  For laplacian, it is an isotropic derivative filter for detecting high frequencies.  It has a 3×3 kernel [-1, -1, -1, -1, 8, -1, -1, -1, -1]. For sharpening, the high frequency result from a laplacian filter is added to an identity.  Please consider reference text below for detail and mathematics.

Bokeh

kernel graph

At the center of the image, an identity kernel is applied.  As the convolution is applied on pixels away from center, a larger RECT function is utilized.  Illustrated in the above kernel graph on the right, black spot marks the identity kernel and lighter intensity mark larger kernels.  Source code is available on Github.

Future work

Currently working on code quality which includes ES6 modules to divide decoder from filter.  Also, large format camera, scheimpflug principle, is being implemented to simulate a plane shift image effect.

References

  1. Digital Image Processing by Rafael C. Gonzalez, Richard E. Woods, Addison Wesley Publishing, 1992.  ISBN 0-201-50803-6
  2. The file formats handbook by Gunter Born, International Thomson computer press, 1995.  ISBN 1-850-32128-0

MachWaves – iOS

Tuesday, January 3rd, 2017

This is a retro of MachWave mobile application written on June 26th, 2014.  The Objective-C, source code for iOS is now on Github.  However, the application is no longer available on iTune for download.  This camera application has a custom view that supports 2 lines being drawn via touch-points (below).screen-shot-2017-01-03-at-10-30-30-am

When there are four touch points, two lines are drawn.  Tt calculates the mach number base on the touch point coordinates and intersecting angle. The values are posted at the top of the photo (below).

screen-shot-2017-01-03-at-10-36-25-am

Link for detail and references.

 

Learning Hadoop basics

Monday, January 2nd, 2017

Greatly appreciate a lesson on Apache Hadoop this fall semester.  Over the winter break, I am posting one assigned problem to help cement my understanding.  This is a personal reference for moving forward to learn Apache Spark and Hadoop image processing interface (HIPI).

This hadoop exercise involves installing on Windows 10, adding JSON library, and performing a page rank.

Windows10 installation

At the time of implementation, I was not aware of Cloudera quickstart VM and Hortonworks’ sandboxes.  So, I embarked on Windows so it can be shared among team members of my class.

I am following ‘s article to install Hadoop 2.7.1 and Java 1.7 on Windows10 without cygwin.  Also, a self reminder to set Java path.  The final step of the article is to start the hadoop file system.

C:\Dev\hadoop-2.7.1\sbin>Start-all.cmd

localhost8088

localhost50070

Adding JSON library

To process JSON input, I need to include an external java library.  From this article on StackOverflowDoug Crockford references a Java JSON library authored by Sean Leary.  With my working directory at c:\Hwork, I place the java files in sub-directory c:\Hwork\org\json\.

json-src

Compiling the JSON library: javac org/json/*.javacompile-json

Build JSON jar file: jar -cvf json.jar org/json/*.class

jar-json

Page Rank

As a beginner, I am using the famous wordcount example in Hadoop documentation as a starting point.  My task is to sort and rank below text file (one example) with multiple lines of JSON.  

{“page”:”http://marys_flowers/main.html”, “list”:[{“rose”:”http://bobs-flowers/page1″},{“rose”:”http://bettys-flowers/rose_page.html”},{“lily”:”http://flowers-r-us/lily”},{“rose”:”http://bobs-flowers/page1″}]}
{“page”:”http://flowerpage/main.html”,”list”:[{“rose”:”http://bobs-flowers/page1″},{“mums”:”http://flowerpage/page2.html”}]}

From the wordcount example, I renamed it pagerank and include the JSON library and change methods’ signatures to handle Text for keys and values (download PageRank.java download PageRank.jar).

import-json

map-signatures

reducer-signatures

The input and output for Map() and Reduce() methods are key and Iteratable values of type Text and IntWriteable by default.  In above screenshots, I edit this to Text for both key and values.  They must be consistent across both methods.  I have to set the job output type in main (below).

setoutput

The map() code is straight forward.  I parse the JSONObject value list into JSONArray and then write each key and value for reduce() to rank.

map-parser

The reduce() code is also straight forward.  I aggregate the values for the same key.  Ranking is a numeric count of values per key (below).

reduce

The output result is as follow.  The second column of integers is the number of values associated with that row key.

1 1 lily http://flowers-r-us/lily
1 1 mums http://flowerpage/page2.html
1 4 rose http://bobs-flowers/page1,http://bobs-flowers/page1,http://bettys-flowers/rose_page.html,http://bobs-flowers/page1

Build + Execution

Compile:   C:\Hwork>javac -classpath c:\dev\hadoop-2.7.1\share\hadoop\common\hadoop-common-2.7.1.jar;c:\dev\hadoop-2.7.1\share\hadoop\mapreduce\hadoop-mapreduce-client-core-2.7.1.jar;c:\dev\hadoop-2.7.1\share\hadoop\common\lib\gson-2.2.4.jar;c:\dev\hadoop-2.7.1\share\hadoop\common\lib\commons-cli-1.2.jar PageRank.java
Build JAR:   C:\Hwork>jar -cvf PageRank.jar *.class

build

CREATE input directory on HDFS and copy input file there for execution.

C:\Dev\hadoop-2.7.1\sbin>hadoop fs -mkdir /in

C:\Dev\hadoop-2.7.1\sbin>hadoop fs -copyFromLocal c:\Hwork\input\input.txt /in

create_in

Make JSON library JAR file available by copying it into a directory that is one of hadoop class path.

C:\Dev\hadoop-2.7.1\sbin>hadoop classpath

classpath

copy JSON library JAR file into common directory;  PageRank code will be able to access the JSON library when executed.
json-dir

Execute PageRank:

C:\Dev\hadoop-2.7.1\sbin>hadoop jar c:\Hwork\PageRank.jar PageRank /in /out

execute

Retrieve output file from HDFS:

C:\Dev\hadoop-2.7.1\sbin>hadoop fs -copyToLocal /out/* c:\Hwork\output

retrieve

Reference: Hadoop, “The Definitive Guide”-4th Edition by Tom White, O’Reilly

ISBN:978-1-491-90163-2