Viewing auto-stereo images on VR headset

July 29th, 2017

Recently upgraded my google cardboard experience with a Utopia360 Virtual Reality 3D headset for $20 bucks from Microcenter.  Really impressed with WebVR, Android cardboard and daydream sdks.  While there is  lots of content in vextex rendering as well as 360 degree images, I am starting from the tradition auto-stereo images.  Thanks to professor Davidhazy from RIT for giving some that were originally photographed on sliver halide film.

In this exercise, I am loading two sets of images, Andy’s portrait and balloon explosion — highspeed.  The code is implemented in javascript, available on Github, StereoView project.  device orientation triggers next image set upload.  This code is tested on a Samsung Note II with Chrome browser.  Try it out here.

Spatial convolution and bokeh

January 17th, 2017

On my path to gaining proficiency in ecmascript6, I am exercising with a windows bitmap decoder, spatial convolution and bokeh effect filter.


Windows Bitmap decoder

This decoder is capable only of loading 24bpp uncompressed RGB image from your local drive at current.  The source code is based on a comprehensive version written in actionscript back in August 1, 2009.  The Windows 3.x Bitmap format (BMP) is defined by four sections below.  Please consider reference text below for detail.

  1. BITMAP File-Header
  2. Bitmap Info (Bitmap Info-header)
  3. Bitmap palette (if exists)
  4. BITMAP-Picture data

Spatial Convolution

Filters, blur, laplacian and sharpen have been implemented for testing.  For blur, a RECT function of 5×5 kernel is set for default and configurable.  For laplacian, it is an isotropic derivative filter for detecting high frequencies.  It has a 3×3 kernel [-1, -1, -1, -1, 8, -1, -1, -1, -1]. For sharpening, the high frequency result from a laplacian filter is added to an identity.  Please consider reference text below for detail and mathematics.


kernel graph

At the center of the image, an identity kernel is applied.  As the convolution is applied on pixels away from center, a larger RECT function is utilized.  Illustrated in the above kernel graph on the right, black spot marks the identity kernel and lighter intensity mark larger kernels.  Source code is available on Github.

Future work

Currently working on code quality which includes ES6 modules to divide decoder from filter.  Also, large format camera, scheimpflug principle, is being implemented to simulate a plane shift image effect.


  1. Digital Image Processing by Rafael C. Gonzalez, Richard E. Woods, Addison Wesley Publishing, 1992.  ISBN 0-201-50803-6
  2. The file formats handbook by Gunter Born, International Thomson computer press, 1995.  ISBN 1-850-32128-0

MachWaves – iOS

January 3rd, 2017

This is a retro of MachWave mobile application written on June 26th, 2014.  The Objective-C, source code for iOS is now on Github.  However, the application is no longer available on iTune for download.  This camera application has a custom view that supports 2 lines being drawn via touch-points (below).screen-shot-2017-01-03-at-10-30-30-am

When there are four touch points, two lines are drawn.  Tt calculates the mach number base on the touch point coordinates and intersecting angle. The values are posted at the top of the photo (below).


Link for detail and references.


Learning Hadoop basics

January 2nd, 2017

Greatly appreciate a lesson on Apache Hadoop this fall semester.  Over the winter break, I am posting one assigned problem to help cement my understanding.  This is a personal reference for moving forward to learn Apache Spark and Hadoop image processing interface (HIPI).

This hadoop exercise involves installing on Windows 10, adding JSON library, and performing a page rank.

Windows10 installation

At the time of implementation, I was not aware of Cloudera quickstart VM and Hortonworks’ sandboxes.  So, I embarked on Windows so it can be shared among team members of my class.

I am following ‘s article to install Hadoop 2.7.1 and Java 1.7 on Windows10 without cygwin.  Also, a self reminder to set Java path.  The final step of the article is to start the hadoop file system.




Adding JSON library

To process JSON input, I need to include an external java library.  From this article on StackOverflowDoug Crockford references a Java JSON library authored by Sean Leary.  With my working directory at c:\Hwork, I place the java files in sub-directory c:\Hwork\org\json\.


Compiling the JSON library: javac org/json/*.javacompile-json

Build JSON jar file: jar -cvf json.jar org/json/*.class


Page Rank

As a beginner, I am using the famous wordcount example in Hadoop documentation as a starting point.  My task is to sort and rank below text file (one example) with multiple lines of JSON.  

{“page”:”http://marys_flowers/main.html”, “list”:[{“rose”:”http://bobs-flowers/page1″},{“rose”:”http://bettys-flowers/rose_page.html”},{“lily”:”http://flowers-r-us/lily”},{“rose”:”http://bobs-flowers/page1″}]}

From the wordcount example, I renamed it pagerank and include the JSON library and change methods’ signatures to handle Text for keys and values (download download PageRank.jar).




The input and output for Map() and Reduce() methods are key and Iteratable values of type Text and IntWriteable by default.  In above screenshots, I edit this to Text for both key and values.  They must be consistent across both methods.  I have to set the job output type in main (below).


The map() code is straight forward.  I parse the JSONObject value list into JSONArray and then write each key and value for reduce() to rank.


The reduce() code is also straight forward.  I aggregate the values for the same key.  Ranking is a numeric count of values per key (below).


The output result is as follow.  The second column of integers is the number of values associated with that row key.

1 1 lily http://flowers-r-us/lily
1 1 mums http://flowerpage/page2.html
1 4 rose http://bobs-flowers/page1,http://bobs-flowers/page1,http://bettys-flowers/rose_page.html,http://bobs-flowers/page1

Build + Execution

Compile:   C:\Hwork>javac -classpath c:\dev\hadoop-2.7.1\share\hadoop\common\hadoop-common-2.7.1.jar;c:\dev\hadoop-2.7.1\share\hadoop\mapreduce\hadoop-mapreduce-client-core-2.7.1.jar;c:\dev\hadoop-2.7.1\share\hadoop\common\lib\gson-2.2.4.jar;c:\dev\hadoop-2.7.1\share\hadoop\common\lib\commons-cli-1.2.jar
Build JAR:   C:\Hwork>jar -cvf PageRank.jar *.class


CREATE input directory on HDFS and copy input file there for execution.

C:\Dev\hadoop-2.7.1\sbin>hadoop fs -mkdir /in

C:\Dev\hadoop-2.7.1\sbin>hadoop fs -copyFromLocal c:\Hwork\input\input.txt /in


Make JSON library JAR file available by copying it into a directory that is one of hadoop class path.

C:\Dev\hadoop-2.7.1\sbin>hadoop classpath


copy JSON library JAR file into common directory;  PageRank code will be able to access the JSON library when executed.

Execute PageRank:

C:\Dev\hadoop-2.7.1\sbin>hadoop jar c:\Hwork\PageRank.jar PageRank /in /out


Retrieve output file from HDFS:

C:\Dev\hadoop-2.7.1\sbin>hadoop fs -copyToLocal /out/* c:\Hwork\output


Reference: Hadoop, “The Definitive Guide”-4th Edition by Tom White, O’Reilly



MSSE spring semester count down

December 29th, 2016


Thanks to my MSSE colleague for providing the requirements.  Over the holidays, I am working on a spring semester countdown website.  Looks like Typescript ?  Feels like a little less.  ECMAScript 6 class leaves me a bit confused about the missing private variables that work so nicely with getter and setter.  OK, thanks for reminding me that javascript is different from OOP development in C++.  After the initial learning curve on prior ECMAScript, prototype seems to make sense with ‘this’ or ‘that’ for scope referencing.  The new ECMAScript 6 class seems to have lost some of that capability.

Thanks to Mike Bostock’s excellent D3.js calendar view.  I am using it to illustrate our final count down of spring semester.  Click anywhere on the calendar view to select the date for calculation.  The results are in percentage, minutes, hours and days.  Note, the selected date currently does NOT resolve the current hour and minutes.  The source code of the website is available on Github.

…. but who’s counting ?



June 23rd, 2016

I am implementing the ascii decoder for an earlier exercise, STL binary decoder, 26Sept2012.  The STL ascii file specification I am using is available on wiki.  A few C# 6.0 feature enhancements have been added (null propagation, string interpolation, expression-bodied members).  Files tested are available in the assets directory under the project solution.  Next step would be to add color decoding, logging and test it more thoroughly with defective STL files.  Decoder needs to be more flexible, currently normal needs to be in the first line.

Screen Shot 2016-06-23 at 11.21.25 AM

Google Cardboard

May 8th, 2016

Few years late on this technology but I was still giddy assembling the Google Cardboard.  To my surprise, the price of this cardboard viewer varied from $5.00 – $20.00.  Then there were many other fancier headsets in the price of hundreds.


I bought the generic brand of cardboard viewer.  It arrived with two magnets, lens and perforated lines, minus the 2D barcode.  There was very little assembly required.


No instruction was necessary although I found a rubber band helpful to keep everything together.  I suppose a few stables would work as well but just in case I need to take it apart for storage.20160508_130639

Next came the Cardboard demo software on Google Play.  iPhone version was available as well.


Before attempting any exciting development project(s), my daughter and I have already found hours of entertainment with the various demos provided.  My favorite was the northern lights under the lighthouse.  It reminded me of SplitRock lighthouse, near Duluth.


Anyone has recommendation on purchasing a ‘real’ viewer ?  For development, is it better with iOS vs Android ?  Saw amazing 3D painting tools with Vocativ on facebook, so cool !!

Reading Google Cardboard and SDK documentation.



Mightex TCE1209-U new pictorial application

August 15th, 2015

As a next step from my last blog – utility app, challenge is to write a demo app replacement.

Thanks to Mightex’s software engineering, there already exists some sample code in C++, C#, VB for interfacing with driver and camera.  You can download them from Mightex TCE1209-U page.


My personal copy of Visual Studio is 2008 and it is able to load the solution CSharp_Application under the SDK directory.  The SDK is written in C++ so I am building this as an x86 app for interop.



The sample application is threaded and already has code for interfacing with the driver.  User interface even has a button to start and stop frame grabs.  All I need to do are three tasks, display video rendering, save buffered image to file and offer a ‘run-on’ mode to save frames indefinitely.

Display Video


Sample code provided an unsafe pointer to each 16 bit pixel during after a frame-grab.  All I have to do is bitblit pixels to screen continuously.  For C#, I choose to use pictureBox as my video display.  I create 2 bitmap buffers to handle the alternating process of render and display.

// create 2 buffers for preview

bmps = new Bitmap[2];

bmps[0] = new Bitmap(picBox.Width, picBox.Height, PixelFormat.Format24bppRgb);
bmps[1] = new Bitmap(picBox.Width, picBox.Height, PixelFormat.Format24bppRgb);
rect = new Rectangle(1, 0, picBox.Width – 1, picBox.Height);


// copy previous buffer data to current except current frame.

g = Graphics.FromImage(bmps[notCurrent]);
g.DrawImage(bmps[current], 0, 0, rect, GraphicsUnit.Pixel);


// sample frame-grab pixels into new buffer
for (i = 0; i < frameSize; i++)
// bit shift – 12 bpp to 8bpp
byte p = (byte)((uint)*frameptr >> 4);

// preview – sample 1 out of 10 pixels
if (i % 10 == 0)
*Bmpptr = p; Bmpptr++;
*Bmpptr = p; Bmpptr++;
*Bmpptr = p; Bmpptr += (Bmpdata.Stride-2);


// copy new buffer into pictureBox

picBox.Image = (Image)bmps[current];

Sized at 500 x 205, it is scaling 10X smaller the specified 2048pixels/frame.  So, I am sampling 1 pixel out of 10 for video preview.  All previous frames (1 to n-1) are copied to (0,0) so as to provide a left moving video.

Buffered Image to file


The main goal of this application is to provide Andy a way to specify image size and then save the image in a common format.  The Buffered tab offer a standard saveFileDialog1 to create file name and destination.  .NET Bitmap class saves the 8bpp image into gif, bmp, jpg, png, tif.

How large can our Bitmap be ?  Here is the Microsoft documentation on System.Drawing.Bitmap.  Constructor signature for width and height are Int32 which has a range of 2,147,483,647.  32bit pointer with 2048 image height leaves us 1048575 pixels for width (minus header and padding).  In my tests, writing a 100,000 x 2048 bitmap crashes at regardless of destination format type.  The largest bitmap I am able to save is 80,000 x 2048 pixels as bmp.


Inherently, the camera is 12bpp and produces 16bpp frames (4bits blank).  Logically I should be able to render and store png16 or bmp16 with indexed colors.  Also, another little quark about working with bitmap 8bpp index palette, one cannot directly set the palette values.  Instead, the following order of value assignments are required.  Read more from Charles Petzold’s article.

// set gray scale palette
ColorPalette pal = buffer.Palette;
for (int i = 0; i < 256; i++)
pal.Entries[i] = Color.FromArgb(255,i,i,i);
buffer.Palette = pal;

Run-On Mode


One advantage of a line scanning camera is that there is no specified image width.  One may wish to scan indefinitely or for a very long time.  So, it would be nice to use this ‘run-on’ mode to continuously scan and save each frame to a separate file.  For combining the frames, use the utility written for the last blog.  I will update it to handle png input next.  It should be noted that run-on mode is a secondary feature because of performance issue.  Writing image to file is a slow process while other frame-grabs get dropped; it takes about 500ms to iterate 1 frame save on my MacBookPro running Vista on VMWare.  We may revisit this feature later if it should be desirable.

To use this feature, you should use an empty directory.  If you prefer otherwise, the selected directory is scanned and displays an error popup if any file(s) with prefix of “frame” is founded.  Sorry I am not offering a choice in name or file format.  I am sticking with png files for now.


Test results

Here are some images I am creating at Lake Calhoun today; the original images are bundled in Github repository under directory exampleResults.




Here is the Microsoft Visual Studio solution source code – CSharp_Application on Github with debug executable.


It is a fun day making panoramic images at lake Calhoun.  I am learning that the software exposure value can speed up the frame rate and change the image brightness.  Unfortunately, the value of 1 locks up the application for unknown reason.


Here is a video demo.  thanks to my better-half for filming.

Next step

1) Update utility to handle png inputs with prefix name “frame”.

2) Debug above mention error with exposure value of “1”. problem in driver, block user from setting value of 1 for now.

3) Debug and implement the ability to use 8bpp gray scale bitmap if possible. done !

4) Sent it back to Andy for feedback !


1) Setting Bitmap Palette – Charles Petzold


Application use direction



  1. plug in camera.
  2. execute application.
  3. select ‘camera1’ in left-upper combo box.
  4. start preview by clicking upper-right-button, ‘start preview’.

Buffered image mode


  1. to save scans by buffering, click on the ‘buffered tab’ (bottom).
  2. enter the numeric line count, (image width).
  3. select or enter destination file name path.
  4. check all the file types that apply (bmp, png, tiff, gif, etc).
  5. if not already selected, select radio button (upper-right) for “buffered image”.
  6. click on button “save” to start recording.
  7. a popup dialog will display when successfully scanned and saved to file.

Run-on mode

  1. to save scans by ‘run-on’, click on the ‘run-on tab’ (bottom).
  2. select a destination directory (preferably empty).
  3. select radio button (upper-right) for “run-on”.
  4. click on button “save’ to start recording.
  5. click on button ‘stop’ (previously ‘save’) or ‘stop preview’ to exit run-on mode.

Mightex TCE1209-U demo app utility

August 9th, 2015

Greatly appreciate my professor, Andy Davidhazy for loaning his line scan camera for an ‘enrichment’ project.

The challenge:  figure out why (possibly fix) the camera-demo app is producing images with poor tonal quality.

Given: Mightext TCE1209-U camera with demo application as well as sample project source in C++, C#, VB.


After much anticipation, camera arrives with 50mm lens adaptor already assembled.  No power supply needed, USB will do!  Software installation includes a device driver for Windows 7 and demo app (available from website).  The demo application display image result similar to an oscilloscope, one line per time interval.  Output options are Windows bitmap or raw ASCII files.



















With Windows bitmap option known to be a problem, I am diving in to work with the ASCII output option.  Here is one of my ‘better’ result from the first try.  Here, I am swiping the camera view across my kitchen.  You can see a blur version of my daughter in silhouette.













For a better test, I printed the following test page with grayscale (short of buying a Kodak IT-8 target).










The ascii output files each contain 2048 lines (camera pixel width) .  Each pixel has a depth from 0 – 4098 (12 bits per pixel) and is represented in a line delimited by a carriage return.  Since displays and common pictorial file formats are in 8bits per pixel, I am writing the utility to requantize 12bpp ASCII into 8bpp binary image.  With .NET library, System.Drawing.Bitmap class offer all of the features I need to manipulate and save the files into bmp, gif, tif, png, jpg and others.  For better visual result, I am applying histogram equalization to minimize the bit depth compression (12bpp to 8bpp).  Here is my result.









The utility app has three tab pages, Source, Image processing and Storage.  Source code in C#, Visual Studio 2008 solution is available on GitHub.  Included in the solution is a debug executable, test source files and results.

Source (ASCII) – user selects the source directory with ASCII files.  Internally, DirectoryInfo retrieves all the FileSystemInfo and sort them by creation time.

Screen Shot 2015-08-09 at 1.21.24 PM








Image processing – Internally, it loads all the source files to find image black point, white point, mode and number of shades.  It also builds a histogram and look-up-table for 12bpp->8bpp conversion.  User may write the histogram to file (ASCII) for more detail analysis.  Also, user may override the histogram-equalization-look-up-table by changing the white point and black point.

Screen Shot 2015-08-09 at 1.21.15 PM








Storage – Internally, it loads all the source files, apply histogram-equalization (above mentioned) to assemble an 8bit per pixel bitmap which is then saved as bmp, gif, tif, png or jpg.

Screen Shot 2015-08-09 at 1.21.42 PM








Conclusion / Next step:

Probably should invest more time investigating the tone reproduction issue.  Instead, a quick utility is devised and moving onward to the next solution by writing a replacement demo app.  Most of the features in this utility will be ported to the next.

Future enhancements are as follow:

1) an auto-gamma correction

2) a display of the cumulative histogram.

3) FFT filter to remove sine wave if the vertical lines in the image are not artifacts by yours truly.

4) save image into png16.

5) high dynamic range feature to blend the additional 4 bits of data (alternative bit depth conversion 12bpp -> 8bpp).

6) write a demo-app replacement that display live image and output better pictorial binary image files.


1) Peripheral photography article by Andrew Davidhazy.

2) Digital Image Processing by Gonzalez, Woods.  – class text – Digital Image processing I with Dr. Rao at Rochester Institute of Technology.

3) My viewer exercise project in C#, Visual Studio 2005

4) Mightex TCE1209-U camera documentation.

5) Photographics Materials & Processes by Dr. Strobel – class text – M&P with Dr. Strobel, Jack Holm, Russ Kraus at Rochester Institute of Technology.







Android Dictation app – Google SpeechRecognizer

August 31st, 2014

Found this great article titled “Develoing Android* Application with Voice Recognition Features”, by Stanislav of Intel.  Included is a SpeechRecognitionHelper class which instantiate and invoke Google SpeechRecongizer service.  Unlike iOS OpenEar library, the free Google SpeechRecognizer service requires internet access but is impressively faster in performance and accuracy.  Apparently offline processing is possible for some devices with Jellybean and later.  Goto Settings->Language and input->Voice search->Offline speech recognition and install your language packet.  Inside the Dictation project Manifest file, comment out this line for internet permission.

<uses-permission android:name=”android.permission.INTERNET” />

Screenshot_2014-09-01-08-35-38 Screenshot_2014-09-01-08-34-50

For features and UI, there are four buttons, Dictate, Last, Clear and Save.  Dictate will invoke the Google SpeechRecognizer service and render/append the interpreted text on to the page.  I am assuming each dictation is a complete sentence and insert a period after.  The Last button will remove your last dictation instance.  The Clear button will remove all dictations.  The save button will write/append all text on screen to file along with a time stamp.  Additional features such as deletion is probably a good idea but not implemented.  Below are some screenshots of the app in operation.

Screenshot_2014-09-01-10-17-59 Screenshot_2014-08-31-19-35-52 Screenshot_2014-08-31-19-37-14 Screenshot_2014-08-31-19-47-20 Screenshot_2014-08-31-19-47-41

Complete Dictation project source maybe found on Github.

Installation of the Dictation app is available on GooglePlay under the yeuchi collection.