Exercise: ReactJS + ES6 + SVG + cubic spline interpolation

November 25th, 2017

This is an upgrade of a previous cubic spline code-kata.  It is now written in ES6 class that is wrapped in ReactJS components to render SVG elements.  The previous version renders as pixels on a canvas. The original source is from Numeric Recipes in C.

For this demo, click anywhere on the SVG to append knot points to the curve.
Screen Shot 2017-11-26 at 5.16.37 PM

This project source is available on Github build with Visual Studio Code and npm.

Viewing auto-stereo images on VR headset

July 29th, 2017

Recently upgraded my google cardboard experience with a Utopia360 Virtual Reality 3D headset for $20 bucks from Microcenter.  Really impressed with WebVR, Android cardboard and daydream sdks.  While there is  lots of content in vextex rendering as well as 360 degree images, I am starting from the tradition auto-stereo images.  Thanks to professor Davidhazy from RIT for giving some that were originally photographed on sliver halide film.

In this exercise, I am loading two sets of images, Andy’s portrait and balloon explosion — highspeed.  The code is implemented in javascript, available on Github, StereoView project.  device orientation triggers next image set upload.  This code is tested on a Samsung Note II with Chrome browser.  Try it out here.
VR

Spatial convolution and bokeh

January 17th, 2017

On my path to gaining proficiency in ecmascript6, I am exercising with a windows bitmap decoder, spatial convolution and bokeh effect filter.

49

Windows Bitmap decoder

This decoder is capable only of loading 24bpp uncompressed RGB image from your local drive at current.  The source code is based on a comprehensive version written in actionscript back in August 1, 2009.  The Windows 3.x Bitmap format (BMP) is defined by four sections below.  Please consider reference text below for detail.

  1. BITMAP File-Header
  2. Bitmap Info (Bitmap Info-header)
  3. Bitmap palette (if exists)
  4. BITMAP-Picture data

Spatial Convolution

Filters, blur, laplacian and sharpen have been implemented for testing.  For blur, a RECT function of 5×5 kernel is set for default and configurable.  For laplacian, it is an isotropic derivative filter for detecting high frequencies.  It has a 3×3 kernel [-1, -1, -1, -1, 8, -1, -1, -1, -1]. For sharpening, the high frequency result from a laplacian filter is added to an identity.  Please consider reference text below for detail and mathematics.

Bokeh

kernel graph

At the center of the image, an identity kernel is applied.  As the convolution is applied on pixels away from center, a larger RECT function is utilized.  Illustrated in the above kernel graph on the right, black spot marks the identity kernel and lighter intensity mark larger kernels.  Source code is available on Github.

Future work

Currently working on code quality which includes ES6 modules to divide decoder from filter.  Also, large format camera, scheimpflug principle, is being implemented to simulate a plane shift image effect.

References

  1. Digital Image Processing by Rafael C. Gonzalez, Richard E. Woods, Addison Wesley Publishing, 1992.  ISBN 0-201-50803-6
  2. The file formats handbook by Gunter Born, International Thomson computer press, 1995.  ISBN 1-850-32128-0

MachWaves – iOS

January 3rd, 2017

This is a retro of MachWave mobile application written on June 26th, 2014.  The Objective-C, source code for iOS is now on Github.  However, the application is no longer available on iTune for download.  This camera application has a custom view that supports 2 lines being drawn via touch-points (below).screen-shot-2017-01-03-at-10-30-30-am

When there are four touch points, two lines are drawn.  Tt calculates the mach number base on the touch point coordinates and intersecting angle. The values are posted at the top of the photo (below).

screen-shot-2017-01-03-at-10-36-25-am

Link for detail and references.

 

Learning Hadoop basics

January 2nd, 2017

Greatly appreciate a lesson on Apache Hadoop this fall semester.  Over the winter break, I am posting one assigned problem to help cement my understanding.  This is a personal reference for moving forward to learn Apache Spark and Hadoop image processing interface (HIPI).

This hadoop exercise involves installing on Windows 10, adding JSON library, and performing a page rank.

Windows10 installation

At the time of implementation, I was not aware of Cloudera quickstart VM and Hortonworks’ sandboxes.  So, I embarked on Windows so it can be shared among team members of my class.

I am following ‘s article to install Hadoop 2.7.1 and Java 1.7 on Windows10 without cygwin.  Also, a self reminder to set Java path.  The final step of the article is to start the hadoop file system.

C:\Dev\hadoop-2.7.1\sbin>Start-all.cmd

localhost8088

localhost50070

Adding JSON library

To process JSON input, I need to include an external java library.  From this article on StackOverflowDoug Crockford references a Java JSON library authored by Sean Leary.  With my working directory at c:\Hwork, I place the java files in sub-directory c:\Hwork\org\json\.

json-src

Compiling the JSON library: javac org/json/*.javacompile-json

Build JSON jar file: jar -cvf json.jar org/json/*.class

jar-json

Page Rank

As a beginner, I am using the famous wordcount example in Hadoop documentation as a starting point.  My task is to sort and rank below text file (one example) with multiple lines of JSON.  

{“page”:”http://marys_flowers/main.html”, “list”:[{“rose”:”http://bobs-flowers/page1″},{“rose”:”http://bettys-flowers/rose_page.html”},{“lily”:”http://flowers-r-us/lily”},{“rose”:”http://bobs-flowers/page1″}]}
{“page”:”http://flowerpage/main.html”,”list”:[{“rose”:”http://bobs-flowers/page1″},{“mums”:”http://flowerpage/page2.html”}]}

From the wordcount example, I renamed it pagerank and include the JSON library and change methods’ signatures to handle Text for keys and values (download PageRank.java download PageRank.jar).

import-json

map-signatures

reducer-signatures

The input and output for Map() and Reduce() methods are key and Iteratable values of type Text and IntWriteable by default.  In above screenshots, I edit this to Text for both key and values.  They must be consistent across both methods.  I have to set the job output type in main (below).

setoutput

The map() code is straight forward.  I parse the JSONObject value list into JSONArray and then write each key and value for reduce() to rank.

map-parser

The reduce() code is also straight forward.  I aggregate the values for the same key.  Ranking is a numeric count of values per key (below).

reduce

The output result is as follow.  The second column of integers is the number of values associated with that row key.

1 1 lily http://flowers-r-us/lily
1 1 mums http://flowerpage/page2.html
1 4 rose http://bobs-flowers/page1,http://bobs-flowers/page1,http://bettys-flowers/rose_page.html,http://bobs-flowers/page1

Build + Execution

Compile:   C:\Hwork>javac -classpath c:\dev\hadoop-2.7.1\share\hadoop\common\hadoop-common-2.7.1.jar;c:\dev\hadoop-2.7.1\share\hadoop\mapreduce\hadoop-mapreduce-client-core-2.7.1.jar;c:\dev\hadoop-2.7.1\share\hadoop\common\lib\gson-2.2.4.jar;c:\dev\hadoop-2.7.1\share\hadoop\common\lib\commons-cli-1.2.jar PageRank.java
Build JAR:   C:\Hwork>jar -cvf PageRank.jar *.class

build

CREATE input directory on HDFS and copy input file there for execution.

C:\Dev\hadoop-2.7.1\sbin>hadoop fs -mkdir /in

C:\Dev\hadoop-2.7.1\sbin>hadoop fs -copyFromLocal c:\Hwork\input\input.txt /in

create_in

Make JSON library JAR file available by copying it into a directory that is one of hadoop class path.

C:\Dev\hadoop-2.7.1\sbin>hadoop classpath

classpath

copy JSON library JAR file into common directory;  PageRank code will be able to access the JSON library when executed.
json-dir

Execute PageRank:

C:\Dev\hadoop-2.7.1\sbin>hadoop jar c:\Hwork\PageRank.jar PageRank /in /out

execute

Retrieve output file from HDFS:

C:\Dev\hadoop-2.7.1\sbin>hadoop fs -copyToLocal /out/* c:\Hwork\output

retrieve

Reference: Hadoop, “The Definitive Guide”-4th Edition by Tom White, O’Reilly

ISBN:978-1-491-90163-2

 

MSSE spring semester count down

December 29th, 2016

screen-shot-2016-12-29-at-9-35-25-pm

Thanks to my MSSE colleague for providing the requirements.  Over the holidays, I am working on a spring semester countdown website.  Looks like Typescript ?  Feels like a little less.  ECMAScript 6 class leaves me a bit confused about the missing private variables that work so nicely with getter and setter.  OK, thanks for reminding me that javascript is different from OOP development in C++.  After the initial learning curve on prior ECMAScript, prototype seems to make sense with ‘this’ or ‘that’ for scope referencing.  The new ECMAScript 6 class seems to have lost some of that capability.

Thanks to Mike Bostock’s excellent D3.js calendar view.  I am using it to illustrate our final count down of spring semester.  Click anywhere on the calendar view to select the date for calculation.  The results are in percentage, minutes, hours and days.  Note, the selected date currently does NOT resolve the current hour and minutes.  The source code of the website is available on Github.

…. but who’s counting ?

 

WPF – STL ASCII Decoder

June 23rd, 2016

I am implementing the ascii decoder for an earlier exercise, STL binary decoder, 26Sept2012.  The STL ascii file specification I am using is available on wiki.  A few C# 6.0 feature enhancements have been added (null propagation, string interpolation, expression-bodied members).  Files tested are available in the assets directory under the project solution.  Next step would be to add color decoding, logging and test it more thoroughly with defective STL files.  Decoder needs to be more flexible, currently normal needs to be in the first line.

Screen Shot 2016-06-23 at 11.21.25 AM

Google Cardboard

May 8th, 2016

Few years late on this technology but I was still giddy assembling the Google Cardboard.  To my surprise, the price of this cardboard viewer varied from $5.00 – $20.00.  Then there were many other fancier headsets in the price of hundreds.

20160507_122632

I bought the generic brand of cardboard viewer.  It arrived with two magnets, lens and perforated lines, minus the 2D barcode.  There was very little assembly required.

20160507_123459

No instruction was necessary although I found a rubber band helpful to keep everything together.  I suppose a few stables would work as well but just in case I need to take it apart for storage.20160508_130639

Next came the Cardboard demo software on Google Play.  iPhone version was available as well.

20160508_131856

Before attempting any exciting development project(s), my daughter and I have already found hours of entertainment with the various demos provided.  My favorite was the northern lights under the lighthouse.  It reminded me of SplitRock lighthouse, near Duluth.

20160508_132013

Anyone has recommendation on purchasing a ‘real’ viewer ?  For development, is it better with iOS vs Android ?  Saw amazing 3D painting tools with Vocativ on facebook, so cool !!

Reading Google Cardboard and SDK documentation.

 

 

Mightex TCE1209-U new pictorial application

August 15th, 2015

As a next step from my last blog – utility app, challenge is to write a demo app replacement.

Thanks to Mightex’s software engineering, there already exists some sample code in C++, C#, VB for interfacing with driver and camera.  You can download them from Mightex TCE1209-U page.

Setup

My personal copy of Visual Studio is 2008 and it is able to load the solution CSharp_Application under the SDK directory.  The SDK is written in C++ so I am building this as an x86 app for interop.

Baseline

original

The sample application is threaded and already has code for interfacing with the driver.  User interface even has a button to start and stop frame grabs.  All I need to do are three tasks, display video rendering, save buffered image to file and offer a ‘run-on’ mode to save frames indefinitely.

Display Video

previewtabSmall

Sample code provided an unsafe pointer to each 16 bit pixel during after a frame-grab.  All I have to do is bitblit pixels to screen continuously.  For C#, I choose to use pictureBox as my video display.  I create 2 bitmap buffers to handle the alternating process of render and display.

// create 2 buffers for preview

bmps = new Bitmap[2];

bmps[0] = new Bitmap(picBox.Width, picBox.Height, PixelFormat.Format24bppRgb);
bmps[1] = new Bitmap(picBox.Width, picBox.Height, PixelFormat.Format24bppRgb);
rect = new Rectangle(1, 0, picBox.Width – 1, picBox.Height);

….

// copy previous buffer data to current except current frame.

g = Graphics.FromImage(bmps[notCurrent]);
g.DrawImage(bmps[current], 0, 0, rect, GraphicsUnit.Pixel);

….

// sample frame-grab pixels into new buffer
for (i = 0; i < frameSize; i++)
{
// bit shift – 12 bpp to 8bpp
byte p = (byte)((uint)*frameptr >> 4);

// preview – sample 1 out of 10 pixels
if (i % 10 == 0)
{
*Bmpptr = p; Bmpptr++;
*Bmpptr = p; Bmpptr++;
*Bmpptr = p; Bmpptr += (Bmpdata.Stride-2);
}

….

// copy new buffer into pictureBox

picBox.Image = (Image)bmps[current];

Sized at 500 x 205, it is scaling 10X smaller the specified 2048pixels/frame.  So, I am sampling 1 pixel out of 10 for video preview.  All previous frames (1 to n-1) are copied to (0,0) so as to provide a left moving video.

Buffered Image to file

bufferedtab

The main goal of this application is to provide Andy a way to specify image size and then save the image in a common format.  The Buffered tab offer a standard saveFileDialog1 to create file name and destination.  .NET Bitmap class saves the 8bpp image into gif, bmp, jpg, png, tif.

How large can our Bitmap be ?  Here is the Microsoft documentation on System.Drawing.Bitmap.  Constructor signature for width and height are Int32 which has a range of 2,147,483,647.  32bit pointer with 2048 image height leaves us 1048575 pixels for width (minus header and padding).  In my tests, writing a 100,000 x 2048 bitmap crashes at bitmap.save() regardless of destination format type.  The largest bitmap I am able to save is 80,000 x 2048 pixels as bmp.

largest

Inherently, the camera is 12bpp and produces 16bpp frames (4bits blank).  Logically I should be able to render and store png16 or bmp16 with indexed colors.  Also, another little quark about working with bitmap 8bpp index palette, one cannot directly set the palette values.  Instead, the following order of value assignments are required.  Read more from Charles Petzold’s article.

// set gray scale palette
ColorPalette pal = buffer.Palette;
for (int i = 0; i < 256; i++)
pal.Entries[i] = Color.FromArgb(255,i,i,i);
buffer.Palette = pal;

Run-On Mode

run-ontab

One advantage of a line scanning camera is that there is no specified image width.  One may wish to scan indefinitely or for a very long time.  So, it would be nice to use this ‘run-on’ mode to continuously scan and save each frame to a separate file.  For combining the frames, use the utility written for the last blog.  I will update it to handle png input next.  It should be noted that run-on mode is a secondary feature because of performance issue.  Writing image to file is a slow process while other frame-grabs get dropped; it takes about 500ms to iterate 1 frame save on my MacBookPro running Vista on VMWare.  We may revisit this feature later if it should be desirable.

To use this feature, you should use an empty directory.  If you prefer otherwise, the selected directory is scanned and displays an error popup if any file(s) with prefix of “frame” is founded.  Sorry I am not offering a choice in name or file format.  I am sticking with png files for now.

framesSmall

Test results

Here are some images I am creating at Lake Calhoun today; the original images are bundled in Github repository under directory exampleResults.

CalhounBeach

CalhounBeach2Small

 

Here is the Microsoft Visual Studio solution source code – CSharp_Application on Github with debug executable.

Conclusion 

It is a fun day making panoramic images at lake Calhoun.  I am learning that the software exposure value can speed up the frame rate and change the image brightness.  Unfortunately, the value of 1 locks up the application for unknown reason.

Demo

Here is a video demo.  thanks to my better-half for filming.

Next step

1) Update utility to handle png inputs with prefix name “frame”.

2) Debug above mention error with exposure value of “1”. problem in driver, block user from setting value of 1 for now.

3) Debug and implement the ability to use 8bpp gray scale bitmap if possible. done !

4) Sent it back to Andy for feedback !

 References

1) Setting Bitmap Palette – Charles Petzold  http://www.charlespetzold.com/pwcs/PaletteChange.html

 

Application use direction

Preview

previewDirection

  1. plug in camera.
  2. execute application.
  3. select ‘camera1’ in left-upper combo box.
  4. start preview by clicking upper-right-button, ‘start preview’.

Buffered image mode

bufferedDocumentation

  1. to save scans by buffering, click on the ‘buffered tab’ (bottom).
  2. enter the numeric line count, (image width).
  3. select or enter destination file name path.
  4. check all the file types that apply (bmp, png, tiff, gif, etc).
  5. if not already selected, select radio button (upper-right) for “buffered image”.
  6. click on button “save” to start recording.
  7. a popup dialog will display when successfully scanned and saved to file.

Run-on mode

  1. to save scans by ‘run-on’, click on the ‘run-on tab’ (bottom).
  2. select a destination directory (preferably empty).
  3. select radio button (upper-right) for “run-on”.
  4. click on button “save’ to start recording.
  5. click on button ‘stop’ (previously ‘save’) or ‘stop preview’ to exit run-on mode.

Mightex TCE1209-U demo app utility

August 9th, 2015

Greatly appreciate my professor, Andy Davidhazy for loaning his line scan camera for an ‘enrichment’ project.

The challenge:  figure out why (possibly fix) the camera-demo app is producing images with poor tonal quality.

Given: Mightext TCE1209-U camera with demo application as well as sample project source in C++, C#, VB.

Setup

After much anticipation, camera arrives with 50mm lens adaptor already assembled.  No power supply needed, USB will do!  Software installation includes a device driver for Windows 7 and demo app (available from website).  The demo application display image result similar to an oscilloscope, one line per time interval.  Output options are Windows bitmap or raw ASCII files.

camera

 

 

 

 

 

 

 

 

MIGHTEX-IMG_4851

 

 

 

 

 

 

 

 

With Windows bitmap option known to be a problem, I am diving in to work with the ASCII output option.  Here is one of my ‘better’ result from the first try.  Here, I am swiping the camera view across my kitchen.  You can see a blur version of my daughter in silhouette.

Image2Small

 

 

 

 

 

 

 

 

 

 

 

For a better test, I printed the following test page with grayscale (short of buying a Kodak IT-8 target).

targetSmall

 

 

 

 

 

 

 

 

The ascii output files each contain 2048 lines (camera pixel width) .  Each pixel has a depth from 0 – 4098 (12 bits per pixel) and is represented in a line delimited by a carriage return.  Since displays and common pictorial file formats are in 8bits per pixel, I am writing the utility to requantize 12bpp ASCII into 8bpp binary image.  With .NET library, System.Drawing.Bitmap class offer all of the features I need to manipulate and save the files into bmp, gif, tif, png, jpg and others.  For better visual result, I am applying histogram equalization to minimize the bit depth compression (12bpp to 8bpp).  Here is my result.

ImageSmall

 

 

 

 

 

 

 

The utility app has three tab pages, Source, Image processing and Storage.  Source code in C#, Visual Studio 2008 solution is available on GitHub.  Included in the solution is a debug executable, test source files and results.

Source (ASCII) – user selects the source directory with ASCII files.  Internally, DirectoryInfo retrieves all the FileSystemInfo and sort them by creation time.

Screen Shot 2015-08-09 at 1.21.24 PM

 

 

 

 

 

 

 

Image processing – Internally, it loads all the source files to find image black point, white point, mode and number of shades.  It also builds a histogram and look-up-table for 12bpp->8bpp conversion.  User may write the histogram to file (ASCII) for more detail analysis.  Also, user may override the histogram-equalization-look-up-table by changing the white point and black point.

Screen Shot 2015-08-09 at 1.21.15 PM

 

 

 

 

 

 

 

Storage – Internally, it loads all the source files, apply histogram-equalization (above mentioned) to assemble an 8bit per pixel bitmap which is then saved as bmp, gif, tif, png or jpg.

Screen Shot 2015-08-09 at 1.21.42 PM

 

 

 

 

 

 

 

Conclusion / Next step:

Probably should invest more time investigating the tone reproduction issue.  Instead, a quick utility is devised and moving onward to the next solution by writing a replacement demo app.  Most of the features in this utility will be ported to the next.

Future enhancements are as follow:

1) an auto-gamma correction

2) a display of the cumulative histogram.

3) FFT filter to remove sine wave if the vertical lines in the image are not artifacts by yours truly.

4) save image into png16.

5) high dynamic range feature to blend the additional 4 bits of data (alternative bit depth conversion 12bpp -> 8bpp).

6) write a demo-app replacement that display live image and output better pictorial binary image files.

References:

1) Peripheral photography article by Andrew Davidhazy.

2) Digital Image Processing by Gonzalez, Woods.  – class text – Digital Image processing I with Dr. Rao at Rochester Institute of Technology.

3) My viewer exercise project in C#, Visual Studio 2005

4) Mightex TCE1209-U camera documentation.

5) Photographics Materials & Processes by Dr. Strobel – class text – M&P with Dr. Strobel, Jack Holm, Russ Kraus at Rochester Institute of Technology.