Augmented Reality on Rotating Objects

ImageDisplacements captures the optical flow field in a video sequence in real time. Via the curl , you can detect a rotating hand movement in front of your built-in camera. Thus, you can increment or decrement the time of a virtual clock in an augmented reality example.

The curl of a vector field determines the inherent amount of rotation.

In[1]:=

listCurl = flow \[Function] ListConvolve[{{0.5`, 0, -0.5`}}, flow[[All, All, 2]], {{1, 2}, {1, 2}}, "Fixed"] - ListConvolve[{{-0.5`}, {0.`}, {0.5`}}, flow[[All, All, 1]], {{1, 2}, {1, 2}}, "Fixed"];

In[2]:=

(* Convert the incoming video frames to gray scale and swap left and \ right to mimic a mirror image. *) preProcessFrame = img \[Function] ImageReflect[ColorConvert[img, "Grayscale"], Left -> Right];

The entire code of this augmented reality example is less than one page.

In[3]:=

DynamicModule[ {frames, flow, curl, mask, time = 0, W = 320, H = 240, w = 240, h = 160, r = 64}, (* specify a sensitive image region *) mask = ImageCompose[ ConstantImage[0, {W, H}], ImageAdjust[Image[GaussianMatrix[r/2]]], {w, h} ]; (* initialize the optical flow acquisition *) frames = Table[preProcessFrame@CurrentImage[], {2}]; flow = First@ImageDisplacements[frames]; Graphics[ {Inset[ (* read and display camera image, determine the curl *) Dynamic[ frames[[1]] = frames[[2]]; frames[[2]] = preProcessFrame@CurrentImage[]; flow = First@ImageDisplacements[frames, 0.7 flow]; curl = Image@listCurl@flow; time -= 2 ImageMeasurements[curl, "Mean", Masking -> mask]; frames[[2]] ], {0, 0}, {0, 0}, {W, H} ], Inset[ (* display clock *) Dynamic[ ClockGauge[ DatePlus[Date[], {Round[5 time, 5], "Minute"}], GaugeStyle -> RGBColor[1, 0.75, 0], LabelStyle -> White, GaugeFaceStyle -> Opacity[0.1] ], UpdateInterval -> 1 ], {w, h}, {0, 0}, {2 r, 2 r} ] }, PlotRange -> {{0, W}, {0, H}}, ImageSize -> {W, H} ] ]

Play Animation

Stop Animation