Optical Flow is a technique for tracking flow of image objects in the scene. The output of Optical Flow is a series of flow 2D vectors which in turn is called as the Flow Field. It has numerous advantages and has been extensively used in self-driving cars, autonomous robots, and in assistive devices for the visually impaired.
The key concept behind Optical Flow is to calculate the intensity displacement through time and space. That said, optical flow can be thought as a function of tracking brightness across time and space. For example, given a point p0at time t0 and its intensity as i0,
i0 = I(p0, t0)
where I is the intensity function of a given point at a given time. Then let’s assume the intensity is i1 for the point p1 at time t1 during which a motion has occurred that made the original point now locate into p1. Now,
i1 = I(p1, t1)
According to Optical Flow, it is considered that,
i0 = i1
In other words it assumes that image brightness (intensity) is independent from camera motion.
Having followed a tutorial in OpenCV, I managed to put together my own Python code for a simple demonstration of Optical Flow using the Lucas Kanade algorithm .
Color tracking is fairly a simple activity in OpenCV and you may find a variety of articles in the Internet which describe different mechanisms and implementations. However the common steps involve the following.
- Convert the source frame into HSV color space.
- Define the upper bounds and lower bounds for the color you intend to track. In my example I mind the blue color and so the lower bound and upper bounds are respectively (100, 150, 0) and (140,255, 255) where the lower bound being the darker blue.
- Find the image mask that contain pixels which fall within the lower and upper bounds as mentioned above.
- Find contours in the mask image.
- Find the maximum contour in terms of the contour area. So that we can omit trivial contours and focus only on the most dominating blue color object in the scene.
- Approximate the maximum contour into a polygon.
- Find the bounding rectangle of the above polygon.
- Draw the rectangle from above step on the source image.
The code that follows the above series of steps is provided below.
#This is an exercise to track blue color from camera.
import numpy as np
cap = cv2.VideoCapture(0)
ret, source = cap.read()
print('Error in video capture')
hsv = cv2.cvtColor(source, cv2.COLOR_BGR2HSV)
lowerblue = np.array([100,150,0])
upperblue = np.array([140,255,255])
mask = cv2.inRange(hsv, lowerblue, upperblue)
#find contours in the mask
im, contours, hierarchy = cv2.findContours(mask, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
maxContour = max(contours, key=cv2.contourArea)
epsilon = 0.1*cv2.arcLength(maxContour, True)
approxpoly = cv2.approxPolyDP(maxContour, epsilon, True)
x,y,w,h = cv2.boundingRect(approxpoly)
cv2.rectangle(source,(x,y),(x+w, y+h),(0,255,0), 2)
if (cv2.waitKey(1) & 0xFF == 113):
The demonstration is shown below.
I love books that add value to our lives and provide us with lot of inspiration. This particular book titled
“How to think like Steve Jobs” happened to grab my attention out of nowhere when I was browsing through the personal library of my wife’s father. It’s true that there are numerous books written by many, under the label of Steve Jobs, and I believe they all share the wisdom of Steve Jobs in overall, no matter how different their stories are.
In the course of reading this book “How to think like Steve Jobs“, I was able to harness an array of material on this man, once known as the hardest-working man in Silicone Valley, and how he transformed the computer and music industries. He was known to lead Apple from its humble beginnings, starting it off in his parents’ garage to the global empire it is today. So the big question for everybody, (especially after his death) was How did he do it? In this book, the author tries to answer this question by drawing out key elements on Job’s life and work, and about his vision towards technology. In fact, the book suggests that everything Jobs produced (Mac, iPod, iPhone, iPad) has the personality, charisma and his style built-in to them. So it’ll be worthy to study his life rather than his products so we could add some value to our lives and view the world from the eyes of a visionary.
In this post I intend highlight some of the key elements from this book, that took my attention, as in below. Continue reading “How to think like Steve Jobs..”
Dr. Abdul Kalam is perhaps India’s most admired technocrat who once led the country’s space rocketry and missile programme into monumental heights. I recently came across his autobiography ‘Wings of Fire‘ which is a fascinating account of his life and work and a story full of inspiration. In spite of being brought up in a rural village and a son of an ordinary boat-owner, his ceaseless courage was ultimately well paid off at the end by becoming the most distinguished person in India – the country’s 11th President. But yet, what captivated me most is his humble and unassuming nature and the way he maintained it throughout his life. Following are some of the useful traits that I could pick up from his deeply passionate personal story, which I believe will enlighten our lives too.
Continue reading “Abdul Kalam: A Valuable Role Model”
It is perhaps surprising to realize that only two things in this world have troubled man’s ingenuity for centuries, i.e. space and time. These two are absolute benchmarks, often used when making a reference to a physcial object in space or when describing a past incident, though it is hard to understand why do we always attribute our actions or events in relation to space and time. For instance, a special event (a birthday perhaps) can be expressed in relation to time by marking that occurence on a calendar either digitally or manually. We are capable of doing this since ’time’ as we know of, is one dimensional. It is somewhat puzzling at this point, should we deal with space in the same manner, because space is three dimensional and it provides freedom for travelling in multiple directions, as opposed to the single dimensional nature of time. These implications led our curiosity to focus on one implicit feature, yet something strange about space – ”How can we mark space?”. I shall later describe the background for arriving at this notion. For the moment let us accept this question and describe its logic by an analogy with our understanding of space.
From a biological point of view, human beings tend to use physical objects for designating places of interest that often help them representing space and constructing three-dimensional cognitive maps [Egerton, 2005]. The mammalian spatial referencing patterns, as described by Egerton  organise physical objects in the form of a trail, for tracing out specific points in their respective environments. Imagine you were exploring an unknown and complex environment and wanted to find your wayback after an exploration. One solution would be to mark your trail with pebbles. The pebbles would persist and you could readily trace-back your path in return, unless an ill-tempered being removes all the pebbles from your sight after you placed them. Extending this concept, imagine we could mark out any point in space, with pebbles that remain persistent over time. In such a way, we could pin-point an arbitrary location – even a point somewherein front of our eyes – freely in any perspective while tracing out complex paths in all 3 dimensions of space. Extending the idea further, if the pebbles could convey information then they could be used to pass messages or communicate information to other travelers. Further still, if pebbles could express relationships with their neighbors, complex process models could be expressed. Continue reading “AR and its Role in Marking Space”
Through my previous post, I highlighted the Augmented Reality’s (AR) potential to function as a novel paradigm in Human-Robot-Interactions (HRI). Marker-less AR seems more plausible for this work, as it can readily mark points in space, without demanding a prior knowledge of the environment. In other words, we can just look at any random environment and mark any point in that environment real time. By placing a virtual marker, we already saw a demonstration on how can we persistently mark space, so that virtual markers remained persistent under changing perspectives of the camera, often like as if they were real.
Now we will continue from that point onwards, and see how can we apply such an AR-based spatial marker platform into HRI. In this article we make a case study in which we assimilate Augmented Reality into robot navigation. Virtual markers are overlaid on the video feed captured by a camera which in turn is mounted on top of the robot. We mark a point in space, just by placing a virtual AR marker, so then the robot automatically navigates to the location we pointed. My hypothesis here is to prove that just by pointing somewhere in space, we could readily perform HRI tasks – especially navigation. But before moving into application specific details let us dive into some background about HRI and marking space.
Continue reading “Spatial Human Robot Interaction Marker Platform (SHRIMP)”
It’s been a while since I last blogged as I kept receiving piles and piles of work. I do realize now, that research is not a leisurely activity although it may seem like, for a person who is looking from outside. Apart from usual activities of experiments & literature surveys, doing tutorials, marking assignments, writing conference papers, and making presentations are enough to crush a PhD student to his limits. Nevertheless all these implications help in testifying one’s potential for research and the love for science. So I thought of putting all that matters aside and write a post for the sake of contributing to the scientific knowledge. In particular I’d like to share some of the work I did in my research while taking a short break off from my studies.
Marker-less Augmented Reality has been my primary source of curiosity from the day I started my PhD journey. With my research I am exploring the ways in which we can apply AR into HRI (Human-Robot Interactions) and further improve the collaborative patterns between the man and the robot. Consequently I came up with a state-of-the-art interface based on a well- known marker-less AR platform named PTAMM (Parallel Tracking and Multiple Mapping). The interface that I brought up has the capability for marking an arbitrary point in space persistently with a virtual object (AR object). What does the term persistence mean? Suppose you have an AR object that can be clearly seen through your camera. Now you change the camera perspective, move the camera to a different location and return to your AR object from a different direction. At this instance you must still see the AR object persistently anchored at its original location. The idea may look simple but systems with such a functionality are still rare, even at the existence of powerful AR frameworks (i.e. PTAM, PTAMM). This is what I describe as Persistent Augmented Reality. My AR interface provides the capability to appear an AR object persistent over time and space, no matter in which direction you change the camera. See the video below. But how does it work? What are the concepts? are the questions that you might wonder at this point.
Continue reading “Parallel Tracking And Global Mapping”