Which path to choose ?

Alrighhtt, I’m back to blogging after a break of 3+ months, I don’t know if people still follow my blog or no, but the last three months has been very eventful in my life, a series of highs and lows, but all is well now :). In this blogpost I will try to sum up in brief whats been going on lately and what are my plans with this blog in the future.

In January 2010, I was in a serious emotional turmoil on what path to choose professionally, I basically had three options

  1. Keep working on PyMT till I find a proper sponsor, this was easiest way to follow as the team was great, I had a pretty good hold on the python langauge and PyMT framework. But I was worried that I was hitting a technical roadblock as I knew only python properly , whereas the world of Multitouch has varied interest and I wanted to explore these other fields too.
  2. I had an offer from NUITeq. The worries that I had were that the platform was completely new, Purely C++ based. Even the team was new, I didn’t know if I would fit well.  Plus I would have to work remotely due to the visa issues. Other then these it was all good.
  3. Quit Multitouch and join a generic XYZ software company in India.

After several weeks of stressful thinking I decided to vote for my technical progress over ease of job and i decided to join NUITeq. I decided I can face all challenges it posed, plus the job offer from NUITeq was very good.

Just when I was all set to begin my new career, I met with an Motorcycle accident. Aftermath: A cracked wrist, surgery necessary 😛 . So I had to undergo a surgery and quit my job plans for next 6 weeks. I gained a metal plate in my hand to hold the bones in place but, lost 6 weeks of precious time 😐 . I’m very grateful to the guys at NUITeq for understanding my situation and deferring the plans till I recover.

My wrist after surgery :P

My wrist after surgery 😛

Once I joined NUITeq, its all been good for me, I’m really liking the C++ SDK, C++ language in general is awesome. The team is really good, my colleagues help me in every other question or doubt I have regarding the framework. I’m really glad I took the decision of joining NUITeq. Just can’t wait to meet these guys in person sometime in the future :). Its been three months now that I’ve been working with NUITeq, time sure flies when we work on something we really love.

My contract got extended with NUITeq, so I will continue to work with them. Soo now the question is what about this blog. I’ve had several ideas, either I will continue blogging about multitouch coding  in this same blog, or I’m hoping to start another blog focusing only on multitouch coding, the mathematics involved, and other tricks which I’m hoping to share with the community. I was hoping to dwell into Android hacking too and maybe port NUIPaint into c++ using some base framework, he he lots and lots of plans :). So all that remains is to make up sometime for this in the weekends :). Lets see how it goes.

Credits: Crossroads photo by DJ Bass @

PS: NUITEQ got nominated for Norrmejerier-stipendiet 2010. Please support by voting at . It just takes a few seconds for voting process 🙂


Pixel Shader Magic

One thing which really fascinated me from really long is use of Pixel shaders to do interesting things in OpenGL. I’ve been trying to decipher the mystery of GLSL from a long time now, this interest in GPU based computation was sparked especially from Thomas’s Example apps in the PyMT trunk( Mandelbrot and Shadowing in Pictures example), although I never understood how it works back then (or might be because I never put much effort in understanding it).

GL Shading Language is a extension to OpenGL using which we can directly compute pixel and graphics on the GPU instead of the CPU. This provides us with tremendous performance benifits since pixel transformations run in parallel. The scope of Pixel shaders is amazing, some of us sometimes dont know how they achieve those stunning effects in latest computer games. Most of the effects like

  • Motion blur
Motion Blur

Motion Blur

  • HDR Lighting

HDR Lighting

HDR Lighting

are all done utilizing the pixel shader capabilities of your graphics card. So I started out by searching some  tutorials on the net and there are several good references that i found out.

GLSL is all about pixel manipulation, usually involving usage of neighboring pixel colors, distance etc etc. Its mathematics. But there are a lot of free GLSL code available online using which you can learn and implement your won shaders.

Once I learnt the basics of GLSL, I decided to put effort to further learn this technology and implement it in NUIPaint as filters. I found out that some common filters like Gaussian Blur, Sharpening, Embossing etc belong to a common group of shaders called as Convolutions Filters. These vary only by a single matrix called as Convolution Matrix. Its very interesting, for more details please check this link .

I implemented a separate class called Filters which will later be added into PyMT core. This class has several functions which take in texture as input, apply GLSL filters to it and return the transformed textures back. In total i implement 7 kinds of filters

  • Gaussian Blur
  • Sharpness
  • Saturation
  • Brightness
  • Contrast
  • Black and White
  • Sepia

Here is a set of screenshots of the Filtered images.

first four

other filters

It works much faster than regular CPU based filters like the ones in Python Imaging Library. If you want to use the glsl filters then here is the link to the source code, each filters vertex and fragment shaders are in separate functions, so you can easily use it in your programs. If you want to use the Filter class into your software then wait for the induction of the class into PyMT core soon 🙂

A Color Picker for PyMT

It’s been long since I last posted about my GSOC work, so today I’ll be writing about how I achieved the color picker. There are several points that comes to the mind when we thing about the colorpicker.

  • How to achieve the gradient color selection area
    • Use a pre-rendered image ?
    • Generate using computer algorithm ? If what are the appropriate opengl calls?
  • What is the algorithm for picking the color from the location ?
    • Use opengl methods to pick the pixel from the point where the user touches ?
    • Or use some algorithmic technique to determine where the user is touching local to the gradient and then calculate the appropriate RGB values

There is a humongous list of resources available for Color pickers especially AJAX based, but my problem was different, I wanted a Quarter Circle Color Picker 🙂 , nowhere in the internet did i find one single open-source quarter circle color pickers, but hey no pain no gain :), so I set out to create a Opengl rendered quarter circle color picker with algorithmic based color picking.

1. Rendering the Gradient Circle

So my first goal was to create a opengl rendered quarter circle, well its very easy to generate a single color circles in opengl like the ones that I have used in my previous widgets, Rendering a gradient one is a very hard task. I know that opengl allows to blend colors by specifying different colors at different vertices of a polygon, but how can one do the same in a circle ?!!, circle has no vertex edges.

A Color Blended Triangle

A Color Blended Triangle


I did not find any way to get around this problem using Circle render, so the alternative solution was to render thin triangles and arrange it in the form of a fan, and when i looked around the net for this one, to my surprise there was one opening call to make such a polygon using triangle fan.

def drawPartialCircle(pos=(0,0), radius=100):
    with gx_begin(GL_TRIANGLE_FAN):
        for angle in range (90,185,5):

So in the above algorithm i’ve chosen the angle range from 90 to 185 because i want it to appear at the bottom right corner. So what the algoritm does it,

  • It generates 5 degree triangles to form a quater circle.
  • The center of the circle is blue so the other two edges of the two radii will be red and green respectively.
  • The mid way of the circumference must be yellow that is RGB =>(1,1,0) , so to we know that only at 45 degrees sin 45 = cos 45.
  • sin 45 = 1/sqrt(2), so to obtain 1 at the center we multiply it by sqrt(2), so we get (1,1,0) at the center.
Opengl rendered Quarter Circle

Opengl rendered Quarter Circle

If you have any queries regarding this algo, feel free to comment.

2. Calculating color at the touch point

I used the similar sine cos formula to calculate the RGB values, The R and G values varies according to the angle made by the line joining the center and the touch point with the horizontal line. Whereas the blue value is a function of the distance touch point from the center. So here goes the algorithm

def calculate_color(self):
        b = 1-self.point_distance/self.size[0]
        r = (sin(radians(self.point_angle))-b)*sqrt(2)
        g = (cos(radians(self.point_angle))-b)*sqrt(2)

where self.size[0] is the radius of the circle.

3. Hue and Saturation

As you can see in the rendered quarter circle above there is no range of black nor white, so this posed a new challenge to use a slider to make a hue saturation variation, HSV[Hue, Saturation, Value]  is an alernative representation of RGB color space, something similar to Rectangular to Polar coordinates conversion.

HSV Representation

HSV Representation

Since I need to vary the color from White>>Color>>Black, the variation has to be done in two steps

  • White >> Color variation, which is nothing but variation of the Saturation, here saturation = 0 implies White
  • Color >> Black which is variation of the Value field

Thomas (My mentor) pointed me to this really nifty function in core python rgb_to_hsv and hsv_to_rgb , which converts between the colorspaces with ease. So I divide the slider into two portions lower limit to middle value for saturation, middle value to upper limit for Value. I chose to take 2 as the maximum range of the slider, so it varies from 0 to 2, and middle i get 1. I developed the following algorithm.

value = rgb_to_hsv(self.slider.slider_color[0],self.slider.slider_color[1],self.slider.slider_color[2])
h,s,v = value[0],value[1],value[2]
if self.slider._value <= 1.0: s = self.slider._value else: v = 2-self.slider._value self.slider.slider_color = hsv_to_rgb(h,s,v) [/sourcecode] Overall this entire widget was very complex to build, but I got to learn alot from this tiny project. If you have queries regarding any of the algorithm that I developed, or if you have suggestions of better algorithm, or anything at all feel free to comment :), it will definitely help me make this project better

Python Bi(y)te

Well i finished my 7th sem exams, finished playing crisis core, both went out good, especially the Game crisis core, amazing ending very emotional.. anyways after finishing these two tasks i set out to learn python. To start with my multitouch application development and to get a hold of opengl.

My brother suggested a book written by and Indian author, Swaroop C H. This book by the name “A Byte of Python“, is a excellent book for Beginners as well as intermediate programmers and the best part is its free, you can also buy the printed version and support the writer. Its really well written, right on point with no gribbrish examples. So if you are looking for a python tutorial look no further, just start with this book.

Ok that’s all i have to write for now, i will write about my progress with opengl and python in my next post 🙂