Hand/Pen tracking Data Gathering

A first series of experiments with a small visualization of accelerometer and gyro data integrated over time to get approximate angle of the hand/pen.

The position of the pen is controlled by the X,Z coordinates of the stylus on the screen.

The angle is controlled by the accelerometer + gyro angles.

The Y position of the pen (UP) is controlled by the z(UP) Degrees/Sec of the accelerometer.

I’m using this basic document as a guide to filter and integrate the accel/gyro data:


The EMG data is also collected for the flexure of the hand.

The next step is to integrate more sensors into the experiment on the hand.  To explore the capabilities of the wacom (tilt and pressure and are able to be gathered from the tablet but it may be beyond my coding abilities, and more research needs to be done.  Possible sensors to explore are proximity sensing for distance to drawing surface, GSR of the hand, and possible relation to heartrate during the drawing process.

If I cannot get pressure data from the stylus directly it might be more fruitful to explore the hands grip on any pen in general using force sensors on the thumb and forefinger.

EMG data on the forearm might prove more reliable then on the hand as well.

While waiting for additional sensors, I will run a series of small experiments to make the data more manageable.  The experiments will be while drawing very specific shapes and in specific ways.

These will be compared to longer drawing sessions of exploratory sketching, technical drawing, and observational drawing.

This entry was posted in Realtime Interactive Visualization, Thesis. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s