Gesture Mouse – Controlling Computer With a Glove

Hello there! Gesture Mouse – Controlling Computer With a Glove.

This instructable is on making a gesture-controlled mouse in the form of a glove.

The initial idea is to integrate this in FPS (First Person Shooter) games and computer pointer control, but it can also be used in other projects like a remote controlled car, mechanical hand, etc.

Hope you like the project. Leave your thoughts and suggestions in the comment section.

Thank you!

 

All of the above listed items can be bought by one’s own choice, albeit keep care on buying the glove.

Step 1: Know How the Sensors Work

Step 3: Readying the Glove

The Glove

The two vital factors of buying the proper glove are as follows:

• The glove should be of the perfect size, i.e., should perfectly fit the hand of the user.
• It should be flexible enough to allow free movement of the fingers.

Using a cloth gardening glove with spaces between the threads would be best for tying the components. Making holes in the leather or any other glove may prove difficult and couldn’t be reused.

Placing the Components

The Flex Sensors

In order to bring the “trigger-pulling” feeling, the flex sensors would be placed on the fingers.

It is one’s choice if they want to put the sensors on inside of the fingers or the outside. It is adviced to put the sensors on the outside of the hand to maximise the movement of the fingers.

To tie these, we’ll put 3-4 cable ties on equal distances through the spaces and tighten them around the sensors by looping it from inside the glove skin and zip it from outside like done in pics 2 and 3.

Ensure keeping the fingers free from ties in the middle of the finger, otherwise the ties would prickle the finger when it is fold.

The MPU-6050

The sensor is placed between the index and middle finger, at the bottom of the flex sensors.

It is apparent that in order to make the glove move the pointer by (in a way) pointing it towards the target area, the sensor would be placed on the edge of one the fingers.

Unfortunately, that is not possible due to the flex sensors. That’s because in order to click buttons, the finger would have to be moved, displacing the sensor, which would in turn displace the pointer.

Therefore, the sensor is placed below the fingers to lessen the pointer displacement but also keeping the “gun effect” to a certain level.

Preferably, the sensor’s y-arrow should point towards the computer screen perpendicularly to produce the effect more efficiently.

For fixing this sensor, the ties would be put through the pcb holes and through the glove layer and tightened up.

The Arduino

The board could be put straight or flipped on the wrist depending on the way its pins have been soldered.

Keep in mind that the pins need to face up to connect wires easily and the micro USB also needs to be clear of the glove for easy USB connection.

For tying, a single long tie could be used to fasten it there diagonally along the Arduino.

Stick a piece of double sided tape before tying.

Your glove should look something like Pic 1 by now.

Step 4: Making the Circuit

After fixing all the components in their places, now’s the time for connecting everything.

MPU-6050’s Connections:

VCC to VCC

GND to GND (any)

SCL to 3

SDA to 2

Use 4 female to female jumpers for connection.

Flex Sensors’ Connections

Left Jumper to GND

Right Simple Jumper to A0 (index sensor) and A1 (middle sensor)

Right +ve Jumper to RAW and VCC

Ensure to tighten the female connectors for better connection by pushing in their metal strips visible from the little hole on them.

Step 5: Programming the Arduino

Before we do any programing, we’ll need to install Arduino IDE onto our computer along with MPU-6050 operating library.

Download the IDE from here : Software | Arduino

And the MPU-6050.h library from here : ElectronicCats/mpu6050: MPU6050 Arduino Library (github.com)

Install the IDE and open the .ino code file in the editor. Include the .zip file from the GitHub repositery into the code.

Afterwards, now lets understand the working of vital portions of the code.

The code starts off by including the following libraries.

• Wire.h : This library allows the microcontroller to communicate with I2C devices. Here, I2c device is referred to MPU-6050 module, which communicates with the SCL and SDA pins.
• I2Cdev.h : This library is used for reading and writing I2C functions.
• MPU-6050.h : Controller library for MPU-6050 module.
• Mouse.h : HID Library for mouse functions.

The Wire.h, I2Cdev.h and Mouse.h generally come pre-installed with the IDE. If not, download and include these libraries from GitHub.

// Include libraries
#include <Wire.h>
#include <I2Cdev.h>
#include <MPU6050.h>
#include <Mouse.h>

setup() initializes I2C communication, MPU-6050 module connection, HID mouse functionality and serial communication.

void setup()
{
  Wire.begin();
  mpu.initialize();
  Mouse.begin();
  Serial.begin(9600); // Begin serial communication for reading sensor readings
}

loop() executes continuously the whole functionality.

void loop()
{
 moveMouse();
 clickMouse();
 printValues();
 delay(rate);
}

moveMouse() gets rotational values from the gyro sensor and feeds them into the variable assigned.

Then, it converts the output values into pointer displacement values in the form of x and y displacement coordinates.

In the end, it calls the Mouse.move() function, which moves according to the displacement values calculated.

void moveMouse()
{
  if (flagIndex == 0 && flagMiddle == 0)
  {
    mpu.getRotation(&gx, &gy, &gz); // Get rotation values from gyroscope

    // Convert output values to pointer displacement
    vx = -(gx + 200) / sense; // Here, change the '200' & '30' values to the approximate +ve values
    vy = (gz + 30) / sense;   // of the values shown by the gyroscope when in rest on the serial monitor.

    Mouse.move(vx, vy); // Move mouse according to gyro rotation
  }
}

clickMouse() uses values from the flex sensors to detect change in bend and take actions defined in the clickIndex() and clickMiddle() functions. Both functions are assigned to the respective flex sensor.

void clickMouse()
{
  // Read flex sensor values
  valIndex = analogRead(indexPin);
  valMiddle = analogRead(middlePin);

  // Perform mouse functions
  clickIndex();
  clickMiddle();
}

clickIndex() makes sure of left mouse clicks based on state of the flex sensors. clickMiddle() also works in a similar fashion. The only difference is of mouse button.

void clickIndex()
{
  if (flagIndex == 0)
  {
    if (valIndex > buffIndex)
    {
      flagIndex = 1;
      Mouse.press(MOUSE_LEFT);
      Mouse.release(MOUSE_LEFT);
    }
  }
  else if (flagIndex == 1)
  {
    if (valIndex < buffIndex)
    {
      flagIndex = 0;
    }
  }
}

After understanding the whole code, now just plug the arduino in, compile the code and upload it.

Open the serial monitor afterwards for viewing the sensor outputs.