Minor Project Final Report

1
GESTICULATION
Report submitted in partial fulfillment
Of the requirement for the degree of
B.Tech.
In
Information Technology Engineering
Under the Supervision of
Ms. Parul Yadav
By
Yogesh Madaan (0411503112)
Kanika Agarwal (05811503112)
To
Bharati Vidyapeeth’s College of Engineering
Paschim Vihar East, New Delhi
Of
Guru Gobind Singh Indraprastha University
Dwarka, New Delhi
Year 2015-2016

2
DECLARATION
This is to certify that Report entitled Gesticulation, which is submitted by us in partial
fulfillment of the requirement for the award of degree B.Tech. in Bharati Vidyapeeth’s
College of Engineering, Paschim Vihar East, New Delhi, of GGSIP University,
Dwarka, New Delhi, comprises only our original work and due acknowledgement has
been made in the text to all other material used.
DATE: NAME OF STUDENTS:
Yogesh Madaan (0411503112)
Kanika Agarwal (05811503112)
APPROVED BY Prof. (Dr.) Vanita Jain
(Head of Department, Information Technology
Bharati Vidyapeeth’s College of Engineering)

3
CERTIFICATE
This is to certify that Report entitled Gesticulation, which is submitted, by Yogesh
Madaan and Kanika Agarwal in partial fulfillment of the requirement for the award
of degree B.Tech. in Information Technology Engineering to BVCOE, Paschim Vihar
East, New Delhi, of GGSIP University, Dwarka, New Delhi is a record of the
candidates’ own work carried out by them under my supervision. The matter embodied
in this report is original and has not been submitted for the award of any other degree.
DATE: SUPERVISOR:
Ms. Parul Yadav

4
ABSTRACT
The Internet of Things (IoT) is – at its most profound levels – about creating digital
representations of real-world objects. It is a phenomenon that draws on rapid
developments within IT to spark insights and to help companies create entirely new
types of services and business areas. Today Devices are becoming more capable and
more broadly utilized, and when they are connected to the internet, they are integrated
into vast numbers of different applications across sectors. Hence, IoT is evolving. [1]
In this project, an effort has been made to implement the above technology that is IoT,
using a hardware device called Myo Gesture Control Armband. This hardware device,
developed by Thalmic Labs, senses the motion of an arm of a human being and acts
accordingly. This project has three modules- The Windows System Module, The
Android OS Module and The Home Automation. These modules involve the integration
of the hardware device with an existing application on a windows system, a special
application development on an android OS supported mobile device, and the operation
of basic home appliances with the hardware device respectively.

5
TABLE OF CONTENTS
Title Page………………………………………………………………........................i
Declaration.....................................................................................................................ii
Certificate………………………… ……………………………………………..…...iii
Abstract……………………………………………………………………..………...iv
Table of Contents…………………………………………………..………………….v
List of Figures……………………………………………………..…………………vii
1. Introduction………………………....……………………………………..………8
1.1. Overview……………………………………………………………………...8
1.2. Existing System……. …………………………………………......................8
1.3. Scope of Proposed Project…………………………………………………....9
2. Pre-Requisite Technologies………………………………………………………10
2.1. Internet of Things……………………………………………………………10
2.2. Myo Gesture Control Armband……………………………………………..12
2.3. Bluetooth Low Energy………………………………………………………12
2.3.1. BLE Platform Support ……………………………………………….13
2.3.2. Technical Details……………………………………………………..14
2.4. Arduino……………………………………………………………………...15
2.4.1. Technical Details…………………………………………………….16
3. Myo Armband………………………………………………………………....…17
3.1. Design……………………………………………………………………….18
3.2. Setup and Calibration……………………………………………………….19
3.3. Performance and use………………………………………………………..20
3.4. Compatibility………………………………………………………………..21
3.5. Features……………………………………………………………………...21
3.6. Highs and Lows……………………………………………………………..22
4. Gesticulation…………………………………………………………………..…23
4.1. Idea………………………………………………………………………….23
4.2. Project Phases…………………………………………………………….…23
4.2.1. Phase 1……………………………………………………………….23
4.2.2. Phase 2……………………………………………………………….24
4.2.3. Phase 3……………………………………………………………….24

6
5. Selfie Camera and Gallery Viewer.……………………………………………..25
6. Smart Assistant……………………………………………………………….…27
7. Home Automation (Prototype)………………………………………………….28
8. Summary………………………………………………………………………..30
9. Conclusion……………………………………………………………………....31
10. Future Scope…………………………………………………………………….32
11. References………………………………………………………………………33
12. Appendix………………………………………………………………………..34

7
LIST OF FIGURES
Fig.1 Type of Gestures of Myo…………………………..............…………………17
Fig.2 Hardware parts of Myo……………………………………………………….19
Fig.3 Flow diagram for windows’ camera application……………………………...26
Fig.4 Flow diagram for android utility application…………………………………27
Fig.5 Flow diagram for Home Automation………………………………………...29

8
INTRODUCTION
The objective of our project is to use technology for a social change. We have
tried to design a platform and set of utility tools that can be used by people with
visual impairment to help them in day to day life with a help of just smartphones
and reduce their effort. So with the vocabulary meaning of gesticulation task
performance can be accomplished with just a set of gestures. With the initial idea
of using a device to create interfaces for youngsters to control their smart devices
from a distance without physical contact with the device, we are now building
an aid to serve as a partner for blind and physically handicapped persons.
1.1 Overview
Today, there are the inventors who work on various hardware devices with some
other hardware devices to make a new one. There are coders, programmers,
software developers, designers who are bringing new innovative ideas and
revolutionizing the world of web. There are brilliant hardware devices, amazing
software; the need of hour is to bring together the two and kick-start a new era
of IoT that is Internet of Things.
An effort has to be made to implement the above technology that is IoT. A hardware
device called MYO, developed by Thalmic Labs, can be used for the same. This device
senses the motion of an arm of a human being and acts accordingly. This feature of
MYO can be used in integration with various software platforms to develop an
innovation. These platforms may include windows, android OS and even other
hardware technology like Arduino.
1.2 Existing System
For the integration of MYO with the windows system, The Thalmic Labs has
provided the whole MYO market where the user can use any utility application
connector he/she wants to operate with the gesture of a hand. For example,
PowerPoint connector is used to switch between the slides of a presentation with
their hand gesture. The user has to download the connector from the MYO market
and add it in the application manager of for MYO, that is the MYO Connect. The

9
MYO connect is the windows application that acts as an interface between the
connector and the other windows application. The Thalmic Labs also provide the
facility for the developers. People can code their own application connector and
use for their application using the MYO Connect.
Just like for windows, The Thalmic Labs also provides application for MYO in
android for the mobile devices. Some of these applications are MYO music and
MYO dialer. It also provides developer options for android. The Myo gesture
control armband is never integrated with Arduino.
1.3 Scope of Proposed Project
In the integration of Myo Gesture control Armband with the windows system a
pre-existing application, camera is used. The Myo is integrated with the camera
so that the user do not have to touch the system to click his photograph, make a
video or view the gallery. All will be done by gesturing the arm in which Myo is
worn.
For android, a special general utility mobile application is developed in which
the user can use his selfie stick to click his photograph without setting any timer.
Also, it has a feature by which the user can feel all the notification of his phone
on his arm and can act accordingly.
An effort is made to generate a basic prototype for such integration through an
android mobile application. This has been done to solve the purpose of home
automation. Through the gesture of a hand having Myo, The electric appliances
like, a tube light, a fan, etc. of a room can be controlled.

10
PRE-REQUISITETECHNOLOGIES
2.1 Internet of Things – IoT
The Internet of Things (IoT) is the network of physical objects or "things" embedded
with electronics, software, sensors, and network connectivity, which enables these
objects to collect and exchange data. The Internet of Things allows objects to be sensed
and controlled remotely across existing network infrastructure, creating opportunities
for more direct integration between the physical world and computer-based systems,
and resulting in improved efficiency, accuracy and economic benefit. Each thing is
uniquely identifiable through its embedded computing system but is able to interoperate
within the existing Internet infrastructure.
IoT offers advanced connectivity of devices, systems, and services that goes beyond
machine-to-machine communications (M2M) and covers a variety of protocols,
domains, and applications. The interconnection of these embedded devices (including
smart objects), is expected to usher in automation in nearly all fields, while also
enabling advanced applications like a Smart Grid, and expanding to the areas such as
smart cities.
"Things," in the IoT sense, can refer to a wide variety of devices such as heart
monitoring implants, biochip transponders on farm animals, electric clams in coastal
waters, automobiles with built-in sensors, or field operation devices that assist
firefighters in search and rescue operations. These devices collect useful data with the
help of various existing technologies and then autonomously flow the data between
other devices. Current market examples include smart thermostat systems and
washer/dryers that use Wi-Fi for remote monitoring.
Besides the plethora of new application areas for Internet connected automation to
expand into, IoT is also expected to generate large amounts of data from diverse
locations that is aggregated very quickly, thereby increasing the need to better index,
store and process such data.
Survey conducted by Mckinsey & Company in June 2015 To get a broader view of the
IoT’s potential benefits estimates that the IoT has a total potential economic impact of
$3.9 trillion to $11.1 trillion a year by 2025. At the top end, that level of value—

11
including the consumer surplus—would be equivalent to about 11 percent of the world
economy (exhibit).[1]
Achieving this kind of impact would require certain conditions to be in place, notably
overcoming the technical, organizational, and regulatory hurdles. In particular,
companies that use IoT technology will play a critical role in developing the right
systems and processes to maximize its value. Among our findings:
 Interoperability between IoT systems is critical. Of the total potential economic
value the IoT enables, interoperability is required for 40 percent on average and
for nearly 60 percent in some settings.
 Currently, most IoT data are not used. For example, on an oilrig that has 30,000
sensors, only 1 percent of the data are examined. That is because this
information is used mostly to detect and control anomalies—not for
optimization and prediction, which provide the greatest value.
 Business-to-business applications will probably capture more value—nearly 70
percent of it—than consumer uses, although consumer applications, such as
fitness monitors and self-driving cars, attract the most attention and can create
significant value, too.
 The IoT has a large potential in developing economies. Still, we estimate that it
will have a higher overall value impact in advanced economies because of the
higher value per use. However, developing economies could generate nearly 40
percent of the IoT’s value, and nearly half in some settings.
 Customers will capture most of the benefits. We estimate that IoT users
(businesses, other organizations, and consumers) could capture 90 percent of
the value that IoT applications generate. For example, in 2025 remote
monitoring could create as much as $1.1 trillion a year in value by improving
the health of chronic-disease patients.

12
 A dynamic industry is evolving around IoT technology. As in other technology
waves, both incumbents and new players have opportunities. Digitization blurs
the lines between technology companies and other types of businesses; makers
of industrial machinery, for example, are creating new business models by using
IoT links and data to offer their products as a service.[1]
2.2 Myo Gesture Control Armband
Myo Gesture Control Armband, launched in June 2013, is a wearable technology
developed by a Canada based start-up named Thalmic Labs. Using a technique known
as electromyography, this little armband is able to read electrical signals from the
muscles in the forearm, map them to gestures made with the hand, and control other
devices with those gestures. The Myo armband gives a touch-free control of technology
with hand gestures and motion. It aims to give a motion control without the need for a
Wii remote, a presentation controller, or anything else that has to be held in hand.
Instead, the armband just has to be slipped like a blood pressure sleeve (or a Nintendo
Power Glove) and things can be controlled with the movements of fingers and wrist.
Most gesture-control systems, like Microsoft’s Kinect, still detect movements with
cameras, which can be thrown off by poor lighting conditions, distance, and simple
obstructions. By drawing gesture information directly from the arm muscles instead of
a camera, Myo circumvents all these problems — and works with devices that do not
have a camera in the first place. It detects five distinct hand gestures to wirelessly
control tech, like music, games, and presentation slides, all with the flick of a wrist.
Myo wraps it in eight different blocks, each of which contains a medical-grade EMG
sensor. The armband also uses a three-axis gyroscope, three-axis accelerometer, and
three-axis magnetometer to sense motion in any direction.
2.3 Bluetooth Low Energy
Bluetooth Low Energy (BLE), sometimes referred to as "Bluetooth Smart", is a light-
weight subset of classic Bluetooth and was introduced as part of the Bluetooth 4.0 core
specification. While there is some overlap with classic Bluetooth, BLE actually has a

13
completely different lineage and was started by Nokia as an in-house project called
'Wibree' before being adopted by the Bluetooth SIG.
There are plenty of wireless protocols out there for engineers and product designers,
but what makes BLE so interesting is that it's almost certainly the easiest way to design
something that can talk to any modern mobile platform out there (iOS, Android,
Windows phones, etc.), and particularly in the case of Apple devices it's the only HW
design option that doesn't require you to jump through endless hoops to be able to
legally market your product for iOS devices.
This guide will give you a quick overview of BLE, specifically how data is organized
in Bluetooth Low Energy, and how devices advertise their presence so that you can
connect to them and start passing data back and forth.
2.3.1 BLE Platform Support
Support for Bluetooth 4.0 and Bluetooth Low Energy (which is a subset of BT 4.0) is
available on most major platforms as of the versions listed below:
 iOS5+ (iOS7+ preferred)
 Android 4.3+ (numerous bug fixes in 4.4+)
 Apple OS X 10.6+
 Windows 8 (XP, Vista and 7 only support Bluetooth 2.1)
 GNU/Linux Vanilla BlueZ 4.93+
Bluetooth Smart extends the use of Bluetooth wireless technology to devices that are
powered by small, coin-cell batteries such as watches and toys. Other devices such as
sports & fitness, health care, keyboards and mice, beacons, wearables and entertainment
devices are enhanced by this version of the technology. In many cases, it makes it
possible to operate these devices for more than a year without recharging.
As with previous versions of the specification, the range of the radio may be optimized
according to application. The majority of Bluetooth devices on the market today include
the basic 30 foot, or 10 meter, range of the Classic Bluetooth radio, but there is no limit
imposed by the Specification. With Bluetooth Smart, manufacturers may choose to

14
optimize range to 200 feet and beyond, particularly for in-home sensor applications
where longer range is a necessity.
Bluetooth Smart features provides:
 Ultra-low peak, average and idle mode power consumption
 Ability to run for years on standard coin-cell batteries
 Lower implementation costs
 Multi-vendor interoperability
 Enhanced range
This enhancement to the Bluetooth Core Specification allows two types of
implementation, dual-mode and single-mode. In a dual-mode implementation,
Bluetooth low energy functionality is integrated into an existing Classic Bluetooth
controller. The resulting architecture shares much of Classic Bluetooth technology's
existing radio and functionality resulting in a minimal cost increase compared to Classic
Bluetooth technology. Additionally, manufacturers can use current Classic Bluetooth
technology (Bluetooth v2.1 + EDR or Bluetooth v3.0 + HS) chips with the new low
energy stack, enhancing the development of Classic Bluetooth enabled devices with
new capabilities.
Single-mode chips, which will enable highly integrated and compact devices, will
feature a lightweight Link Layer providing ultra-low power idle mode operation, simple
device discovery, and reliable point-to-multipoint data transfer with advanced power-
save and secure encrypted connections at the lowest possible cost. The Link Layer in
these controllers will enable Internet connected sensors to schedule Bluetooth low
energy traffic between Bluetooth transmissions.
2.3.2 Technical Details
 Data Transfers – Bluetooth Smart (low energy) supports very short data packets
(8 octet minimum up to 27 octets maximum) that are transferred at 1 Mbps. All
connections use advanced sniff-sub rating to achieve ultra low duty cycles
 Frequency Hopping – Bluetooth Smart (low energy) uses the adaptive
frequency hopping common to all versions of Bluetooth technology to minimize

15
interference from other technologies in the 2.4 GHz ISM Band. Efficient multi-
path benefits increase the link budgets and range
 Host Control – Bluetooth Smart (low energy) places a significant amount of
intelligence in the controller, which allows the host to sleep for longer periods
of time and be woken up by the controller only when the host needs to perform
some action. This allows for the greatest current savings since the host is
assumed to consume more power than the controller
 Latency - Bluetooth Smart (low energy) can support connection setup and data
transfer as low as 3ms, allowing an application to form a connection and then
transfer authenticated data in few milliseconds for a short communication burst
before quickly tearing down the connection
 Range – Increased modulation index provides a possible range for Bluetooth
Smart (low energy) of over 100 meters
 Robustness – Bluetooth Smart (low energy) uses a strong 24 bit CRC on all
packets ensuring the maximum robustness against interference
 Strong Security – Full AES-128 encryption using CCM to provide strong
encryption and authentication of data packets
 Topology – Bluetooth Smart (low energy) uses a 32 bit access address on every
packet for each slave, allowing billions of devices to be connected. The
technology is optimized for one-to-one connections while allowing one-to-
many connections using a star topology. [4]
2.4 Arduino
The Micro is a microcontroller board based on the ATmega32U4, developed in
conjunction with Adafruit. It has 20 digital input/output pins (of which 7 can be used
as PWM outputs and 12 as analog inputs), a 16 MHz crystal oscillator, a micro USB
connection, an ICSP header, and a reset button. It contains everything needed to support
the microcontroller; simply connect it to a computer with a micro USB cable to get
started. It has a form factor that enables it to be easily placed on a breadboard.
The Micro board is similar to the Arduino Leonardo in that the ATmega32U4 has built-
in USB communication, eliminating the need for a secondary processor. This allows
the Micro to appear to a connected computer as a mouse and keyboard, in addition to a

16
virtual (CDC) serial / COM port. It also has other implications for the behavior of the
board; these are detailed on the getting started page.
2.4.1 Technical specs
Microcontroller ATmega32U4
Operating Voltage 5V
Input Voltage (recommended) 7-12V
Input Voltage (limit) 6-20V
Digital I/O Pins 20
PWM Channels 7
Analog Input Channels 12
DC Current per I/O Pin 20 mA
DC Current for 3.3V Pin 50 mA
Flash Memory
32 KB (ATmega32U4)
of which 4 KB used by bootloader
SRAM 2.5 KB (ATmega32U4)
EEPROM 1 KB (ATmega32U4)
Clock Speed 16 MHz
Length 48 mm
Width 18 mm
Weight 13 g

17
MYO ARMBAND
Myo Gesture Control Armband, launched in June 2013, is a wearable technology
developed by a Canada based start-up named Thalmic Labs. Using a technique known
as electromyography, this little armband is able to read electrical signals from the
muscles in the forearm, map them to gestures made with the hand, and control other
devices with those gestures. The Myo armband gives a touch-free control of technology
with hand gestures and motion. It aims to give a motion control without the need for a
Wii remote, a presentation controller, or anything else that has to be held in hand.
Instead, the armband just has to be slipped like a blood pressure sleeve (or a Nintendo
Power Glove) and things can be controlled with the movements of fingers and wrist.
Most gesture-control systems, like Microsoft’s Kinect, still detect movements with
cameras, which can be thrown off by poor lighting conditions, distance, and simple
obstructions. By drawing gesture information directly from the arm muscles instead of
a camera, Myo circumvents all these problems — and works with devices that do not
have a camera in the first place. It detects five distinct hand gestures to wirelessly
control tech, like music, games, and presentation slides, all with the flick of a wrist.
Fig. 1

18
Myo wraps it in eight different blocks, each of which contains a medical-grade EMG
sensor. The armband also uses a three-axis gyroscope, three-axis accelerometer, and
three-axis magnetometer to sense motion in any direction.
Muscle activity and motion readings are handled by an onboard ARM Cortex M4
Processor, which communicates with your devices via Bluetooth. For devices that do
not have Bluetooth functionality built in, Myo also comes with a Bluetooth dongle that
plugs into any USB port.
Details on battery size are not readily available, but Thalmic claims the lithium-ion cell
inside is big enough to keep Myo going for a full day of continued use.
One can also map gestures to key strokes for customized control, or dig into the open
API and free, easy-to-use SDK to create scripts and applications that put technology at
the fingertips. The device already has a sizable library of apps and plugins to offer. So
far, the Armband has been used to serve the purpose PowerPoint presentations
and media playback for apps like Netflix, Windows Media Player, and
iTunes. [2]
3.1Design
Myo has undergone some major design changes since it was first conceived, but the
final version is about as polished as you can hope for. It is relatively slim and sleek
considering how many sensors are stuffed inside its hull, and despite being a somewhat
geeky piece of gear; it does have an undeniably badass look to it.
The band itself is a series of plastic, rectangular "pods," held together by a
perimeter of rubber lining. On the underside of each pod there are three medical
grade stainless steel EMG sensors that detect the electric impulses in the arm's
muscles. The design allows the Armband to fit a multitude of differently sized
forearms.

19
Fig. 2
One of the pods sports two indicator LEDs. One is simply for the Myo logo,
while the other, rectangular light turns green to indicate the band is charging,
and light blue when it is on and connected.
As for the cold, hard measurements, the Myo comes in at 7.5 inches for
circumference, expandable to 13 inches. It weighs only 3.3 ounces, and each
individual pod has a thickness of 0.45 inches. It is available in black or white.[3]
3.2 Setup and calibration
Setting up Myo is a strange and unfamiliar process, but despite being unlike you’ve
ever used before, getting the armband up and running is pretty straightforward.
Once you’ve got it all charged up (via the included micro USB cable), all that’s left to
do is establish a Bluetooth connection with your computer and run through the
calibration process. Thalmic has put together a series of short videos to guide you
through the process, which make it a breeze to get going.

20
After you slip Myo onto your arm, you’ll need to wait for a couple minutes for the
sensors to warm up. The setup video explains that during this time, Myo is forming a
strong electrical connection with the muscles in your forearm. After this bond is
established, Myo vibrates to let you know it’s ready for sync.
To sync Myo and start using it, you need to perform Thalmic’s special “sync gesture,”
which involves holding your arm parallel to the ground, rotating your wrist outward,
and then swinging your forearm out 90 degrees like it’s on a hinge.
If this gesture works and Myo vibrates again, you’re off to the races.
3.3 Performance and use
Simply put, Myo’s gesture-recognition capabilities are everything they’re chalked up
to be — so long as you have the sensors calibrated properly. That’s the trick. It might
take a couple tries to get everything perfect, but once you do, Myo works like a dream.
The first thing we tried out was Thalmic’s presentation demo, which allows you to
navigate through slideshows using gesture controls like tapping fingers together,
waving your hand right or left, and opening and closing your fist. At first we had some
trouble getting finger taps to register unless they were a bit exaggerated, but after a
quick recalibration, Myo recognized all our motions instantaneously, even if the taps
were light and quick.
After that, we tried it out on iTunes. It was awesome. Wave left and right to skip tracks,
play and pause by spreading your fingers, and roll your fist to crank up the volume.
Even when we were across the room, the gesture commands were recognized on the
first try every time, and their corresponding action was executed without any sort of
delay. So long as you’re within Bluetooth range of whatever you’re controlling, Myo
is extremely responsive.
If there’s a downside to all this, it’s probably the fact that, in its current state, Myo can
only recognize five different gestures and two types of motion. It’s not quite as dynamic
as we had initially thought, and can only read finger taps, left or right waves, opening
your hand, making a fist, and any up, down, left, right, and roll movements.
The sensors are likely capable of more –like, say, a “trigger pull” motion you make
with your index finger, or pointing with two fingers instead of one– but right now it’s

21
limited to the aforementioned gestures and motions. That’s not to say Myo feels
incomplete or incapable in any way, though. You can do a lot with the available
commands — we just expected more gestures to be available out of the box.
3.4 Compatibility
Flipping through your iTunes playlists and Keynote presentations is just the beginning.
Thalmic put Myo in the hands of developers long before the general release, so now
that it’s officially launched, the device already has a sizable library of apps and plugins
to offer.
In the Myo Marketplace, you can download integrations for all kinds of apps, games,
and utilities — Netflix, Spotify, YouTube, Minecraft, Civilization V, Kerbal Space
Program, and tons more. There’s even one that allows you to control a Parrot AR drone
like they showed in the pitch video two years ago.
Thalmic doesn’t really place any limits on how Myo can be used or what it can be used
for, so developers have basically run wild with it – which is awesome, but you’ll
definitely find that some of the integrations are much more practical than others.
One of the first applications I downloaded after setup was the Mac Desktop Utility,
which allows you to do things like switch between desktops, take a screenshot, or show
OS X’s Launchpad by using gestures. Cool, no doubt, but when you’ve already got a
keyboard and a trackpad at your fingertips, it’s pretty unnecessary to throw gesture
control into the mix.
For other things, however, Myo makes a whole lot of sense. Linked up to media players
like iTunes or Spotify, it functions as a sort of wearable hands-free controller. The other
night I wore it while I cooked dinner and didn’t have to whip out my phone to skip
tracks on Spotify – I could just wave my hand in the air over the stove and continue
sautéing my onions.
3.5 Features
Along with the Armband, the whole package is also provided with a standard
micro USB cable for charging and a USB Bluetooth adapter for pairing the device

22
with the computer. It also comes with ten sizing clips that can be used to readjust
the band if it doesn't fit the arm properly.
The Armband uses a three-axis accelerometer, a three-axis magnetometer, and a
three-axis gyroscope to sense motion. An onboard ARM Cortex M4 processor
talks with any connected devices via Bluetooth. The band is compatible with PCs
running Windows 7, Windows 8, and Mac OS X 10.8 and above. It also works
with Bluetooth 4.0-equipped mobile devices running iOS 7.0 and higher, as well
as Android 4.3 and above.
According to Thalmic Labs, a built-in rechargeable lithium-ion battery can keep
the Myo going for one full day on a single charge.
3.6 Highs and Lows of the Device
Highs
 Easy setup
 Expandable design
 Broad compatibility
 Super fast and responsive
 Growing library of apps/integrations
Lows
 Limited range of gestures
 Perfect calibration takes a few tries[3]

23
GESTICULATION
4.1 Idea
Gesticulation is a project in which an effort has been made to implement IoT using a
hardware device called Myo Gesture Control Armband. This hardware device,
developed by Thalmic Labs, senses the motion of an arm of a human being and acts
accordingly. This project has three modules- The Windows System Module, The
Android OS Module and The Home Automation Module. These modules involve the
integration of the hardware device with an existing application on a windows system, a
special application development on an android OS supported mobile device, and the
operation of basic home appliances with the hardware device respectively.
With the initial idea of using a device to create interfaces for youngsters to control their
smart devices from a distance without physical contact with the device we are now
building an aid to serve as a partner for blind and physically handicapped persons.
The idea is to use technology for a social change. We have tried to design a platform
and set of utility tools that can be used by people with visual impairment to help them
in day to day life with a help of their smartphones and this gesture control hardware
device, MYO and reduce their effort. So now various tasks of everyday life can be
accomplished with just a set of a few hand gestures.
4.2 Project Phases
4.2.1 Phase 1
The idea is very simple and unique with try to do ordinary task in a creative and smart
way using technology and gadgets. With the help of utility camera a gallery can be
smartly controlled from you hands from a distance apart removing the difficulty of
switching images/ slides during a slide show. The mobile version of the application is
targeting the youngster that are normally seen capturing a selfie using a stick which can
now only be completed by putting a phone at a distance and opening your fist.

24
4.2.2 Phase 2
The smart application is always running with you establishing a smart and battery
efficient Bluetooth connection with you smart device to assist you immediately. The
application gets triggered whenever some receiver is triggered.. The functionalities of
the application includes
• Handle various receivers of the mobile (Call Receiver, Message receiver)
• Accept and reject calls from a distance
• Set phone to speaker mode
• Read out incoming text messages loudly
• Handle notifications
4.2.3 Phase 3
Continuing with the most basic and commonly used IOT idea of making your home
smart we have designed a prototype to help unpaired people in controlling their home
appliances remotely with their virtual presence. The is just a prototype and we are
working on the research for this is order to create a stable product. The idea is to create
switch boards that have integrated wifi modules in them that will be handled by a home
server .The home server in return will be connected with various clients (Android phone
in our case) . The armband will be in continuous connection with the app and will
interact with the switch relay to control the appliances.

25
SELFIE CAMERA AND GALLERY VIEWER
The idea is very simple and unique with an effort made to be able to do ordinary
task in a creative and smart way using technology and gadgets. With the help of
this module, utility camera and gallery can be smartly controlled by your hands
from a distance apart removing the difficulty of switching images/ slides during
a slide show. The mobile version of the application is targeting the youngster
that are normally seen capturing a selfie using a stick, which can now be
completed by putting a phone at a distance and opening your fist.
In the windows version of this module the effort is made to allow the user to use
the camera of the system without actually touching the system. All they have to
do is look into the camera, smile, twist the hand a little and the photo is clicked.
Through this tool the user can also make a video, can look into the gallery of all
his photos, play a video, increase and decrease the brightness of the image before
clicking and much more.
The module is providing support for two different type of platforms namely
Windows operating system and Android operating system. In windows system
default application “Camera.exe” and “Photos.exe” will be used and we have
provided MYO support by developing a connector which runs in background
under the surveillance of Myo Connect which handles the direct connection with
the MYO armband and manages its different connections states and different
pose actions. The connector gets notified about the change of states of armband
and acts accordingly. With mapping of various keyboard keys and mouse
controls the application is controlled with the hand gestures.
The working in android application is slightly different in case of mobile
application where direct connection is established between the mobile
application and MYO armband. Thalmic labs have provided the OS SDK to
interact directly with the armband. In order to have an efficient performance in
terms of battery BLE(Bluetooth low energy ) connection has been established

26
rather than the classical Bluetooth connection between the armband and the
application.
Fig. 3

27
SMART ASSISTANT
This module is based only on Android platform.
The smart application is always running with you establishing a smart and battery
efficient Bluetooth connection with your smart device to assist you immediately.
The application gets triggered whenever some receiver is triggered. The
functionalities of the application include
• Handle various receivers of the mobile (Call Receiver, Message receiver)
• Accept and reject calls from a distance
• Set phone to speaker mode
• Read out incoming text messages loudly
• Handle notifications
Moving on the technical specification of the application it has support same as
BLE for android OS so it will be supported for Android OS version greater than
4.3 thus targeting the 76% of total android users.
Apart from call and message alerts the application will handle all the third party
notifications like WhatsApp messages and Hangout notifications. We have
designed the different broadcast receivers for different tasks. A background
service is designed which starts running automatically as soon as phone boots up
and tries to connect to the nearest armband.
Fig. 4

28
HOME AUTOMATION (PROTOTYPE)
Home automation is the use of one or more computers to control basic home functions
and features automatically and sometimes remotely. An automated home is sometimes
called a smart home .
Home automation can include the scheduling and automatic operation of water
sprinkling, heating and air conditioning, window coverings, security systems, lighting,
and food preparation appliances. Home automation may also allow vital home functions
to be controlled remotely from anywhere in the world using a computer connected to
the Internet. Besides the functions already mentioned, remote control can be extended
to telephones and answering machines, fax machines, amateur radios and other
communications equipment, and home robot s such as automatic vacuum cleaners.
Continuing with the most basic and commonly used IOT idea of making your
home smart, we have designed a prototype to help impaired people in controlling
their home appliances remotely with their virtual presence. The is just a
prototype and we are working on the research for this in order to create a stable
product. The idea is to create switchboards that have integrated Wi-Fi modules
in them that will be handled by a home server. The home server in return will be
connected with various clients (Android phone in our case). The armband will
be in continuous connection with the app and will interact with the switch relay
to control the appliances.

30
SUMMARY
Today, there are the inventors who work on various hardware devices with some
other hardware devices to make a new one. There are coders, programmers,
software developers, designers who are bringing new innovative ideas and
revolutionizing the world of web. There are brilliant hardware devices, amazing
software; the need of hour is to bring together the two and kick-start a new era
of IoT that is Internet of Things.
In this project, a hardware device called Myo Gesture control Armband is
integrated with a pre-existing application on the windows system, a special
application for android mobile devices, and with yet another technology called
Arduino.
In the integration of Myo Gesture control Armband with the windows system a
pre-existing application, camera is used. The Myo is integrated with the camera
so that the user do not have to touch the system to click his photograph, make a
video or view the gallery. All will be done by gesturing the arm in which Myo is
worn.
For android, a special general utility mobile application is developed in which
the user can use his selfie stick to click his photograph without setting any timer.
Also, it has a feature by which the user can feel all the notification of his phone
on his arm and can act accordingly.
In the final module, the Myo gesture control armband is integrated with Arduino
through an android mobile application. This has been done to solve the purpose
of home automation. Through the gesture of a hand having Myo, The electric
appliances like, a tube light, a fan, etc. of a room can be controlled.

31
CONCLUSION
Through this report it is concluded that the hardware device Myo Gesture Control
Armband has been studied properly. It has been integrated with a windows
system, an android mobile device and with Arduino in order to achieve following
targets
 Control the predefined applications for Windows application with
keyboard mapping of MYO.
 Handle calls and read phone notification with a gesture of MYO.
 Control the various home appliances using the armband.
On an overall review MYO has turned to be a great device with so much potential
to exist in the market but still it has some shortcomings which is needed to be
overcome for better performance.

32
FUTURE SCOPE
• Home Automation
• Gaming
• Blind Teaching
• E-Learning
• Motion Detection

33
REFRENCES
[1] Dr. Ovidiu Vermesan and Dr. Peter Friess , Internet of Things: Converging
Technologies for Smart Environments and Integrated Ecosystems. River Publishers
2013, pp. 20-100
[2] Mithileysh Sathiyanarayanan, Tobias Mulling and Bushra Nazir, Controlling a
Robot Using a Wearable Device (MYO), School of Computing, Engineering and
Mathematics, UK, 2015, pp. 1-3
[3] Drew Prindle, Myo Gesture Control Armband Review, Digital Trends, 2015
[4] Josie Hughes, Jize Yan and Kenichi Soga, Development of Wireless Sensor
Network using Bluetooth Low Energy (BLE) for Construction Noise Monitoring, June
15, pp.1394-1400

34
APPENDIX
9.1 Lua Script
scriptId = 'kanika.myoscript.camera'
scriptTitle = "Camera Connector"
scriptDetailsUrl = ""
temp=0
function onForegroundWindowChange(app, title)
myo.debug("onForegroundWindowChange: " .. app .. ", " .. title)
if(title=="Camera") then
temp =0
end
if(title=="Photos") then
temp=1
end

35
return platform == "Windows" and (app == "ApplicationFrameHost.exe" or
title == "Camera")
end
function activeAppName()
return "Camera"
end
function performAction()
myo.keyboard("space", "press")
end
function switchTab()
myo.keyboard("tab", "press")
end
function cancelFocus()
myo.keyboard("escape","press")
end
function switchOptions()
myo.keyboard("up_arrow","press")
end
function nextPhoto()
myo.keyboard("right_arrow","press")
end

36
function previousPhoto()
myo.keyboard("left_arrow","press")
end
function closePhotos()
myo.keyboard("alt+f4","press")
end
function conditionallySwapWave(pose)
if myo.getArm() == "left" then
if pose == "waveIn" then
pose = "waveOut"
elseif pose == "waveOut" then
pose = "waveIn"
end
end
return pose
end
function onPoseEdge(pose, edge)
myo.debug("onPoseEdge: " .. pose .. ", " .. edge)
if(temp==0) then

37
if (pose == "fist") then
if (edge == "on") then
performAction()
myo.setLockingPolicy("standard")
end
if edge == "off" then
myo.unlock("timed")
end
end
if (pose == "waveOut") then
switchTab()
myo.setLockingPolicy("none")
end
myo.unlock("timed")
end
end
if (pose == "waveIn") then

38
switchOptions()
end
myo.unlock("timed")
end
end
if (pose == "fingersSpread") then
cancelFocus()
end
myo.unlock("timed")
end
end
end
if(temp==1) then
if (pose == "fist") then

39
performAction()
end
myo.unlock("timed")
end
end
if (pose == "waveOut") then
nextPhoto();
end
myo.unlock("timed")
end
end
if (pose == "waveIn") then
previousPhoto();

40
end
myo.unlock("timed")
end
end
if (pose == "fingersSpread") then
closePhotos()
end
myo.unlock("timed")
end
end
end
end
9.2 Code snippet for background services that handle various gestures
private DeviceListener mListener = new AbstractDeviceListener() {
@Override

41
public void onConnect(Myo myo, long timestamp) {
Log.e("myo
connected",""+Hub.getInstance().getConnectedDevices().get(0).getName());
eventBus.post(new String("Connected"));
}
@Override
public void onDisconnect(Myo myo, long timestamp) {
eventBus.post(new String("Disconnected"));
}
// onPose() is called whenever the Myo detects that the person wearing it has
changed their pose, for example,
// making a fist, or not making a fist anymore.
@Override
public void onPose(Myo myo, long timestamp, Pose pose) {
// Show the name of the pose in a toast.
switch (pose) {
case UNKNOWN:
break;
case REST:
case DOUBLE_TAP:
switch (myo.getArm()) {
case LEFT:
break;
case RIGHT:
break;
}
break;
case FIST:
break;
case WAVE_IN:
break;

42
case WAVE_OUT:
break;
case FINGERS_SPREAD:
break;
}
if (pose != Pose.UNKNOWN && pose != Pose.REST) {
// Tell the Myo to stay unlocked until told otherwise. We do that here so you
can
// hold the poses without the Myo becoming locked.
myo.unlock(Myo.UnlockType.HOLD);
// Notify the Myo that the pose has resulted in an action, in this case changing
// the text on the screen. The Myo will vibrate.
myo.notifyUserAction();
} else {
// Tell the Myo to stay unlocked only for a short period. This allows the Myo
to
// stay unlocked while poses are being performed, but lock after inactivity.
myo.unlock(Myo.UnlockType.TIMED);
}
EventBus eventBus = EventBus.getDefault();
eventBus.post(pose);
Log.e("posting event",""+pose);
}
@Override
public void onArmSync(Myo myo, long timestamp, Arm arm, XDirection
xDirection) {
super.onArmSync(myo, timestamp, arm, xDirection);
eventBus.post(new String("Synced"));
}

43
@Override
public void onArmUnsync(Myo myo, long timestamp) {
super.onArmUnsync(myo, timestamp);
eventBus.post(new String("Unsynced"));
}
@Override
public void onUnlock(Myo myo, long timestamp) {
super.onUnlock(myo, timestamp);
eventBus.post(new String("Unlocked"));
}
@Override
public void onLock(Myo myo, long timestamp) {
super.onLock(myo, timestamp);
eventBus.post(new String("Locked"));
}
};
9.3 Code snippet for call receiver
public class IncomingCallReceiver extends BroadcastReceiver {
Context context;
private int mInterval = 4000; // 5 seconds by default, can be changed later
private Handler mHandler;
private AudioManager myAudioManager;
public int mod;
public static boolean isRinging = false;
public static Intent intent ;

44
@Override
public void onReceive(Context context, Intent intent){
try{
String state = intent.getStringExtra(TelephonyManager.EXTRA_STATE);
mHandler =new Handler();
myAudioManager =
(AudioManager)context.getSystemService(Context.AUDIO_SERVICE);
mod=myAudioManager.getRingerMode();
this.intent = intent;
if(state.equals(TelephonyManager.EXTRA_STATE_RINGING)){
Toast.makeText(context, "Phone Is Ringing",
Toast.LENGTH_LONG).show();
startRepeatingTask();
myAudioManager.setRingerMode(AudioManager.RINGER_MODE_SILENT);
isRinging = true;
Log.e("ringing","true");
}
if(state.equals(TelephonyManager.EXTRA_STATE_OFFHOOK)){
Toast.makeText(context, "Call Recieved", Toast.LENGTH_LONG).show();
stopRepeatingTask();
myAudioManager.setRingerMode(AudioManager.RINGER_MODE_VIBRATE);
isRinging = false;
Log.e("ringin","false");
}
if (state.equals(TelephonyManager.EXTRA_STATE_IDLE)){
Toast.makeText(context, "Phone Is Idle", Toast.LENGTH_LONG).show();
stopRepeatingTask();
myAudioManager.setRingerMode(AudioManager.RINGER_MODE_VIBRATE);

45
isRinging = false;
Log.e("ringin","false");
}
}
catch(Exception e){e.printStackTrace();}
}
Runnable mStatusChecker = new Runnable() {
@Override
public void run() {
if(Hub.getInstance().getConnectedDevices().size()>0)
{
if(isRinging){
Hub.getInstance().getConnectedDevices().get(0).vibrate(Myo.VibrationType.LONG);
Log.e("vibration","sent");}
}
mHandler.postDelayed(mStatusChecker, mInterval);
}
};
void startRepeatingTask() {
mStatusChecker.run();
}
void stopRepeatingTask() {
mHandler.removeCallbacks(mStatusChecker);
}
}
9.4 Code Snippet for usb serial connection
private SerialInputOutputManager mSerialIoManager;

46
private final SerialInputOutputManager.Listener mListener =
new SerialInputOutputManager.Listener() {
@Override
public void onRunError(Exception e) {
Log.d(TAG, "Runner stopped.");
}
@Override
public void onNewData(final byte[] data) {
SerialConsoleActivity.this.runOnUiThread(new Runnable() {
@Override
public void run() {
SerialConsoleActivity.this.updateReceivedData(data);
}
});
}
};
@Override
protected void onResume() {
super.onResume();
Log.d(TAG, "Resumed, port=" + sPort);
if (sPort == null) {
mTitleTextView.setText("No serial device.");
} else {
final UsbManager usbManager = (UsbManager)
getSystemService(Context.USB_SERVICE);
UsbDeviceConnection connection =
usbManager.openDevice(sPort.getDriver().getDevice());
if (connection == null) {
mTitleTextView.setText("Opening device failed");
return;

47
}
try {
sPort.open(connection);
sPort.setParameters(115200, 8, UsbSerialPort.STOPBITS_1,
UsbSerialPort.PARITY_NONE);
} catch (IOException e) {
Log.e(TAG, "Error setting up device: " + e.getMessage(), e);
mTitleTextView.setText("Error opening device: " + e.getMessage());
try {
sPort.close();
} catch (IOException e2) {
// Ignore.
}
sPort = null;
return;
}
mTitleTextView.setText("Serial device: " +
sPort.getClass().getSimpleName());
}
onDeviceStateChange();
}

Minor Project Final Report

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Semelhante a Minor Project Final Report

Semelhante a Minor Project Final Report (20)

Minor Project Final Report