With the demonstration over, Ed and I finished writing the Work in Progress Paper. A clear and well written paper is crucial to allow us to submit our work to CHI, and I believe we have achieved this.
We also made a CHI video to demonstrate our device. Joe took the role of camera man, Ed and Tim were our actors, Rob did the final video editing, and I did the voice over. The video demonstrates our final device, as well as presenting future directions that our research could take us. Take a look at our final video below.
I would like to thank Sue Ann Seah for all the feedback and help she has provided throughout our project. Overall this course has been a really enjoyable experience and I am very proud of what Famous 5ive have achieved.
Our demonstration went brilliantly. We were really happy to have so many of our peers interested in our device. Even a few of our markers were willing to be controlled by us!
Quite a few people came back to us with new ideas of how we could take this concept further, proving that it really is an interesting device with lots of potential.
I’m really pleased with our demonstration; it certainly had the “shock” factor.
Today Ed and I worked on writing up the Work in Progress Paper. I did some more background reading on research that has been done prior to our work. We have found at least 8 useful papers that we can reference in our paper. I also read some previous Work in Progress Papers to get a general idea as to the formatting and content. As some papers were more technical, I tended to focus on the papers which write more about the concept they are presenting because this is similar to our paper. We also decided on the subheadings including “Abstract”, “Introduction”, “Background”, “Guided Gaming” and “Conclusion and Future Work”.
Joe worked with Sue on building the circuit that would become our interactive device. Joe has modified the TENs unit to be controlled by the Arduino. The controls are fairly simple; two push switches control the duration of the activated electrical stimulation, and the electrical power delivered by the circuit is determined by two potentiometer sliders. Joe has put a lot of thought and effort into building this device so that it is safe to use, and because of this there were a few redesigns of the circuitry along the way. The photo below shows the final circuitry that makes up our device.
Tim and Rob wrote some code for the Arduino and built the box with the laser cutter and then attached the sliders and push buttons. Admittedly I am very pleased with our final device.
In workshop 5 the group split up to work on different parts of our project.
1. Joe worked with Sue on finalising the design of the circuitry needed to control the TENs unit. He also cut and soldered the wires that leave the TENs unit and connect to the electrodes so that it can connect to an Arduino.
2. Ed and I looked up the format for the Work in Progress Paper. We also made a draft plan of what we intend to write and given ourselves the goal as to finish the first draft in a few days time.
3. Tim and Rob will have the task of making our interactive device ‘game-like’. As a group we have brainstormed a few ideas of what game would be possible; for example a game in which the controller has to get their player to throw a ball in a bucket, or a game of darts using a magnetic dart board. They will work on making the demo engaging.
Workshop 4 was all about discovering the capabilities of our TENS unit, in terms of seeing if we could cause movement with electrical stimulation.
We worked out the muscles that needed to contract based on papers we had read and the results from other people’s work (for example following what the people in PossessedHand had done).
Attached the electrodes to these muscles.
Started with a small stimulation and gradually increased and changed the pulse width, pulse rate/frequency and the intensity of the signal.
We found that we were about to move Tim’s fingers pretty accurately. We also tried this on Sri however we must not have placed the electrodes in the ideal place on his arm because we got his hand moving rather than his fingers.
Tim’s leg was pretty hard to control, possibly for the fact that it is a large muscle and will require more electrodes to contract it.
The most successful control we achieved was by attaching the electrodes to Tim’s bicep and triceps. By connecting different electrodes to both muscles and alternating the current to each, we were able to move Tim’s forearm up and down fairly accurately.
Because of this success, we are now considering a throwing game or a game of darts which uses the bicep and triceps muscles.
The next stage that we have to think about is hacking the TENS unit so that we can control it with an Arduino. Our current idea is to run each of the outputs of the Tens Unit into a MOSFET each, and have the gate of the MOSFET connected to an output pin of the Arduino. Setting the Tens unit to the highest setting we need, we should be able to vary the intensity of the pulses anywhere from off to full power, using the MOSFET.
We will meet with Sue again soon to get working on building the system.
Concept: By using an array of 14 electrode pairs (28 pads) arranged around the arm, the motion of 16 joints in the hand can be precisely controlled.
Points to take away:
The PossessedHand team showcase a system which can be calibrated to an individual user who has no prior knowledge of the system, potentially opening their product up for sale to non-expert users. They also show that they can control precise dextrous activities by targeting seven muscles in the forearm to control individual fingers. In the context of our project, we can take away several of their discoveries as a base to work from. For example, they use a pulse width of 0.2ms in order to be similar to the human nervous signals. They also use a pulse frequency of 40Hz and in doing so minimize the pain felt by the wearers of the electrode belt as it mimics human impulse to move muscles. From here they simply change the voltage in order to control the angle of various joints.
Figure 1: Graph showing the relation between joint angle and voltage.
PossessedHand also gives advice on electrode pad selection with the view to reducing pain felt, recommending the use of pads at least 10x10mm or larger with a solid conductive gel between the pad and the skin.
2. Muscle-propelled force feedback: bringing force feedback to mobile devices using electrical stimulation
Unlike the Possessed Hand, where electrical muscle stimulation (EMS) is used to guide and control hand gestures, here electrodes are used to induce involuntary contractions to hinder movement. This used in conjunction with a gesture based video game played on a mobile phone, where the contractions handicap the player at certain points. In a game where the player controlled an aeroplane by tilting the phone, gusts of wind would trigger EMS to add resistance to an arm, forcing the player to steer with one hand. Another game, which was multiplayer, allowed players to handicap the other by activating power-ups.
The hardware required fit in a box to the back of the phone. An arduino uno was used, powered by a 9V battery, and communcated with the phone via Bluetooth. A battery powered signal generator with amplifiers induced the EMS through the electrodes.
While in our project we intend to guide the player with EMS, not impede them, the controller, who is guiding the player, could punish or disrupt a player attempting to move against their wishes. In a multiplayer setting, we could also implement power ups to trigger handicaps on opposing players, or have obstructing digital elements that hinder or stop players from moving into certain areas or in certain directions.
3. Supporting Interaction in Public Space with Electrical Muscle Stimulation
The group designed a game where a user plays in front of a screen and is provided with haptic feedback using electrical muscle stimulation (EMS). EMS technology offers a wide variety of haptic feedback, ranging from a small tickle to large muscle contraction. This allows the impact of different intensities of feedback to be investigated in depth.
Due to the integration of a Microsoft Kinect, the user is capable of interacting with the game using body movements. As the user touches a bubble, the display application sends a signal to the EMS control system which then provides an electrical impulse contracting the muscles of the user as feedback. An Arduino is used to control duration and intensity of feedback (typically 260 s. and from 50 to 70 Hz) and communication is established between the Arduino and display node using WiFi.
In the discussion they concluded that some challenges arise with this new kind of technology. For example, the amount of electricity generated needs to be adapted to each user. Applying too strong currents results in pain whereas too little results in no effect at all. We will definitely need to consider this for our project.
4. Influence of electrical stimulation frequency on skeletal muscle force and fatigue
The aim of the study is to determine the impact of electrical stimulation frequency on muscle force and fatigue and propose application modalities in regards to sports or clinical rehabilitation training.
Testing on 26 healthy adults, the maximal voluntary contraction (MVC) force of the quadriceps femoris muscle was recorded before, during and after electrical stimulation. The adults underwent 20-minute sessions; 60 rounds of 5-second contractions separated by 15-second rest periods. Three different frequencies were used: 100, 50 and 20 Hz, corresponding to their maximum tolerable intensity. The results they found were that with a maximum frequency of 100 Hz, they obtained 71% of the MVC force at the beginning of the session. The decrease in muscle force at the end of the session is significantly greater for the highest frequencies respectively 27, 33 and 38% of MVC for 100, 50 and 20 Hz.
To conclude, higher intensity and higher frequency induce stronger muscular contractions, but also a stronger decline in force. This study indicates that the higher the frequency we use, the faster fatigue sets in. They propose that 100 Hz frequency should only be applied to maximal muscle force in athletic training programs and that according to the slower onset of fatigue, 20 Hz should be used for rehabilitation. In relation to our project, we wouldn’t want the player to get fatigued too quickly, so we will look at using at using more force but at a lower frequency.
5. Ergonomic and Electronic Designing of Muscle Stimulator
This paper discusses the existing uses of electronic stimulators for muscles in the contexts of muscle toning and in therapy. It discusses the safety and the risks of devices currently available and then goes on to describe methods of increasing the safety in a new device, covering both hardware and software features. It evaluates the risk of a device with these features using the European Commission’s RAPEX standard.
Then goes on outline the merits of the SiC (Silicon Carbide) MOSFET over a standard silocon MOSFET, in the context of this device.
Points to take away:
Several features described (such as a software timeout to prevent accidental overuse) that we can implement into our software side to increase the safety.
With safety features implemented, RAPEX evaluation gives these devices a risk level of very low for health adults. Vulnerable people (elderly and injured) are at a higher risk of harm.
SiC MOSFET appropriate for use in electro-stimulation devices, due to it’s high switching frequency (important as we’re doing high frequency pulses) and lower operating temperature, so less wasted energy (longer battery life) and no need for large heatsink.
6. Effects of Electrical Muscle Stimulation on Body Composition, Muscle Strength, and Physical Appearance
As a response to a large amount of marketing of Electrical Muscle Stimulation (EMS) devices as a way to “increase muscle strength, to decrease body weight and body fat, and to improve muscle firmness and tone”, the paper details an 8 week test of their effectiveness, using metrics including bodyweight, isometric and isokinetic strength, and self-perception, amongst others, tested both before and after the programme. They tested 27 young adults, 16 using the EMS devices and 11 doing sham stimulation sessions, as a control group. The results showed that the EMS devices produced no significant changes. The only significant changes in tests were also present in the control, and so were not down the device.
Points to take away:
Even with extended use of the EMS devices, we won’t be changing the muscular build of anyone playing.
The devices can be used several times a week for extended sessions without causing any damage or problems, in healthy young adults.
7. Imaginary Reality Gaming: Ball Games Without a Ball
This paper showcases team sports that have been adapted to allow for the introduction of virtual game elements. These allow for mechanics traditionally found in video games, such as power ups and unrealistic physics, to be complement features of team sports such as physical exertion and immediate social interaction, creating new game dynamics.
Past papers have tried to combine physical and virtual gameplay using augmented reality screens, either handheld or head mounted; however these had the disadvantage of players viewing the game through screens with lag, limited resolution and peripherial vision, losing social aspects such as being able to read body language or facial expression in other players.
To avoid these hindrances, the state of the virtual elements are only hinted at by sparse use of audio cues (used in the demonstration to indicate who has ownership of the ball) and doing away with screens altogether.
We will be expanding upon this concept by guiding or controlling the players using electrical stimulation, as well as deliberately hindering some of the players senses (such as sight or hearing) allowing an external controller or AI to use the player as an avatar.
8. The Relationship Between Electromyography and Muscle Force
Rough idea: Discuss methods and problems with moving the leg with electrical stimuli.
Points to take away:
In the majority of people, the power needed to move a specific muscle is similar across different people, but can be drastically different across different muscles (tends to be roughly proportional to the muscle length).
Moving a limb with muscle stimulation isn’t just about contracting muscles, when going past a certain angle it is more about extending the opposing muscle.
See graph below. The black line represents force needed to contract muscle, and red line is the force needed to extend the opposing muscle.
The strength needed at any one time when moving a limb with a muscle changes. The graph below shows the force needed by the quadricep when straightening the leg. As you can see, the force needed changes as the leg gradually becomes straighter, and as such the current needed from an electrical stimuli would change.
9. Restoration of Grasp following Paralysis through Brain-Controlled Stimulation of Muscles
Rough idea: The idea is that paralysed people can use brain signals to control electrical stimuli that control their paralysed muscles.
Points to take away:
In this paper, the first step is to record the movement of a non-paralysed subject to get the correct muscles to stimulate and the correct brain signals to trigger the movement. The image below shows the brain signals and muscle feedback from a simple movement.
To move a limb, both contracting and extending muscles need to be stimulated.
In order to stimulate a muscle, you need to apply the electronic stimulus in waves. The frequency seems to vary depending on the muscle, the amount of movement and the length of stimulation. This paper is about moving monkeys arms (could be similar to ao humans) and they use a firing rate of between 25 or 30 Hz.
The current used in the experiments in the this paper used a current of 2-8 mA, but due to the fact that this is on monkeys, this could probably be increased to about 20-40 mA for human arms.
10. Programmed Six-Channel Electrical Stimulator for Complex Stimulation of Leg Muscles During Walking
Concept: This paper discusses the design process for a 6-channel electrical stimulator for assistance of a person’s gait.
Points to take away:
For walking it is necessary to have a six-channel TENS device as there is a need to be able to coordinate 6 muscles in order to control gait effectively. Note that this paper is intended for stroke patients and therefore some of the muscle contraction may simply be for stability which would be present without the need for contraction in someone with fully functioning legs. They note that even among stroke patients it is rare that all 6 channels are necessary, however they also note that any spare channels may be used for abduction of the hip (not shown in diagram below).
Figure 2: Diagram of the muscles in the legs which need to be controlled for walking and balance.
The paper has shown that it is possible to aid a stroke patient both in the stance(standing) and swing(moving) phases of walking simply by contracting the correct muscles with the TENS machine. This is encouraging as it means that our system does not necessarily have to have limitations and there will be a lot of room for improvements to our basic system.
Figure 3: A stroke patient walking with 5 stimulated muscles.
Finally, the paper was not without some drawbacks, for example they found that each person reacts differently to the system and therefore a lot calibration is required not only in terms of electrode location but also stimulation patterns and synchronisation was different for people with different gaits.
As an aside, here is an interesting new development in the gaming industry which utilises the same technology as we plan to use in order to simulate bullet impact and other collisions through EMS:
From this video we can see that EMS is starting to become more widespread.The ARAIG suit aims to give a player feedback to their gaming experience and EMS is used alongside simple vibration pads in order to provide this.
References
Tamaki, E., Miyaki, T., Rekimoto, J. PossessedHand: Techniques for controlling human hands using electrical muscles stimuli. CHI 2011, 2011, Vancouver, BC, Canada.
Pedro Lopes, Lars Butzmann, and Patrick Baudisch. 2013. Muscle-propelled force feedback: bringing force feedback to mobile devices using electrical stimulation. In Proceedings of the 4th Augmented Human International Conference (AH ’13). ACM, New York, NY, USA, 231-232.
Pfeiffer, M., Schneegass, S., Alt, F., Supporting Interaction in Public Space with Electrical Muscle Stimulation. UbiComp’13 Adjunct, Zurich, Switzerland, 2013.
Dreibati, B., Lavet, C., Pinti, A., Poumarat, G. Influence of electrical stimulation frequency on skeletal muscle force and fatigue. Phys Rehabil Med. 2010. 52(4):266-71, 271-7.
Lukic, P., Zunjic, A. Ergonomic and Electronic Designing of Muscle Stimulator. 2010.
Porcari, J., et al. Effects of Electrical Muscle Stimulation on Body Composition, Muscle Strength, and Physical Appearance. Journal of Strength and Conditioning Research, 2002, 16(2), 165-172.
Baudisch, P., et al. Imaginary Reality Gaming: Ball Games Without a Ball. UIST’13, Oct 2013, St Andrews, UK.
Kuriki, H., et al. The Relationship Between Electromyography and Muscle Force, EMG Methods for Evaluating Muscle and Nerve Function, Mr. Mark Schwartz (Ed.), 2012.
Ethier, C., et al. Restoration of grasp following paralysis through brain-controlled stimulation of muscles. Nature, Vol. 485, Issue 7398, April 2012.
Strojnik, A., et al. Programmed Six-Channel Electrical Stimulator for Complex Stimulation of Leg Muscles During Walking. IEEE Trans on Biomedical Engineering, Vol. BME-26, No. 2, Feb 1979.
After doing a bit more research into our project, I came across 3 related videos.
Electrodes turn your Eyelids into 3D Shutter Glasses. The most bizzarre application of EMS is this video where the creator claims to see in 3D by using electrical pulses to rapidly and alternatively close each eyelid. Electrodes turn your eyelids into 3D shutter glasses.
Arduino Controlled Arm Movement. Pulses are from prerecorded brain readings translated into an on/off signal that controls the Arduino, which in turn controls the arm. It could be interesting to look at how the brain could stimulate the muscle into moving. In the long term, an application for this could be used to allow paralysed patients to move again! Brain controlled arm movements.
Golem. Bennet Schatz created a prototype systems that uses EMS to turn his arm into a golem controlled by a computer for the purpose of manipulating a sculptural medium such as clay. Although the finished result isn’t really ‘art’, it is another good example for an application of EMS. Golem arm used to sculpt clay.
Our third workshop was dedicated to finding out what components we will need to buy/get hold of for next weeks workshop so that we can start building the system. We spent some time researching systems that have been designed to contract muscles using similar electrode pads that we have bought.
TENS units can be used to control the stimulation to the electrode pads (as I have previously mentioned in another blog). The problem with these machines is that they provide a pulsing electric current. For our project we will need a sustained current and a greater control of how much current is provided to each electrode than a simple TENS unit can provide us. Therefore we are looking into how to ‘hack’ a TENS unit or build a circuit very similar. I spent most of the workshop looking for circuit diagrams on the internet and according to my other team member the diagram below looked plausible[1].
Between now and the next workshop we need to:
Buy components to build the circuit.
Research into which muscles we want to contract and where we will have to place the electrodes. For example, how many electrodes and at which places will they need to be placed on the arm to stimulate an elbow bending?
Look into health and safety for our project. What is a safe amount of current to apply to the muscles? Too little current and the muscles will not be stimulated but too much and it could be painful and potentially harmful to the user.
Collect 10 research papers related to our project.
Last week we arranged a meeting with our supervisor Sue to help refine our idea. After a discuss in the Monday lecture we decided that we will still create a Controller-Player game, however we will build a system around muscle stimulation rather than our other idea of using vibro-tactile feedback.
This means that our research has gone a different direction and that we are now looking at papers in muscle stimulation and rehabilitation devices to control muscles.
A interesting paper named “PossessedHand: A Hand Gesture Manipulation System using Electrical Stimuli” demonstrates that the use of electrical stimuli can control the 16 joints in the hand. The application they propose is to help beginners learn how to play musical instruments. If we can replicate something similar in our proof of concept paper but aimed more at a gaming environment then we will have contributed to this field of research.
After our discuss with Sue we came up with a few things to buy for next week labs including a) suitable pads/electrodes, b) crimped wires (male-male, male-female and female-female) and c) gel for the electrodes. The setup used for the possessed hand can be seen below.
I also looked at TENS units (Transcutaneous Electrical Nerve Stimulation) because we will need these or a similar set up to control the electrodes.
References:
Tamaki, E., Miyaki, T., Rekimoto, J., PossessedHand: Techniques for Controlling Human Hands using Electrical Muscles Stimuli. CHI 2011.
During the second workshop the team collectively narrowed our idea down, deciding to pursue the Guided Gaming project. The overview of this project is that there will be two sets of people called ‘controllers’ and ‘players’ with the ‘controllers’ using a controller to guide a blindfolded player through a game. When we first drew up the initial concept, we were keen on creating a football based guided game, however upon further thought, we came to the conclusion that the combination of blindfolds and a football was likely to be too difficult for the players. Therefore we have refined our idea to a guided obstacle course, with the potential for two players to race around the course. The ‘players’ are given commands via a variety of sensory stimulations including (but not limited to) vibrations, led lights, sounds and potentially even electric currents.
We have left the project open for further expansion, with the possibility of using an Xbox Kinect as an input method for the ‘controllers’ to use to guide the ‘players’. Another potential expansion would be to create an AI which would replace the need for human ‘controllers’. Though safety concerns would have to be met, a further expansion could have the ‘players’’ muscles stimulated by directly applying a current through electrodes on the skin; this would remove all responsibility to react and move due to input from the ‘controller’. Research into what equipment and power supply to be capable of this would be required.
Requested Components:
During the workshop, we came up with a list of items which we will probably need in order to create this system:
2x wireless Arduino boards. Each arduino board will require
9V battery adapter
http://playground.arduino.cc/Learning/9VBatteryAdapter this article describes how to construct an adapter to power an Arduino board from a 9V battery. It requires a 9V battery clip and a 2.1mm DC Male Power socket (as well as solder and heat shrink tubing.)
there are several that came with the Arduino kit, and otherwise are extremely cheap
Expansion: AI player
Camera. This could be a Webcam or Kinect.
Markers for players to wear, to be identified by the camera
If we assume there will be a maximum of 12 motors per player, a transceiver module, and that we will be provided with an arduino, wiring, battery, etc, each player will have a max cost of around £20.
Miranda Aperghis 
