Reproducing biological motion in a robotic arm

Automatic capture of an amputee's own natural biological arm motion for embedding into their motorised prosthetic arm is a goal that could potentially aid in the use of this type of therapeutic device. In 2007, of the approximately 301 million people in the USA, about 1.7 million were living with limb loss, many of which are upper limb amputees. Quality of life of many of these amputees may be improved through the use of a motorised prosthetic arm, but there are currently limitations of being able to embed into these arms the desired natural motion of the amputee. This research aims to investigate capturing a single-arm amputee's natural motion from their remaining biological arm, and automatically translating this into a control algorithm for prosthetic arm motion that may be activated on command. This was done by developing an imaging system for capturing natural arm motion, replaying of the motion on a prosthetic arm, and assessing the performance, functionality and useability of the developed system. Capturing of natural arm motion was done by developing a custom developed stereo imaging system. The imaging system comprised a portable four-mirror single camera stereographic camera unit housing incorporating a one megapixel monochrome industrial camera capable of infra-red imaging. The system had a capture area of suitable size for tracking a person's arm motion whilst requiring minimal setup time and being relatively inexpensive. A series of wireless infrared tracking markers suitable for being worn were designed and constructed. These markers comprised of a series of infrared LEDs in the form of a band‚ÄövÑvp that can be worn around sections of the arm being tracked, and a marker control board. The marker control boards comprised of a microcontroller, an Xbee wireless module and other basic circuitry to provide power to the control board and marker chain. The tracking markers could be turned on and off wirelessly from the control PC by a series of serial commands. A LabVIEW based implementation of a 3D motion capture and replay system was created and interfaced with the tracking markers and stereographic camera unit. The 3D motion capture system received images from the industrial camera, and processed these to detect the location of the markers within the images. These locations, when used as inputs to lookup tables, allowed the motion capture system to locate the markers in 3D real-world coordinates. A motorised prosthetic arm was also interfaced to the system. This arm consisted of a carbon-fibre shell, with three embedded motors used to rotate the elbow in two axes and the wrist in a single axis. The arm was retrofitted with off the shelf servo control boards, allowing all three motors to be controlled through a single USB cable connected to the control PC. The wrist was not used in this research and so only the two elbow motors were used. The stereo imaging system used a look-up table to determine 3D joint positions from marker position in the stereo images. Accuracy of different interpolation methods were compared to determine which to use in the final system, with cubic interpolation giving better results more often than the linear alternative (~45%, ~62% and ~64% more in the X,Y and Z axes respectively). Further, when comparing average errors, cubic interpolation showed over 10% error reduction for all three axes using a 1cm interpolation resolution, over linear interpolation with the same resolution. The combined system was tested to determine how accurately a single semi-randomly placed marker could be located. This single point testing showed errors of less than 1.5 cm in the X axis for over two-thirds of the time, with errors less than 3cm in the Y axis also over two-thirds of the time. The Z axis exhibited errors of less than 5cm just under two-thirds of the time. An artificial arm fabricated from cardboard with adjustable flexion and rotation was then used with the combined system to determine how well multiple points could be located, and hence determine the accuracy for resolving the arm's flexion and rotation angles. This multipoint testing showed errors of less than 5¬¨‚àû for the majority of the time. Both single and multipoint testing showed that a purely proportional line of best fit was generally quite close to being a 1:1 relationship between the true values and the predicted output values. In tests involving patients, patients' arms were tracked and simultaneously recorded. The recorded motion was immediately replayed in real-time‚ÄövÑvp to control the motorised prosthetic arm synchronously with the patient's movements. Patients rated the system through a questionnaire. Patient trials showed that desired arm motions could be generated more accurately for a patient's real arm rather than their prosthetic arm, and that it was possible to satisfactorily record motion and accurately replay it through a prosthetic arm. Patients reported that the system was good and enjoyable, but could benefit from further refinements such as increased speed and ease of use. Single point, multi-point, and patient based results could theoretically be improved by using a higher resolution camera (to increase the pixel:cm ratio), a higher quality mirror unit (to remove mirror based distortions) and finer resolution measurements to increase the amount of data available from which to create the lookup tables. The results of patient trials could be further improved by implementing faster image processing techniques or more powerful hardware to increase the rate at which frames can be captured, leading to a higher availability of motion data. The system including the four-mirror stereographic camera unit, software and tracking markers was designed successfully, and shown to be able to track, record, and replay motion. Patients found that the system was usable, enjoyable and worked well, and that use of such a system in embedding their desired arm motion into a prosthetic arm would be feasible and of potential benefit.