Thrust 3: Effectiveness
Title
3A.1: Multimodal Human Machine Interfaces
Project Leader
Prof. Wayne Book (Georgia Tech)
Statement of Project Goals
This project will establish the relationship between the user interface and fuel efficiency for a relevant range of dynamic system behaviors. The interfaces used will incorporate both traditional and experimental interface devices and sensory modalities. Prediction of the relative performance of interface approaches will be enabled for a range of applications. Implications for usability will be addressed through collaboration with researchers at NCAT.
Project Role in Support of Strategic Plan
Fluid power devices will be used more effectively, thereby reducing working time and hence the energy consumption (efficiency barrier). New and existing devices will be able to safely perform their intended functions under human direction without undue workload on the operator (safety and human machine interface barrier).
Description and Explanation of Research Approach
Fluid power applications potentially range from huge mobile excavators to compact wearable tools for home use. In most of those systems, human operators directly interact with machines. The necessary communication between humans and machines directly impact system performance1. Coordinated control and other more intuitive interfaces have been shown to reduce operator errors and speed up completion time. However, the impact of intuitive operator interfaces on fuel efficiency is unknown. This research will compare how fuel efficient operators are with a standard versus a novel HMI. It will also examine the differences based upon the type of machine (pump or valve controlled) and the characteristic behaviors that may be inherent or imposed on these differences.
Traditional human machine interfaces often rely solely on the visual modality as a path of communication between humans and machines. However, when operator's workload is heavy, the number of channels that are available for communication between the operator and the machine become more and more crucial. In practice, the visual modality or seeing, and the audition modality or hearing are the most commonly employed. Other modalities through which the machine can send information to the human include feel (sense of pressure and its variations) and proprioception (the body's awareness of its own geometry and forces).
The challenge before us is the multiple criteria for evaluating the performance of a fluid power system: energy efficiency, productivity, user acceptance, accuracy of motion, safety.... The improvement of one of these metrics has a consequence on the others. It is possible to quantitatively measure energy efficiency as is proposed here, while usability is a more qualitative issue but one which must be simultaneously evaluated. The expertise of NCAT researchers have already started to address this issue and their contribution will continue to be essential.
Another challenge for some scenarios is that the operating environment is extremely complex and not readily controlled for testing. Georgia Tech has created an excavator simulator which minimizes the variation between test runs and will be further augmented to include operator motion. Concepts will be reinforced with simulation and physical verification on the compact rescue robot, TB4.
 |  |  |
Figure 1: The full scale virtual excavator (left & center) and view of the simulated trenching work cycle (right).
See video of the Virtual Excavator in operation.
System behavior depends on the dynamics of all components and their interaction. In many applications of fluid power the most complex interaction includes the human operator. To the extent the human operator is understood, Project 3A1 applies the technology to the context of current and potential applications of fluid power. Often a lack of a fundamental understanding of system behavior hampers design of improved interfaces, limiting industry to cut and try experimentation and crude rules of thumb. This lack of understanding is exposed and addressed in an associated project.
Figure 2: John Deere Tractor-Loader-Backhoe used to study
control performance degradation caused by biodynamic feedthrough.
Fluid power devices often involve many degrees of freedom under direct control of a human operator. Complex and variable human dynamics therefore interact with complex machine dynamics to compromise system performance and stability. The use of haptics (tactile or force feedback) is at the forefront of this challenge, because it is by the sense of touch that we intuitively comprehend the effect of manual control actions. Often in machine control, the motion of the controlled machine excites motion of the human operator, which is transmitted back into the control device causing unwanted command input and sometimes instability. This phenomenon is termed biodynamic feedthrough. In operation of backhoes and excavators, it causes control performance degradation. This interaction has been observed in our experiments and by our industrial members (John Deere, Bobcat, Caterpillar). We are modeling this interaction to predict the conditions under which it will occur and understand its effects on the control system dynamics. Full-scale implementation of a small John Deere Tractor-Loader-Backhoe in our laboratory recreates the problem and will verify our models. Structural and control solutions are being sought. This project is appropriate for a center in fluid power. Because of the high forces many hydraulic devices must exert, this is a problem not experienced with other haptic interfaces (e.g., electric drives) such as in surgery or virtual reality.
See a video example of biodynamic feedthrough.
References
1. Lyons, L., Nuschke, P., Jiang, X., "Multimodal interaction in augmented reality of fluid power applications: current status and research perspective", The 7th Human in Complex Systems and the 1st Topical Symposium on Sensemaking, Greenbelt, MD. November 17-18, 2006.
2. Enes, A., Book, W., "Blended Shared Control of Zermelo's Navigation Problem", ACC2010: IEEE American Control Conference, Baltimore, MD, 2010. (submitted)
3. Enes, A. and Book, W., "Recursive Algorithm for Motion Primitive Estimation", in proceedings of IEEE International Conference on Robotics and Automation, Shanghai, China, May 2011 (submitted).
4. Enes, A. and Book, W., "Optimum Point to Point Motion of Net Velocity Constrained Manipulators," 2010 IEEE Conference on Decision and Control, Atlanta, Dec. 15-17 2010.
5. Wang, L., Book, W., Huggins, J., "A Control Approach with Application to Variable Displacement Pump", AIM 2009: IEEE/ASME International Conference on Advanced Intelligent Mechatronics, Singapore, 2009.
6. Wang. L., Book, W., Huggins, J., "Adaptive Robust Control of Hydraulic Robots with Recursive Least Squares", DSCC 2009: Dynamic Systems and Control Conference, Hollywood, 2009.
7. Enes, A. and Book, W., "Sharing Control can Increase Excavator Productivity," 52nd National Conference on Fluid Power (IFPE 2011), Las Vegas, March 2011 (to appear).
8. Wang, Longke and Wayne Book, "Applying Singular Perturbation Theory to Hydraulic Pump Control", Accepted pending revisions, IEEE/ASME Transactions on Mechatronics.
9. Wang, Longke, Wayne Book and James Huggins, "A Hydraulic Circuit for Single-rod Cylinders," Accepted ASME J. Dynamic Systems, Measurement and Controls.
10. Humphreys, H. C. and W. J. Book, "Possible Methods for Biodynamic Feedthrough Compensation in Backhoe Operation," in Fluid Power Net International 6th Annual PhD Symposium West Lafayette, IN, July 2010.
11. Humphreys, Heather, Wayne Book and James Huggins, "Compensation for Biodynamic Feedthrough in Backhoe Operation by Cab Vibration Control", in proceedings of IEEE International Conference on Robotics and Automation, Shanghai, China, May 2011 (submitted).