DISCLOSURE / FILE
A November 2010 Defense Intelligence Reference Document (DIA-08-1011-002), produced under AAWSAP, that sketches a provisional cockpit for a hypothetical vehicle with control over gravity and inertia and faster-than-light capability, while stating that the underlying physics does not yet exist.
DOW-UAP-D144, AAWSAP DIRD, Cockpits in the Era of Breakthrough Flight, November 2010
A November 2010 Defense Intelligence Reference Document (DIA-08-1011-002), produced under AAWSAP, that sketches a provisional cockpit for a hypothetical vehicle with control over gravity and inertia and faster-than-light capability, while stating that the underlying physics does not yet exist.
The Defense Intelligence Agency's Technology Warning Division (DWO-4) issued this 57-page DIRD on 1 November 2010 as one of the advanced technology reports produced under the Advanced Aerospace Weapon System Applications Program, with the author redacted as 'AAP Person 82.' Using the 2009 AIAA textbook Frontiers of Propulsion Science as its main reference, it asks what piloting would look like if a craft could move in six independent degrees of freedom, hold the crew cabin between 0 g and 1 g regardless of external acceleration, and exceed light speed within a 100-light-year radius of the Sun. The answer is a single-pilot cockpit with no windows, a pair of six-degree-of-freedom joysticks, fixed physical panels in the 'tunnel vision' zone, and a virtual surround display fed by exterior cameras. The report states on page 15 that the physics and engineering to create such situations do not yet exist, and on page 7 that its predictions were generated solely by the author and had not been debated with other scientists.
It must also be stressed that these interpretive predictions and cockpit implications are solely generated by the author and, thus, have not yet been published or debated with other scientists and engineers.p.7
Probably the most significant and perplexing difference for breakthrough-era cockpits is that the sensations of motion inside the vehicle will not necessarily match the motion of the vehicle itself.p.8
It should be emphasized that these notions are at the level of thought experiments, as opposed to being mature theory.p.14
To be explicit, the physics and engineering to create such situations do not yet exist.p.15
In other words, do not expect windows.p.15
For the sake of bracketing the scope of coverage, this study assumes that practical star flight will be limited to a 100-light-year radius around our Sun.p.6
A survey of Boeing 757 and 767 pilots in 1989 revealed that over half felt their workload had actually increased by the high levels of automation introduced into these cockpits.p.29
Regardless of the sophistication of the technology, a weak link in these systems is the human mind itself; that is, thoughts can wander.p.33
Extracted text · Page 1
Page 1 · born-digital extraction
UNCLASSIFIED//FOR OFFl@lslll:. WOE OF.I:¥ Defense Intelligence Reference Document Defense Futures 01 November 2010 ICOD : 8 Ju ly 2010 DI A-08- 1011 - 002 Cockpits in the Era of Breakthrough Flight UNCLASSIFIED//FOR 8FFl@IAI:. WOE 0 .. 1:.lf
Page 2 · born-digital extraction
UNCLASSIFIED//F8R 8FFIOIAL YSE BHLY Cockpits in the Era of Breakthrough Flight The Defense Intelligence Reference Document provides non-substantive but authoritative reference information related to intelli ence to ics or methodolo ies. Prepared by: Technology Warning Division (DW0-4) Defense Warning Office Directorate for Analysis Defense Intelligence Agency Author: AAP Person 82 Administrative Notes: (U) COPYRIGHT WARNING: Further dissemination of the photographs in this publication is not authorized . This product is one in a series of advanced technology reports produced in FY 2009 under the Defense Intelligence Agency, Defense Warning Office's Advanced Aerospace Weapon System Applications AAWSA Pro ram. Comments or questions pertaining to this document should be addressed to .... , - ...- ___-.. _,M_ E,__Person AAP Person 1 AWSA Program Manager, Defense Intelligence Agency, ATTN: JUIAF - DI/DWO-3,1 •• g , as mg on, DC 20340-5100. UNCLASSIFIEDf,/liOA OliliiliCiliOL Plltli Otill¥
Page 57 · born-digital extraction
UNCLASSIFIED/ fFOR OFFI@IAI:: HSE OPtl::Y • Woods (1993) . 36 Five References apply: • Bass, L. & Dewan, P. (Eds.). (1993) . • Elm, W. C. & Woods, D. D. (1985) . • Johannesen, L. & Woods, D. D. (1991). • Potter, S. S. & Woods, D. D. (1991). • Potter, S. S. et al. (1992) . 37 Grifantini, (2010) 38 Two references apply: • Oran, Daniel. (1991). Apply Psychology to Product Design. Machine Design. Nov. 7, 37-40 . • Stokes, Wickens, & Kite, K. (1990). 39 Woods (1984). 40 Thee references apply: • Stokes, Wickens, & Kite, K. (1990). • Sarter, N. & Woods, D. (1992) • Woods (1993) . 41 Degani & Wiener (1990). 42 Hall, Kenji. (2007). 43 Two references apply: • Cohen, A. & Chen, E. (1999). Six Degree-of-Freedom HAPTIC System as a Desktop Virtual Prototyping Interface, Cambridge, MA: SensAble Technologies, Retrieved from <http ://citeseerx. ist.psu .edu/viewdoc/download?doi= 10.1.1.122.1783&rep= repl&type=pdf> • Van der Vaart, A. J. M. (1995). Arm movements in operating rotary controls (Doctoral dissertation) . Techniche Universiteit Delft, Netherlands, Del~se Univ. Pres. 44 Four references apply: • Hall, Kenji. (2007). • Pavlovic', V. I., Sharma, R. & Huang, T. S. (1997) . Visual Interpretation of Hand Gestures for Human Computer Interaction : A Review . In IEEE Transactions on Pattern Analysis and Machine Intelligence, 19, 677- 695 • Sutter, J. D. (2010). • Underkoffler, J. (2010). 45 Morgan, B. ( 1985). DOD enlists voice control and expert systems. Electronic Products, Oct. 1, 65 -67. 46 Genik (2009) 47 Coombes, L. F. E. (2005). Control In The Sky: The Evolution and History of The Aircraft Cockpit, Pen & Sword Aviation : Leo Cooper Limited. 48 Proffitt, D. & Kauiser, M. (1991) . Observer Properties for Understanding Dynamical Displays: Capacities, Limitations, and Defaults. NASA TM 102812. Ames Research Center, Moffett Field, CA. 49 Coombes (2005). 50 Grifantini (2010). 51 Multiple possibilities: • SO Great Examples of Data Visualization. (2009) • McCandless (2009). • Shneiderman, B. (1997). • Tu~e, E. (1991) . • (VUE). (2008). 52 Van der Vaart (1995). UNCLASSIFIED/fFOA. OFFICIO ls. Plili Olhlls.¥ 52
Concatenated page-by-page transcript. Born-digital pages came through pdf.js; scanned pages were transcribed by Claude vision OCR. Pages marked unreadable failed multiple OCR retries (heavy redaction, microfilm artifacts, or blank separators) and are kept in place for audit.
UNCLASSIFIED//FOR OFFl@lslll:. WOE OF.I:¥ Defense Intelligence Reference Document Defense Futures 01 November 2010 ICOD : 8 Ju ly 2010 DI A-08- 1011 - 002 Cockpits in the Era of Breakthrough Flight UNCLASSIFIED//FOR 8FFl@IAI:. WOE 0 .. 1:.lf
UNCLASSIFIED//F8R 8FFIOIAL YSE BHLY Cockpits in the Era of Breakthrough Flight The Defense Intelligence Reference Document provides non-substantive but authoritative reference information related to intelli ence to ics or methodolo ies. Prepared by: Technology Warning Division (DW0-4) Defense Warning Office Directorate for Analysis Defense Intelligence Agency Author: AAP Person 82 Administrative Notes: (U) COPYRIGHT WARNING: Further dissemination of the photographs in this publication is not authorized . This product is one in a series of advanced technology reports produced in FY 2009 under the Defense Intelligence Agency, Defense Warning Office's Advanced Aerospace Weapon System Applications AAWSA Pro ram. Comments or questions pertaining to this document should be addressed to .... , - ...- ___-.. _,M_ E,__Person AAP Person 1 AWSA Program Manager, Defense Intelligence Agency, ATTN: JUIAF - DI/DWO-3,1 •• g , as mg on, DC 20340-5100. UNCLASSIFIEDf,/liOA OliliiliCiliOL Plltli Otill¥
UNCLASSIFIED/ ,'FOR OFFI@IAL WSE ONLY Contents Introduction .. ......................................................................................................... iv Chapter 1 : Predicting Implications of Propulsion Breakthroughs ........................... 1 MARCH OF PROGRESS: REVOLUTIONARY PROPULSION PHYSICS ....................... 1 VEHICLE and COCKPIT IMPLICATIONS ............................................................... 2 Chapter 2 : Human-Machine Interface Lessons ..................................................... 18 HUMAN PERCEPTION NORMS ........................................................................... 18 DESIGN FOR STRESS ........... ..... .. ........... ......... ...... .. ........ .... .......... ........ .... ....... ..... . 21 DEVICES TO CONVEY INFORMATION ................................................................ 23 DEVICES FOR RECEIVING PILOT COMMANDS ................................................... 27 CONTEMPORARY AIRCRAFT COCKPITS ............................................................. 28 Chapter 3 : Provisional Cockpit for Breakthrough Flight .................... .................... 32 FLIGHT MODES ................................................................................................. 32 PHYSICAL DISPLAVS .................................................................... ... ............... .. 34 VIRTUAL SURROUND DISPLAY ......................................................................... 40 CO NTROLS ........................................................................................................ 41 Chapter 4 : Future Work ........................................................................................ 43 MULTIPLE FLIGHT REGI ME GUIDANCE CONVENTIO NS ..................................... 43 VECTOR MOTION DISPLAY ............................................................................... 43 VECTOR MOTION CONTROL .............................................................................. 43 OPTIMUM MIX OF CONTROL METHODS ............................................................. 44 Appendix A: Annotated Bibliography ......... .. ......................................................... 45 Appendix B: Endnotes .......................................................................................... SO Figures Figure 1. Six Independent Degrees of Freedom ...................................................... 3 Figure 2. Comparing Conventions of Aircraft Motion .............................................. 5 UNCLASSIFIED/ fFOA. OFFICIO Is. flili Ol'lls.¥ ii
UNCLASSIFIED/ ,'FOR OFFI@IAI:: ~SE OPtl::Y Figure 3. Necessary Distinction Between External & Internal Force Environments. 6 Figure 4. Warp Drive ............................................................................................... 8 Figure 5. Hypothetical Gravitational Bias Drive ....................................................... 9 Figure 6. Inertial Frame Bias Drive and Vehicle Zones .......................................... 10 Figure 7. Typical Science Fiction Orientations ....................................................... 12 Figure 8. Cosmic Microwaves as Universal Motion Reference Frame ..................... 15 Figure 9. Human Fields of View ............................................................................. 20 Fig ure 10. Flight Deck of Contemporary Aircraft ................................................... 29 Figure 11. Contemporary Primary Flight Display ................................................... 30 Figure 12. Space Cockpit Visions Ci rca 1959 ....... .... ...... .. ................................... ... 31 Figure 13. Provisional Breakthrough- Era Cockpit .................................................. 33 Figure 14. Functional Designation of Physical Cockpit Panels ......... ....... .... ..... ....... 34 Tables Table 1 : Comparing Reaction Time to Distance Traversed at Various Speeds .. ... .. 13 UNCLASSIFIED/fFOA. OFFICIO ls. Plili ONls.¥ iii
UNCLASSIFIED/ ,'FOR OFFI@IAL WSE ONLY Cockpits in the Era of Breakthrough Flight Introduction Responding to the request to explore forefront science relevant to future cockpits for any form of aerospace craft and/or deep-space craft that is propelled by any unspecified advanced or breakthrough propulsion physics, th is report offers a provisional cockpit design that employs the following: • Predictions of propulsion physics breakthroughs. • Lessons of human-machine interface. • Emerging technology for displays and controls. This study discusses the implications of breakthrough propulsion, includ ing the mastery over gravitational and inertial forces and the prospect for faster-than-light spaceflight. The main reference used to predict these possibilities is the book Frontiers of Propulsion Science [Millis & Davis, 2009]. Although the breakthroughs discussed in this book are not imminent, enough progress has been made to allow for thoughtful speculation about their characteristics and possible implementations. How these advances may affect future cockpits is described, and this is the central message of this study. The most significant differences from legacy cockpits are identified and then used to set the baseline design requirements. Additionally, substantial lessons about human-machi ne interfaces are reviewed and applied to this notional cockpit. Most of this progress relies on better accommodating the norms and limitations of human perception-lessons that do not change even when vehicle characteristics change. Recent advancements in the use of hand gestures for commands are also included, as well as advancements in brain-machine interfaces. In this conceptual study of far-future possibilities, these technologies are assumed to have reached fu ll maturity, with one exception: in order to focus this study on future cockpits, the options for brain implants and for transhumanism-where humans are reengineered to adapt to new requirements-are not considered. Next-step investigations are suggested to refine the ideas presented herein. A caveat is that advances in cockpits for breakthrough flight might be further advanced by taking advantage of the gaming industry techniques or through science fiction speculation. Note: All projections in this report are based on public domain informati on. UNCLASSIFIED/fFOA. OFFICIO ls. Plili ONls.¥ iv
UNCLASSIFIED/ ,'FOR OFFI@IAL WSE QptLY Chapter 1: Predicting Implications of Propulsion Breakthroughs MARCH OF PROGRESS: REVOLUTIONARY PROPULSION PHYSICS Breakthroughs in propulsion physics (such as the control over gravitational or inertia l forces, propellant-less space drives, and even faster-than-light travel) are not imm inent; however, enough progress has been made to allow for thoughtful speculation about their nature and implications. As a preview, the implications to cockpit design include added degrees of motion, combination of operational regimes (near ground, orbit, and beyond), greater range of speed (from zero-speed hover to beyond light speed), and loss of familiar motion cues (pilot's inertia and visual cues) resulting from the separation of external and internal environments. The primary reference used to predict these possibilities is the book Frontiers of Propulsion Science [Millis & Davis, 2009], 1 particu larly chapters 3, 4, and 15. This book may be the first-ever scholarly compilation of science pertaining to breakthrough flight-methods sufficiently advanced to enable human voyages to other star systems. The book examines a wide range of works, offering introductory explanations and comparisons between approaches and identifying high -priority unknowns needing deeper study. References to specific ideas and issues cite that book and other orig inal works. Setting Ideal Performance as Design Target This report focuses on the most sign ificant likely differences between contemporary cockpits and cockpits in the era of breakthrough flight. Possibilities that imply the most demanding changes are considered first, and explanations of the correlations between the propulsion characteristics and resulting cockpit features are provided . Looking to the far future, this study evaluates the impact of having achieved the following breakthrough advancements: • Control over gravitational and inertial forces: - The craft is propelled by interact ing with t he properties of the space-t ime and/ or inertial frames su rround ing the craft- and can accelerate at g levels beyond human endura nce . - The environment inside a craft can be sustained anywhere between 0 g and 1 g (m inimum range) without regard for either the motion of the craft or its outside gravitational environment. • Faster-than -light (FTL) speeds are possible by hav ing mastered control over t hose aspects of nature that impose the light-speed limit. However, due to reasonable re lativistic projections of the energy required for propulsion coupled with the limits of the human lifespan, it is reasonable to expect that travels will be limited to within our galaxy. For the sake of bracketing the scope of coverage, this study assumes that practical star flight will be limited to a 100-light-year radius around our Sun. Even with this constraint, thousands of star systems are within that range. UNCLASSIFIED/fFOA: OFFICIO la Plili Oralla¥ 1
UNCLASSIFIED/ ,'FOR OFFI@IAI:: WSE ONl::Y • The energy supply for these features resides on the vehicle and is cons idered to have a dynamic interplay with the motion of the vehicle . The energy can be transferred to and from the environment surrounding the craft as a consequence of the propulsive maneuvers. Sanity Check on Predictions Objectively, the propulsion physics pred ictions offered in this report should be interpreted as informed conjectures or, at best, well-reasoned speculations. Absent of verified theories and engineering implementations, it is premature to consider this first study as the last word on this topic. Further progress will likely reduce the span of options and provide greater insight into implementation details. It must also be stressed that these interpretive predictions and cockpit implications are solely generated by the author and, thus, have not yet been published or debated with other scientists and engineers. Therefore, the reader should consider these predictions to be an initial step into the process. VEHICLE AND COCKPIT IMPLICATIONS ~deally, it is desirable to have a vehicle that can move in any direction, at any speed, in both air and space, without limitations. These features imply the need to have technological mastery over the forces of gravity and inertia and mastery over those aspects of nature that impose the light-speed limit. Based on projections of the underlying physics, such abilities would have secondary characteristics that affect how such motions are monitored and controlled. Degrees of Freedom Unlike an aircraft, whose motion consists basically of deviations from constant forward motion, or a helicopter, whose motion is dominated by t he dynamics of its main rotors, a breakthrough propulsion vehicle would allow the full six degrees of freedom, including the ability to remain fixed relative to a desired reference. For example, if we start with the situation of a vehicle hovering over the ground, the breakthrough vehicle should be able to change its orientation (yaw, pitch, or ro ll) without affecting its altitude or lateral position. Similarly, it should be able move up/down or laterally without the need to induce pitch or roll maneuvers (Figure 1). Such novel motion leads to two major differences from legacy cockpits: • Independent control inputs are needed for the full six degrees of freedom (yaw, pitch, and roll; and laterally, x [fore-aft], y [ left-rig ht], and z [up-dow n]). • New display methods are required to convey position, orientation, and motion for all those degrees of freedom. The control methods need not copy legacy methods from airplanes or helicopters methods that are based on the mechanisms of their origin (Figure 2). Instead, future cockpit designs are now free to use control methods tailored to the natural action/reaction of pilots, whi le t he vehicle's interfaces perform the function of converting pilot inputs to drive the veh icle's motion. Whether such a system consists of UNCLASSIFIED/fFOA. OFFICIO ls. Plili ONls.¥ 2
UNCLASSIFIED/ ,'FOR OFFI@IAL WSE QptLY a single joystick with six degrees of freedom, some sort of gesture-based system, or one that has those degrees of freedom dispersed across multiple pilot inputs (e.g ., head motion, legs and feet, and arms and hands) remains open for future study. As a provisional baseline, th is report chooses the option of having a pair of six-degree-of freedom joysticks, one for both the left and right hands and located at the edge of the cockpit chair's arm rests. Th ree equally avalla ble recti linear axes of motion Three equally available rotational axes of motion Figure 1, Six Independent Degrees of Freedom. [Graphic : A. Szames] Note : the vehicle shown is strictly hypothetical and is a combination of three 1960s science fiction vehicles: Seaview submarine, Galileo shuttle, and Amtronic car. Simi lar to requiring new control methods, new display methods are also required to convey more information than in legacy cockpits. In addition to the complete six degrees of freedom, these motions will take place near the Earth's surface, in orbit, and in deep space. A key difference spanning those regimes is the traditional role played by a gravitational field as a reference for orientation and motion. Since a gravitational reference will not always be present, and yet is extremely important when it is present, the new display system must accommodate all regimes in a way that feels natural to the pilots. These particular challenges are addressed in the section on Mixed Operational Regimes. Separation of Internal and External Environments Probably the most significant and perplexing difference for breakthrough-era cockpits is that the sensations of motion inside the vehicle will not necessarily match the motion of the vehicle itself. This is both a consequence of the method of propulsion as well as cockpit features designed to ensure crew survival. UNCLASSIFIED/fFOA: OFFICIO la Plili Oralla¥ 3
UNCLASSIFIED/ fFOR OFFI@IAI:: HSE OPtl::Y As illustrated in Figure 2, when planning for breakthrough flight, there is no need to constrain designs to match the legacy conventions derived from prior vehicles. In the case of both the airplane and the helicopter, the control inputs available to the pilot are specific to the mechanisms of the control surfaces. When projecting breakthough flight, it is assumed that the controls will be tailored to match the natural characteristics of pilots, and the propulsion system will be designed to perform accordingly. UNCLASSIFIED/fFOA. OFFICIO ls. Plili Olhlls.¥ 4
UNCLASSIFIED/ ,'FOR OFFI@IAL WSE QptLY 2A 28 2C Figure 2. Comparing Conventions of Aircraft Motion. [Cred it : A. Szames] UNCLASSIFIED/fFOA: OFFICIO la Plili Oralla¥ 5
UNCLASSIFIED/ ,'FOR OFFI@IAL WSE QptLY To make t his easier to grasp and to provi de a provisional co ncept , imagine tha t t he vehi cl e is partit ioned into concent ric sections as shown in Figure 3. The central vol ume is for the crew, where it is required that t he gravitational and inertial forces be sustained within su rvivable levels. The inner she ll, or inner hull, su rround in g that reg ion conta ins wha t ever devices provide that safe environment. The out er shell conta ins the propulsion devices that induce the desired motion of the craft relative to the externa l space. The reg ion between t he two hull shells is a provisional separation for ana lyzing the int eraction between those two fu nctions. Coordinate system of the Crew cabin where inertial external environment and gravitational forces are maintained at normal, safe conditions Outer shell whose propulsion interacts with provides a safe internal external environment force environment Figure 3. Necessary Distinction Between External and Internal Force Environments. [ Grap hic: A. Szames] For analytica l purposes, it is advant ageous to consider these volumes and control surfaces (the hu ll shells) as separate and t hen use t hem to define boundary cond itions . I n addition to its utility for assessing future cockpit designs, t his multisectioned vehicle baseline is valuable for gedanken experiments about revolutionary propulsion concepts . Why This Is Odd Conventionally, gravitationa l and inertial effects permeate through everyth ing , so the notion of having different conditions inside and outside of the craft runs contrary to UNCLASSIFIED/fFOA: OFFICIO la Plili Oralla¥ 6
UNCLASSIFIED/ ,'FOR OFFI@IAI:: ~SE OPtl::Y experience. If the gravitational environment outside the craft is 1 g, that same 1 g is expected inside as well. Similarly for accelerations, the entire mass of the vehicle, including its occupants, will experience the same inertial reactions as the vehicle accelerates. Upon the advent of breakthrough propulsion, mastery over gravitational and inertial forces will have been achieved. This implies, for example, that it is possible for a vehicle to accelerate at extreme g's while its crew remains within survivable limits or that, while cruising in deep space (absent of any acceleration or gravitational field), the crew can enjoy the familiar, constant 1 g upward. Because such possibilities run so contrary to established experience, it is difficult to comprehend how such things can be achieved and then contemplate the consequences that these advances impose onto other systems of the vehicle-in this case, how they affect cockpit designs. Not all known approaches to propulsion physics evoke the need for a double hull. 2 Versions that simply suggest a new thrusting mechanism and reaction mass would only need the double hull if also addressing how to provide a safe acceleration environment for the crew. As stated earlier, however, the most significant possible impacts are considered for this study. Therefore, two examples are described next for how breakthrough propulsion would require this provisional double-hull configuration. Why Double-Hull Needed: Example 1 (Warp Drive) The Alcubierre warp drive, which uses the physics from the Riemannian geometry of Einstein's general relativity, creates a "warp bubble" around the cralt, and then this bubble of space-time is moved through the surrounding space-time . This effect is created (in theory) by expanding space-time behind the vehicle and contracting space time in front. The vehicle within the bubble feels no acceleration forces. As illustrated in Figure 4, 3 the high peaks represent expanding space-time, while the low peaks represent contracting space-time. Note that the inner region is flat (meaning that the vehicle does not experience any acceleration forces), that the region far from the propulsion effect is also flat, and that these two regions are separate. UNCLASSIFIED/fFOA. OFFICIO ls. Plili ONls.¥ 7
UNCLASSIFIED/ ,'FOR OFFI@IAI:: ~SE OPtl::Y Figure 4. Warp Drive. Taken directly from the Frontiers book, this image has become the iconic representation for a warp drive : the "York Extrinsic Time Plot." In theory, the outer shell is presumed to create the propulsive effect of warping space time outside the craft without affecting the inside. In short, the Alcubierre warp drive creates a separation of space-time environments inside and outside of the craft, although the exact details remain uncertain. Although there is no explicit function for the inner hull in this situation, this Alcubierre warp drive at least illustrates the concept of separation of these outer and inner environments. When planning future cockpits, the outer and inner inertial frames need to be treated as two distinct zones. The physics related to such considerations is still evolving 4 and beyond the scope of this report. Why Double Hull Needed: Example 2 {Field Space Drive) Another class of conceptual propulsion is a "field drive"-a subset of "space drives" where a spacecraft is propelled " ... using only the interactions between the spacecraft and its surrounding space ... " 5 Instead of using the Riemannian geometry of Einstein's general relativity, these approaches use the physics of fields and scalar potentials. 6 While several variants exist, the "Bias Drive" concept is selected here to illustrate the relevance of the double-hull configuration, specifically in the context of modifying the scalar potential that defines an inertial frame. By altering the properties of the surrounding inertial scalar, a gradient in that scalar is induced which, in turn, induces gravitational-like forces on matter located in that gradient. UNCLASSIFIED/fFOA. OFFICIO ls. Plili ONls.¥ 8UNCLASSIFIED/ fFOR OFFI@IAI:: HSE OPtl::Y Figure 5 represents one version of this effect/ where the vehicle and its contents would be located at the steep gradient and therefore would jointly experience acceleration forces. Taken directly from the Frontiers book, this image represents what would happen if it were possible to asymmetrically modify Newton's gravitational constant to induce gravitational gradients that would then, in turn, accelerate the vehicle. Despite the similarity to the distortions in the warp drive (Figure 4), these surfaces represent scalar potentials of gravitational fields. Figure 6 is a modified version of this where the double-hull configuration of Figure 3 is imposed, along with the condition that the inner region remains unaffected. In Figure 6, the locations of the inner and outer shell are identified on the figure. The notion of the gravitational bias drive has been modified here to illustrate the concept of having two separate zones of inertial and gravitational forces. In this case, the plotted surface represents a scalar potential of an inertial frame, where a gradient has the same effect as a gravitational field. The smaller central flat area is the inside of the vehicle where no acceleration forces are felt. The outer edges of the diagram are also flat, representing the unaffected space sufficiently far from the propulsive effect. The distorted regions in between represent the effects of both the outer and inner hull shells. The outer hull shell induces a gradient that propels the vehicle, and the inner shell acts to prevent those distortions from reaching the crew cabin. It should be emphasized that these notions are at the level of thought experiments, as opposed to being mature theory. a) Multiplicative modification, B, Eq. (39a) b) Exponential modification, b, Eq. (39b) Figure 5. Hypothetical Gravitational Bias Drive. In short, this Inertial Frame Bias Drive has affected the space both outside and inside of the craft to propel the veh icle and to keep its crew isolated from the resulting acceleration forces. For example, if the outer hull creates a 10-g field outside the craft, the inner hull would compensate to create an opposing field such that the crew cabin is free of the 10-g acceleration forces. It can, therefore, be speculated that-if such field affecting physics is discovered-the inner shell could create a 1-g field when the craft is coasting in 0-g deep space. UNCLASSIFIED/fFOA. OFFICIO ls. Plili Olhlls.¥ 9
UNCLASSIFIED/ ,'FOR OFFI@IAL WSE QptLY Outer Hull Inner Hull Crew Safe Zone Figure 6. Inertial Frame Bias Drive and Vehicle Zones. [Credit : M. Millis] To be explicit, the physics and engineering to create such situations do not yet exist. The related physics can be categorized as still being at steps one and two of the scientific method: defining the problem and collecting data. 8 Among many other issues eluding discovery and resolution, major issues include momentum conservation, the role played by inertial frames, and methods to affect gravitational and inertial properties of matter and space. Secondary Consequences Pertinent to cockpit design, the normal sensations of motion inside the cockpit will likely not be the same. In contrast to the advantage of shielding the crew from harsh maneuvers, this shielding removes sensations of motion (seat-of-pants feeling) that pilots use to help judge the motion of their vehicle. This detriment is compounded by the likelihood of inducing motion sickness, since the visual cues of the vehicle's real motion will be different from that felt by the pilot. A difference between visual and vestibular cues is a cause of motion sickness. The option of allowing a certain portion of the vehicle's g-loading to be transferred to the cockpit can be considered as a mitigation strategy. Accordingly, cockpit controls to affect such changes are required. Also, it is likely that this double-hull notion would prevent direct visual contact between the occupants and the environment outside the vehicle, or perhaps distort such visual cues beyond easy interpretation. In other words, do not expect windows. Without the famil iar visual and vestibular cues directly available to the pilot, it becomes vitally important for the cockpit displays to provide re liable and instinctive cues for the pilot to aptly judge the position, orientation, and motion of the vehicle. Mixed Operational Regimes A major desirable feature sought from propulsion breakthroughs is the ability to move from the surface of the Earth directly into space. This implies that the veh icle's displays and controls must readily encompass motion near the Earth's surface, ascent into space, UNCLASSIFIED/fFOA: OFFICIO la Plili Oralla¥ 10
UNCLASSIFIED/ ,'FOR OFFI@IAI:: ~SE OPtl::Y transitions into and out of orbits, and long-duration sustained cruising in a 0-g environment. As alluded to earlier, this deviates from prior displays where the Earth's gravitational field is available from which to gauge orientation. Similarly, the notion of an altimeter takes on a whole new meaning in this context. While visual cues for "up" are instinctively clear near the surface of the Earth (or even in closed rooms where 1 g is present), for a true breakthrough vehicle, these will be special conditions amongst a greater span of possibilities. A particular consequence of these added operational regimes is that unfamiliar situations are presented that must be made easy for the pilot to comprehend. Human instincts of motion and perception are honed from living in a 1-g environment with the majority of motions constrained to the (comparatively) two-dimensional ground. Also, lacking eyes in the back of our heads, our natural sense of attention is focused forward. While these instinctual characteristics serve well in travel near the ground, they do not apply to orbits or to deep-space flight. Orbit Orbits around the Earth-or any gravitating body, for that matter-present stable, constant energy situations. Orbits are convenient parking locations. A vehicle does not need to expend energy to stay in orbit indefinitely (unless drag forces from the atmosphere or long extensions of the vehicle come into play). Orbits, therefore, are common trajectories to select when loitering near gravitating bodies. But so far in the course of human evolution, developing an innate sense of placing a vehicle into an orbit does not exist. Although a human can instinctively run at just the right speed and direction to catch a ball thrown toward them, such natural instincts do not apply to placing a vehicle in orbit. Therefore, display systems will be required to provide readily interpretable cues for the pilots to transition into orbital flight. This implies presenting the natural relations between orbital altitude and orbital speed. This challenge is compounded since such cues must naturally blend with the motion cues used when flying near the surface. Deep-Space and Interstellar Flight Deep-space flight adds yet another challenge; namely, the almost total absence of familiar cues for motion, position, and orientation. Given the extremely large distances between astronomical objects and that relativistic effects do not become significant (>1% distortions) until reaching beyond 10% of light speed, the view outside the craft will appear stationary-even when traveling at 60 million miles per hour (9% c). The display systems that are tied to the navigation references (to be discussed later) must convey motion to the pilots in a natural manner despite the absence of familiar human cues. Compounding the absence of a sense of motion, there is an absence of orientation. There is no dominant direction for "up" during deep-space flight. If some form of artificial or synthetic gravity is provided for long-duration crew health, then that internal 1 g will create the most dominant sense of "up" for the crew, and the display UNCLASSIFIED/fFOA. OFFICIO ls. Plili ONls.¥ 11
UNCLASSIFIED/ ,'FOR OFFI@IAI:: ~SE OPtl::Y system that conveys the spacecraft's orientation relative to the external space will have to be clea r enough to overcome this prejudicial sense of orientation. Notice, for example, that in almost all science fiction stories, spacecraft move laterally (forward) relative to the veh icles' internal sense of "up" {Figure 7). Motion along the z axis is seldom mentioned. Althoug h this is a natural extension of how we move relative to the surface of the Earth, it is not the only scenario. In contrast, consider a rocket whose 1-g orientation is aligned with its major axis of motion. Th is is a consequence of its propulsive thrust. In other words, at least two conventions for direction during deep space flight are possible : the notion of lateral motion across a landscape (where the internal 1 g is at right ang les to flight), or vertical motion with an astronomica l range (where the internal 1 g is coincident with the direction of flight). Conven iently matching film studio cond itions, the interiors of fictional spacecraft provide a comfortable 1-g environment for the crew. They also fo ll ow the terrestrial convention of motion: their major direction of motion is forward (a lateral motion), even though they are experiencing an acceleration force of 1 g upward (their internal, synthetic gravity). These two directions, up and forward, are at a right angle. In contrast, the thrusting direction and the internal g-axis of a rocket are in the same direction. The choice of orientation for real deep-space motion is a subject for further study. FORWARD (Externally) Figure 7. Typical Science Fiction Orientations. [Images: A. Szames] Since propulsion breakthroughs have not yet been discovered, there is no way of knowing if the propulsion methods themselves will dictate the choice of orientation. Therefore, to plan for the uncertain future, this is a choice worthy of deeper study. Is the natural human instinct for forward-dominated motion a better human-machine interaction than the upward-dominated motion that might be dictated by the propulsion method? Such an assessment must also consider how well the convention works when transitioning from deep-space flight into orbit, then landing, and then back again to deep-space flight. Once any convention is set into place, it will be difficult to change later. UNCLASSIFIED/,<FOA. OFFICIO ls Plili ONI.¥ 12UNCLASSIFIED/ ,'FOR OFFI@IAI:: WSE ONl::Y Crew Size Considerations The last aspect to t ake into account as a conseq uence of m ixed ope rational regimes is that of the crew size. For short-durati on missions (less than a few hours), it is reasonable to conceive of vehi cles with only one pil ot. For more complex missions, add it ional crew will be required, and thus addit ional displays and controls specific to thei r tasks wil l be req uired . Finally, for long-duration missions, sufficient crew will be required to carry out its mission and maintain optimal vehicle performa nce. These changes-for accommodating the roles and responsibilities of crew in relation to the overall mission-are likely to be the same as those disti nctions in trad itional vehicles (e.g. , cars versus cruise ships). Those changes typica lly incl ude a hierarchica l orga nization, wh ich is independent of the issues of propulsion physics . Essential elements wi ll include monitoring and controll ing the 1-g internal life-support environment as wel l as ensuri ng the long-t erm health of t he crew . Full Span of Speeds I n add ition to inertial effects previously addressed, t he implicat ions due to high speed remain. Accommodating the reaction t ime of the pilot is critical. The extreme high speed of breakthrough spacecraft will demand that aut omated flight controls take precedence over the pilot's manual flig ht control. Automated controls for aircraft and even for automobiles are an ever-improving technology. For breakthrough fli ght, these technolog ies will be mandatory and will also have to include options for maneuvering near ground, into orbits, and t hrough deep space. This should come as no surp rise, since the advantages of having automated flight controls warrant their use even if pilot reaction times were not an issue. Table 1. Comparing Reaction Time to Distance Traversed at Various Speeds 1 Speed Distance Traversed in l Second mph km/h C Feet Meters Miles Km Walking 2 3 3 1 Driving Around Town 40 64 60 18 Commercial Air Flight 500 800 730 220 Hyperson ic Flight Low Earth Orbit 4,000 17,500 6,400 28,000 0.00001 0.00003 5,900 26,000 1,800 7,800 1 5 2 8 Deep-Space Probe 35,000 56,000 0.00005 51,000 16,000 10 16 Non relativist ic Flight 60 Million 97 Million 0.09 89 Million 27 Million 17,000 27,000 Relativist ic Flight 400 Million 650 Million 0.60 590 Mi llion 180 Million 110 Thousand 180 Thousand 1 The distances traversed while waiting for the pilot to react are reasonable for speeds slower than hypersonic fl ight. If traveling at hypersonic speeds near the ground, however, the situation is different. At some point, regardless of the skill of the pilot, an automated system will be needed . Also note the huge disparity between the fastest achieved speeds (deep-space probe) in comparison to nonrelativistic flight. This disparity of three orders of magnitude is a clear statement about the state of our technology when contemplating deep-space flight. UNC LASSIFIE D/fFOA. OFFICIO ls. Plili ONls.¥ 13
UNCLASSIFIED/ ,'FOR OFFI@IAI:: ~SE OPtl::Y Extreme Relativistic 660 Million 1.07 Billion 0.99 970 Million 270 Million 180 Thousand 297 Thousand Light Speed 670 Million 1.08 Billion 1 980 Million 300 Million 190 Thousand 300 Thousand Faster Than Light? 13 Billion 22 Billion 20 202 Billion 6 Billion 4 Million 6 Million Table 1 is designed to put a pilot's reaction time into perspective; it compares distances traversed during the one second it takes the pilot to scan and comprehend his displays ("dwell time") and then react. 9 In addition, it will take time for those commanded changes to take effect, but those durations are not known. The distances shown in the table are those traversed before any corrective actions are initiated. If these distances are determined to be excessive, then automated flight controls are mandatory. Another aspect resulting from the effects of ultrahigh vehicle speed is the so-called "relativistic twin paradox. 1110 Because of relativistic effects, there will be a mismatch between the time measured aboard the craft and that measured at its base of departure. The equations to track this situation are well established. 11 The challenge is how to present this information so that both the crew and the mission personnel at the base can easily comprehend the implications. More provocative than the implications of relativistic speeds is the possibility of faster than-light (FTL) travel. Beyond the perplexing issues of causal violations and closed time-like curves inherent with all FTL notions to date, 12 there is the question of tracking position, orientation, and motion when beyond light speed. It is reasonable to assume that when a vehicle is traveling FTL, the normal flow of electromagnetic waves (i.e., light) to and from the craft will be cut off. To better visualize this, consider the Doppler shifts as a vehicle approaches light speed. The colors of light heading into the flight path will be shifted to such a short wavelength that it will cease to be detectable. Similarly, the light approaching the rear of the craft will red-shift so much that it also ceases to be detectable. Again, do not count on windows. Navigation References The main difference between navigating with existing vehicles and breakthrough vehicles is that the breakthrough vehicles will have to navigate in deep space and around other astronomical bodies where GPS systems and location beacons do not exist. Another major difference is that the physics of the propulsion methods might distort or block information that is traditionally used for navigation. Inertial Navigation (Acceleration Measurement) In inertial guidance systems, accelerometers and ring laser gyros accurately track the changes in the vehicle's motion (lateral and rotational accelerations). These signals are integrated to keep track of position by evaluating changes in both velocity and acceleration. In the case of the double hull, where the inertial effects might be different inside of the craft, these tools become more diffi cult to apply. If we assume that the physics and UNCLASSIFIED/,<FOA OFFICIO ls Plili ONI.¥ 14UNCLASSIFIED/ ,'FOR OFFI@IAL WSE QptLY technology for manipulating inertial fields ( or for warping space-time) can accurately track these effects, then that knowledge may compensate to keep these tools viable. Design of future guidance systems must address this issue. Absolute Velocity - Universal Speedometer Conven iently, nature provides another reference frame for deep-space navigation. The cosmic microwave background is a reference frame against which velocity can be measured relative to the mean rest frame of the universe. By comparing fore/aft Doppler shifts relative to th is highly isotropic and homogeneous rad iation, velocity measurements can be derived. For example, the net velocity of the Earth's motion relative to this background has been measured to be 365 km/s. 13 However, the cosm ic microwave background will not be detectable at FTL speeds. As illustrated in Figure 8, although many of the pictures of the cosmic microwave background radiation remove the prominent dipole moment shown in this graphic (the major color difference), it is precisely this dipole-the difference between fore/aft Doppler shifts-that provides a navigation reference for deep-space flight. The projection of this image is a spherical shell that has been opened and flattened. The Doppler shift corresponds to the Earth's motion of over 1.3 million km/h relative to the mean rest frame of the universe. The Earth moves in the direction away from the red and toward the blue. Figure 8. Cosmic Microwaves as Universal Motion Reference Frame. [Cred it : NASA] Position-Reference Star Trackers For deep-space flight, the star trackers that have been developed for existing spacecraft may still prove viable, new instrumentation will probably be required. Given the enormous expanse of space, the apparent locations of stars will not vary that much .14 Even those that do appear to move-our closest stars-are known well enough so that software can take into account how those positions will change as the vehicle's position changes. Due to Doppler shifting, as noted previously, checking positions relative to the stars will only be possible at sublight speed. For speeds approaching light speed, corrections will be required for relativistic effects. Such effects will probably not become apparent until traveling well beyond about 9% c, UNCLASSIFIED/fFOA: OFFICIO la Plili Oralla¥ 15
UNCLASSIFIED/ ,'FOR OFFI@IAI:: ~SE OPtl::Y which is a speed that is still three orders of magnitude beyond the highest speeds achieved to date. Another modification for star trackers will be required for FTL travel. In essence, with FTL flight, the vehicle arrives at the destination ahead of time-in an unfamiliar way. To understand this, recall that all information we see from the cosmos is old. Those images have taken a while to reach us, and the reality at their point of emission has continued forward in time. For example, when we see sunlight, the image is more than 8 minutes old. The images we see from Alpha Centauri show what it looked like over 4 years ago. Thus, if we could zip to Alpha Centauri instantly, over 4 years of time would have elapsed since we last looked at it. Alpha Centauri's condition will be a surprise upon arrival. Therefore, any star tracker to accompany FTL flight must take into account the trajectories of astronomical objects so that their positions can be accurately predicted to correspond to the correct time of arrival in both spatial and temporal coordinates. There is no known precedent for this situation. In support of the forgoing discussion, we are speculating that heretofore unknown advances in physics regarding the quantum vacuum and the nature of inertial frames will result in new motion-detection technology. In researching future propulsion breakthroughs, the utility of sensing and affecting such phenomena is pertinent. Compilations of Implications The following list is a compi lation of the characteristics discussed in this section about the possible features associated with breakthrough flight. While the list is admittedly incomplete, it conveys the most significant differences compared to conventiona l methods of flight. • Six degrees of independent motion/orientation: - Translational motion: fore/aft, left/right, up/down. - Rotational (orientation): pitch, yaw, roll. • Distinct inner and outer environments for inertial and gravitational forces. • Speeds encompassing zero, subrelativistic ( <0.1 c), relativistic (0.1 c ~ v < 1.0 c), and beyond light-speed, yet expecting a travel limit of about a 100-light-year radius around the Sun. • Three flight regimes: - Near the surface of gravitating body (where gravitational direction provides natural orientation). - Orbits around a gravitating body (where cues for entering orbit are required for the pilot). - Deep-space flight (without obvious orientation cues or obvious sense of motion). UNCLASSIFIED/fFOA. OFFICIO ls. Plili ONls.¥ 16
UNCLASSIFIED/ fFOR OFFI@IAI:: HSE OPtl::Y • Navigational information sources: - Position: • Master reference taken relative to starting position (Sun-Earth system). • Current location taken from the following: o Star tracker: • Modified to handle 3D database of star locations for deep-space motion (yet less than 100-light-year radius around the Sun). • Predictive trajectories (to extrapolate positions for FTL travel). • Note: Can only take star tracker readings at sublight speed. o First integration on measured velocity and relative to point of departure. o Second integration on measured accelerations since point of departure. - Velocity: • Directly measured from cosmic microwave background Doppler shifts, where velocity is relative to the mean rest frame of the universe. • First integration on measured accelerations since point of departure. - Accelerations: • Linear: o Accelerometers with corrections calculated based on the influence of the propulsion methods that affect gravitational and inertial forces. o Differentiation of velocity changes as measured using the cosmic microwave background. • Rotational: o Gyros (e.g., ring laser gyros) with correction inputs from the propulsion methods that affect gravitational and inertial forces. o Orientation as inferred from star tracker measurements. UNCLASSIFIED/fFOA. OFFICIO ls. Plili Olhlls.¥ 17
UNCLASSIFIED/ ,'FOR OFFI@IAI:: WSE ONl::Y Chapter 2: Human-Machine Interface Lessons Over recent decades, substantial improvements have been made to human-machine interfaces. 15 Most of this progress relies on better accommodating the norms and limits of human perception-lessons that do not change even when vehicle characteristics change. These lessons are reviewed in this chapter and then applied in the conceptual design offered in the third chapter. Examples of such characteristics include reaction times, tunnel vision under stress, instinctual association with position, interpretations of displayed colors, and lessons learned from interactions with display and control technologies. Recent progress on augmented rea lity displays, 16 voice control, 17 gesture-based computer inputs, 18 and brain-machine interfaces (BMis) 19 are also considered. In this study of far-future possibilities, these technologies are assumed to have reached full maturity. Instead of going into the details of their status, only their implications will be addressed here. One exception was made when considering emerging technologies-specifically the notion of modifying humans for breakthrough flight. This exception includes brain implants for BMis and reengineering humans (transhumanism) to adapt to new requirements. 20 Rather than requiring humans to be reengineered for breakthrough flight, this study focuses on adapting the cockpit to address natural human characteristics. This forces attention on the cockpit design requirements. Numerous references about human factors were consulted, focusing on those details most relevant to this study. Since similar assertions were echoed in many of these references, it is not always clear how to trace a given assertion to a specific reference. Instead, an annotated bibliography is included at the end of this report that has short descriptions of each reference. HUMAN PERCEPTION NORMS Physical Object Analogs Human interpretation mechanisms are rooted in the paradigm of a physical environment. Mimicking a physical environment in a display and control system enhances comprehension and allows features to be recognized with less effort t han when translating dial s, bar graphs, or alphanumeric displays. 21 Th is not only pertains to perceptions of motion, but also to t he comprehension of a vehicle's operation: understanding its lim its, supplies of consumables, malfunction modes, and other parameters . Accordingly, humans naturally remember where to look for a particular piece of information or what direction to flip a particular switch. Humans are also adept at subconsciously applying models of social behavior and causal events. Hierarchies, similar to societal organizations, offer a natural model with which to categorize systems and subsystems. Causal relations (cause leads effect) match well with procedures and operational flow diagrams. For more abstract concepts and complex data sets, a combination of hierarchical and causal relations can be used, typically taking the form UNCLASSIFIED/fFOA. OFFICIO ls. Plili ONls.¥ 18
UNCLASSIFIED/ /FOR OFFICIAL USE ONLY of conceptual maps. Numerous templates for these tools are readily available and many are adaptable to gesture-based interaction .22 Color Color is convenient for labeling different categories of information, providing status indications, and for adding redundant (corroborating) information to minimize errors. Humans mentally process color information first and do so more accurately and with less effort than when distinguishing shapes or alphanumeric indicators. The upper limit on the recommended number of colors to simultaneously display is only about 4 to 12 colors. To augment this number, it is acceptable to adjust the saturation or brightness of a given color to represent gradations, such as terrain height or the criticality of a variable. 23 Although research has not progressed to the point where absolute recommendations can be made for the use of colors, the tripartite system is a widely recognized and easily discernable status indicator: • Red = danger. • Yellow = caution. • Green = safe to proceed. Conveniently, fewer pilots have colorblindness than the general public, and thus color coded displays can be used more effectively than with the general public (data: 8% of males and 0.4% of females are colorblind to red-green or blue-yellow distinctions). 24 Other lessons regarding the use of color include the following :25 • Use strong colors sparingly: strong colors draw attention, and too many of them can overstimu late the user. • Yellow is a rare color since it is simultaneously soft and intense. • The color of objects is more discernable than the color of text or lines. • Use soft colors (grays, pastels) for backgrounds and large areas. • A good color choice is one where that color is not conspicuous. • Use soft contrasts or graduated contrasts between areas, since hard contrasts can appear to vibrate. • Avoid having the color white next to strong colors since it is too much of a contrast. • Avoid framing a box of text since that creates clutter. Instead, place the text in a box with a different fill color. • If needed to strengthen boarders between softly colored areas, frame the area with a darker shade of its fill color. UNCLASSIFIED/fFOA. OFFICIO ls. Plili ONls.¥ 19
UNCLASSIFIED/ ,'FOR OFFI@IAL WSE QptLY Fields of View There is a difference between what humans are physically capable of seeing and where they naturally concentrate their attention. Using a reference point that is stra ight ahead and level with the eyes, the normal full field of view extends laterally (left/right) ±100°, upward about +60°, and downward about -75°. Of course, turning the head can extend the lateral reach directly behind, but that rearward view would be in the edge of periphery vision. This full span, however, is not where humans normally direct their attention. With the head in a fixed position, the normal field of view spans left/right ±100°. The periphery is most sensitive to motion even when discrete images cannot be resolved. The region shown that spans roughly ±60° is the area that can be easily scanned by moving the eyes and whose information can be mentally processed in parallel. A much narrower field, about ±6 °, is where humans can discern details and begin processing information serially. This zone can be aimed anywhere in the normal field of view. Finally, the region of focus where alphanumerics can be read is only about ±2°, which corresponds to only one square inch at a distance of 2 feet. Another characteristic of human vision is the tendency to fixate in the upper right corner of a given display. In the figure insert, the percentages shown refer to the proportion of total time that the eye tends to fixate on those regions. Figure 9. Human Fields of View. [Graphic: A. Sza mes] UNCLASSIFIED/fFOA: OFFICIO la Plili Oralla¥ 20
UNCLASSIFIED/ ,'FOR OFFI@IAL WSE ONLY Peripheral Vision Peripheral vision has been used with some success to display rates of change and artificial horizons in aircraft. Evidence indicates that peripheral vision can process spatial information in parallel without appreciable mental attention. Also, evidence indicates that peripheral spatial cues (e.g., artificial horizon laser trace) may still be subconsciously processed during stress-induced tunnel vision, even though the pilot is no longer consciously noticing it. 26 Attention In the absence of stimuli, visual attention is spread evenly across the full field of view, and that information is processed subconsciously in parallel. But in the event of motion or noise, visual attention aims toward those changes, and then that information is processed serially. In t he context of cockpits, this means that all the panels and windows that are normally in the field of view are processed subconsciously in parallel, and a change in any one of those will be noticed, thus drawing attention to that change. Once attention has been triggered, the information is processed more serially. Blinking lig hts are a common way to draw attention. Sound can also be used, and the combination of sound and lig hts is common ly used in malfunction enunciator panels. It is possible, however, to saturate the pilot with too many blinking lights and sounds. Although firm values are not established, it is recommended to keep such functions to a minimum-preferably tied to the highest-priority status indications. Another method to draw the attention is through physical feeling. Vibrations or a change in feel of the vehicle will get the attention of the pilot, and with experience, the correlation between physical sensations and the status of the vehicle can become second nature. Historica lly, there are many instances where the pilots were innately able to sense changes in operating condition of the vehicle just through feel. In addition to naturally created sensations, having deliberate vibrations built into t he seat is another option for sending information to the pilot. Force-feedback controls have also been found helpful, where the degree of resistance or vibration fed back through a control (e.g., joystick and peda ls) provides interpretable information that can be mentally processed in parallel. Upon the advent of control over gravitational and inertial forces, it will likely become possible to deliberately provide the pilot with vestibular cues-mimicking inertial accelerations in association with the external conditions, but at survivable levels. DESIGN FOR STRESS The most important time for the pilot-machine interface to work optimally is in moments of crisis. Thus, as a starting point for cockpit design, it is best to focus on the highest-priority information and controls and to present those displays and controls in a manner that accommodates human norms during stress. 27 Accordingly, this section covers human limits and errors and advice for providing alarms and response options. UNCLASSIFIED/ fFOA. OFFICIO Is. flili Ol'lls.¥ 21
UNCLASSIFIED/ ,'FOR OFFI@IAL WSE QptLY Dwell and Reaction Time It takes about 0.4 to 0.6 seconds of dwell time at a particular display to extract the necessary quantitative information and at least 0.125 to 0.2 seconds for a qualitative recheck to reaffirm the reading has not significantly changed . In addition, the pilot needs another 0.125 to 0.2 seconds to act on that information. 28 Taken together, this creates a total response time of 0. 7 to 1.0 seconds between first looking at a display and commanding the appropriate response. As an aside, displays whose update rates exceed this dwell time cannot be accurately read. This means that displays shou ld be slowed down to the rate at which humans can absorb their information, rough ly a half-second. Tunnel Vision While under stress, humans t end to narrow their visual attention, often fixating on a single central display or task : the greater the stress, the greater the narrowing . Humans are oblivious to this effect as it is happening. (As an aside, hypoxia - breathing insufficient levels of oxygen-induces the same effect and can be used to simulate this cond ition during train ing.) Periphera l vision is ignored, although some evidence suggests that spatial peripheral cues are subconsciously retained. This tunnel vision tendency cannot be prevented, but can be accommodated by providing the most critical information on the central display in a natural symbolic format. 29 Forgetful Visual Scanning Humans tend to lose track of when they last looked at a particular area of information, sometimes forgetting to recheck parameters when needed. 30 This is where automated checklists can help, 31 as well as vehicle management systems that process the vehicle's situation and then recommend the best corrective measures. Limiting Options Humans tend to be able to retain only about 3 to 10 different items in short-term memory. Some studies focus on seven items as the optimum. Thus, it is recommended in quick-selection menus, or when designing hierarchical categorizations, to have no more than four to seven items per level. For groupings of information that are not needed during critical moments, these constraints can be relaxed, but with the added consequence of requiring significantly more browsing time. Adaptive Display Errors In computer displays having mu ltiple layered windows, or where the display changes in different situations, it is common that a user will lose track of how their current display image relates to the whole system. 32 Th is problem is called "getting lost" or referred to as a "keyhole" error. To prevent this error, it is best to simultaneously display some schematic indicator of where the user is in the system. Th is requires a pictorial representation of the relations of the system's operational windows . This situation is consistent with the evidence that humans tend to rely on a physical model for where to UNCLASSIFIED/fFOA: OFFICIO la Plili Oralla¥ 22
UNCLASSIFIED/ ,'FOR OFFI@IAL WSE QptLY look for information rather than being proficient at remembering a sequence of steps from which to retrieve information. In systems having different operating modes, a common error is for an operator to execute a command sequence that is inappropriate for the mode they are currently using but entirely correct for a different mode. 33 This type of error is called a "mode error." An example of this is when a pilot enters a new heading for the autopilot to follow when the plane is not in autopilot mode. To prevent this error, it is advised to have the most critical operating modes as fixed, physical displays. Other Distractions It is important to remember that the cockpit might not always offer a smooth, distraction-free ride. Buffeting can cause a finger to press the wrong button (or a gesture-based command to misdirect), or the eyes might not be able to resolve a particular value. Excessive use of audible alarms and blinking lights can saturate the pilot. Other activities or distractions available to the pilot must be taken into account so that the most critical functions are easy to find and operate, including sufficiently large buttons and text. Alarms and Responses Traditionally, physical enunciator panels combined with audible alarms and blinking lights were used to highlight malfunctions. In addition to fixed enunciator panels, more complex systems can now computationally analyze a number of variables and only present the most pertinent values and alarm states. DEVICES TO CONVEY IN FORMATION Observation and interpretation of displays is always secondary to the primary task of actually operating the system. Thus, it is important that displays be designed to first serve the user and to take on as much of the information-processing burden as possible. Also, it is suggested that the displays and control input devices should be designed according to the operator's preferences, as opposed to the more common practice of designing based on the system engineer's expectations. The following paragraphs describe lessons learned regarding various methods of conveying information to the pilot. 34 Fixed and Adaptive Displays Fixed displays, such as the gauges of older aircraft, do not change the type of information they display nor do they morph their format. These can be dials, bar graphs, an artificial horizon, or simple indicator lights. Such fixed displays can be projected with computerized screens where several interrelated values can be combined into a single, computed value. Adaptive displays, on the other hand, can change their formats or change the topics displayed to fit operational situations or individuals. Both have their respective optimum uses. During the advent of adaptive displays (circa 1990s), they tended to be overused. The advantage is that the information can be tailored to minimize the volume of information displayed at any one time. The disadvantages are that such systems require more UNCLASSIFIED/fFOA: OFFICIO la Plili Oralla¥ 23
UNCLASSIFIED/ ,'FOR OFFI@IAI:: ~SE OPtl::Y effort on the part of the operator and are more prone to mode errors and keyhole effects, particularly during high-stress situations. It was also found that automatically changing display formats added anxiety and mistrust of the system, because the user was now burdened with the additional task of determining why the display just changed. A survey of Boeing 757 and 767 pilots in 1989 revealed that over half felt their workload had actually increased by the high levels of automation introduced into these cockpits. In practice, users forced adaptive displays into fixed displays to improve their utility. 35 The ultimate lesson is that the most critical displays should be fixed to take advantage of the naturally efficient search method where humans use mental models that are analogous to physical objects. For other functions that are less time-critical and where further details are required (e.g., entering destinations and way points into a flight plan), adaptive displays are more advantageous because they require less console space. Other lessons include the following: 36 • Following the same recommendations for the individual elements of adaptive displays that are given for graphical and alphanumeric display components. • Matching the performance characteristics of the displays to the operator's own internal model of the system, and they should be structured to accommodate a user's knowledge. • Packaging information into appropriate, meaningful clusters. • Representing logical and causal relationships should be implemented. • Using "fewer displayed variables with many states" is recommended over the display of "many variables with few states." This necessitates arranging variables in a hierarchy where the lower-level variables are combined into higher-level variables. • Using color to improve the speed of human comprehension. • Accessing back-up information should be easy (familiar procedure), and it should be easy to browse through the system without disturbing its operation. Virtual Displays and Augmented Reality An extension of the adaptive screen is a virtual display where the information is projected into a field of view without the need for a physical display screen. These can function as adaptive displays or be used with gesture-based input systems. Although far-future cockpit technology is still being researched, this study assumes it to be fully matured, such that any information can be displayed clearly in the pilot's full field of view. That being said, the other recommendations about accommodating human norms and the lessons of adaptive displays should be heeded. By that, it is meant that the critical information should remain consistent, and the functions that can tolerate slower user actions can be accommodated with pop-up screens. Also, rather than relying solely on a virtual display, the critical displays should still have a physical, redundant set of displays. UNCLASSIFIED/,<FOA OFFICIO ls Plili ONI.¥ 24
UNCLASSIFIED/ ,'FOR OFFI@IAI:: WSE ONl::Y A key feature of virtua l displays will be the inclusion of augmented reality. This means that key elements in the field of view are highlighted with color or framing to draw attention 37 to them. This includes adding obvious border lines to objects for which the pilot needs to be aware . Similarly, a virtual display of the environment outside the craft could be augmented to revea l cha racteristics outside the range of normal human vision such as infrared and ultraviolet light, or objects too small or fast to normally be noticed. Graphical and Alphanumeric Representations Pictorial displays in combination with alphanumeric displays have been found to be an effective way to convey information to a pilot. A pictorial representation of the item or cond ition being controlled (maps, artificial horizons, graphical representations of vehicle stores, and others) takes advantage of the interpretation paradigm of physical models, and thus enhances comprehension and allows features to be recognized with less effort. Although reading alphanumeric displays are more time consum ing and workload intensive, they provide more accurate values and are more easily relayed verbally to secondary users (i.e., the value can be read aloud). It has also been found effective to place the alphanumeric values in the upper-right corner of the graphical display. Other lessons include the following: 38 • Using consistent text, format, placements, axes, and other parameters. 39 Consistency has been found to be more desirable than optimized displays. 40 • Displaying parameters relative to their expected va lues rather than just displaying the alphanumeric va lue. Some indication of the criticality of an off-nominal reading also needs to be displayed. • Using zero-value points as a reference and showing range settings on graphical displays to indicate how a given value compares to the desired or pred icted value; all ranges should be roughly the same size if possible. • Displaying nonessential symbols as half-intensity helps declutter displays. Enunciators Historically, all the major alarm indicators for a system were grouped together into what is called the "enunciator panel." Usually, this panel is an array of rectangular lights with each representing a possible alarm or critical-state variable (e.g ., 5 x 5 arrays of 25 discrete alarm channels). Frequently, the lights use the tripartite color convention, blink to alert, and are connected to an audible alarm. There is usually some means to acknowledge the alarm and thus stop the distracting blinking and noise without ca nceling the alert status indication (e.g., goes from blinking with audible alarm to just lit in red or yellow). Old hardware displays unintentionally maintained a manageable number of alarm states as an inadvertent, but positive, consequence of the difficulty of adding enunciator segments. With computerized systems, however, th is number can grow to be overwhelming. Conversely, with computerized systems, analysis of vehicle operations can reduce a large number of variab les into a composite key status variable for alarm states and offer pre-determined options of emergency responses to the pilot. Hence, the concept of an enunciator can be quite useful if following t hese recommendations: 25 UNCLASSIFIED/fFOA. OFFICIO ls. Plili ONls.¥
UNCLASSIFIED/ ,'FOR OFFI@IAI:: WSE ONl::Y • Limit the number of alarms. • Display alarms in a dedicated, predictable location. • Link alarms to a pictorial representation of the system to help recognize root causes, causal relations, and where to focus corrective action. • Provide a means to acknowledge the attention-getting aspect of the alarm without turning off its alert status indication. • In the case of multiple triggered alarms, the system's most prominent display should be the highest hierarchical alarm, the highest causal alarm, or the location on which to focus corrective action. • Link the alarm clearly to the control response options. Checklists Checklists are a standard and useful tool in the operation of vehicle systems. They can be displayed on adaptive displays where the list corresponds to the vehicle's current status (e.g., preflight readiness or postflight checks). Recommended features of a checklist should include the following :41 • Display the desired and current value with each checklist item. • Require that buttons/switches have to be touched to acknowledge t hat each item has been checked off. This helps avoid skipping items and helps the user keep a sense of involvement in the process. • Include a completion call at the end of the checklist. • For long lists, subdivide using the following guides: - List critical items first. - Use geographical (i.e., physical location) flow or pictorial flow to help users retain relation between the model of the system and the procedures being performed in the checklist. - Use parallel tasks between the onboard operation and the ground operations to maintain fluency between pilot and mission control communications. - Include buffers in checklists to provide recovery time for anomalies. Decouple tasks if possible so that t he failure to meet a given checklist item does not leave a parallel item in an open state. - Standardize checklists within the system (assuming nested, hierarchical, or multiple checklists) to minimize the mental burden necessary to extract information when going from one format to another. UNCLASSIFIED/fFOA. OFFICIO ls. Plili ONls.¥ 26
UNCLASSIFIED/ ,'FOR OFFI@IAI:: WSE ONl::Y DEVICES FOR RECEIVING PILOT COMMANDS In much the same way that displays are configured to match the dominant norms of human behavior, so too are the command functions. As evidenced by the gaming industry, the motions of the pilot's hands-whether by a joystick or a Wii controller 42 - mimic the intended physical motions of the object under control. Some of these control technologies are described below. Physical Controls Devices such as joysticks, toggle switches, thumb wheels, rotary switches, and even keyboards will still be mandatory in cockpits of the future. This is based on needing fixed locations for the most critical displays and controls. Additionally, tactile feedback helps the pilot know that their command has been entered. In moments of crisis, a human can react quickly to reach for just the right switch and detect the sensation when that switch is flipped. Joysticks take advantage of human nature, where hand motions mimic the intended motions of the object under control. Joysticks and pedals with force-feedback or vibration feedback add another element of information that humans can process in parallel-feedback that would not be possible with virtual controls . 43 Despite advances in other data-input technologies (e.g., voice), it is likely that there will be times when a keyboard is required, but its routine use is not expected. Keyboards are an efficient way to accurately enter alphanumeric data and especially narrative text. Conversely, keyboard use is time consuming, physically requires a large space (can be stowed, however), and is subject to errors during vibration or buffeting. Such errors are reduced when having some physical support to help anchor the hands. Gesture-Based Inputs By the time that propulsion breakthroughs become viable, it is likely that gesture-based commands will have evolved past the current systems' problems of misinterpreting wayward motions and will have become an effective way to replace the mouse for cursor control. In addition, it is expected that more options will be available through gestures than through existing mouse buttons (right click, left click, and scrolling). 44 When used in combination with a voice-command system, it is expected to be an effective tool for the more complex and varied instruction sets, such as navigation. For example, the notion of being able to point to a locati on on an expansive virtual map and say, "go there," seems an ideal implementation. Furthermore, the use of gesture-based inputs in analyzing new data seems appropriate, provided that the lessons from adaptive displays are heeded. Although fascinating, gesture-based commands are dependent on how well the information that they are manipulating is organized. Therefore, these inputs might be prone to the same keyhole errors and mode errors of adaptive displays. Prior lessons regarding quick emergency commands should also be heeded, specifically where all critical commands have dedicated physical controls. Gesture-based commands can be redund ant, but again, the physical control should be the dominant source of critical inputs . Consider the event of buffeting, where the position of an UNCLASSIFIED/fFOA. OFFICIO ls. Plili ONls.¥ 27
UNCLASSIFIED/ ,'FOR OFFI@IAI:: WSE ONl::Y extended arm is likely to waver. This would induce significant errors in the gesture system. Voice Commands Voice commands are promising for hands-free and parallel task initiation. 45 By the time that propulsion breakthroughs become viable, it is likely that voice-command technology will have solved the problem of voice recognition (including when stressed) and the problem of accommodating the myriad of ways to ask for the same thing. That having been considered, voice commands are projected to be an excellent augmentation to the set of other cockpit controls, particularly with the more complex and varied instruction sets, such as navigating within gesture-based systems. Based on lessons from human-to-human communication, however, it is advised to have the pilot's commands repeated back to them as a form of confirmation. Brain-Machine Interface Although research is currently in the initial stages, 46 th is study of far-future cockpits assumes that this technology is fully matured, such that the pilots can issue commands with their thoughts. A limiting constraint here is that only non-invasive techniques are used (avoiding the need to modify the pilots). Regardless of the sophistication of the technology, a weak link in these systems is the human mind itself; that is, thoughts can wander. Much more study is required to determine the implications of distractions, loss of concentration, fright, levels of mental discipline, and many others. CONTEMPORARY AIRCRAFT COCKPITS For comparative reference, some features of contemporary aircraft cockpits are examined next. Key displays and controls are becoming standardized, although the placement still varies across manufacturers and models. 47 Regardless of differences, the functions are representative of similar functions to consider for breakthrough-era cockpits. The layout of the Airbus A380 Flight Deck, shown in Figure 10, has major flight functions located in specific areas. There is a combination of fixed and adaptable displays. The following items of interest are identified: • Primary flight display. • Navigation display. • Flight mode control. • Vehicle status indicators. • Joystick, throttles, and keypads. UNCLASSIFIED/fFOA. OFFICIO ls. Plili ONls.¥ 28
UNCLASSIFIED/ ,'FOR OFFI@IAI:: WSE ONl::Y Flight Mode Con Figure 10. Flight Deck of Contemporary Aircraft. As one example, the Airbus A380 Flight deck contains displays and controls that are ind icative of displays and controls that will be needed in breakthrough-era craft. Image source : < http ://www .flickr.com/photos/83823904@N00/641S6219 /> (September 2007) Primary Flight Display The primary flight display (PFD) provides information about the vehicle's orientation (pitch, roll, and compass heading), motion (speed and rate of climb), and altitude. It is centrally located in front of the co-pilot's seat, and a smaller version is available to the right of the pilot's navigation display. Figure 11 shows a close-up of a PFD that also shows its heritage from the mechanical instruments of older aircraft that have now been merged into a single computer-controlled instrument. This instrument is vital to the safe operation of an airplane. It displays orientation (pitch, roll, compass heading), motion (forward airspeed, rate of climb), and position relative to a critical axis (altitude). This particular graphic was selected because it also shows the lineage to the individual mechanical displays of prior aircraft; artificial horizon, airspeed, compass heading, altimeter, and rate of climb. Per many of the recommendations from human factors research, a blend of graphical and alphanumeric elements are now used, and t he central orientation graphic, t he "artificial horizon," is a direct physical analogy to the physical world. UNCLASSIFIED/fFOA. OFFICIO ls. Plili ONls.¥ 29
UNCLASSIFIED/ ,'FOR OFFI@IAL WSE QptLY Figure 11. Contemporary Primary Flight Display. Source: < http:/ / www.freshgasflow .com/physics/avia tion_and_anest hesia/ aviation_comparison .html> An analogous display will be required for the breakthrough-era craft, but with modifications to accommodate the full six degrees of freedom and the additional flight regimes of orbit and deep space. For example, the artificial horizon and altimeter have meaning near the surface of a gravitating body, but it is not clear how effective such references will be in flight regimes of orbit or in deep-space flight. These orientation cues as well as compass headings are Earth dependent. Far from Earth, the value of maintaining these cues is unknown, and substitute cues have not been identified . Other orientation standards wi ll have to be developed for breakthrough-era craft that will still be consistent with this aircraft convention when operating near the surface of a gravitating body. Navigation Display The next most significant display is for navigation. Near Earth, this takes the form of a map that can be augmented to show terra in and weather. Due to its importance, it has a prominent place on the flight deck. Numerous related controls and displays are avai lable for allowing the entry of destinations and waypoints into the system to augment this one display screen . Breakthrough-era craft will require additiona l navigation cues; these may include those that will direct pilots into orbits and those for navigating in deep space. UNCLASSIFIED/fFOA: OFFICIO la Plili Oralla¥ 30
UNCLASSIFIED/ ,'FOR OFFI@IAI:: WSE ONl::Y Flight-Mode Control In contemporary aircraft, the flight-mode control panel is often located prominently in the glare shield . It contains displays and controls for the autopilot and for setting headings, speeds, and other operational parameters. In breakthrough-era craft, there will likely be no need for a glare shield, since all images of the exterior will be projected indirectly; thus, the intensity of the light can be limited. Nonetheless, prominent location for the flight -mode display is critical. having a Vehicle Status Indicators The status of critical operating pa rameters of the vehicle, such as its fuel level and eng ine temperature, is essentia l. Note the prominence of those displays in the center of the flight deck. However, other than knowing that similar displays and controls will be required for breakthrough -era cockpits, it is too early to speculate on their details. Joystick, Throttles, and Keypads Some sort of physical controls are required to fly the aircraft. On contemporary aircraft, these include the joystick (side-mounted), rudder pedals, and throttle controls. Fine tun ing controls that manage features such as trim settings and flight heading can be located on separate panels. The details of these controls are dependent on the methods of flight. For a breakthrough craft, it is reasonable to speculate that six-degree-of freedom controls will be accommodated by whatever method is most natural for pilots. Figure 12. Space Cockpit Visions Circa 1959 . Taken directly from Seifert's 1959 book, Space Technology, this is an artist's rendering of a "space travel control statio n." It is offered here to convey the provisional nature of far future speculations, including the speculatio ns offered in t his report. UNCLASSIFIED/fFOA. OFFICIO ls. Plili ONls.¥ 31
UNCLASSIFIED/ ,'FOR OFFI@IAL WSE QptLY Chapter 3: Provisional Cockpit for Breakthrough Flight Applying the strategy of using the most demanding situations to guide designs, the provisional cockpit presented here assumes a single pilot. Without any crew amongst whom to disburse the workload, such a cockpit must be more efficient and effective. The "design for stress" approach is also applied, which drives the design to have actual physical displays and controls that are concentrated in a "tunnel vision" zone. The elements on these physical panels are predictably fixed, owing to lessons of human factors. Newer technology such as virtual reality displays with augmented reality and gesture based inputs are also included, along with voice and thought commands. Based on lessons from adaptive display errors, and considering the limits of human concentration, limits are imposed on the use of voice and thought commands . The overall configuration is shown in Figure 13 and consists of a modest set of physical panels surrounding a seated pilot, along with a large virtual display surrounding the entire area. The most critical displays and controls are condensed into the forwardmost panels. FLIGHT MODES The most fundamental flight mode is the full manual mode. Each subsequent flight mode offers increasing levels of automated assistance, up to and including a fully autonomous control mode that can operate even when the pilot is incapacitated. A provisional set of flight modes (whose buttons and indicators are presented across the central flight mode panel from left to right) include the following: • Full manual. • Manual w/safe assist. • Interactive assist. • Fully autonomous (keyed to emergency responses). Full Manual As the name suggests, this is where the pilot has full control over the motion of the vehicle through joysticks, switches, gestures, voice, and thought. This is the most difficult flight mode for the pilot, and accordingly provides the best situation from which to design the guidance displays and to set the requirements for the various control inputs. In subsequent studies, 48 this would be the flight mode to specify when addressing the challenges of guiding the pilot for entry into orbits and navigating in difficult situations. UNCLASSIFIED/fFOA: OFFICIO la Plili Oralla¥ 32
UNCLASSIFIED/ ,'FOR OFFI@IAL WSE QptLY ·----- Figure 13. Provisional Breakthrough-Era Cockpit. The spherical zone around the cockpit center represents the virtual surround display onto which images of the environment outside the vehicle are displayed. Keeping with lessons of prior human interfaces, a min imal set of physical displays are also provided, with the most important ones located deliberately in the "tunnel vision" zone in front of the pilot. [Graphic: A. Szames with background image through M. Cherfi] Manual With Safe Assist To prevent the pilot from inadvertently flying into a hazardous situation, this mode augments the manual flight controls by taking over to prevent collisions or to prevent entry into hazardous zones (e.g., dangerous radiation or gravitational fields too strong to escape). When the system must take over, it would be accompanied by force feedback into the joysticks and obvious indications on the display. UNCLASSIFIED/fFOA: OFFICIO la Plili Oralla¥ 33
UNCLASSIFIED/ ,'FOR OFFI@IAI:: WSE ONl::Y Interactive Assist This is a further automated augmentation of the prior flight mode. In th is case, fo llowing commands from the pilot (voice, gesture, or thought), the pilot can request for the flight assistant to fly the veh icle as desired. In this mode, it is most likely that the pi lot wi ll not be touching the joysticks, but rather issuing "drive to" commands to the automated system . Fully Autonomous In the event that t he pilot is incapacitated or performing non-flight tasks, the fully auto nomous mode flies the veh icle in whatever flight command mode was last entered. Th is means that the system follows t hrough with previously entered comma nds, in clud ing a selection of preset emergency responses, such as "returning to base" or "flying to t he nearest med ical facil ity." These emergency response fu nctions may have a dedicated physical panel. PHYSICAL DISPLAYS In keeping with human-factors lessons, particularly the use of physical analogies of human perception, the critica l displays and controls will be anchored in physical hardware in fixed locations. The primary reason for providing an actua l physical set of displays and controls is to minimize errors during stressful situations. Thi s includes tunnel vision, mode errors, and keyho le errors. The controls also include tactile feedback. Fo llowing the precedents of modern fl ight decks and human-factors lessons, each of the panels will have a dominant fu nction as identified in Figure 14. Although t he look and fee l of these panels has not yet been conjectured, t heir likely functions and features can be estimated. Vector Motion Flight Assist Navigation Display s E Interactive Display E R Display Survival Emergency Flight Mode Enunciator Response Cabin Comm. Control Control Vehicle Status External Pullout Internal Sensing Conditions Keyboard Dual Six-Degree Joysticks Figure 14. Functional Designation of Physical Cockpit Panels. Basic layout of the physical displays and controls . The function of each is described in the text. [Credit: M. Millis] UNCLASSIFIED/fFOA. OFFICIO ls. Plili ONls.¥ 34
UNCLASSIFIED/ ,'FOR OFFI@IAL WSE QptLY Vector Motion Panel (Primary Flight Display) The most critical, survival-dependant displays are in the "tunnel vision" zone, directly in front of the pilot. The main panel at the center of the display is analogous to the primary flight display (PFD) of contemporary aircraft (shown previously in Figure 11). In this case, however, it must be modified to accommodate the full six degrees of freedom and the additional flight regimes of orbit and deep space. Because of these modifications, and to distinguish from conventional aircraft displays, this text provisionally uses the term "vector motion" display. The main distinction between an aircraft PFD and this vector motion display is that the aircraft PFD is biased toward flight near the surface of a gravitating body and for which navigation conventions are firmly established (e.g., "up" and magnetic compass heading). The vector motion display, which must accommodate orbits and deep-space flight will also feature conventional functional subsets for near-Earth navigation. Since we have no established standards for deep-space navigation that are applicable beyond our solar system. Devising and designing such a system and the appropriate display requires further work. Orientation standards will have to be developed for breakthrough-era craft that are still consistent with this Earth-surface convention. The main axes adopted for this display would also be superimposed onto the virtual surround display. This display will include information to ensure a safe trajectory, avoid collisions, and handle other situations. In addition, visual cues for assisting a pilot to safely enter a correct orbit are required. Although it is expected that such maneuvers would be handled by an automated system, the more demanding condition, in which the pilot will have to manually enter orbit, presents an appropriate design goal for the display and is a subject for future work. Navigation Display Panel The navigation display is responsible for showing the pilot his location relative to base and all other points of interest. Following the common practice of aviation, this display is located immediately to the left of the central motion display. The navigation display will need a new coordinate system to address deep-space flight, transition into orbits, and maneuvering near to and landing on gravitating bodies. Such an orientation and guidance system does not yet exist. It is expected that most gravitating bodies can be conventionally mapped using systems analogous to those used for the Earth, where the rotation axis is a defining characteristic. The caveat is that not all gravitating bodies are spherica l (asteroids and small moons). Flight-Assist Panel The flight-ass ist panel is located to the right of the vector motion display. Because of the variety of functions this panel must perform, it must be in the form of an adaptive display. This panel provides information and accepts pilot commands to assist in UNCLASSIFIED/fFOA: OFFICIO la Plili Oralla¥ 35
UNCLASSIFIED/ ,'FOR OFFI@IAI:: ~SE OPtl::Y operating the vehicle and carrying out the mission. It is functionally similar to the "flight management system" 49 of commercial aircraft. The pilot needs critical information in real time that can be easily interpreted. If anything goes wrong, the pilot needs to know the options and probable consequences of each option. A simplified depiction of the nature of the emergency is also critical. Predicted consequences and risks of each response option should also be succinctly displayed as part of the adaptive display options. The pilot also needs to know, unambiguously and quickly, how to select any option. The envisioned functions include the following: • Automated assessment and display of the overall vehicle operational status. • Automated assessment and display of pilot's health and functional status. (If the system detects that the pilot is incapacitated, it automatically engages the "get medical assistance" emergency response mode.) • Provide short-list of anomaly/emergency response options • In emergency, time-critical situations, the system will automatically choose its top recommended response if the pilot does not enter in a command within the required time. • Checklists for various procedures (e.g., pre- and post-flight or diagnostics). • Monitor progress relative to mission goals and timeline. • Provide access to other databases (e.g., something akin to an Internet search). • Provide the option to project a larger version of the display on the virtual surround, whereupon gesture-based commands can be used. When designing this panel, the lessons discussed in Chapter 2 regarding adaptive displays and graphical and alphanumeric representations should be followed. This includes hierarchical organization, limit of displayed choices of four to seven, simultaneous display of the current mode of the display relative to a model of the system, and sound and verbal accompaniment for displayed information. Additionally, the functions of the surviva l enunciator, flight mode panel, and emergency response panel are all tied into the flight assistant, but these fixed panels provide only the most critical alarm and response options. One of the emergency response panel buttons allows for selection of whatever response option is currently displayed on the flight assistant panel, while the other emergency response buttons are fixed functions. Survival Enunciator Panel On the left side of the flight mode panel, between the center and left main panels, is the survival enunciator. For this limited -size enunciator, only those alarms that are of a life-threatening nature are listed. Less critical alarms are either handled by the flight assistant or the side panels. Audio cues are also recommended for this enunciator, with the added feature that the direction of the source of the sound corresponds to the location of the alarm. UNCLASSIFIED/fFOA. OFFICIO ls. Plili ONls.¥ 36
UNCLASSIFIED/ ,'FOR OFFI@IAL WSE QptLY Copying the general organization of the other major panels, this tall and narrow enunciator is divided into three sections: • External Alarms (Upper Left): Alarm indicators corresponding to the most life threatening conditions (limit of 4 to 10) that could occur external to the vehicle, such as impending collisions, hazardous radiation, or other critical situations. • Vehicle Alarms (Upper Right): Alarm indicators corresponding to the most life threatening conditions (limit of 4 to 10) that could occur internally from the vehicle itself, such as catastrophic equipment malfunctions. • Cabin Alarms (Bottom): Alarm indicators corresponding to the most life threatening conditions of the cabin's life support (limit of 4 to 10), such as loss of air pressure, excessive internal g's, temperature extremes, or pilot health conditions. Emergency Response Panel The narrow vertical panel between the center and right panel is the emergency response panel: a fixed series of buttons for preprogrammed emergency responses, including at least one variable button tied to the recommendations on the flight assist display. These are deliberately set into the tunnel vision field of view. Each of these responses is programmed to perform without needing additional inputs from the pilot. A provisional set of these preprogrammed emergency response buttons is as follows: • Evasive collision avoidance. • Escape to safe loiter position. • Get medical assistance. • Effect automated repair. • Return to base. • Flight-assist recommendation. One of the emergency response panel buttons is to select whatever response option is currently displayed on the assistant panel, while the other emergency response buttons are fixed functions. Flight-Mode Panel Because selecting any of these emergency responses will change the flight mode into "fully autonomous" mode, the flight-mode panel is between the emergency response panel and the survival enunciator. Because the fl ight mode is a critical piece of information, it is also located between the vector motion display and the vehicle status panel. In aircraft, the flight-mode controls are often on the glare shield, but in the breakthrough-era craft, there will not be a need for a glare shield. The light in the cabin is only from the display systems whose brightness can be set to prevent glare on the consoles. UNCLASSIFIED/fFOA: OFFICIO la Plili Oralla¥ 37
UNCLASSIFIED/ ,'FOR OFFI@IAL WSE QptLY Vehicle Status Panel Below the vector motion panel is the veh icle status panel that conta ins t he most critical vehicle status and controls. This is ana logous to where fuel levels and engine temperatu res would be displayed. In this centra l panel, only the most critical and highest hierarchical status cond itions would be displayed in a fixed format. More detailed, lower-level statuses and controls are on the right side panel and avai lable through the flight-assist system . The proximity of this panel to the survival enunciator, the flight-mode panel, and the emergency response panel is deliberate. Also flanking either side of this vehicle status panel are the cabin controls (on the left) and the communication controls (on the right). Cabin Control Panel The cabin controls are not just for temperature and pressure of the pilot's cabin, but also for the inertial and gravitational environment inside the cabin . This panel is del iberately located just below the section of the surviva l enunciator that displays alarm states for the cabin, and just adjacent to the veh icle status panel, with the expectation that related vehicle functions are displayed near the cabin controls. The novel fu nction of including controls for the internal inertial and gravitational forces is in keeping with the recommendation of the double-hull configuration from Chapter 1. If the pilot wishes to feel a portion of the vehicle's acceleration, that can be dialed in. If the pilot wishes to select a nominal background gravitational field of O g or 1 g (or other options), that can be done, too. Communication Panel The communication panel is the primary selection point for communicating with base. According ly, it is located just below the flight-assist system and the emergency response panel. An alphanumeric key pad serves both the communication panel and the flight-assist system. Available settings would likely include selection of communication methods, channels, and related functions. Further details are dependent upon factors that cannot yet be predicted . Pullout Keyboard For more intensive alphanumeric data entry, and when t ime allows, a pullout computer keyboard is available . External Sensing Panel One of the two side panels flanking the pilot seat is the externa l sensing panel. It includes settings for external sensors, navigation cues, and related functions. It also includes controls for how the information obta ined from the external sensors is processed, displayed, and recorded. UNCLASSIFIED/fFOA: OFFICIO la Plili Oralla¥ 38
UNCLASSIFIED/ ,'FOR OFFI@IAI:: ~SE OPtl::Y Both side panels are deliberately not in the tunnel vision zone, since they provide add it iona l functions that are less time critical. Each side panel is a mix of fixed and adaptive displays. The convention is that the panel on the left (consistent with the navigation panel on the left) is for displays and controls related to the envi ronment outside the craft, while the panel on the right is associated with the function of the craft itself. Only some of the sensors and functions are known at this time, and it is expected that fu rther advances in sensor technology might provide other abilities that are not yet foreseeable . Following the requirements from Chapter 1, this panel will at least have controls for the fol lowing : • External visible spectrum cameras that provide input to the virtual surround display . • Augmented spectrum cameras (e.g., infrared or ultraviolet). • Star tracker. • Cosmic microwave background sensors. • Inertial sensors and their related integrations for velocity and position. • Object detection and trajectory determination. Internal Conditions Panel The right panel flanking the pilot seat has more detailed displays and controls regarding the vehicle's functions, such as energy, propulsion, and computation. Similar to the left side panel, it is a mix of fixed and adaptive displays. This panel provides deeper levels of detail for the elements displayed on the vehicle status panel, along with control settings for the same. The functions of this panel are closely integrated with the flight-assist system, especially in regard to recommending corrective actions to vehicle malfunctions. Absent the technology for the vehicle's breakthrough propulsion and power systems, further details of this panel cannot be suggested at this time. Periphery Motion Displays The last two physical panels are the periphery motion displays, which are located on either side of the cockpit in the pilot's periphery vision. They are a physical back-up to the virtual surround display, where each offers more intense moving grid lines. The panels simply display a grid whose motion is synced with the motion of the craft relative to the surrounding space. In this way, the pilot gets subtle cues to motions that can be mentally processed in parallel with the other displays. Such an additional motion display is expected to become necessary because of the added degrees of freedom available for the vehicle's motion. This can be particularly useful during hover, UNCLASSIFIED/,<FOA. OFFICIO ls Plili ONI.¥ 39
UNCLASSIFIED/ ,'FOR OFFI@IAL WSE QptLY where careful changes in position are required that are difficult to infer from just looking at external landmarks. VIRTUAL SURROUND DISPLAY Responding to the likelihood that direct visual contact to the outside environment will not be available, images of the outside will have to be relayed to the pilot via exterior cameras and interior displays. Taking advantage of foreseeable advances, such images are likely to be in the form of virtual displays. This also allows the most critical motion and orientation cues, along with augmented reality functions, to be superimposed on these images. At this point, the method of virtual display is not important (e.g., helmet mounted, projected on screen, or contact lens). The issues addressed here discuss what is required rather than the details of how those functions are provided. Exterior Visuals Since direct visual contact to the outside environment will not be available, images of the outside will have to be relayed to the pilot. A system of cameras that can see the external environment are required, plus whatever processing is required to seamlessly merge these images for projection on the virtual surround display. Additionally, it would be advantageous to be able to focus in on and magnify the view of any particular area of the display. Points of Interest Taking advantage of the technology for augmented reality, 50 where objects of interest can be highlighted on a display, this function will be included in the virtual surround display to flag such things as points of interest, objects, and the trajectories of objects. Overlays Ideally, it would be advantageous to be able to superimpose false-color images representing light spectra beyond normal human perception, such as infrared or ultraviolet light. Data Call Up With such a large projection area available, physical panels can also be projected onto manipulated with gesture-based commands. techniques of visual information display. 51 any the This feature of the adaptive displays from the virtual surround display and then can employ ever-evolving Sound Augmentation Audio cues shall be included so that the location of objects of interest and alarm states can be spatially inferred, and redundant information to their locations can be projected by visual displays. In addition, the vehicle's flight-assist system can speak information to the pilot. Techniques of using different sounds, as well as the manipulation of a given sound (i.e., changing the repetition rate of a given beeping tone), will be used. UNCLASSIFIED/fFOA: OFFICIO la Plili Oralla¥ 40
UNCLASSIFIED/ ,'FOR OFFI@IAL WSE ONLY CONTROLS Similar to the blend of physical and virtual displays, the control methods will also be primarily physical controls (joystick), with augmentations possible through gesture, voice, and thought commands. Vector Motion: Six-Degree Joysticks Adapting to human norms of physical analogies and taking advantage of force-feedback features, the primary control for the vehicle's motion will be a pair of six-degree-of freedom joysticks, one for both the left and right hands, located at the edge of the arm rests. By moving either hand to mimic the orientation or direction of the desired motion, the craft will respond accordingly. Studies suggest that force-proportional, instead of displacement- proportional, operation of joysticks is optimum. Accordingly, the joystick does not require much space to encompass its motions. The intensity of the force input corresponds with the intensity of the resulting motion of the craft. There is, however, a limit to the span of forces that can be input from the hand compared to the span of intensity of the vehicle's propulsion. For example, consider that the hand can resolve speed settings of ±2 mph, but only over a span of zero to 45 mph, while the vehicle is capable of speeds from zero to relativistic. Obviously, some additional input, analogous to a throttle, is necessary. The challenge for the breakthrough craft is that such a throttle is an option for all three linear axes, not just the primary axis of motion. With such uncertainty, and not having any simulations run on test subjects, it is uncertain how best to provide the directional intensity control. Provisionally, it is recommended to have a trigger and/or a combination of buttons that the user selects to convey the maximum intensity of the propulsion system that corresponds to the maximum force input to the joystick. For example, with a light trigger input, the full joystick force might only correspond to a speed comparable to driving a car. With a heaver trigger input or buttons selected to provide deep-space flight speeds, the maximum force on the joystick might correspond to 0. 9 c. Accommodating such uncertainties would be a subject of future study. Owing to lessons of natural relaxed positions of wrist rotation, 52 the neutral palm (with the thumb-side facing inward and forward) is at 35° upward relative to the horizontal and 25° forward. This deviates from the normally upward-pointed joystick, but recall that contemporary flight joysticks are just for rotational axes' command inputs. Gesture Commands Gesture-augmented commands are used in conjunction with the virtual surround display to identify and act on objects in that display. For example, the pilot can point to an object of interest, which is then highlighted, and issue the voice command, "go there." This would be an option in the interactive assist flight mode. Other options for the gesture-based commands are difficult to define at this time, due to the uncertainty of actually operating such a vehicle. Regardless, however, provisions to account for buffeting and other possible input errors would need to be included in future plans. UNCLASSIFIED/ fFOA. OFFICIO Is. flili Ol'lls.¥ 41
UNCLASSIFIED/ ,'FOR OFFI@IAL WSE ONLY Voice Commands Voice commands are projected to be an excellent augmentation to the set of other cockpit controls, particularly with the more complex and varied instruction sets, such as navigating within gesture-based information interfaces. Based on lessons from human-to-human voice communication, it is advisable to have commands from the pilot repeated back by the flight-assist system as a form of confirmation. Thought Commands Thought commands are projected to be a faster method to input commands, but face the underlying limit of the concentration ability of the human. In the case of the breakthrough-era vehicle, the use of thought commands can be considered comparable to the use of voice commands. Accordingly, some form of confirmation from the flight assist system is needed, which the pilot must acknowledge before that command is implemented. UNCLASSIFIED/fFOA. OFFICIO ls. Plili ONls.¥ 42
UNCLASSIFIED/ ,'FOR OFFI@IAL WSE QptLY Chapter 4: Future Work Given the incomplete body of knowledge concerning futuristic propulsion and maintaining cogn izance of ongoing human-factors research, it is probably premature to engage in specific research on cockpits for propulsion physics. Instead, more insights are likely to be gained from relevant physics research. A caveat is that advances in cockpits for breakthrough flight might be further advanced by taking advantage of the gaming industry or through science fiction speculation. Without the actual technology for breakthrough flight, a game simulation or detailed science fiction show/movie could be used as the context around which to explore such options. A concern with this approach, however, is that the underlying stories serve the primary function of enterta inment as opposed to user efficiency. It is conceivable to encounter a guidance system more intended to create dramatic tension ("wow effect") than ease of use. MULTIPLE FLIGHT REGIME GUIDANCE CONVENTIONS The breakthrough vehicle will operate in regimes for which guidance standards do not yet exist, specifically orbit insertion and deep-space (interstellar) travel. Although motion near the surface of a gravitating body can copy the standards of aircraft flight (primary flight display and terrestrial navigation standards), further work is required to explore and select the best options for orbit and deep-space flight. Although it is expected that orbit insertion maneuvers would be handled by an automated system, the more demanding cond ition to use as a design target is to have a display system that can guide a pilot to manually enter a stable orbit. Choosing the convention for the primary axis of deep-space motion will require a trade study to determine if the natural human instinct for forward-dominated motion offers a better human-machine interface than the propulsion-dominant option for upward dominated motion. Such an assessment must also consider how well the convention works when transitioning from deep-space flight into an orbit, then landing, and then back again to deep-space flight. Once any convention is set into place, it will be difficult to change later. VECTOR MOTION DISPLAY Once the development of those guidance conventions is further along, a method to clearly display those conventions for the pilot would have to be developed. A complication that these future displays will encounter is the need for a seamless transition between these three conventions: flight above a gravitating body, orbit insertion, and deep-space flight beyond our solar system. VECTOR MOTION CONTROL Adding three linear axes (plus yaw) to the classic two-rotational degree-of-freedom joystick is a significant change. Although six-degree joysticks are ava ilable commercially, they are oriented toward computer interfaces rather than commanding the motion of a vehicle. To determine the optimum configuration for an actual six degree vehicle control, simulations would likely be required. Perhaps one venue is to UNCLASSIFIED/fFOA: OFFICIO la Plili Oralla¥ 43UNCLASSIFIED/ ,'FOR OFFI@IAL WSE ONLY explore the vehicle controls for explorati on submarines that also have the full six degrees of freedom. Another critica l detail to be resolved with commanding a breakthrough vehicle is how best to accommodate the "throttle." Should each linear axis have the same speed capabi lity, or should on ly the primary motion axis have the fu ll span of zero to maximum speed? With that characteristic specified, it will also be necessary to determ ine how best to provide adjustments to cover the speed ranges of each axis . OPTIMUM MIX OF CONTROL METHODS A variety of methods have been considered for commanding the breakthrough-era vehicle. What is not known is the degree and in what combinations they shou ld be used, and whether constraints shou ld be placed on some of them. The potential use of confirmation feedback for both voice and thought commands is suggested, but future research will need to determ ine whether (and how) such functions shou ld be employed . UNCLASSIFIED/fFOA. OFFICIO ls. Plili ONls.¥ 44
UNCLASSIFIED/ ,'FOR OFFI@IAL WSE ONLY Appendix A: Annotated Bibliography References used in the course of th is report are provided here along with a short description of each. Although not all were explicitly cited, each influenced this study. • 50 Great Examples of Data Visualization. (2009, June) . Web Designer Depot. Retrieved from < http://www . webdesig ne rd epot .com/2009/06/50-great-examples-of-da ta-visualization > Article with examples and links to dozens of different ways to visually convey the interrelationships of multiple items - visua lization techniques suitable for future gesture- based interfaces and/or art. • Ausubel, J. & Marchetti, C. (2001). The Evolution of Transport. The Industrial Physicist, April/May, 20- 24. Article (5 pg) gives historic data on human travel instincts compared to transportation technology - with projections aimed to advocate Maglev trains. The data appears constant over history that people average 1 hr/day of travel, typically round trips from home (80% of the time) and where school and work are the temporary destinations. Also, and again consistent over time, th e resources devoted to travel is 12-15% of total resources. This means for wa lking (2.5 km/h), the village territory is 20 km 2 . Driving (40km/h), territory is 1,000 km 2 . For flying (600km/h), the territory was unspecified due to differences in the proportion of population using air flight and the differences in time devoted (different travel reasons) . • Bass, L. & Dewan, P. (Eds.). (1993). Trends in Software, User Interface Software, New York, NY: John Wiley & Sons. Book (216 pg) Addresses usabil ity attributes and then how these are tested and improved. • Cohen, A. & Chen, E. (1999). Six Degree-of-Freedom HAPTIC System as a Desktop Virtual Prototyping I nterface, Cambridge, MA: SensAble Technologies, Retrieved from < http://citeseerx . ist.psu .edu/viewdoc/download?doi= 10.1.1.122.1783&rep= repl&type=pdf> Technical Advertisement (2 pg) Mechanical version of a six-deg -interface that offers force-feedback on all 6 motions. Although not adaptable to cockpit joysticks, the issue of tactile feedback is relevant. • Coombes, L. F. E. (2005). Control in the Sky: The Evolution and History of The Aircraft Cockpit, Pen & Sword Aviation : Leo Cooper Limited . Book (320 pg) A historical overview of the developments of the cockpit. Contains significant examples of contemporary cockpit displays. • Degani, A. & Wiener, E. (1990). Human Factors of Flight-Deck Checklists : The Normal Checklist, NASA CR - 177549, University of Miami, Coral Gables, FL. Report (67 pg) Focuses on aircraft checklists and presents recommendations for improvements. • Dismukes, R. (1991). Aerospace human factors research division - Code FL. Internal document, NASA Ames Research Center, Moffett Field, CA. Report (83 pg) listing the projects and personnel of that division during that year. • Elm, W. C. & Woods, D. D. (1985). Getting Lost : A Case Study in Interface Design. In Proceedings: Twenty Ninth Annual Meeting of the Human Factor Society, 927-931. Report (5 pg) Applies principles of spatial data management to solve the 'getting lost' and 'mode error' phenomena frequently encountered with multi-display networks. • Genik II, R. J. (2009/ March/ 23) . Technological Approaches to Controlling External Devices in the Absence of Limb-operated Interfaces, Defense Intelligence Reference Document, DIA-08-1003-012 (unclassified). Report (31 pg) summaries the variety of approaches for having the brain directly control machines, called, Brain -Machine Interface (BM!), without the need for mechanical actions. While long-term projections suggest that implants might ultimately become viable, the most likely near-term embodiments are noninvasive means, such as the existing commercial products Mindset, by NeuroSky, and nia, by OCZ Technology. Accuracy is improving as this technology is further researched. UNCLASSIFIED/fFOA. OFFICIO ls. Plili ONls.¥ 45
UNCLASSIFIED/ fFOR OFFI@IAI:: HSE OPtl::Y • Grifantini, K. (2010) . GM Develops Augmented Reality Windshield. Technology Review, MIT, March 17. Retrieved from < http://www.technologyreview.com/blog/editors/24936/?n lid= 2828&a =f> News note on an advanced version or next generation of heads-up display to outline the road, pinpoint obstacles (people and signs), even in bad weather. Pertinent excerpt: "To turn the entire windshield into a transparent display, GM uses a special type of glass coated with red-emitting and blue-emitting phosphors--a clear synthetic material that glows when it is excited by ultraviolet light. The phosphor display, created by Superimaging, is activated by tiny, ultraviolet lasers bouncing off mirrors bundled near the windshield. Three cameras track a driver's head and eyes to determine where she is looking." • Hall, Kenji. (2007) . The Big Ideas Behind Nintendo's Wii. Business Week . Retrieved from < http://www . businessweek. com/technology /content/nov2006/tc20061116_7 50580 .htm> News note on the development of the Wii gaming interface which is a device-dependant gesture-based interface. • Hoover, G. (1992). Stop Saturating Pilots with Alphanumeric Hieroglyphics. Aviation Week and Space Technology, June. 15, 98-99. Editorial (2 pg) Asserting opinions by an accomplished researcher on what pilots need versus what flight instrument designers were providing at that time. In short, with the advent of computer capabilities, many displays were simply providing data in numeric form rather than in some form of interpretable graphic format: "Nature does not use symbols, letters or numbers to convey information. As a result, the human being has evolved as an effective differentiator, but a poor integrator." • Joels, Kennedy, & Larkin. (1982). The Space Shuttle Operator's Manual, New York, NY: Ballantine Books. Book (179 pg) Public oriented book illustrating the displays and controls of the Space Shuttle, which itself is tightly coupled to the legacy aircraft methods. • Johannesen, L. & Woods, D. D. {1991). Human Interaction with Intelligent Systems: Trends, Problems and New Directions. In Conference Proceedings 1991 IEEE International Conference on Systems, Man, and Cybernetics, 1337-1341. Article (5 pg) Examines lessons and trends and emphasizing the need to have a visual model for the process being monitored. • McCandless, David . (2009) . The Visual Miscel/aneum . New York, NY: HarperCollins. Book (255 pg) Illustrating numerous ways to graphically convey complex information in readily interpretable forms. • Millis, M. & Davis, E. (Eds.) . (2009). Frontiers of Propulsion Science, Volume 227 of Progress in Astronautics and Aeronautics, Reston, VA : American Institute of Aeronautics and Astronautics (AIAA). Book {739 pg) First scholarly textbook about the physics issues, progress, and next-step research regarding such game-changing advances as control of gravitation or inertia, propellant-less space drives, faster-than-light travel, and related energy requirements. Includes physics regarding the quantum vacuum, space-time warping, quantum non-locality, cosmology, and others. Includes chapters to clearly document null-results of commonly recurring proposals, and a chapter about how to conduct such revolutionary research in contemporary circumstances. • Moravec H. (2000). Robot: Mere Machine to Transcendent Mind . Oxford University Press . Book (240 pg) Fitting introduction to the notion of transhumanism, plus other predictions about the ascent of artificial life. This is an updated version of his earlier seminal book, Mind Children : The Future of Robot and Human Intelligence. • Morgan, B. (1985) . DOD enlists voice control and expert systems. Electronic Products, Oct. 1, 65-67 . Article (3 pg) Relevant in reflecting the early trends to actually apply voice control in military cockpits, citing Wright-Patterson AFB as a leadi ng center. • Moskowitz, 5. & Devereux, W. (1970). Navigational Aspect of Transstellar Space Flight. In F. Ordway III (Ed .), Advances in Space Science and Technology. Academic Press, Vol.10, 75-126 . UNCLASSIFIED/fFOA. OFFICIO ls. Plili Olhlls.¥ 46UNCLASSIFIED/ fFOR OFFI@IAI:: HSE OPtl::Y Excerpt (15 pg) NASA and Kallsman Instrument Corporation (Syosset, NY) discuss how stellar charts change over distances traveled and address Doppler shihs. • Oran, Daniel. (1991). Apply Psychology to Product Design. Machine Design. Nov. 7, 37- 40. Article (4 pg) Illustrates how to use different switch locations, groupings, and shapes to more readily convey functions. • Pavlovic', V. I., Sharma, R. & Huang, T. S. (1997) . Visual Interpretation of Hand Gestures for Human Computer Interaction : A Review . In IEEE Transactions on Pattern Analysis and Machine Intelligence, 19, 677- 695 Article (36 pgs) Summarizes recent progress toward using gestures of the arms and hands to interact with computers. Hand gestures, including those with different finger positions, can be used. These features are likely to mature soon into commercial products. • Russo, D., Tillman, B., & Pickett, L. (2009). Human Factors Engineering Standards at NASA, Presented at the Human Factors and Ergonomics Society Conference (22-26 Sep. 2008, Manhattan, NY) . NASA Johnson Space Center, Houston TX. Quoting Abstract : "NASA has begun a new approach to human factors design standards. For years NASA STD-3000, Manned Systems Integration Standards, has been a source of human factors design guidance for space systems. In order to better meet the needs of the system developers, NASA is revising its human factors standards system. NASA-STD-3000 will be replaced by two documents: set of broad human systems design standards (including both human factors and medical topics) and a human factors design handbook. At the present time the standards document is in final review with some disagreement on several critical issues. The handbook is progressing with November 2008 as the anticipated completion date." • Potter, S. S. & Woods, D. D. (1991). Event Driven Timeline Displays: Beyond Message Lists in Human Intelligent System Interaction. In Conference Proceedings 1991 IEEE International Conference on Systems, Man, and Cybernetics, 1283-1288. Article (6 pg) Examines lessons and trends of fault monitoring, suggesting that timelines of events be visually conveyed, incl uding causal connections. • Potter, S. S. et al. (1992). Visualization of Dynamic Processes: Function -Based Displays for Human Intelligent System Interaction. In Proceedings 1992 IEEE International Conference on Systems, Man, and Cybernetics, 1283- 1288. Article (6 pg) Although in the context of monitoring processes, it suggests that visual models of the processes (instead of alphanumeric displays) are crucial. • Proffitt, D. & Kauiser, M. (1991). Observer Properties for Understanding Dynamical Displays: Capacities, Limitations, and Defaults . NASA TM 102812. Ames Research Center, Moffett Field, CA. Report (15 pg) Examined human ability to understand and predict motion in the display of dynamic systems. They found that both physics- educated people and naive people perform about equally on the intuitive level: particle motion is relatively accurate, but rotational kinematics, especially that concerning moments of inertia, is relatively poor. • Reason, J. (1990). Human Error. Cambridge, UK: Cambridge University Press . Book (316 pg) By examining major accidents, the correlation between psychology and the operation of hazardous technologies is described. As such, it presents a wealth of lessons of mistakes to avoid. • Sarter, N. & Woods, D. (1992) . Mode error in supervisory control of automated systems . In Proceeding of the Human Factors Society 36 th Annual Meeting. Atlanta GA, Oct. Report (5 pg) Illustrates and defines mode error: "When an action entirely appropriate for a situation is performed, except that this is not the current situation." - Norman 1981. Describes how systems that have command sequences, which change their pattern in different operating modes, is a problem . • Seifert, H. (Ed) . (1959). Space Technology. New York, NY : Wiley & Sons . UNCLASSIFIED/fFOA. OFFICIO ls. Plili Olhlls.¥ 47
UNCLASSIFIED/ fFOR OFFI@IAI:: HSE OPtl::Y Book (several hundred pages) Early scholarly textbook about space technology in general, including interesting projections on displays and controls (Figure 29 - 16). Despite the age of this text, much of its foundational coverage is still valid. • Stanton, N., et al. (2005). Human Factors Methods; A Practical Guide For Engineering and Design . Aldershot, Hampshire: Ashgate Publishing Limited . Book (571 pg) Title says it all. • Shneiderman, B. (1997). Designing the User Interface : Strategies for Effective Human -Computer Interaction, 3rd edition. Boston, MA: Addison-Wesley Longman Publishing. Book (639 pg) Covers a variety investigations, in particular the "Visual Information Seeking" (VIS) principles by which people look for information . Like so many other references, it discusses how people use visual models either directly or indirectly analogous to physical objects to arrange information. Also goes into detail about o Time to learn o Retention time o Rate of errors o Error recovery o Cultural and individual differences • Stokes, A., Wickens, C. & Kite, K. (1990). Display Technology - Human Factors Concepts. Warrendale, PA: Society of Automotive Engineers, Inc. Book (167 pgs) This was a major source of information for this study. Funded by General Motors, this is a compilations of a variety of display research results, focusing on systems for pilots and drivers. • Sutter, J. D. (2010/ June/ 18). New tech moves beyond the mouse, keyboard and screen . CNN. Retrieved from < http://www.cnn.com/2010/TECH/in novation/06/18/natural .user.interface. mouse/index. html?hpt=C2 >. News article describing a number of emerging technolog ies for the computer interface, including voice, gesture commands, brain-controlled computing, eye-tracking software, and even having the images appear virtually around you via special contact lenses. Cites the new Microsoft Kinect gesture-based gaming system, and John Underkoffler - the science advisor to the 2002 movie, Minority Report, where provocative interactions between the user and data are shown. Also reminds us that voice commands and touch screens are now common with cell phones . Regard less of such advances, the keyboard is still likely to be around for those lingering tasks that still cannot be comfortably done with the new "Natural Interface" technology. Also discusses the difficulty with such systems, citing the tiring aspect of gesture based systems, the likelihood of misinterpretation of motions, and the lack of tactile feedback. • Tufte, E. (1991) . Envisioning Information . Cheshire, CT: Graphics Press . Book (126 pg) Comprehensive examples about graphically presenting information, including bad examples to avoid. Useful for general rules on use of color, shading, lines, etc. Although focused on static display of information (print material instead of dynamic displays), many of the lessons of human perception equally apply. It does not address dynamic displays or system integration. • Tilley, A. R., & Dreyfuss, H. (2001). The Measure of Man and Woman : Human Factors in Design . Wiley. Book (104 pg) Comprehensive information regarding anthropometric charts - dimensions and positions of humans, male and female, to be used as references when designing chairs, consoles, or other human factors equipment. This builds on decades of work from designer Henry Dreyfuss who first started publishing in this field in 1955. • Underkoffler, J. (2010). John Underkoffler Points to the Future of User Interfaces. TED Talks, June 2, 2010, Retrieved from < http://www.huffingtonpost.com/tedtalks/john-underkoffler-points_b_597 SO 1.html>> Lecture (YouTube) About the status and projections of gesture-based user interfaces. Underkoffler was the science advisor to the 2002 movie, Minority Report, where provocative interactions between a user and data are shown. He is also a 15-year veteran of MIT's Media Laboratory. The key take-away points from this lecture include projections that gesture-based interfaces will be available in products before 2015, and will no longer require gloves or other devices to be worn by the user. Errors, however, such as the computer misinterpreting an unintended motion as a command input, are likely to remain issues. UNCLASSIFIED/fFOA. OFFICIO ls. Plili Olhlls.¥ 48
UNCLASSIFIED/ fFOR OFFI@IAI:: HSE OPtl::Y Although not mentioned in the lecture (advertising is not allowed at TED talks), Underkoffler's Oblong Industries is already working to make those systems real. • Van der Vaart, A. J. M. (1995). Arm movements in operating rotary controls (Doctoral dissertation). Techniche Universiteit Delft, Netherlands, Delftse Univ. Pres. Dissertation (169 pg) Intense detail into arm and hand motions when dealing with controls such as knobs and joysticks - including experimental test methods. Sections both in English and Dutch . Many useful diagrams for those designing the actual mechanics of controls. Suitable information if designing a new six-degree of freedom joystick. • Visual Understanding Environment (VUE) . (2008). Tufts University. Site developed by Academic Technology. Retrieved from < http://vue.tufts.edu /> Downloadable, open source software to create flexible tools for managing and integrating digital resources in support of teaching, learn ing and research. "VUE provides a flexible visual environment for structuring, presenting, and sharing digital information ." • Wan, C, et al. (1991). 'Vise: A Visual Programming Environment for Data Analysis and Visualization ." Presented at the 1999/Nov. Conference of Association for Computing Machinery (ACM) User Interface Software and Technology (National Information Display Laboratory) . Report (7 pg) Pertinent in the context of reflecting the degree of difficulty when presenting information about the health of an electromechanical system, where comprehending and reacting to the data is not straightforward. Hierarchical organization covered. • Webb, P. (Ed.). (1964). SECTION 17. VISION, Bioastronautics Data Book. NASA SP-3006. Washington DC : Scientific and Technical Information Division. Section from Book containing data on human vision, field of view, effects of lighting, etc. • Woods, D. D. (1984). Visual momentum: a concept to improve the cognitive coupling of person and computer. International Journal of Man-Machine Studies, 21, 229-244. Report (16 pg), Examines the problem of getting lost when working with the multiple display windows that are now common in computer displays. The taxonomy is also useful. • Woods, D. D. (1988). The Significance Messages Concept for Intelligent Adaptive Data Display. Working Paper Series No. 1988-015. Columbus OH: Ohio State University. Report (21 pg) Discusses how intelligent monitoring systems can take on the function of combining a number of variables to calculated a condensed and meaningful "Significant Message" to the user. For example, instead of just displaying various states of a system, the intelligent system takes all those values to give the user a key message, such as "Pump 1 at reduced capacity ." • Woods, D. D. (1993). The price of flexibility. In Hefley & Murray (Eds.), Proceeding of the International Workshop on Intelligent User Interfaces. Association for Computing Machinery (ACM). Report (7 pg) Illustrates how adding flexibility to information displays increased the burden on the users. Users dealt with this by setting a default fixed display and by avoiding using the systems during high stress moments. A list of corrective measures is provided for designing intelligent user interfaces. UNCLASSIFIED/fFOA. OFFICIO ls. Plili Olhlls.¥ 49
UNCLASSIFIED/ ,'FOR OFFI@IAL WSE ONLY Appendix B: Endnotes L Millis, Marc & Davis, Eric. (eds). (2009). Frontiers of Propulsion Science, Volume 227 of Progress in Astronautics and Aeronautics, Reston, VA : American Institute of Aeronautics and Astronautics (AIAA) . 2 Mi llis & Davis (2009) p. 150, Specifically refer to Table-6. Methods in the first 6 rows of t hat table would not require the double hull config uratio n. Also refer to the "gravitational control" methods discussed in the subsequent chapter, 4, for more approaches. 3 Millis & Dav is (2009) p. 487-489. 4 Millis & Davis (2009) Chapter 15. 5 Mill is & Davis (2009) p. 127. 6 Millis & Davis (2009) Chapter 3. 7 Millis & Davis (2009) p. 166. 8 Millis & Davis (2009) Chapters 3-13, & 18. 9 Stokes, A., Wickens, C. & Kite, K. (1990). Display Technology - Human Factors Concepts, Warrendale, PA : Society of Automotive Engineers, Inc. to Millis & Davis (2009) p. 460. 11 Millis & Davis (2009) p. 456-459. 12 Millis & Davis (2009) p. 496-499. 13 Millis & Davis (2009) p. 132. 14 Moskowitz, S. & Devereux, W. (1970). Navigational Aspect of Transstellar Space Flight. In F. Ordway III (Ed.), Advances in Space Science and Technology, Academic Press, Vol. 10, 75-126 15 Numerous references pertain to the overall study of human factors. The more sweeping of these include: • Dismukes, R. (1991) . Aerospace human factors research division - Code FL. Internal document, NASA Ames Research Center, Moffett Field, CA. • Russo, D., Tillman, 8 ., & Pickett, L. (2009) . Human Factors Engineering Standards at NASA, Presented at the Human Factors and Ergonomics Society Conference (22-26 Sep. 2008, Manhattan, NY). NASA Johnson Space Center, Houston TX. • Reason, J. (1990). Human Error. Cambridge, UK : Cambridge University Press. • Stanton, N., et al. (2005). Human Factors Methods; A Practical Guide For Engineering and Design. Aldershot, Hampshire : Ashgate Publishing Limited . • Shneiderman, B. (1997) . Designing the User Interface: Strategies for Effective Human-Computer Interaction, 3rd edition. Boston, MA : Addison-Wesley Longman Publishing. • Stokes, Wickens, & Kite, K. (1990) . • Tilley, A. R., & Dreyfuss, H. (2001). The Measure of Man and Woman: Human Factors in Design . Wiley. • Webb, P. (Ed.). (1964). Bioastronautics Data Book. NASA SP-3006. Washington DC: Scientific and Technical Information Division . 16 Grifantini, K. (2010). GM Develops Augmented Reality Windshield. Technology Review, MIT, March 17. Retrieved from < http://www .technologyreview. com/blog/editors/24936/?nlid =2828&a =f> 17 Morgan, B. ( 1985). DOD enlists voice control and expert systems. Electronic Products, Oct. 1, 65 - 67. 18 Only a few gesture-based computer articles were consulted considering the assumption in this study that these would be mature technologies by the time that propulsion breakthroughs become possible. • Hall, Kenji. (2007) . The Big Ideas Behind Nintendo's Wii. Business Week . Retrieved from < http://www .businessweek. com/technology/content/nov2006/tc20061116_ 750580 .htm> • Sutter, J. D. (2010/ June/ 18). New tech moves beyond the mouse, keyboard and screen . CNN. Retrieved < http://www.cnn .com/2010/TECH/in novation/06/18/natural .user .i nterface.mouse/index. html?hpt=C2 >. • Underkoffler, J. (2010). John Underkoffler Points to the Future of User Interfaces . TED Talks, June 2, 2010, Retrieved from < http://www.huffingtonpost.com/tedtalks/john-underkoffler-points_b_ 597501.html > > ' 9 Only two Brain-Machine interface articles were consulted since the assumption of this study that these would be mature technologies by the time that propulsion breakthroughs become possible. • Genik II, R. J. (2009/ March/ 23). Technological Approaches to Controlling External Devices in the Absence of Limb-operated Interfaces, Defense Intelligence Reference Document, DIA-08-1003-012 (unclassified). • Sutter (2010). 20 Moravec H. (2000) Robot: Mere Machine to Transcendent Mind: Oxford University Press 21 Two references apply to this endnote: • Stokes, Wickens, & Kite, K. (1990). • Hoover, G. (1992). Stop Saturating Pilots with Alphanumeric Hieroglyphics. Aviation Week and Space Technology, June. 15, 98-99. 50 UNCLASSIFIED/fFOA. OFFICIO ls. Plili ONls.¥
UNCLASSIFIED/ fFOR OFFI@IAI:: HSE OPtl::Y 22 Several references are available on the broad category of different method to display complex information in graphical formats. These are just some noteworthy examples: • 50 Great Examples of Data Visualization. (2009, June). Web Designer Depot. Retrieved from < http://www . webdesig nerdepot.com/2009/06/50-great-examples-of-data-visualization > • McCandless, David . (2009). The Visual Misce/laneum. New York, NY: HarperCollins. • Shneiderman, B. (1997). Designing the User Interface: Strategies for Effective Human-Computer Interaction, 3rd edition. Boston, MA: Addison - Wesley Longman Publishing. • Tufte, E. (1991) . Envisioning Information. Cheshire, CT: Graphics Press. • Visual Understanding Environment (VUE) . (2008). Tufts University. Site developed by Academic Technology . Retrieved from < http://vue.tufts.edu /> 23 Stokes, Wickens, & Kite, K. (1990). 24 Stokes, Wickens, & Kite, K. (1990) . 25 Tufte, E. ( 1991) . 26 Stokes, Wickens, & Kite, K. (1990) . 27 Reason, J. (1990). 28 Stokes, Wickens, & Kite, K. (1990) . 29 Stokes, Wickens, & Kite, K. (1990) . 30 Stokes, Wickens, & Kite, K. (1990) . 31 Degani, A. & Wiener, E. (1990) . Human Factors of Flight-Deck Checklists: The Normal Checklist, NASA CR 177549, University of Miami, Coral Gables, FL 32 Several references apply: • Bass, L. & Dewan, P. (Eds.). (1993). Trends in Software, User Interface Software, New York, NY: John Wiley & Sons. • Elm, W. C. & Woods, D. D. (1985). Getting Lost: A Case Study in Interface Design. In Proceedings: Twenty Ninth Annual Meeting of the Human Factor Society, 927- 931. • Hoover, G. (1992). • Johannesen, L. & Woods, D. D. (1991) . Human Interaction with Intelligent Systems: Trends, Problems and New Directions. In Conference Proceedings 1991 IEEE International Conference on Systems, Man, and Cybernetics, 1337- 1341. • Potter, S. S. & Woods, D. D. (1991). Event Driven Timeline Displays: Beyond Message Lists in Human Intelligent System Interaction. In Conference Proceedings 1991 IEEE International Conference on Systems, Man, and Cybernetics, 1283-1288. • Potter, S. S. et al. (1992) . Visualization of Dynamic Processes : Function-Based Displays for Human Intell igent System Interaction. In Proceedings 1992 IEEE International Conference on Systems, Man, and Cybernetics, 1283-1288. • Sarter, N. & Woods, D. (1992) . Mode error in supervisory control of automated systems . In Proceeding of the Human Factors Society 36th Annual Meeting. Atlanta GA, Oct. • Wan, C, et al. (1991) . 'Vise: A Visual Programming Environment for Data Analysis and Visualization." Presented at the 1999/Nov. Conference of Association for Computing Machinery (ACM)User Interface Software and Technology (National Information Display Laboratory). • Woods, D. D. (1984). • Woods, D. D. (1988). The Significance Messages Concept for Intelligent Adaptive Data Display . Working Paper Series No. 1988-015. Columbus OH: Ohio State University . • Woods, D. D. (1993). The price of flexibility. In Hefley & Murray (Eds.), Proceeding of the International Workshop on Intelligent User Interfaces. Association for Computing Machinery (ACM). 33 Regarding adaptive displays, a number of references are pertinent: • Hoover, G. (1992). • Sarter, N. & Woods, D. (1992) . Mode error in supervisory control of automated systems . In Proceeding of the Human Factors Society 36th Annual Meeting. Atlanta GA, Oct. • Woods, D. D. (1984). • Woods, D. D. (1988). • Woods, D. D. (1993). • Stokes, Wickens, & Kite, K. (1990). 34 Three references apply: • Stokes, Wickens, & Kite, K. (1990). • Hoover, G. (1992). • Shneiderman, B. (1997) . 35 Three references apply: • Stokes, Wickens, & Kite, K. (1990) . • Woods (1984). 51 UNCLASSIFIED/fFOA. OFFICIO ls. Plili Olhlls.¥
UNCLASSIFIED/ fFOR OFFI@IAI:: HSE OPtl::Y • Woods (1993) . 36 Five References apply: • Bass, L. & Dewan, P. (Eds.). (1993) . • Elm, W. C. & Woods, D. D. (1985) . • Johannesen, L. & Woods, D. D. (1991). • Potter, S. S. & Woods, D. D. (1991). • Potter, S. S. et al. (1992) . 37 Grifantini, (2010) 38 Two references apply: • Oran, Daniel. (1991). Apply Psychology to Product Design. Machine Design. Nov. 7, 37-40 . • Stokes, Wickens, & Kite, K. (1990). 39 Woods (1984). 40 Thee references apply: • Stokes, Wickens, & Kite, K. (1990). • Sarter, N. & Woods, D. (1992) • Woods (1993) . 41 Degani & Wiener (1990). 42 Hall, Kenji. (2007). 43 Two references apply: • Cohen, A. & Chen, E. (1999). Six Degree-of-Freedom HAPTIC System as a Desktop Virtual Prototyping Interface, Cambridge, MA: SensAble Technologies, Retrieved from <http ://citeseerx. ist.psu .edu/viewdoc/download?doi= 10.1.1.122.1783&rep= repl&type=pdf> • Van der Vaart, A. J. M. (1995). Arm movements in operating rotary controls (Doctoral dissertation) . Techniche Universiteit Delft, Netherlands, Del~se Univ. Pres. 44 Four references apply: • Hall, Kenji. (2007). • Pavlovic', V. I., Sharma, R. & Huang, T. S. (1997) . Visual Interpretation of Hand Gestures for Human Computer Interaction : A Review . In IEEE Transactions on Pattern Analysis and Machine Intelligence, 19, 677- 695 • Sutter, J. D. (2010). • Underkoffler, J. (2010). 45 Morgan, B. ( 1985). DOD enlists voice control and expert systems. Electronic Products, Oct. 1, 65 -67. 46 Genik (2009) 47 Coombes, L. F. E. (2005). Control In The Sky: The Evolution and History of The Aircraft Cockpit, Pen & Sword Aviation : Leo Cooper Limited. 48 Proffitt, D. & Kauiser, M. (1991) . Observer Properties for Understanding Dynamical Displays: Capacities, Limitations, and Defaults. NASA TM 102812. Ames Research Center, Moffett Field, CA. 49 Coombes (2005). 50 Grifantini (2010). 51 Multiple possibilities: • SO Great Examples of Data Visualization. (2009) • McCandless (2009). • Shneiderman, B. (1997). • Tu~e, E. (1991) . • (VUE). (2008). 52 Van der Vaart (1995). UNCLASSIFIED/fFOA. OFFICIO ls. Plili Olhlls.¥ 52