6. References

In Enhancing Human-Agent Teaming by ieeebrain3 Comments

      • Abbink DA, Mulder M, Boer ER. Haptic shared control: smoothly shifting control authority? Cogn Technol Work. 2012;14:19–28.
      • Baard S, Kozlowski S, DeShon R, Biswas S, Braun M, Rench T, Pearce M, Bo D, Piolet Y. Assessing team process dynamics using wearable sensors: An innovative methodology for team research. Paper presented at the annual conference for the Society for Industrial and Organizational Psychology, San Diego, CA.
      • Bell ST, Outland N. Team composition over time. In E. Salas, W. Vessey, & L. Landon (Eds.) Team Dynamics Over Time. Emerald Publishing Limited; 2017. p. 3–27.
      • Billings CE. Human-centered aircraft automation: A concept and guidelines; 1991. NASA Technical Memorandum 103885. Moffett Field, CA: NASA-Ames Research Center.
      • Bohannon A, Waytowich N, Lawhern V, Sadler B, Lance BJ. Collaborative image triage with humans and computer vision. In: Systems, Man, and Cybernetics (SMC), 2016 IEEE International Conference on IEEE; 2016.
      • Bonato P. Wearable sensors and systems. IEEE Eng Med Biol Mag. 2010;29:25–36.
      • Burke CS, Stagl KC, Salas E, Pierce L, Kendall D. Understanding team adaptation: A conceptual analysis and model. J Appl Psychol. 2006;91:1189.
      • Cannon-Bowers JA, Salas E. A framework for developing team performance measures in training; 1997. In M. T. Brannick, E. Salas, & C. Prince (Eds.), Series in applied psychology. Team performance assessment and measurement: Theory, methods, and applications (pp. 45-62). Mahwah, NJ, US: Lawrence Erlbaum Associates Publishers.
      • Cannon-Bowers JA, Tannenbaum SI, Salas E, Volpe CE. Defining competencies and establishing team training requirements. Team Eff Decis Mak Organ. 1995;333:380.
      • Chen JY, Barnes MJ. Human–agent teaming for multirobot control: a review of human factors issues. IEEE Trans Hum-Mach Syst. 2014;44:13–29.
      • Cooke NJ. Team cognition as interaction. Curr Dir Psychol Sci. 2015;24:415–419.
      • Cooke NJ, Gorman JC, Winner JL, Durso F. Team cognition. Handb Appl Cogn. 2007;2:239–268.
      • Crandall JW, Goodrich MA. Experiments in adjustable autonomy. In: systems, man, and cybernetics. IEEE International Conference on, 2001. p. 1624–1629.
      • Crandall JW, Goodrich MA. Characterizing efficiency of human robot interaction: A case study of shared-control teleoperation. In: Intelligent robots and systems. IEEE/RSJ International Conference on, 2002. p. 1290–1295.
      • Cummings M, Clare A. Holistic modelling for human-autonomous system interaction. Theor Issues Ergon Sci. 2015;16:214–231.
      • Cummings MM. Man versus Machine or Man+ Machine? IEEE Intell Syst. 2014; 29:62–69.
      • Dekker SW, Woods DD. MABA-MABA or abracadabra? Progress on human–automation co-ordination. Cogn Technol Work. 2002;4:240–244.
      • Department of the Army. The US Army learning concept for training and education 2020–2040. Department of the Army; Fort Eustis (CA); 2017. RADOC PAM 525-8-2.
      • Deslauriers L, Schelew E, Wieman C. Improved learning in a large-enrollment physics class. Science. 2011;332:862–864.
      • Duchon A, Orvis KL, DeCostanza AH, Rench T, Wade H, Sullivan S, Ziemkiewicz C. The command operations dashboard: a common operating picture of the operators. 19th Int Cmd and Ctrl Research and Tech Symp; 2014; Alexandria, VA.
      • Fitts PM. Human engineering for an effective air-navigation and traffic-control system, 1951. National Research Council, Division of Anthropology and Psychology, Committee on Aviation Psychology.
      • Foerster J, Assael IA, de Freitas N, Whiteson S. Learning to communicate with deep multi-agent reinforcement learning. In: Advances in Neural Information Processing Systems; 2016. p. 2137–2145.
      • Fong T, Thorpe C, Baur C. Multi-robot remote driving with collaborative control. IEEE Trans Ind Electron. 2003a;50:699–704.
      • Fong T, Thorpe C, Baur C. Robot, asker of questions. Robot Auton Syst. 2003b;42:235–243.
      • Garcia JO, Brooks J, Kerick S, Johnson T, Mullen TR, Vettel JM. Estimating direction in brain-behavior interactions: Proactive and reactive brain states in driving. NeuroImage. 2017;150:239–249.
      • Gerkey BP, Matarić MJ. A formal analysis and taxonomy of task allocation in multi-robot systems. Int J Robot Res. 2004;23:939–954.
      • Goldberg B, Brawner K, Sottilare R, Tarr R, Billings DR, Malone N. Use of evidence-based strategies to enhance the extensibility of adaptive tutoring technologies. In: The Interservice/Industry Training, Simulation & Education Conference (I/ITSEC); 2012.
      • Goodwin GA, Murphy JS, Hruska M, Consulting QI. Developing persistent, interoperable learner models in GIFT. In: Generalized Intelligent Framework for Tutoring (GIFT) Users Symposium (GIFTSym3); 2015. p. 27.
      • Grossman R, Salas E. The transfer of training: what really matters. Int J Train Dev. 2011;15:103–120.
      • Hake RR. Interactive-engagement vs. traditional methods: a six-thousand-student survey of mechanics test data for introductory physics courses, 1998.
      • Hinton G, Deng L, Yu D, Dahl GE, Mohamed A, Jaitly N, Senior A, Vanhoucke V, Nguyen P, Sainath TN, et al. Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups. IEEE Signal Process Mag. 2012;29:82–97.
      • Horn Z. Assessment and Development Strategies for Team Rapid Cognitive Effectiveness. Woburn (MA): Aptima Inc.; 2014. Technical Report 1656.
      • Ilgen DR, Hollenbeck JR, Johnson M, Jundt D. Teams in organizations: From input-process-output models to IMOI models. Annu Rev Psychol. 2005;56:517–543.
      • Kalia AK, Buchler N, DeCostanza A, Singh MP. Computing team process measures from the structure and content of broadcast collaborative communications. IEEE Trans Comput Soc Syst; 2017.
      • Kogut B, Zander U. What firms do? Coordination, identity, and learning. Organ Sci. 1996;7:502–518.
      • Kohn LT, Corrigan HM, Donaldson MS. To err is human: building a safer health system. Washington (DC): National Academies Press; 2000.
      • Kozlowski S, Grand JA, Baard SK, Pearce M. Teams, teamwork, and team effectiveness: Implications for human systems integration. Handb Hum Syst Integr. 2015:535–552.
      • Kozlowski SW, Chao GT, Grand JA, Braun MT, Kuljanin G. Advancing multilevel research design: Capturing the dynamics of emergence. Organ Res Methods. 2013;16:581–615.
      • Kozlowski SW, Klein KJ. A multilevel approach to theory and research in organizations: Contextual, temporal, and emergent processes; 2000. In: Klein K.J. and Kozlowski, S.W.J., Eds., Multilevel Theory, Research, and Methods in Organizations: Foundations, Extensions, and New Directions, Jossey-Bass, San Francisco, 3-90.
      • Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems; 2012. p. 1097–1105.
      • LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015;521:436–444.
      • Lyons JB. Being Transparent about Transparency: A model for human–robot interaction. 2013. In Sofge, D., Kruijff, G. J., Lawless, W. F. (Eds.), Trust and autonomous systems: Papers from the AAAI Spring Symposium (Tech. Rep. SS-13-07). Menlo Park, CA: AAAI Press. Google Scholar
      • Mait J, Poree D, Prater J, Reynolds P, Stepp D, West BJ, Kott A, Swami A, DeCostanza AH, Franaszczuk PJ, McDowell, K, Pierkarski B, Sadler BM, Carter R, Zabinski J. 2015 Army science planning and strategy meeting series: outcomes and conclusions. Aberdeen Proving Ground (MD): Army Research Laboratory (US); 2017. ARL Technical Report No.: ARL-SR-0390.
      • Marathe AR, Metcalfe JS, Lance BJ, Lukos JR, Jangraw D, Lai K-T, Touryan J, Stump E, Sadler BM, Nothwang WD, McDowell K. The privileged sensing framework: A principled approach to improved human-autonomy integration. Theor Issues Ergon Sci. 2017.
      • Marks MA, Mathieu JE, Zaccaro SJ. A temporally based framework and taxonomy of team processes. Acad Manage Rev. 2001;26:356–376.
      • Mazur E. Farewell, lecture. Science. 2009;323:50–51.
      • McEwan D, Ruissen GR, Eys MA, Zumbo BD, Beauchamp MR. The effectiveness of teamwork training on teamwork behaviors and team performance: a systematic review and meta-analysis of controlled interventions. 2017. PloS One 12:e0169604.
      • Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, Graves A, Riedmiller M, Fidjeland AK, Ostrovski G et al.. Human-level control through deep reinforcement learning. Nature. 2015;518:529–533.
      • Nedic A, Ozdaglar A, Parrilo PA. Constrained consensus and optimization in multi-agent networks. IEEE Trans Autom Control. 2010;55:922–938.
      • Olguın DO, Gloor PA, Pentland AS. Capturing individual and group behavior with wearable sensors. In: Proceedings of the 2009 AAAI Spring Symposium on Human Behavior Modeling, 2009. Stanford, CA.
      • Orvis K, DeCostanza A, Duchon A. Developing systems-based performance measures: a rational approach. 2013. Proceedings of the Interservice/Industry Training, Simulation & Education Conference, Orlando, FL.
      • Palazzolo ET. Transactive memory theory. In: Scott CR, Lewis L, editors. International encyclopedia of organizational communication. Chichester: Wiley Blackwell; 2017.
      • Parasuraman R, Sheridan TB, Wickens CD. A model for types and levels of human interaction with automation. IEEE Trans Syst Man Cybern Part Syst Hum. 2000;30:286–297.
      • Piekarski B, Sadler B, Young S, Nothwang W, Rao R. Research and vision for intelligent systems for 2025 and beyond. Small Wars Journal Mad Science. [accessed 2016]. http://smallwarsjournal.com/jrnl/art/research-and-vision-for-intelligent-systems-for-2025-and-beyond
      • Rose DH, Meyer A. Teaching every student in the digital age: universal design for learning. ERIC. 2002.
      • Rosen MA, Wildman JL, Salas E, Rayne S. Measuring team dynamics in the wild. In: Hollingshead A, Poole MS, editors. Research methods for studying groups: A guide to approaches, tools, and technologies. New York (NY): Taylor & Francis; 2012. p. 386–417.
      • Salas E, DiazGranados D, Weaver SJ, King H. Does team training work? Principles for health care. Acad Emerg Med. 2008;15:1002–1009.
      • Salas E, Rosen MA, Burke CS, Goodwin GF. The wisdom of collectives in organizations: An update of the teamwork competencies. Team Eff Complex Organ Cross-Discip Perspect Approaches. 2009:39–79.
      • Salas E, Sims DE, Burke CS. Is there a “big five” in teamwork? Small Group Res. 2005;36:555–599.
      • Salas E, Stagl KC, Burke CS, Goodwin GF. Fostering team effectiveness in organizations: Toward an integrative theoretical framework. In: Nebraska Symposium on Motivation; 2007. p. 185.
      • Schecter A, Contractor N. Modeling the joint dynamics of relational events and individual states. In: Advances in social networks analysis and mining (ASONAM), 2016 IEEE/ACM International Conference on, 2016. p. 1087–1094.
      • Sellner B, Heger FW, Hiatt LM, Simmons R, Singh S. Coordinated multiagent teams and sliding autonomy for large-scale assembly. Proc IEEE. 2006;94:1425–1444.
      • Senge P. The fifth discipline: The art and science of the learning organization. N Y Curr Doubleday; 1990.
      • Sheridan TB. Telerobotics, automation, and human supervisory control. MIT press; 1992.
      • Sheridan TB. Function allocation: algorithm, alchemy or apostasy? Int J Hum-Comput Stud. 2000;52:203–216.
      • Shuffler ML, Salas E, Xavier LF. The design, delivery and evaluation of crew resource management training. In: Crew Resource Management. 2nd ed. Elsevier; 2010. p. 205–232.
      • Socher R. Recursive deep learning for natural language processing and computer vision, 2014. PhD Thesis, Computer Science Department, Stanford University
      • Silver D, Huang A, Maddison CJ, Guez A, Sifre L, Van Den Driessche G, Dieleman S. Mastering the game of Go with deep neural networks and tree search. Nature. 2016;529(7587):484–489.
      • Ranjan R, et al. Deep learning for understanding faces: machines may be just as good, or better, than humans. IEEE Signal Processing Magazine. 2018;35:66–83.
      • Sottilare RA, Brawner KW, Sinatra AM, Johnston JH. An updated concept for a Generalized Intelligent Framework for Tutoring (GIFT). Gift Org. 2017.
      • Stone P, Kaminka GA, Kraus S, & Rosenschein JS. Ad Hoc autonomous agent teams: collaboration without pre-coordination. In Proceedings of the Twenty-Fourth Conference on Artificial Intelligence, AAAI, July 2010.
      • Sukhbaatar S, Szlam, A, & Fergus R. Learning multiagent communication with backpropagation. In: Advances in Neural Information Processing Systems; 2016; p. 2244–2252.
      • Suri N, Tortonesi M, Michaelis J, Budulas P, Benincasa G, Russell S. Stefanelli C. Winkler, R. Analyzing the applicability of Internet of Things to the battlefield environment. Proc. of ICMCIS Conference. 2016. Brussels.
      • Telesford QK, Lynall M-E, Vettel J, Miller MB, Grafton ST, Bassett DS. Detection of functional brain network reconfiguration during task-driven cognitive states. Neuroimage. 2016;142:198–210.
      • Van de Ven AH, Delbecq AL, Koenig Jr, R. Determinants of coordination modes within organizations. Am Sociol Rev. 1976:322–338.
      • VanLehn K. The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems. Educ Psychol. 2011;46:197–221.
      • Warnell G, Waytowich N, Lawhern V, Stone P. Deep TAMER: Interactive Agent Shaping in High-Dimensional State Spaces. 2017. ArXiv Prepr ArXiv170910163.
      • Weaver SJ, Dy SM, Rosen MA. Team-training in healthcare: a narrative synthesis of the literature. BMJ Qual Saf. 2014;23:359–372.
      • Wegner DM. Transactive memory: A contemporary analysis of the group mind. In: Mullen B, Goethals GR, editors. Theories of Group Behavior. New York (NY): Springer; 1986. p. 185–208.
      • Woods DD. Cognitive Technologies: The design of joint human-machine cognitive systems. AI Mag. 1985;6:86.
      • Woods DD, Branlat M. Hollnagel’s test: being “in control” of highly interdependent multi-layered networked systems. Cogn Technol Work. 2010;12:95–101.
      • Yuste R, Goering S, Aguera y Arcas B, Bi G, Carmena JM, Carter A, et al. Four ethical priorities for neurotechnologies and AI. Nature. 2017;551:159–163.
Previous Section

Comments

    1. Thanks Joann! Do you have specific recommendations in mind? Would love to hear more of your thoughts, as an expert in teaming.

  2. The preferred medium of communication in human-agent teams should be natural language (NL). Also, agents should be able to learn through dialog with the human members of their teams. A prerequisite is that the agents possess a model of their world (including tasks, team organization and roles of each team member) that supports learning and autonomous decision making. Much, if not all of the above cannot be addressed through machine learning alone: a) an instance of learning cannot rely on extensive training data; b) in decision-making, having humans use commands formulated in a highly stylized sublanguage that would enable direct triggering of particular operations by the agent is impractical; same applies to communication in an artificial language.

    Work on developing language-endowed intelligent agents that are members of human-agent teams is ongoing. I’d be glad to discuss this further if there is interest.

Leave a Comment