TECHNOLOGY AREA(S): Air Platform, Ground/Sea Vehicles, Human Systems

ACQUISITION PROGRAM: Support of FNC/INP programs with distributed unmanned systems

OBJECTIVE: To develop and demonstrate a human interface and related decision support tools that allow human management of swarms of up to 100 unmanned vehicle systems in which communications are highly limited, attrition can occur, and individual swarm members may not have accurate state information about themselves and/or others.

DESCRIPTION: The last decade has seen substantial advances in the design of supervisory control interfaces that allow a single operator to manage multiple unmanned systems based on higher-level mission criteria such as objectives, constraints, priorities, allowable risks, and the level of autonomy for decision-making. Frequently, this is done through a combination of map, timeline, and vehicle status displays. However, existing multiple Unmanned Vehicle System supervisory control interfaces are typically designed under assumptions that may not be valid for swarms. Swarming systems are here defined as systems which are scalable to large numbers of platforms and utilize decentralized control to enable robust collective behaviors despite only limited communications and state information. (e.g., for some types of methods, each vehicle would have the ability to intermittently keep track of or communicate with their nearest neighbors only). In contrast, assumptions for existing interfaces include: (1) relatively good communications with each entity or at least a good ability to project the future actions of each individual system, (2) a manageable number of discreet entities (e.g., individuals, cohesive groups that move in close formation, etc.), (3) well-defined methods to convey user intent and adjust as needed, and (4) a good user mental model of the automation. In comparison, a key reason to use swarming methods is that communications are limited, individual members may lack reliable state information about themselves and others, and swarm size and composition may change due to attrition. Swarming algorithms as well may demonstrate emergent behaviors (e.g., dynamic subgroup formation/dissolution) in which the group collectively completes a mission task in a way in which certain aspects of the overall group characteristics are predictable (e.g., boundaries, distribution over areas, etc.), but individual actions within the group are difficult to comprehend and predict. Finally, the types of control inputs that exist for different types of swarming concepts may bring new challenges to designing appropriate human interaction methods. Ways of controlling a swarm may include higher level mission criteria, more direct group control/influence, local control/influence of subgroups/individuals, and group or subgroup parameter adjustment to shape behaviors.

The goal of this effort is to develop and demonstrate a human interface and related decision support tools that enable users to more effectively understand, predict, shape, and redirect the behaviors/capabilities of highly decentralized systems to meet mission demands including a better understanding of (1) what are appropriate levels of user interaction with a swarm, (2) what metaphors are most effective for humans to use in managing these types of systems, (3) how should controls, displays, and decision support be designed to facilitate swarm management and tasking, and (4) what are the best times/modes for the operator to interact with the swarm and what kind of decision support/displays will best help the user infer or be guided to them. The focus of this effort is on humans interacting with swarms remotely via a computer interface of some type. The development of new platforms, network communications, or hardware of any kind is outside the scope of this topic. Existing capabilities for multi-modal inputs and new display concepts may be leveraged, but the focus is not on developing new multi-modal input systems like speech, sketch, or gesture or new display concepts such as 3-D audio/video or virtual/augmented reality. Concepts that require reliable, highly connected, and/or high-bandwidth communications with the swarm or near-perfect state information about the swarm are outside of the scope of this effort.

PHASE I: Develop a concept to demonstrate a human interface and related decision support tools of swarms of up to 100 unmanned vehicle systems. Perform initial limited structured human factors analysis to begin examining what is an appropriate level of interaction with swarms, and what are the best times/modes for the operator to interact with the swarm. A limited set of swarming methods may be used for this initial phase. A limited scope of platforms, environments, and mission tasks may be chosen for Phase I, but the chosen ones should also demonstrate the broader applicability of the concept. Mission tasks of interest include but are not limited to maritime or littoral environmental sensing/sampling, surveillance, search, tracking, and force protection. The system concept should support implementation within appropriate open architecture frameworks. It is preferred that the swarming algorithms used as a baseline be those found in the open literature. Based on this, develop a preliminary user interaction concept focusing on those new elements which appear most promising. This could be a static mockup or include some limited functionality by leveraging existing prerecorded data, limited-fidelity simulation elements and/or hardware elements as appropriate within the limited scope of the Phase I. Use this initial concept to perform a cognitive walkthrough at minimum. Develop experimentation plans and metrics to evaluate the system in Phase II and consider options for how the approach could integrate with a future swarming system.

PHASE II: Perform a more extensive structured human factors analysis of the domain to understand the specific warfighter interaction needs and constraints for swarm management, iterative development and evaluation of the human interface concept and related decision support tools with a broader range of swarming methods and mission tasks, and final development of a software prototype and evaluation of its ability to support swarm management. As much as possible, the Phase II design should be compatible with open architectures to be applicable to multiple naval operating environments. Phase II tasks should continue advancing our understanding of: What is an appropriate level of interaction with a swarm, what metaphors are most effective for humans to use in managing these types of systems, how can controls, displays, and decision support be designed to support swarm management, and what are the best times/modes for the operator to interact with the swarm and how can the human infer or be guided to them. Final evaluation should include integration of the prototype with simulation and/or hardware elements with sufficient autonomy components to perform laboratory operator in-the-loop demonstrations and comparisons with benchmarks. Demonstrations with live assets may be used when of value, but are not required. Revise evaluation metrics and interface concepts as necessary. Ensuring that the demonstrations have representative complexity of the challenges of future swarm operations is of more importance than a very high degree of fidelity to an existing system.

PHASE III DUAL USE APPLICATIONS: Continue software development of the prototype as plug-in capabilities for relevant open architectures and address any unique requirements for interoperability with particular target domain(s), perform a more formal systems integration task to provide effective software interfaces to particular naval control stations and assets, perform component testing and operator evaluations, and participate in integrated demonstrations of autonomous system operations. Private Sector Commercial Potential: This capability could be used in a broad range of military and civilian security and first responder applications of unmanned systems and in other applications involving management of distributed automated systems, such as agriculture and scientific research.

REFERENCES:

• C. E. Harriott, Adriane E. Seiffert, S. T. Hayes and J. A. Adams (2014) “Biologically-Inspired Human-Swarm Interaction Metrics,” Proceedings of the Human Factors and Ergonomics Society’s Annual Meeting.
• D. Brown, S. Kerman, and M. A. Goodrich. Human-Swarm Interactions Based on Managing Attractors. In ACM/IEEE International Conference on Human-Robot Interactions. March 2014.
• Nagavalli, S., Luo, L., Chakraborty, N., Sycara, K., Neglect Benevolence in Human Control of Robotic Swarms, International Conference on Robotics and Automation (ICRA), Hong Kong, China, May 31-June 7, 2014.
• Walker, P. Amirpour S. Chakraborty, N., Lewis M., Sycara, K. Control of Swarms with Multiple Leader Agents, International Conference on Systems, Man and Cybernetics, San Diego CA, October 5-8, 2014.
• Luo, R., Chakraborty, N., Sycara, K. Supervisory Control for Cost-Effective Redistribution of Robotic Swarms, International Conference on Systems, Man and Cybernetics, San Diego CA, October 5-8, 2014.
• D. Brown, S. Kerman, and M. A. Goodrich. Limited Bandwidth Recognition of Collective Behaviors in Bio-Inspired Swarms. Proceedings of AAMAS, May 2014, Paris France.
• Sean T. Hayes and Julie A. Adams. Human-Swarm Interaction: Sources of Uncertainty. In Proceedings of the 9th ACM/IEEE International Conference on Human-Robot Interaction, pages 170-171, 2014.
• D. S. Brown, S.-Y. Jung, and M. A. Goodrich, Balancing human and inter-agent influences for shared control of bio-inspired collectives. Proceedings of IEEE International Conference on Systems, Man, and Cybernetics. October, 2014, San Diego.
• Human Control of Bioinspired Swarms: Papers from the 2012 AAAI Fall Symposium (Michael Lewis, Katia Sycara, Paul Scerri, Michael Goodrich, Marc Steinberg, Program Cochairs), Technical Report FS-12-04. Published by The AAAI Press, Menlo Park, California
• J McLurkin, J Smith, J Frankel, D Sotkowitz, Speaking Swarmish: Human-Robot Interface Design for Large Swarms of Autonomous Mobile Robots, AAAI Spring Symposium, 2006.
• M Steinberg, Biologically-inspired approaches for self-organization, adaptation, and collaboration of heterogeneous autonomous systems, SPIE Defense, Security, and Intelligence, 2011.

KEYWORDS: human interaction; swarming; unmanned systems; autonomy; unmanned air system; unmanned sea surface system; autonomous undersea system

** TOPIC AUTHOR (TPOC) **

DoD Notice:   Between April 22, 2016 and May 22, 2016 you may talk directly with the Topic Authors (TPOC) to ask technical questions about the topics. Their contact information is listed above. For reasons of competitive fairness, direct communication between proposers and topic authors is not allowed starting May 23, 2016 , when DoD begins accepting proposals for this solicitation.

However, proposers may still submit written questions about solicitation topics through the DoD's SBIR/STTR Interactive Topic Information System (SITIS), in which the questioner and respondent remain anonymous and all questions and answers are posted electronically for general viewing until the solicitation closes. All proposers are advised to monitor SITIS (16.2 Q&A) during the solicitation period for questions and answers, and other significant information, relevant to the SBIR 16.2 topic under which they are proposing.

If you have general questions about DoD SBIR program, please contact the DoD SBIR Help Desk at 800-348-0787 or [email protected]