Artificial Intelligence for Anti-Submarine Warfare Training

Navy SBIR 20.2 - Topic N202-091

Naval Air Systems Command (NAVAIR) - Ms. Donna Attick [email protected]

Opens: June 3, 2020 - Closes: July 2, 2020 (12:00 pm ET)

 

 

N202-091       TITLE: Artificial Intelligence for Anti-Submarine Warfare Training

 

RT&L FOCUS AREA(S): Artificial Intelligence/ Machine Learning, General Warfighting Requirements (GWR)

TECHNOLOGY AREA(S): Air Platform, Information Systems, Human Systems

 

The technology within this topic is restricted under the International Traffic in Arms Regulation (ITAR), 22 CFR Parts 120-130, which controls the export and import of defense-related material and services, including export of sensitive technical data, or the Export Administration Regulation (EAR), 15 CFR Parts 730-774, which controls dual use items. Offerors must disclose any proposed use of foreign nationals (FNs), their country(ies) of origin, the type of visa or work permit possessed, and the statement of work (SOW) tasks intended for accomplishment by the FN(s) in accordance with section 3.5 of the Announcement. Offerors are advised foreign nationals proposed to perform on this topic may be restricted due to the technical data under US Export Control Laws.

 

OBJECTIVE: Augment the Navy Anti-Submarine Warfare (ASW) capability via the addition of an artificial intelligence (AI) training aid that can also assist as an operational decision-support tool.

 

DESCRIPTION: Artificial Intelligence (AI) has increased dramatically over the last five years. This SBIR topic seeks a phased approach to implementing a tactical and evolving Anti-Submarine Warfare (ASW) application with AI technology, specifically the application of AI technology to passive ASW analysis. AI augmentation should be aggressively pursued to increase the manning and training aspects of the AO (operational availability) equation across the Navy.

�

This topic is not a case for replacing an acoustic operator with a machine, but rather should be seen as an attempt to point out why AI assistant technology during training and operations would be beneficial not just to the operator, but to the overall ability of the U.S. Navy to conduct ASW operations. The importance of the human operator�s ability to adapt to new information, answer questions about information, and truly analyze an ASW target�s characteristics and behavior cannot be overstated. The human acoustic operator is responsible for providing target-quality data, under any conditions and in any circumstance, to complete the kill chain. That job is not one that machines can currently perform unaided by a human, but is one that machines can augment effectively. Although the AI assistance can provide analytical tools on top of the data, the human operator will still be the one to review the outputs of the algorithms and question whether it makes sense or not.

 

Acoustic operators go through approximately two years of initial training, and 18-24 months of advanced, hands-on training at the squadron, to become a basic qualified �operator.� In order to become proficient, or even experienced, the operator needs years of additional submarine contact time. In order to build proficiency, the operator spends long hours on deployment or detachments, conducting ASW on adversary submarines and observing the differences between his or her training and the data that he or she gains while on-station. With additional capabilities of the Maritime Patrol community coming online, the demands on a given acoustic operator have only increased.�

 

With these increased demands, we must recognize the limitations of human acoustic operators when analyzing acoustic data, especially before initially locating a subsurface target, when unsure of when and where contact will appear. As human beings, they can only apply their cognitive process to information from a single sonobuoy at a time. Experience and training may allow the operator to process the information faster and more accurately, but no matter how quickly they work through the information they can only process a limited number of sonobuoys at one time. Additionally, human operators must choose to allocate their cognitive effort into three categories: speed, accuracy, and quantity. At any given time, an operator may choose to allocate their effort to two of the three categories. If an operator desires to analyze a large quantity of data with a high degree of accuracy, they will sacrifice speed. If they wish to analyze the same set of data quickly, they will sacrifice accuracy. If instead they desire to analyze data with a high degree of accuracy and a high measure of speed, they must reduce the quantity of data they analyze. Although there are exceptions with expert acoustic operators, the majority of acoustic warfare operators (AWOs) are limited by these constraints.

 

Fortunately, AI that is trained on representative data sets can assist AWOs in the training and operational context. Initially, an AI assistant could provide novice AWOs with explanations and reasoning why it notices a contact within a set of acoustic data. Additionally, confidence values with those explanations would further involve the human trainee in the situation. Flag officers have recently requested that information that AI systems output should come with a confidence factor as standard practice. Although the trainee would have the final say in all matters, the AI can help scaffold the training so that the trainee has useful information about multiple different variables, and the reasoning why confidence values are what they are. For example, given a training scenario in a body of water, the AI could inform the trainee that there is a 60% match based on acoustic signature and that it is an �X� class submarine. It must also inform that overall confidence value is only 40% because it is rare that the �X� class submarine is ever in this particular body of water. This �pulling back of the curtain� via traceable and explainable AI can ensure the student is privy to more of the information and situation than a human could experience un-assisted. This will also assist in deterring adversarial AI practices that can confuse and trick AI systems into believing something that is not true. If successful during training, this AI capability could also be used in operational settings and updated, via modifying its own code as it learns, in real-time to ensure AWOs conducting operations have all the relevant information. This would allow the assistant to improve the ability of crews to do ASW on station as it mitigates a limitation of human operators. Where a human must choose between speed, accuracy, and quantity, an AI need not make such a choice. When �plugged-in� to the aircraft�s ASW data stream, an AI with sufficient processing power is able to analyze all ASW data in real-time with accuracy (within the parameters of its training). Although the topic is primarily focused on the training aspect, successful implementation of the technology certainly has use beyond training to help all Maritime Patrol aircrew become more effective at their primary role of ASW.

 

The final solution must meet AI ethics principles outlined by the Department of Defense (Responsible, Equitable, Traceable, Reliable, Governable) [Ref 7] and will meet Risk Management Framework, Cyber and Navy Marine Corp Internet (NMCI) guidance [Ref 6].

 

Work produced in Phase II may become classified. Note: The prospective contractor(s) must be U.S. owned and operated with no foreign influence as defined by DoD 5220.22-M, National Industrial Security Program Operating Manual, unless acceptable mitigating procedures can and have been implemented and approved by the Defense Counterintelligence and Security Agency (DCSA). The selected contractor and/or subcontractor must be able to acquire and maintain a secret level facility and Personnel Security Clearances. This will allow contractor personnel to perform on advanced phases of this project as set forth by DCSA and NAVAIR in order to gain access to classified information pertaining to the national defense of the United States and its allies; this will be an inherent requirement. The selected company will be required to safeguard classified material IAW DoD 5220.22-M during the advanced phases of this contract.

 

PHASE I: Design, develop, and demonstrate feasibility of AI techniques, methods, and models, and determine the optimum approach for this topic. Demonstrate the feasibility of the proposed training capability within an undersea (or relevant) domain with publicly available training data. The early system should demonstrate a form of explainability and confidence values in outputs provided. Outline future concepts for the inclusion of self-tuning algorithm parameters. AI ethics principles outlined by the Department of Defense will be adhered to (Responsible, Equitable, Traceable, Reliable, Governable). The Phase I effort will include prototype plans to be developed under Phase II.

 

PHASE II: Develop and prototype the proposed solution to integrate into a Navy ASW sample training environment. The prototype capability should ingest real-world training data. Demonstration of the prototype AI assistant, that has been trained with real-world data, providing explainable AI recommendations with confidence values to novice AWOs conducting training using low to medium fidelity training devices or simulators. Considerations of how the standalone training device may use continuous data from an operating environment to refine the algorithm�s parameters should be included in a Phase II demonstration. Continued adherence to AI ethics principles outlined by the Department of Defense will be required (Responsible, Equitable, Traceable, Reliable, Governable).

Work in Phase II may become classified. Please see note in the Description section.

 

PHASE III DUAL USE APPLICATIONS: Refine the capability from the Phase II final demonstration and show consistent reliability in a known performance envelop. A Phase III capability must include and demonstrate a function to self-tune its own algorithm based on new data inputs. Integrate within current acoustic warfare operator training and go through verification and validation testing, as well as effectiveness and usability testing. Continue to adhere to AI ethics principles outlined by the Department of Defense (Responsible, Equitable, Traceable, Reliable, Governable). Contractors will harden the software architecture and implement Risk Management Framework guidelines to support IA compliance, including requirements to allow installation on SIPRNet or Navy Internet Protocol Networks (NIPRNet) if appropriate for transition path. The final capability will be required to exist in a representative training environment with all information assurance and cyber security requirement approvals. Final steps will investigate the level of effort required to convert the AI assistant to an operational assistant in the field.

Developing AI decision-support tools is beneficial for a wide variety of domains and commercial industries. Techniques and procedures used to develop acoustic training may be immediately of use to other domains, such as oceanography and survey research organizations that study the world�s oceans and other bodies of water. Additionally, the technical approaches may be applicable to industries outside of pure acoustics, as the AI techniques that will be implemented will cater to somewhat fuzzy training data sets that may or may not be flush with data samples. As the topic may involve image recognition, as well as acoustic recognition algorithms, the results could also be applicable to areas involving surveillance and automatic image classification.

 

REFERENCES:

1. Russell, S. J. & Norvig, P. �Artificial Intelligence: A Modern Approach.� Pearson Education Limited, 2016. https://www.amazon.com/Artificial-Intelligence-Modern-Approach-3rd/dp/0136042597

 

2. Duda, R. O., Hart, P. E. & Stork, D. G. �Pattern classification.� John Wiley & Sons, November 9, 2012. https://books.google.com/books/about/Pattern_Classification.html?id=Br33IRC3PkQC

 

3. Goodfellow, I., Bengio, Y. & Courville, A. �Deep learning.� MIT Press, 2016. https://mitpress.mit.edu/books/deep-learning

 

4. National Science & Technology Council, Artificial Intelligence Research & Development Interagency Working Group, Subcommittee on Networking & Information Technology Research & Development, Subcommittee on Machine Learning & Artificial Intelligence, and the Select Committee on Artificial Intelligence of the National Science & Technology Council. �2016-2019 Progress Report: Advancing Artificial Intelligence R&D.�� https://www.nitrd.gov/pubs/AI-Research-and-Development-Progress-Report-2016-2019.pdf

 

5. Select Committee on Artificial Intelligence of the National Science & Technology Council.� �The National Artificial Intelligence Research and Development Strategic Plan: 2019 Update.� https://www.nitrd.gov/pubs/National-AI-RD-Strategy-2019.pdf

 

6. �Risk Management Framework (RMF) Overview: https://csrc.nist.gov/projects/risk-management/risk-management-framework-(RMF)-Overview

 

7. �AI Principles: Recommendations on the Ethical Use of Artificial Intelligence by the Department of Defense.� Defense Innovation Board. https://media.defense.noclick_gov/2019/Oct/31/2002204458/-1/-1/0/DIB_AI_PRINCIPLES_PRIMARY_DOCUMENT.PDF

 

 

KEYWORDS: AI, Artificial Intelligence, Anti-Submarine Warfare, ASW, Machine Learning, Decision-Support, Explainability