N222-119 TITLE: Next Generation Infantry Heads-up Displays for Close-Air Support

OUSD (R&E) MODERNIZATION PRIORITY: General Warfighting Requirements (GWR)

TECHNOLOGY AREA(S): Human Systems; Information Systems

OBJECTIVE: Develop next-generation daytime heads-up displays (HUDs) to provide training aids, operational tools, and situation awareness (SA) visualizations to improve the speed and quality of decision making by Marine Corps Ground Forces, specifically for close-air support (CAS) and call-for-fire (CFF).

DESCRIPTION: Ground forces must make rapid decisions in complex situations, such as requesting CAS, deconflicting airspace, and providing target information. In these situations, keeping heads-up and aware of the changing dynamics is critical. HUDs take advantage of augmented reality (AR) technologies to overlay information onto the battlefield and enhance SA. While HUD and AR systems have made progress in the past several years [Refs 1, 2], further innovation is required to develop systems for ground forces conducting CAS during daytime training and operations [Refs 3, 4]. Proposed solutions are sought to refine hardware and software requirements for Marine Corps use cases and deliver functional HUDs or HUD prototypes for next-generation AR HUD systems that can serve both as training aids and operational tools in CAS scenarios.

These systems must have maximum utility to Marines while maintaining survivability in a variety of complex environments. The display must be unobtrusive and mountable on existing Marine Corps helmet Night Vision Goggle (NVG) rails. The general device requirements are: (1) a low-cost (< $10,000) optical or video-see through HUD that is rugged (e.g., for outdoor use); (2) has a small form-factor; (3) is very low weight; (4) has ultra-low electronic power requirements; and (5) is capable of high-resolution operation. Specific device optical requirements include: (1) field-of-view (FOV) approaching 120 degrees width and 80 degrees height; (2) a blended, high-resolution 60 pixel/degree Field of View (FOV) across the foveated display area; and (3) a head-mounted display (HMD) with a refresh frame rate above 90 Hz. For requirements of form-factor size and weight, power requirements, and high-resolution operation (general device requirements 2-5), we are not identifying specific targets in this topic call. The solicitors expect performers to make trade-offs between the listed requirements and justify their decisions during Phase I. Priority should be given to higher resolution, lower latency, and smaller size and weight (in that order).

Proposals must detail how hardware and software systems will address physical ergonomics [Ref 5] and cognitive performance (i.e., situation awareness, decision making [Ref 6]) concerns for use in training and operations by Marine Corps Infantry. Proposals do not need to detail development of a complete AR system, but they must describe how they will investigate and evaluate their proposed hardware and software innovation. Development should be done with technologies that have little-to-no licensing fees for development or execution (e.g., Unity), and focus primarily on HUD systems, not AR-related technologies (e.g., tracking, object insertion, etc.). The training and operational use case of interest is daytime Marine Corps CFF and CAS missions.

PHASE I: Develop a concept for a low-cost (< $10,000), high-performance HUD to superimpose computer-generated information on an individual�s view of the real world. Demonstrate the feasibility of the selected concept (hardware/software HUD-centric system) to meet Marine Corps infantry needs through a set of specific Phase I deliverables.

Standard deliverables that are a part of every SBIR Phase I contract include: (1) kick-off brief; (2) progress reports; and (3) a final report. Additional deliverables include: (1) an initial prototype; (2) a computer aided design (CAD) mechanical design package showing the top-level device and all major sub-assemblies anticipated; and (3) trade-off design decisions and associated justification for system design and human factors considerations.

PHASE II: Develop at least two working proof-of-concept HUDs for the Marine Corps. Conduct critical design reviews. Demonstrate that initial capabilities are sufficient for existing AR training applications. Facilitate evaluation of the prototypes to determine their capability to meet Marine Corps needs and requirements for an augmented reality HUD.

Deliverables include: (1) a final bill-of-materials (BOM); (2) all CAD drawings, hardware schematics, software source code; and (3) at least two proof of concept devices for evaluation.

PHASE III DUAL USE APPLICATIONS: Support the Marine Corps in transitioning the HUD system. Support the Marine Corps with integrating the HUD into existing AR training devices. Assist with certifying and qualifying the HUD system for Marine Corps use. Assist in writing Marine Corps device user manual(s) and system specifications/materials. As appropriate, focus on scaling up manufacturing capabilities and commercialization plans. Specific examples of commercial markets that could use this technology include manufacturing, law enforcement, and other hands-on tasks in time-critical domains.

REFERENCES:

1. M. Sizintsev, A. Rajvanshi, H. -P. Chiu, K. Kaighn, S. Samarasekera and D. P. Snyder, "Multi-Sensor Fusion for Motion Estimation in Visually-Degraded Environments," 2019 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR), 2019, pp. 7-14, doi: 10.1109/SSRR.2019.8848958.
2. Rozman, J. (2020). The Synthetic Training Environment. Spotlight SL, 20-6.
3. Schaffer, R., Cullen, S., Cerritelli, L., Kumar, R., Samarasekera, S., Sizintsev, M. Branzoi, V. (2015). Mobile augmented reality for force-on-force training. Interservice/Industry Training, Simulation and Education Conference Proceedings.
4. Samarasekera, S., Kumar, R., Zhu, Z., Branzoi, V., Vitovitch, N., Villamil, R., Garrity, P. (2014.) Live augmented reality-based weapon training for dismounts. Interservice/Industry Training, Simulation and Education Conference Proceedings.
5. Rebensky, S., Carroll, M., Bennett, W., & Hu, X. (2021). Impact of Heads-up Displays on Small Unmanned Aircraft System Operator Situation Awareness and Performance: A Simulated Study. International Journal of Human�Computer Interaction, 1-13
6. Wickens, C. D., & Alexander, A. L. (2009). Attentional tunneling and task management in synthetic vision displays. The International Journal of Aviation Psychology, 19(2), 182-199.

KEYWORDS: Augmented Reality; AR; Virtual Reality; VR; Heads-up-display; HUD; Training; Infantry; Close-Air Support; CAS; call-for-fire; CFF

** TOPIC Q&A **

Questions answered 6/3/22 Q1. Are you looking for a HUD system that converts the view a warfighter currently sees on their tactical phone to a heads up screen? Sort of like a giant transparent floating screen in front of or to the side of the operator. Or are you looking for the user interface to take the incoming tactical data and display it in Augmented Reality in the real-world. Where, for example, the warfighter could look downrange and see target symbology hoverning in the air at the correct coordinates and look left and see the location of a friendly force. If the you are looking for AR style, is the development of this UI software and display methodology in scope or out of scope for this SBIR. It seems like some amount of UI or demo UI is in scope since you mention Unity. But mainly I ask because an AR solution will require additional sensors to give the HUD its pose or position in the real-world and I did not want to burden the system with that SWAP and cost if it is not necessary. A1. The current preference from end users is a monocular system and 60 pixels/degree across the monocular system. The HUD functionality is under development by different performers managed by the ONR TPOC (a news release can be found here - https://www.onr.navy.mil/Media-Center/Press-Releases/2021/ONR-Tests-Virtual-Training-Systems-for-JTACs-Fire-Support-Marines). We primarily care about the HUD hardware itself.

Questions answered 5/27/22 Q1. Just to be clear, this is for a see-through AR type daytime HUD correct (for example you can see through it when it is powered off)? And not a MR (mixed reality) style HUD which is more like a VR rig with low latency cameras strapped to the front. I'm asking because hitting the FOV numbers specified is two orders of magnitude harder in an AR type HUD than a MR rig. A1. Yes, this SBIR topic is for a see-through AR-type daytime HUD. It is not expected that you can see through it if the display is powered off. We are not looking for an MR-style HUD in the way you described it.

Questions answered 5/10/22 Q1. What type of heads-up display is of interest? A1. This topic is only focused on a daytime heads-up display unit.