N231-015 TITLE: Back End Data Lake and Microservices (BEDLAM) Strategy for Battle Management Aid (BMA) Development

OUSD (R&E) CRITICAL TECHNOLOGY AREA(S): Artificial Intelligence (AI)/Machine Learning (ML)

The technology within this topic is restricted under the International Traffic in Arms Regulation (ITAR), 22 CFR Parts 120-130, which controls the export and import of defense-related material and services, including export of sensitive technical data, or the Export Administration Regulation (EAR), 15 CFR Parts 730-774, which controls dual use items. Offerors must disclose any proposed use of foreign nationals (FNs), their country(ies) of origin, the type of visa or work permit possessed, and the statement of work (SOW) tasks intended for accomplishment by the FN(s) in accordance with the Announcement. Offerors are advised foreign nationals proposed to perform on this topic may be restricted due to the technical data under US Export Control Laws.

OBJECTIVE: Develop a prototype integrated multiple Battle Management Aid (BMA) data lake, and expose the data available to developers for reuse while maintaining proper security boundaries for the software applications to protect intellectual property rights of all developers.

DESCRIPTION: The U.S. Government is in need of a method to standardize and add desired data and microservices into a common repository for use and reuse. Data sources are often common between applications but the data is delivered to the application as needed, such that a common data source and common data delivery occurs asynchronously and takes up available bandwidth for intra-application sharing multiple times. Similarly, developers often develop BMA in nongovernment-controlled repositories, which, despite inherent common microservices, may not use the same sources for those microservices (e.g., time servers, network protocols, etc.). As developers deliver code into the U.S. Government�s environment, re-development is often required to integrate a replacement set of microservices over the original baseline to adhere to the environment�s available source requiring rework to recode to use the U.S. Government source.

In addition, current-state microservices are typically limited to basic data such as time or position. As increasingly complex BMAs are developed, the potential for reuse, and therefore optimization, of shared data across BMAs is limited without a unified data strategy for development/security/operations (DevSecOps) environments. This SBIR topic seeks to take advantage of a subset of known data requirements across current BMAs, to include more complex data available as an output from a multitude of existing applications and leverage the current-state DevSecOps environment to provide that data as one of the available microservices for development. As the U.S. Government seeks to require developers not only to deliver, but actually develop in the U.S. Government�s DevSecOps environment, a more flexible back end data lake to enable sharing of data across BMAs in the same environment is desired. An added benefit is that as this government purpose rights data lake is rendered, it should facilitate application porting between environments without imposing an overhead cost, because all used commercial DevSecOps environments may adopt and expand it.

This SBIR topic seeks to enable management of big data by standardizing the mechanism for delivering data repeatedly to multiple applications on a shared network, using data fields common between differing applications. This is also an enabler for network-aware applications to be developed, because one of the common data fields will natively become the required information to interface with a routing stack, and to build the request for data into a common mechanism across the same network. As capabilities like Communications-as-a-Service (CaaS), which is a Program of Record requiring compliant applications to proliferate, are fielded, the same applications may be useful across different platforms with minimal Non-Recurring Engineering to integrate them, and this data lake could become a commoditized government furnished software product to all developers in the future.

The desired solution should be flexible and adaptable. As a given developer requests data and access to data sources, the back end data lake will optimize the computational, storage, and communications bottlenecks inherent in a large monolithic traditional development where feasible, to enable the same data to be accessible to multiple developers, whether contractor or government. The goal is to avoid duplicative storage of data� � and therefore the design might include data translation capabilities as infrastructure the data lake will host. The data lake should also provide an adequate mechanism for the U.S. Government to write BMA requirements into contractual efforts to leverage it, such as an application programmable interface (API) such as RESTful. Note, REST is an Architecture, not a Standard, but rather, it's an architectural style that provides constraints that guide API design. Many APIs do not conform to every element of REST, which has caused some to use the term RESTful to describe the most common types of APIs.

As an example task, many applications exist that enable specific physical layer control over a tactical radio, but these are typically developed by the radio manufacturer, necessitating a license cost for any network controller leveraging that particular application. For next-generation Naval Tactical Grid (NTG) applications, the Department of Navy (DON) must optimize the backend data lake supporting the front end graphical user interface for control and management of many network applications, including, but not limited to, the physical layer devices, crypto, and legacy interfaces using the available standardized interfaces that are becoming required, such as Secure Network Management Protocol (SNMP) for Common Data Link (CDL) systems (Bandwidth Efficient CDL Rev B specification is classified, this is the protocol required), Dynamic Link Enhancement Protocol (DLEP) for Tactical radios, and so forth; for translating to legacy interfaces at the data layer before delivering the requested data to the user interface layer.

Developing a single data lake that can be a repository for relevant data coming from all hosted applications, cognitively recognize when duplicative data is being requested via a services-oriented architecture, such as Communications-as-a-Service (CaaS).

The proposed solution should minimize the bandwidth of duplicate data traversing the network backbone, support all BMA developers in providing a unified data management strategy during development in a DevSecOps environment by enabling a common data lake to then provide microservices (e.g., map data, timing, navigation signals, own-ship position, etc.) across applications, rather than uniquely requesting that data across. This is often a bandwidth constrained communications backbone during wartime operations.

Proposed solutions should support standardized network management protocols such as simple network management protocol (SNMP) and adhere to government-defined data models such as the YANG data model. Proposed solutions should support Containers-as-a-Service (CaaS) development, to provide a solution for an initial data lake that could be supported in a DevSecOps environment such as Overmatch Software Armory (OSA) to support all BMA developers toward the end state of a common microservices architecture and data lake. Design should consider acquisition constraints for current-state processes for fielding new systems and applications to a shipboard environment in the strategy for implementation.

Work produced in Phase II may become classified. Note: The prospective contractor(s) must be U.S. owned and operated with no foreign influence as defined by DoD 5220.22-M, National Industrial Security Program Operating Manual, unless acceptable mitigating procedures can and have been implemented and approved by the Defense Counterintelligence and Security Agency (DCSA) formerly Defense Security Service (DSS). The selected contractor must be able to acquire and maintain a secret level facility and Personnel Security Clearances. This will allow contractor personnel to perform on advanced phases of this project as set forth by DCSA and NAVAIR in order to gain access to classified information pertaining to the national defense of the United States and its allies; this will be an inherent requirement. The selected company will be required to safeguard classified material IAW DoD 5220.22-M during the advanced phases of this contract.

PHASE I: Develop and demonstrate feasibility of a design citing industry standard methods for merging together the superset of data inputs and outputs from a sample set of existing applications and rendering that into a backend data lake with an accessible API for use in development of new applications. Methods for manipulating data into multiple requested formats from the raw repository state, providing micro services requests to and from applications, methods of enabling parallel processing, and methods of data management to minimize redundancy and optimize network performance for multiple data requests of the same time to different endpoints, are all in scope of this effort. A proposed implementation plan, including a mechanism for publishing new data sources, formats, and micro services coded to specific applications that can be tailored, should be included. The Phase I effort will include prototype plans to be developed under Phase II.

PHASE II: Develop a prototype data lake solution and reference implementation of BMA developer resource requests and automated delivery of requested data. An example using existing BMAs is not required, but would provide a meaningful deliverable. Implement into U.S. Government DevSecOps environment (specified by the topic�s Technical Point of Contact) and support BMA development in response to a validated fleet requirement specified by the government at kickoff with the microservices and data lake on the back end as prototype.

Work in Phase II may become classified. Please see note in Description paragraph.

PHASE III DUAL USE APPLICATIONS: Integrate government-specified third-party developers in refactoring, new development, or interfacing of at least two government-specified, third-party-developed BMAs to prove out the concept and continue to refine the application from Phase II to at least two service-level platform systems across the joint community in response to a validated fleet requirement.

Private sector has an equal, if not greater, requirement for big data analytics and real-time performance (e.g., analysis of market trends driving a decision to invest or divest in a given stock, fund, or sector).

REFERENCES:

1.       Mapeso, R. (2020, September 18). Why data lakes are more powerful for the DOD than commercial industry. Nextgov. https://www.nextgov.com/ideas/2020/09/why-data-lakes-are-more-powerful-dod-commercial-industry/168475/

2.       Boyd, A. (2020, September 29). Air Force wants novel ideas for building "data scientist�s ecosystems" at operations centers. Nextgov. https://www.nextgov.com/analytics-data/2020/09/air-force-wants-novel-ideas-building-data-scientists-ecosystems-operations-centers/168859/

3.       Haystead, John. (1997, October 1). Show me the data: High-speed commercial serial buses square off for real-time, military and aerospace applications. Military & Aerospace Electronics. https://www.militaryaerospace.com/computers/article/16710126/show-me-the-data-highspeed-commercial-serial-buses-square-off-for-realtime-military-and-aerospace-applications

4.       Tek, M. (2017, April 12). MIL-STD-1553B in avionics: where data networking has been and where it�s going. Intelligent Aerospace. https://www.intelligent-aerospace.com/commercial/article/16544804/milstd1553b-in-avionics-where-data-networking-has-been-and-where-its-going

5.       Department of Defense. (2006, February 28). DoD 5220.22-M National Industrial Security Program Operating Manual (Incorporating Change 2, May 18, 2016). Department of Defense. https://www.esd.whs.mil/portals/54/documents/dd/issuances/dodm/522022m.pdf

 

KEYWORDS: Data Management; Data Strategy; Big Data; Optimization; Software Development; Battle Management Aid

---