Read The Use of Integrated Architectures to Support Agent Based Simulation: An Initial Invesigation text version

THE USE OF INTEGRATED ARCHITECTURES TO SUPPORT AGENT BASED SIMULATION: AN INITIAL INVESTIGATION

THESIS Andrew W. Zinn, Captain, USAF AFIT/GSE/ENY/04-M01 DEPARTMENT OF THE AIR FORCE AIR UNIVERSITY

AIR FORCE INSTITUTE OF TECHNOLOGY

Wright-Patterson Air Force Base, Ohio

APPROVED FOR PUBLIC RELEASE; DISTRIBUTION UNLIMITED

The views expressed in this thesis are those of the author and do not reflect the official policy or position of the United States Air Force, Department of Defense, or the United States Government.

AFIT/GSE/ENY/04-M01

THE USE OF INTEGRATED ARCHITECTURES TO SUPPORT AGENT BASED SIMULATION: AN INITIAL INVESTIGATION

THESIS

Presented to the Faculty Department of Aeronautics and Astronautics Graduate School of Engineering and Management Air Force Institute of Technology Air University Air Education and Training Command In Partial Fulfillment of the Requirements for the Degree of Master of Science in Systems Engineering

Andrew W. Zinn, BS Captain, USAF March 2004

DISTRIBUTION STATEMENT PENDING

AFIT/GSE/ENY/04-M01

THE USE OF INTEGRATED ARCHITECTURES TO SUPPORT AGENT BASED SIMULATION: AN INITIAL INVESTIGATION

Andrew W. Zinn, BS Captain, USAF

Approved:

____________________________________ David R. Jacques (Chairman) ____________________________________ John O. Miller (Member) ____________________________________ Larry B. Rainey (Member)

____________ date

date ____________ date

AFIT/GSE/ENY/04-M01 Abstract

Architecture descriptions following the DoD Architectural Framework (DoDAF) will be used to aid in the acquisition of all major defense information systems. One of the primary purposes of these architectures is to help conduct military-worth analysis. Recent work in operations analysis of information driven combat is showing that agent based simulation technology is needed to understand the military value of Command, Control, Communications, Computers, Intelligence, and Reconnaissance (C4ISR) systems. This thesis investigates the utility of architecture descriptions based on the DoDAF to provide the needed data for agent based simulation. This is accomplished by means of a case study where data from a proposed Air Operations Center architecture is used in the combat model System Effectiveness Analysis Simulation (SEAS). The research concludes that the DoDAF, if implemented properly, does provide the needed information. A process for taking information from a DoDAF architecture and importing it into agent based simulation is proposed.

iv

AFIT/GSE/ENY/04-M01

To my wife

v

Acknowledgments

I would like to thank my advisor, Dr. David Jacques, for his guidance throughout this effort. Comments and suggestion for improvement by Dr. Jacques, as well as Dr. J.O. Miller, Lt Col Colombi and Dr. Larry Rainey contributed greatly to the quality of this document and the research in general. I would especially like to thank my sponsor, Mr. Robert Weber, who had the foresight and vision to propose this topic. His expertise in combat simulation and systems development was a key resource, and the background information he provided was indispensable. I would also like to thank Jay Vittori, Dr. Tom Pawlowski, Steve Ring, Scott Surer, Lt Col Mark Hall, and others in the system architectures community who took the time to lend me advice and guidance. Greg DeStefano, my partner on this research, deserves the credit for turning theory into results. A special thanks goes out to Greg for his hard work, analyst insight, and modeling skills. Finally, I would like to thank my wife. Her encouragement and support helped me keep things in perspective when the going got tough. Andrew W. Zinn

vi

Table of Contents Page Abstract .............................................................................................................................. iv Table of Contents.............................................................................................................. vii List of Figures .................................................................................................................... ix List of Tables ..................................................................................................................... xi I. Introduction and Problem Statement .............................................................................. 1 1.1 Introduction.............................................................................................................. 1 1.2 Problem Formulation ............................................................................................... 5 1.2.1 General Problem .................................................................................................5 1.2.2 Specific Problem................................................................................................6 1.3 Relevant Activity ..................................................................................................... 7 1.4 Scope of Work ....................................................................................................... 11 II. Background ................................................................................................................. 14 2.1 Background on Architectures in the DoD.............................................................. 14 2.1.1 The DoD Architectural Framework Version 1.0 .............................................18 2.1.2 The CADM and DARS....................................................................................29 2.1.3 DoDAF Guidance ............................................................................................30 2.2 Popkin System Architect........................................................................................ 32 2.2.1 IDEF3 Notation ...............................................................................................35 2.3 The AOC Architecture........................................................................................... 36 2.4 The C4I Support Plan............................................................................................. 39 2.5 Background on DoD Modeling and Simulation .................................................... 39 2.5.1 SEAS................................................................................................................41 2.6 Summary ................................................................................................................ 44 III. Methodology .............................................................................................................. 45 3.1 Introduction............................................................................................................ 45 3.2 Agents in SEAS ..................................................................................................... 45 3.3 The Kosovo War File............................................................................................. 48 3.4 Applying Previous Research to the Case Study..................................................... 49 3.5 Fitness of the DoDAF to support Agent Based Simulation................................... 52 3.6 Choosing the Architecture(s) ................................................................................. 53 3.7 Setting up the Case Study ...................................................................................... 56 3.7.1 Mapping the General Attributes ......................................................................57

vii

3.7.2 Mapping the Communications.........................................................................59 3.7.3 Mapping the Orders .........................................................................................62 3.8 Summary ................................................................................................................ 68 IV. Results and Analysis.................................................................................................. 70 4.1 Results.................................................................................................................... 70 4.1.1 General Attributes............................................................................................70 4.1.2 Communications ..............................................................................................71 4.1.3 Orders ..............................................................................................................75 4.1.4 The Modified SEAS War File .........................................................................80 4.1.5 Using the New War File ..................................................................................83 4.2 Analysis.................................................................................................................. 86 4.2.1 Missing Pieces .................................................................................................87 4.2.2 Potential for Automation .................................................................................88 4.2.3 Comparison to DoDAF's "Architectural Uses" Figure 2-2.............................89 V. Conclusions and Recommendations ........................................................................... 91 5.1 Conclusions............................................................................................................ 91 5.2 Recommendations.................................................................................................. 92 5.2.1 Recommendations for Architecture Developers..............................................92 5.2.2 Recommendations for Tool Vendors...............................................................94 5.2.3 Recommendations for Further Research .........................................................94 Appendix A: DoDAF Products list (from section 2.2 of Volume II) .............................. 96 Appendix B: DoDAF's Architecture Products by Use (from section 2.2 of Volume II) 97 Bibliography ..................................................................................................................... 98 Vita.................................................................................................................................. 102

viii

List of Figures Page Figure 1 Requirements and Acquisition Process Depiction (DoD 5000.2, 2003:3).......... 2 Figure 2 Architecture Impact Areas (AF-CIO/A https://cao.hanscom.af.mil/af-cio.htm). 4 Figure 3 Using Architectures in SE and Acquisition (Dickerson and Soules, 2000) ........ 8 Figure 4 EAMA Viewchart (Pawlowski et al, 2003:slide 24) ......................................... 10 Figure 5 Zachman Framework (www.zifa.com).............................................................. 17 Figure 6 Three Views of Architecture (DoDAF, 2003)................................................... 20 Figure 7 IDEF0 Model Syntax (Levis, 2003:3-26).......................................................... 24 Figure 8 Integrated Architectures (DoDAF Vol I, 2003:Fig 3-3).................................... 25 Figure 9 Building Architectures (DoDAF Volume I, 2003:5.2)...................................... 31 Figure 10 IDEF3 Symbols (Mayer et al, 1995:22) .......................................................... 36 Figure 11 Example of IDEF3 diagram (Mayer et al, 1995:40)........................................ 36 Figure 12 Agent Hierarchy (from DeStefano, 2004:2.3.3.1)........................................... 42 Figure 13 SEAS Top-Level Event Processing (Gonzales et al, 2001:56) ....................... 43 Figure 14 The 3 Entities in SEAS (www.teamseas.com SEAS_Agents.ppt:3)............... 45 Figure 15 A Conceptual SEAS Agent (www.teamseas.com SEAS_Agents.ppt:8) ........ 46 Figure 16 The Synthesis Phase (Levis and Wagenhals, 2000) ........................................ 50 Figure 17 Mapping General Attributes from a DoDAF Architecture to SEAS............... 59 Figure 18 Sample SV-2 (DoDAF Volume II, 2003:5-13) ............................................... 61 Figure 19 Mapping Communications from a DoDAF Architecture to SEAS ................. 62 Figure 20 Example of IDEF3 diagram (Mayer et al, 1995:40)........................................ 65 Figure 21 Sample OV-5 diagram for "Select Contractor" ............................................... 65

ix

Page Figure 22 Example of OV-5 Report + Pseudocode from OV-6a..................................... 66 Figure 23 Mapping Orders from a DoDAF Architecture to SEAS .................................. 68 Figure 24 Mapping DoDAF Products to Agent Based Simulation................................... 69 Figure 25 OV-5 Activity Report Generated by Popkin SA for TCT Activity 6.............. 76 Figure 26 IDEF3 diagram for "Conduct Dynamic Assessment of Target" (TCT 2005 Architecture, 2003:OV-6a) ....................................................................................... 78 Figure 27 Comparison of Baseline vs. Modified War File - Kosovar Casualties (DeStefano, 2004:4.3)............................................................................................... 84 Figure 28 Sorties Flown - TCT vs. Baseline War File (DeStefano, 2004:4.3)................ 86

x

List of Tables Page Table 1. Popkin DoDAF Product Implementation (Popkin, 2003:slide 45).................... 33 Table 2 Sample SV-6 ....................................................................................................... 60 Table 3 Modified Sample SV-6 ....................................................................................... 61 Table 4 System Data Exchange Spreadsheet (AOC Architecture, 2003:SV-6) ............. 71 Table 5 Modified Information Exchange Matrix (AOC Architecture, 2003:OV-3 with SV-2)......................................................................................................................... 74 Table 6 Paired T-Test Results for "All Agents Killed" (DeStefano, 2004:4.3)............... 85

xi

THE USE OF INTEGRATED ARCHITECTURES TO SUPPORT AGENT BASED SIMULATION: AN INITIAL INVESTIGATION

I. Introduction and Problem Statement

1.1 Introduction The Department of Defense has committed itself to the use of integrated architectures to aid in the development of all future weapons systems. These architectures must follow the guidelines laid out in the Command, Control, Communications, Computers, Intelligence, and Reconnaissance (C4ISR) Architecture Framework (soon to be superseded by the DoD Architecture Framework v 1.0, known as the DoDAF). The C4ISR Framework version 2.0 states that, "The purpose of C4ISR architectures is to improve capabilities by enabling the quick synthesis of `go-to-war' requirements with sound investments leading to the rapid employment of improved operational capabilities and enabling the efficient engineering of warrior systems." Recently, the DoD revised the acquisition policy memorandums, known as the 5000 series. DoD 5000.1, in reference to the interrelationships between weapons systems, makes it policy that "Joint concepts and integrated architectures shall be used to characterize these interrelationships." (DoD 5000.1, 2003:5). The new DoD 5000.2, entitled "Operation of the Defense Acquisition System" explicitly states the DoD policy toward architectures: The Under Secretary of Defense (Acquisition, Technology, and Logistics) (USD(AT&L)), the Assistant Secretary of Defense for Command, Control, Communications, and Intelligence (ASD(C3I)), the Joint Staff, the Military Departments, the Defense Agencies,

1

Combatant Commanders, and other appropriate DoD Components shall work collaboratively to develop joint integrated architectures for capability areas as agreed to by the Joint Staff. (DoD 5000.2, 2003:2) The 5000.2 goes on to say that these architectures should be used for developing roadmaps to "conduct capability assessments, guide systems development, and define the associated investment plans as the basis for aligning resources and as an input to the Defense Planning Guidance, Program Objective Memorandum Development (POM), and Program and Budget Reviews" (DoD 5000.2, 2003:3). In other words, architectures will provide much of the foundation upon which the rest of the acquisition process depends. Figure 1 illustrates the role of architectures in system development. Exactly how architectures will be used in this process is the subject of ongoing research.

Overarching Policy NSS/NMS/Joint vision Joint Concept of Operations

Functional Area Analysis

Functional Area Functional Concept Integrated Architecture Feedback

D O T M L P F

- Materiel Process Analysis of Materiel Approaches

MS B

CDD MS A JROC DAB AoA ICD CD Concept Refinement Technology Development CPD

MS C

DOTLPF Process

JROC DAB

JROC

DAB

Demo

Increment 1 MS B Demo MS C Increment 2 MS B Demo MS C Increment 3

Figure 1 Requirements and Acquisition Process Depiction (DoD 5000.2, 2003:3)

The defense communities' commitment to the use of architectures in system development is not only backed up by policy but also by law, including the Clinger-

2

Cohen Act and the Office of Management and Budget's Circular A-130, Management of Federal Information Resources. A complete list of architectural "Governance and Guidance" can be found in section 2.1 of the Air Force Enterprise Architecture Framework Version 2.0 or in section 2 of the new DoDAF, Volume I. A tell-tale indication that architectures will be a part of future systems planning comes out of the recent DoD transformation movement. As part of the transformation, the Joint Capabilities Integration and Development System (JCIDS) was created to replace the old Requirements Generation System. The new JCIDS points to integrated architectures as the primary means to evaluate systems prior to milestone decision points. Furthermore, the DoD states that integrated architectures will completely replace the Capstone Requirement Document once architectures "are fully available and mature" (DoD Business Transformation Brief, 2003:slide 13). As was shown in Figure 1, architectures will be required for every high level review in a product's life cycle as, "Integrated architecture products must be included in mandatory appendixes for the Initial Capabilities Document (ICD), Capability Development Document (CDD), and Capability Production Document (CPD)" (DoDAF, 2003:2.3.2). Clearly, the DoD recognizes that integrated architectures will provide the foundation for JCIDS analysis in the future. The means by which this analysis will be performed are still being investigated. The architecture mandate is well known within the defense community; nevertheless, the full benefits of using architectures are not. Most often, the complexity and required interoperability of modern day "systems of systems" is the reason sighted for the use of architectures. This, however, is just one of the benefits of the use of

3

architectures. The Air Force Chief Architect's (AFCAO) website lists three key impact areas where the use of architectures provides real benefit, shown in Figure 2:

Figure 2. Architecture Impact Areas (AF-CIO/A at https://cao.hanscom.af.mil/af-cio.htm)

Often forgotten are the benefits offered in the programming and planning stage, of which "mil-worth analysis" is a part. The AFCAO lists M&S as a potential way to harness the information in an architecture to aid in the planning of a system. Furthermore, the C4ISR Framework v 2.0 states that architectures built under the framework can be compared to judge military utility and aid decision makers. Dr. Alexander Levis, the Chief Scientist of the Air Force, notes that this is a worthy goal, but the framework lacks explicit guidance on how to achieve it. He presents his own ideas on how to do this: "The derivation of an executable model of the architecture from the three views and the associated integrated dictionary provides a basis for understanding the interrelationships among the various architecture products and establishes the foundation for implementing a process for assessing and comparing architectures" (Levis and Wagenhals, 2000:226).

4

An executable architecture (referred to by Levis as an executable model) is defined by the DoDAF as "use of dynamic simulation software to evaluate architecture models" (DoDAF, 2003:7.3). Levis goes on to assert the value of the executable model, "To fully describe and understand the dynamic aspects of the system requires an executable model" (Levis and Wagenhals, 2000:228). Essentially, the static diagrams presented in the architecture do not adequately convey, "the logical, behavioral, and performance properties of the architecture." Thus, to Levis, the creation of the executable model is essential to be able to portray a dynamic system and measure its utility versus other systems. To implement the executable model, Levis (along with Handley and Zaidi) has advocated the use of the Colored Petri Net (CPN) (Handley et al, 2000). CPNs are a means to create an executable model that ". . . combines all the information in the various static models or views into one model that can illustrate dynamic behavior: the flow of data, the conditions under which functions are performed, and the order in which they are performed. Behavior analysis and performance evaluation can then be carried out using scenarios consistent with the operational concept." (Handley et al, 2000:15). 1.2 Problem Formulation 1.2.1 General Problem Research by Levis and Wagenhals has shown that data from architectural products can be used to generate an executable CPN (Wagenhals et al, 2000). Relatively little work has been put into taking data from the architectures of future weapon systems and exercising them in a constructive combat simulation environment. This capability would have far reaching effects on the ability of planners to evaluate and shape the future force structure.

5

1.2.2 Specific Problem While the CPN is certainly a valuable tool for understanding the dynamic behavior of a system, it falls short in its ability to model a combat environment where the rules of engagement (ROE) are changing and the enemy model is learning and evolving. Clearly, to evaluate the military worth of a future system, you must do so in a simulated combat environment ­ one in which the enemy has his own architecture. The CPN is not able to effectively model a variable environment like a battlefield (Klügl et al, 2002). To address these concerns, one must look toward the Modeling and Simulation (M&S) community, which has been taking on these issues for decades. Lately, the M&S community has been undergoing a transformation of its own. Recent work in operations analysis of information driven combat is showing that agent based simulation technology is needed to understand the military value of C4ISR systems when matched with suitable operations concepts (Gonzales et al, 2001 and Weber, 2003). Older "aggregated" models based on Lanchester equations were not originally designed to model the effects of C4ISR systems or take into account the opposing forces' concepts of operations (CONOPS). More background on this issue will be provided in Chapter Two of this document. To address this problem, the System Effectiveness Analysis Simulation (SEAS) was developed for the Air Force's Space and Missile Systems Center (SMC). Independent studies have confirmed that SEAS, partially due to its ability to characterize "hierarchies of interacting agents that can adapt their behavior," can successfully model the impact of Information Superiority (IS) on the battlefield (Gonzales et al, 2001:11). Unfortunately, the accurate representation of C4ISR systems in agent based simulation software such as SEAS constitutes a significant interface problem. Because agent based

6

models like SEAS model systems at a high level of detail, programming them "by hand" requires extensive research and exhaustive testing (Weber, 2003). Given the emergence of architectures in the DoD mentioned at the beginning of this thesis, it is logical to look to architectures as a source for the needed simulation data. The research explained below has looked into doing just that. 1.3 Relevant Activity Supporting Levis' assertion on the value of the executable model is a study by the US Navy, which has begun to use architectures not only to aid in the design of new systems, but also to improve the acquisition process. In a study conducted in 1999, the Navy showed that C4ISR architectures can be used effectively "to enable a capabilities based approach to the planning and acquisition of DoD families of systems." (Dickerson and Soules, 1999:2). The study documented the Navy's Time Critical Targeting process in a Fleet Battle Experiment using DoDAF products. The results were then successfully used to influence the next POM cycle. The below figure summarizes the process used by Dickerson and Soules. Note that the "3rd Order Analysis: Dynamic Interoperability" is a key step in forming the acquisition strategy.

7

Figure 3. Using Architectures in SE and Acquisition (Dickerson and Soules, 2000)

There is a huge push in the simulation community to become more interoperable with C4ISR systems. The C4ISR Technical Reference Model (TRM) is currently being developed with the goal of becoming the standard frame of reference between C4ISR and Modeling and Simulation (M&S) systems (Griffin, 2003:2.2). In a position paper for the 2003 International Conference on Grand Challenges for M&S (ICGCMS '03), Dr. Andreas Tolk proposed the grand challenge of "merging of C4ISR and M&S components into heterogeneous systems supporting the Warfighter" (Tolk, 2003:1). These and other efforts have the top level goal of increasing interoperability between the two communities, but are focusing on the interface of existing C4ISR systems with virtual simulations. Left largely unaddressed is the possibility of building constructive combat simulations based

8

on architectural data, a capability which would have significant impact on both communities. Pertinent research in this area is currently being conducted by Pawlowski, Barr, and Ring at MITRE Corporation with the Executable Architecture Methodology for Analysis (EAMA). Recognizing the limits imposed by the static nature of most architecture products, EAMA is developing a methodology to convert static architecture products into executable architectures to: · · · Support dynamic analysis of a system or capability Measure process performance Measure organizational work efforts & resource utilization over time (Pawlowski et al, 2003:slide 5)

EAMA uses architectural products as the foundation for a Petri-Net based executable model, much like Levis has advocated above. The Petri-Net is executed in a business process model called Bonapart, which uses a communications network model (NS2) to simulate communications delays. The goal of all this is to essentially provide a time delay estimate to Eagle, a top level combat simulation. Figure 4 gives an example of how this might work for a simulated artillery duel; the horizontal axis represents time. The figure shows how analysts can use the results to determine Measures of Force Effectiveness (MOFEs) ­ in this case the ability to destroy enemy artillery, and Measures of Effectiveness (MOEs) ­ in this case the time required to process and conduct counterbattery fire.

9

Mitre

Model Interactions & Sample Measures of Merit

MOFE - Target Destroyed

Eagle Operational Context

E1 E2 - Enemy Artillery March Order Key Event E1 (Enemy Artillery Fire) Triggers Mission Thread A1 - A4 (Counterfire)

OA MOE - Overall Staff Process Time

Bonapart

OA Process Model

Target Data Sent A1 Target Data Received TIDAT Sent A2 TIDAT Received Fire Mission Sent DIVARTY FCE A3 Fire Mission Received A4

Mission Thread Complete

OA MOP Staff Bottlenecks

MLRS

NS2 SA Network Model

Radar

G2

SA MOP Network Bottlenecks

Node in Comms Network

24

Figure 4 - EAMA's "Modal Interactions and Sample Measures of Merit" (Pawlowski, 2003:slide 24)

The EAMA process generates it's own Measures of Performance (MOPs) from Bonapart and NS2 simulations, which were built with the aid of architectural products. For example, the time it takes to send target information over the network is a potential bottleneck - it is collected as an MOP, and used to calculate the overall process time (which is collected as the above mentioned MOE). EAMA allows for a high resolution look at important details like the identification of communications bottlenecks and areas for improvement for internal business processes. Furthermore, unlike the isolated PetriNet executable models, exercising the architecture in Eagle allows it to be tested against a simulated enemy. However, since Eagle is a deterministic combat model, the application

10

of this federation of simulations to the evaluation of C4ISR architectures is not as useful as one using a stochastic combat model such as SEAS which plays system errors and flawed perceptions of the various agents. Whenever assessing risk of undesirable outcomes is important, the analyst needs to start with a stochastic simulation of the dynamic activity (Weber, 2003). Furthermore, what if one is only interested in a top level evaluation of a weapons system? In a higher level analysis, MOFEs like "time to halt" or "loss/kill ratio" take precedence over 2nd and 3rd order MOPs like communications delays and processing times. Often, the objective is to evaluate how a new weapons system fits into the existing battlespace. In this case, the performance parameters of the weapons system could potentially be taken directly from the architecture. One would expect that the transition from architecture to simulation could, therefore, be less complex and theoretically go directly to the combat simulation model. Agent based simulations like SEAS are ideally suited for conducting this top level analysis of first order effects (Gonzales et al, 2001:54). The above researchers recognize the full potential of architectures can only be reached through the help of modeling and simulation. While the creation of executable models has been going on for some time, the debate goes on as to the best way to use them to aid decision makers in the DoD. The solution will require a collaborative effort between the systems engineering and operations research community to bridge the gap between defense systems architectures and M&S. 1.4 Scope of Work Planners and operations analysts would benefit greatly from a methodology that would guide the transfer of information from a system's architectural description into an

11

agent based simulation for further analysis of military value in specific conflict scenarios. The combined and coordinated use of the new DoD Architectural Framework Template (the DoDAF) representations from the engineering community in conjunction with simulation of operational dynamics in agent based software offers great promise in creating for the first time a collaborative, quantitative infrastructure which bridges the current chasm between the system acquisition community and that of the operational warfighting commands. This thesis investigates the feasibility of using information from a weapon system's DoDAF based architecture to model the system in an agent based simulation. Popkin System Architect, a commercially available architecting tool, and SEAS are the primary software involved. Key questions being addressed include, "what data is required to model in SEAS, and do architectures (as defined by the DoDAF) provide it?" The problem is attacked in three ways. First, the DoD Architectural Framework is reviewed to asses its theoretical potential to provide data for agent based simulation. Next, several current DoD architectures are evaluated to determine if actual architectures are capturing relevant data required for simulation. One of these architectures will be chosen to help validate initial theories. Simultaneously, Capt Greg DeStefano will be researching exactly what is required in order to adequately model a weapons system in an agent based simulation. Capt DeStefano's research (DeStefano, 2004) and this research will be a collaborative effort approaching the problem from the Operational Analyst's viewpoint and the Systems Engineer's viewpoint, respectively. Finally, a case study is conducted where data from the Air Operations Center (AOC) Architecture will be taken from Popkin System Architect and imported into a SEAS scenario based upon the

12

Kosovo conflict. Conclusions are drawn on the best way to conduct the "mapping" of information from architecture to simulation and the potential for automating the process. Chapter 2 of this thesis reviews the use of architectures in the DoD and the rise of agent based simulation. Chapter 2 also provides background on Popkin System Architect and SEAS. Chapter 3 outlines the proposed methodology that will be used in the case study, and provides a detailed description of the AOC Architecture and Kosovo SEAS scenario. Chapter 4 provides the results of the case study. In order to determine if the changes made to the existing SEAS model were significant, a comparison with the baseline model is described. Chapter 4 also contains an analysis of the results, including a look at the potential to automate the process. Chapter 5 will offer recommendations for improving the tie between architectures and simulation and also proposes future research in this area.

13

II. Background 2.1 Background on Architectures in the DoD The series of events that culminated with the DoD mandate for the use of architectures started with the lessons learned from the Persian Gulf War of 1991. According to Dr. Levis, the primary drivers were uncertainty and interoperability (Levis, 2003:1­55). The operations in the Gulf showed that there is a wide degree of uncertainty for requirements due to a constantly changing battlefield. While older platforms were being used in ways never envisioned, newer technology was being applied piecemeal. In the words of Dam and Willis, "the systems associated with C4ISR were becoming increasingly complex and rapidly obsolete due to the explosion of capabilities provided by the information technology revolution" (Dam and Willis, 2002:1). This contributed to Levis's other main driver: interoperability problems. The interoperability problem was evident in Joint and Coalition operations as well as with communication within the DoD (Levis, 2003:1-55). A classic example of service specific systems not talking to each other was how paper copies of the Air Tasking Order (ATO) had to be flown from the command center in Riyadh to the decks of aircraft carriers. Clearly, there was a major problem conducting joint operations. In order to fix this, joint planning was required, but this was also lacking. According to Dam and Willis, "the services, commands, and DoD agencies had a penchant for designing and developing their own independent capabilities and solutions despite extraordinary similarities among the requirements, performance criteria, and applications contexts. At the end of these independent trails, the resulting systems would not work well together further complicating difficult joint operations"

14

(Dam and Willis, 2002:1). The information age had changed the nature of warfare, and the acquisition process needed to adapt. As Eberhardt Rechtin, one of the leaders in the field of Systems Architecting, said in 1991: "The proliferation of information system architectures, as we shall see is accelerating. As a result, some of the newest approaches to architecting have come from information systems as the conventional approaches to searching for solution ­ trial, error, and analysis ­ prove inadequate." (Rechtin, 1991:10) Of course, trial and error on the battlefield can lead to disaster and loss of life. With the lessons of the Gulf War fresh on the mind, the DoD began looking for a solution. It was in this context that the services were studying the science of acquiring and using information systems. Three important studies were conducted between 1993 and 1995 that pointed to architectures as the answer (Levis, 2003:1-56). Realizing that architectures had the potential to address the issues of uncertainty and interoperability and recognizing that architectures should be standardized across the services, the DoD took the first steps to forming a framework. In an October 1995 Memo, the Deputy Secretary of Defense stated, "I am directing the acceleration of the development of C4I integration and architecture efforts through the creation of a DoD-wide C4I Integrated Product Team." the goals of which would include, "to define and develop better means and processes to ensure C4I capabilities most effectively meet the needs of the warfighters" (Nimz, 2000:8). This direction resulted in the creation of the Architecture Working Group (AWG). The AWG had the daunting task of figuring out what a DoD architecture should look like, but they weren't starting from scratch. The DoD was not the first organization to apply architectures to the design of information systems. Rechtin states that, "Systems Architecting is the process of creating

15

(conceptualizing, designing, and building) unprecedented, complex systems" (Rechtin, 1991). According to IEEE an architecture is, "the fundamental organization of a system embodied in its components, their relationships to each other and to the environment and the principles guiding its design and evolution" (IEEE STD 1471, 2000). The AWG needed to bring standardization and structure to DoD organizations and services who "traditionally developed their C4ISR architectures using techniques, vocabularies, and presentation schemes that suited their unique needs and purposes" (C4ISR AF v 2.0, 1997:1-2). This would be done by creating a framework that would set standards based on best practices, and allow different architectures to be compared. Several architectural frameworks were already in existence at the time. The business community often uses the Zachman Framework, created by IBM researcher John Zachman in the 1980s. According to Zachman, the framework was created to help companies in "dealing with the complexities and dynamics of the Information Age Enterprise" (Zachman, 1997). The framework (shown below) is essentially a matrix of 36 cells representing the what (data), how (function), where (network), who (people), when (time), and why (motivation) at 6 different levels from context down to detail.

16

Figure 5. Zachman Framework (www.zifa.com)

While the Zachman and other frameworks had proven themselves in the business world, the DoD required something different. Most frameworks are set up to aide in the selling of goods or services to make a profit; this is not relevant to the DoD. At this time, congress passed a comprehensive bill designed to reform and streamline the federal acquisition system known as the Information Technology Management Reform Act (or Clinger-Cohen). The Clinger-Cohen Act mandated the use of architectures in executivelevel departments to "develop, maintain, and facilitate integrated IT (Information Technology)" (Levis, 2003:1-8). The Office of Management and Budget (OMB) circular A-130 further defined this mandate by requiring the architectures to have a framework. Now the use of architectures had not only been recommended but essentially made law. In June 1996 the AWG released the C4ISR Architectural Framework version 1.0. After a year and a half of much needed additions and revisions, version 2.0 was released. The

17

Undersecretary of Defense for Acquisition and Technology then mandated the use of the new framework in a February 1998 memo saying, "We see the C4ISR Architecture Framework as a critical element of the strategic direction in the Department, and accordingly direct that all ongoing and planned C4ISR or related architectures be developed in accordance with Version 2.0. Existing C4ISR architectures will be redescribed in accordance with the Framework during appropriate revision cycles." (USD 23 Feb 1998 Memorandum). 2.1.1 The DoD Architectural Framework Version 1.0 Following the 1998 Memorandum, DoD agencies began building architectures according to the C4ISR Architectural Framework v 2.0. While the overall response was positive, there were many suggestions for improvement and requests for more guidance. Taking these into account, the Architecture Framework Working Group (AFWG) released a major revision in late 2003. To better reflect the wide applicability of the framework, the "C4ISR" was dropped and the document was renamed the DoD Architectural Framework Version 1.0 (hereafter referred to as the DoDAF ­ which is currently in "Final Draft" form). Instead of designating certain products as "essential" or "supporting," the DoDAF suggests products based on the intended use of the architecture (and specifies a minimum set required for the architecture to qualify as an "integrated architecture"). The changes to the framework largely enhance the user friendliness and better explain the applicability of architectures. For example, in response to the increasing popularity of the Unified Modeling Language (UML), the DoDAF gives examples of types of UML representations that can be used for each architectural product. The core remained largely unchanged, such that minimum effort should be required to make an architecture created under C4ISR 2.0 compliant with the new DoDAF. The DoDAF consist of three parts ­ two volumes and a deskbook. The first

18

volume contains background, definitions and guidelines while the second volume describes each product in detail. The DoDAF Deskbook provides supplementary information, such as processes and implementation recommendations. The DoDAF does not dictate modeling methodology or format. In fact, Levis and Wagenhals have shown that architectures can be developed under the framework using Structured Analysis and Object Oriented approaches (Levis and Wagenhals, 2000). One challenge in building an architectural framework is dealing with the fact that different people view a system from different perspectives. The commander of an operational fighter squadron views his airplane differently than the chief engineer in charge of its design. The former is concerned with concepts of operation (CONOPS) while the latter may focus on the integration of subsystems, for example. Yet, they are both dealing with the same "architecture." In 1994, the Army Science Board proposed the solution of having one architecture with three views: Operational, Systems, and Technical. When data is properly shared across the three views, an "integrated architecture" is born. According to the DoDAF: An architecture description is defined to be an integrated architecture when products and their constituent architecture data elements are developed such that architecture data elements defined in one view are the same (i.e., same names, definitions, and values) as architecture data elements referenced in another view. (DoDAF, 2003:1.5) In other words, the views are consistent. A key task in the engineering of a system is mapping operational activities to system functions. To show both sides (operational and systems) an integrated architecture is required. The DoDAF provides for this, as we shall see. This is one of the reasons that Levis insists that the responsibility for generating each view not be divided. "There is one architecture with multiple views. Consequently,

19

choose one Architect responsible for all the architecture views ­ not an Operational Architect and a Systems Architect" (Levis, 2003:1-65). The figure below shows the relationships between the three views.

Figure 6. Three Views of Architecture (DoDAF, 2003)

Some aspects of the architecture encompass each of the three views; they are called the All-Views (AV) products. This document will give a brief overview of the products, and a complete list of the products is provided in Appendix A. For more detail, reference the DoDAF Volume II. The order by which they are presented and numbered is not necessarily the order in which they are typically created, as will be shown later. The first AV document (AV-1) is the Overview and Summary Information. The AV-1 "provides executive-level summary information in a consistent form that allows quick reference and comparison among architectures" (DoDAF, 2003:3.1.1). An AV-1 is usually presented in a report format, and must contain the following information: product definition, purpose, and detailed description. The product detailed description consists of

20

architecture product identification (i.e. project name, architect's name), scope, purpose and viewpoint, context, tools and file formats used, and findings (DoDAF, 2003:3.1.1). Clearly this is an evolving product, as the "findings" section would not be started until the effort is well underway. However, it is very important that the scope, purpose and viewpoint are defined up front. Maier and Rechtin say that "Systems Architecting is a process driven by a client's purpose or purposes" (Maier and Rechtin, 2002:10). According to Levis, a well defined purpose is crucial: "Having a specific and commonly understood purpose before starting to build an architecture greatly increases the efficiency of the effort and utility of the resulting architecture. The purpose determines how wide the scope needs to be, which characteristics need to be captured, and what timeframes need to be considered. This principle applies equally to the development of an architecture as a whole and to the development of any portion or view of an architecture" (Levis, 2003:2-56). The new DoDAF supports this in its Figure 2-2 "Architecture Products by Use" matrix that provides guidance on what products should be developed based on the intended use of the architecture (Appendix B in this document). Consequently, clearly defining the purpose of the architecture early (during AV-1 development) will help in creating a useful architecture and prevent the creation of products that do not add value. AV-2 is the Integrated Dictionary (sometimes referred to as the Data Dictionary). It is the repository for all the definitions of terms used in the architecture. The AV-2 also includes taxonomies (common definitions) and metadata (information about architecture data types). The AV-2 is crucial to understanding an architecture, as it "enables the set of architecture products to stand alone, allowing them to be read and understood with minimal reference to outside resources" (DoDAF, 2003:3.2.1).

21

Though simultaneous development of multiple views is feasible, most architects begin with the operational view. According to the DoDAF, the operational view is "a description of the tasks and activities, operational elements, and information exchanges required to accomplish DoD missions" (DoDAF, 2003:1-2). The High-Level Operational Concept Graphic (OV-1) is simply a graphical representation of the CONOPS, with accompanying descriptive text. The OV-1 is a good product to show the scope of the architecture (what is internal and what is external) and is intended to be briefed at high levels in the chain of command. The Operational Node Connectivity Description (OV-2) graphically depicts dependencies between nodes to exchange information. The DoDAF describes a node as, "element of the operational architecture that produces, consumes, or processes information" (DoDAF, 2003: 4.2.1). Common nodes include organizations, people, locations, etc . . . Nodes can be internal or external to the system, and are connected by needlines that represent one or many information exchanges. These information exchanges are detailed in the Operational Information Exchange Matrix (OV-3). It identifies, "who exchanges what information, with whom, why the information is necessary, and how the information exchange must occur." (DoDAF cites CJCSI 6212.01B, 2000). To accomplish this, the matrix relates the information exchanges to the nodes and activities as well. For this reason, it is usually created after the activity model, OV-5. The Organizational Relationships Chart (OV-4) is the stereotypical "org-chart." It is typically a diagram that shows the structure of the operational unit in "tree" format. The DoDAF does not limit the relationships to supervisory or command, stating,

22

"Architects should feel free to define any kinds of relationships necessary and important within their architecture to support the goals of the architecture." (DoDAF, 2003:4.4.1). At the heart of the architecture is OV-5, the Operational Activity Model. It describes the activities that occur to accomplish the mission (as depicted in OV-1). It is a diagram that contains one or more activities connected by inputs and outputs. The most common method used to generate an OV-5 is the Integration Definition for Function Modeling (IDEF0), though the DoDAF does not require it. IDEF0 allows the depiction of controls and mechanisms for the activities (see FIPS 183 for more information on IDEF0). Controls are "factors that affect the way that the activity is performed" (DoDAF, 2003: 4.5.1) and, according to IDEF0 convention, every activity must have at least one control. Mechanisms are the systems that perform the activities; note that it may be premature to add these early in the OV development as it may restrict the system design (DoDAF, 2003:4.5.1). In IDEF0 diagrams, Inputs, Controls, Outputs and Mechanisms are collectively referred to as ICOM arrows. Like many diagrams in the framework, the OV-5 can be decomposed in to many other diagrams. For example, the top level activity (called A0) alone would make up the "context diagram." It may then be decomposed into three lower level (or "detailed level") activities, A1, A2, and A3 which are depicted in another diagram. A1, A2, and A3 are a decomposition of A0. The activities at the lowest level of decomposition are called the "leaf activities." Figure 7 shows a decomposed IDEF0 diagram; inputs come in from the right (I1, I2), outputs exit from the left (O1, O2, O3), controls come in from the top (C1, C2, C3), and mechanisms come in from the bottom (M1).

23

Figure 7 IDEF0 Model Syntax (Levis, 2003:3-26)

The leaf activities in the OV-5 would be what the system engineer (SE) uses to perform a functional analysis. Many in the SE community consider it to be the most critical product in the DoDAF. The leaf activities from OV-5, combined with the information on operational nodes from OV-2, join to form the OV-3. A common mistake made in architectural development is to have one product decomposed down several layers while other products are neglected. This leads to architectures that are far from integrated. The DoDAF cautions against this in the case of the activity model, stating, "The decomposition levels and the amount of detail shown on OV-5 should be aligned with the operational nodes that are responsible for conducting the operational activities

24

(shown on corresponding OV-2 products)." (DoDAF, 2003, 4.5.1). This is one of the steps on the way to an integrated architecture (see slide below).

Figure 8. Integrated Architectures (DoDAF Vol I, 2003:Fig 3-3)

The OV-6 products describe the dynamic behavior of the system. According to the DoDAF: "The dynamic behavior referred to here concerns the timing and sequencing of events that capture operational behavior of a business process or mission thread for example" (4.6.1). This is accomplished via the following three documents. The OV-6a is the Operational Rules Model. These rules are often simple IF-THEN-ELSE statements tied to activities in the OV-5 (only leaf activities should be described by rules). The Operational State Transition Description (OV-6b) is a diagram that shows various states the system can be in and the events that causes it to transition between states. A template often used is the familiar "statechart," which marks transitions with the "Event [Guard

25

Condition]/Action" block. Event timing is best described by the OV-6c, the Operational Event-Trace Description. It models the time history of nodal information exchanges of a particular scenario. UML sequence diagrams and IDEF3 are the most common formats used (DoDAF, 2003:4.6.11). A major thrust of this research is to look at the OV-6 products and determine their utility in modeling an architecture using agent based simulation. The last OV diagram is the Logical Data Model (OV-7). The OV-7 "defines the architecture domain's system data types (or entities) and the relationships among the system data types" (DoDAF, 2003:4.7.1). Those familiar with software engineering will recognize this as the Entity-Relationship (ER) diagram. The IDEF1X convention is frequently used to model this product. The Systems View contains eleven products that describe the systems that support operations. For this reason, work is usually started on the SV products during, but not before, OV product development. The SV-1, Systems Interface Description, is a good example of this. It shows the system nodes that support the operational nodes in the OV2 (Operational Node Connectivity Diagram) (DoDAF, 2003:5.1.1). It is only logical, therefore, that the SV-1 would be created after the OV-2. The Systems Communications Description (SV-2) provides information about how systems communicate with each other, including the type of media involved. According to the DoDAF, "SV-2 shows the communications details of SV-1 interfaces that automate aspects of the needlines represented in OV-2 (5.2.1)." Communication is documented between internal nodes and entities external to the system.

26

The SV-3 Systems-Systems Matrix provides more information about the system interfaces diagramed in the SV-1. Like an N2 diagram, systems are arranged in the rows and columns, and the relation between the systems is shown in the corresponding cell. This product may be useful for identifying redundancies. The rough equivalent to the OV-5 in the systems view is the Systems Functionality Description (SV-4). Unlike the OV-5, the SV-4 is intended to describe system functions and often shows the passing of data from system to system. For this reason, the data flow diagram (DFD) is often the model of choice instead of IDEF0. Like the OV-5, the SV-4 is often decomposed into several child diagrams to show the functionality of subsystems. The Operational Activity to Systems Function Traceability Matrix (SV-5) maps operational activities to system functions. The activity is often referred to as "functional allocation" and is a key step in the systems engineering process. As an example, an automobile's architecture would allocate the stopping function to the brakes and the safety function to the seatbelt, airbag, and other components. "Such a matrix allows decision makers and planners to quickly identify stovepiped systems, redundant/duplicative systems, gaps in capability, and possible future investment strategies all in accordance with the time stamp given to the architecture" (DoDAF, 2003:5.5.1). While in development, the ability of a system to fulfill a particular activity isn't always black and white. For this reason, the DoDAF outlines a red/yellow/green "stoplight" system that adjusts depending on the maturity of the system. For the automobile example, proven safety components like seatbelts and airbags may be marked

27

with a green light while a new Heads-Up Display that is still on the drawing board may be marked as yellow. The Systems Data Exchange Matrix (SV-6) describes the data exchanged between systems. Because it is an SV product, the emphasis is on automated information exchanges. At a minimum, the SV-6 usually describes "how the system data exchange is implemented, in system-specific details covering periodicity, timeliness, throughput, size, information assurance, and security characteristics of the exchange" (DoDAF, 2003:5.6.1). These are reflected as columns in the matrix. The Systems Performance Parameters Matrix (SV-7) is simply a listing of quantitative performance parameters for the systems. They are usually annotated as threshold and objective, and tied to a specific time frame. Examples of common parameters include availability, response time, and detection accuracy. The SV-8 is the System Evolution Description. It is essentially a timeline that shows when the system will evolve over a period of time. Often this timeline will denote milestones such as the release of new versions. Supporting the SV-8's evolution description is the SV-9, System Technology Forecast. SV-9 attempts, within reason, to forecast changes in technology relevant to the architecture. It is typically created in a spreadsheet. Just as OV-6 a, b, and c represented the operational dynamic model of the operational view, SV-10 a, b, and c follow the same structure. SV-10a describes the rules that constrain the system, SV-10b shows system changes in a state transition diagram, and SV-10c diagrams how system data exchanges occur as time passes. Like the OV-6, the SV-10 b and c diagrams must pertain to a specific scenario (different scenarios =

28

different diagrams). The SV dynamic models differ from their OV counterparts, however, as they deal with system states and system functions rather than operational states and activities. The last SV product is the Physical Schema (SV-11). It essentially shows how the OV-7 will be implemented. The SV-11 can take many forms, including a class diagram (familiar to those with experience in UML). According to the DoDAF, the SV11 is "of the architecture products closest to actual system design in the Framework" (5.11.1). The Technical View consists of two products. The TV provides general technical guidance for the OV and SV as well as technical and engineering standards for the SV. TV-1, the Technical Standards Profile, is a collection of standards that are relevant to the architecture. For military systems, the Joint Technical Architecture (JTA) is often cited in TV-1. The Technical Standards Forecast (TV-2) documents how the standards in TV1 are likely to change. Because TV-2 and SV-8 both deal with changing technology as time passes, the DoDAF gives the architect the option of combining the products into one document (DoDAF, 2003:6.2.1). 2.1.2 The CADM and DARS Much has been written in the DoDAF on the benefits of "integrated architectures" that use common data across products. Furthermore, the DoDAF stresses that one of the major goals of the framework is to ensure interoperability. The DoD realized shortly after C4ISR 2.0 was released in 1997 that these goals were not being achieved. Architectures were being created using a myriad of different tools that did not reference a common database. For example, a drawing tool like Visio might be used to create OV-5

29

while the OV-3 was created from scratch in Excel. Commercial products were emerging at that time that used automated tools to create diagrams and used a "repository based" architecture to store common data. It would be unwise, however, for the DoD to mandate a specific commercial product. Something needed to be done to help standardize how architectural data is stored so that comparisons could be made between architectures built using different tools ­ the Framework needed a data model of its own. The DoDAF v 1.0 addresses this with the Core Architecture Data Model (CADM). "CADM provides the logical basis for moving architectures from compendiums of documents, spreadsheets, and graphics to architecture data that can be stored in architecture data repositories and manipulated with automated tools" (DoDAF Vol I, 2003:6.2). This is accomplished by providing a data element table for each product in Volume II of the DoDAF. Once a DoD architecture is "CADM-compliant," it may be stored in the DoD Architecture Repository System (DARS). When operational, users will export their CADM-compliant architectural data generated in a commercial tool into XML format. XML stands for "eXtensible Markup Language" and is becoming a worldwide standard and is nonproprietary. After being parsed, the XML is uploaded into the DARS repository. From there, it is available to authorized users who can then reverse the process and view the architecture in a CADM-compliant product of their choosing. 2.1.3 DoDAF Guidance The DoDAF also presents guidance on how to build and eventually use architectures (Figure 9). This six step process is to be seen as guidance only, as specific situations may lead the architect to tailor this process.

30

Figure 9. Building Architectures (DoDAF Volume I, 2003:5.2)

This research, of course, is focused on Step 6, "Use Architecture for Intended Purpose." On this, the DoDAF says, "The architecture description will have been built with a particular purpose in mind. As stated in the discussion of Step 1, the ultimate purpose may be to support investment decisions, requirements identification, system acquisition, interoperability evaluation, operations assessment, or some other purpose. The architecture description facilitates and enables these purposes but does not provide conclusions or answers. For that, human and possibly automated analysis must be applied" (DoDAF Vol I, 2003:5.2). The transition from the architecture to the so-called "automated analysis" is what this thesis is investigating. One of the benefits cited by the DoDAF for using the CADM to standardize repository based architectures is the "ability to use multiple architecture tools and modeling, simulation, and analysis tools" (6.3). Because these compliant

31

architectures will all be stored in a common place, using a common language (XML), with a common data model, the potential exists for an automatic export to M&S tools. In fact, research has already been done on interfacing XML with M&S by Paul Fishwick (Fishwick, 2002). 2.2 Popkin System Architect The previous section mentioned commercial products that aide in architecture development through the use of a data repository. One such product is System Architect by Popkin Software from New York City. Popkin Software was founded by Jan Popkin in 1986 and began providing tools to aide companies with business process modeling through the use of enterprise architectures. After the C4ISR Architectural Framework was released, there was demand for a tool that could help generate the myriad of architectural products and store common data in a central repository. To meet this demand, Popkin added a C4ISR "option" to System Architect designed specifically for architectures using the framework. When starting a version of System Architect purchased with the DoDAF (formally C4ISR) option, the user is greeted with the familiar framework matrix that can be used to browse through the OVs, SVs and TVs. System Architect can be set up so that the architecture's data repository (referred to by Popkin as an "encyclopedia") is resident on central server, and under configuration control. Much like software development, members of the architecting team can "check out" particular products for modification. This allows several team members to work on the same encyclopedia at the same time. Because of this and other features, many DoD organizations, including the Air Force, designated Popkin System Architect as their "preferred tool."

32

As previously mentioned, the DoD does not specify methodologies, tools, or processes for architectural products. For example, nowhere in the DoDAF does it say you must use IDEF0 for OV-5. In response to criticism of C4ISR v 2.0, DoDAF v 1.0 seems to have gone out of its way to not dictate certain methodologies. To aide in describing certain products, the Framework sometimes shows examples or "templates" that abide by specific methodologies but cautions, "Where applicable, the templates for the Framework products reference industry standard methodologies and techniques, although there is no requirement to comply with the template's chosen standard" (DoDAF Vol II, 2003:2.4.4). Therefore, it is up to the architect to determine the form of the products. Likewise, it is up to the tool vendor to decide which automated modeling technique to support. The table below shows how Popkin System Architect has chosen to support the DoDAF.

Table 1. Popkin DoDAF Product Implementation (Popkin, 2003:slide 45)

Product C4ISR Name AV-1 Overview and Summary Information AV-2 Integrated Dictionary AV-3 Capability Maturity Profile OV-1 OV-2 OV-3 OV-4 OV-5 OV-6a OV-6b OV-6c OV-7 SV-1 SV-2 SV-3 SV-4 SV-5 SV-6 SV-7 SV-8 SV-9 SV-10a SV-10b SV-10c SV-11 High-Level Operational Concept Description Operational Node Connectivity Description Operational Information Exchange Matrix Organizational Relationships Chart Activity Model Operational Rules Model Operational State Transition Description Operational Event/Trace Description Logical Data Model System Interface Description Systems Communications Description Systems(2) Matrix Systems Functionality Description Operational Activity to System Function Traceability Matrix System Data Exchange Matrix System Performance Parameters Matrix System Evolution Description System Technology Forecast System Rules Model Systems State Transition Description Systems Event/Trace Description Physical Data Model System Architect Solution AV-1 Overview and Summary Information Definition* The System Architect Repository None - Future OV-1 Operational Concept Diagram* OV-2 Operational Node Connectivity Diagram* OV-3 Operational Information Exchange Report* OV-4 Organization Chart Diagram* OV-5 Activity Model Diagram and Node Tree Diagram OV-6a Operational Rules Model Diagram OV-6b Operational State Transition Diagram* OV-6c Operational Event/Trace Diagram* OV-7 Logical Data Model Diagram SV-1 System Interface Diagram* SV-2 Systems Communication Diagram* SV-3 System to System Matrix and Report* SV-4 Data Flow Diagram and Decomposition Diagram SV-5 System Function to Operational Activity Matrix and Report* SV-6 System Data Exchange Report* Performance Relate Definitions and Report* Derived from use of SA repository information Technology & Technology Area Definitions + Report* Property on System Function using BNF Syntax SV-10b Systems State Transition Diagram* SV-10c Systems Event/Trace Diagram* SV-11 Physical Data Model Diagram

TV-1 Technical Architecture Profile Technical Architecture Profile Definitions + Report* TV-2 Standards Technology Forecast Standards Technology Forcast Definitions + Report* *Capability added with C4ISR extension to System Archtiect

33

The "System Architect Solutions" that are described as diagrams are created in a completely different manner than those described as reports. Diagram solutions are manually drawn in a System Architect workspace using symbols that are specific to the product being created (i.e. an OV-2 Node Diagram will have Nodes and Needlines as symbols). These symbols represent data that may or may not already exist in the underlying architecture encyclopedia ­ they are called "definitions" by Popkin. If the definitions already exists, Popkin provides the developer with a convenient list of "choices" that can simply be dragged and dropped onto the workspace. Proprietary features allow the drawing process to be partially automated for some products (for example, the OV-4 symbols automatically snap to an Org Chart like form). When generating reports, unlike diagrams, it is required that definitions already exist in the encyclopedia (or else the report would be empty, of course). The data can be imported from a variety of sources, or manually input via dialogs (this is often done often during diagram creation, as the above paragraph describes). While diagrams are viewed inside System Architect, reports are commonly generated as stand alone documents in various formats such as Word, Excel, HTML, or XML (recall that the DARS plans to use XML format). While report products like OV-3 are usually simple spreadsheets with column headings defined by the user, diagram products must follow a previously defined format. Therefore, Popkin chose several industry standard methods to portray the diagram products. OV-5 is modeled in Popkin using the IDEF0, a familiar standard for those in the systems engineering community. Popkin's method to portray the OV-6a diagram is slightly less conventional. Rather than providing a dialog to describe the rules model for

34

each activity, Popkin choose to use a diagram to portray the OV-6a Operational Rules Model. The format chosen for the diagram was IDEF3, which is less familiar to engineers than IDEF0. Because the OV-6 will prove to be a key player in this research, a brief description of IDEF3 notation is given below. 2.2.1 IDEF3 Notation IDEF3 was created to aid in business systems modeling by capturing "descriptions of sequences of activities" (Mayer, 1995:1). For a particular scenario, an IDEF3 diagram can show you which processes precede others, which occur in parallel, where decision points exist, etc. Thus, as Mayer indicates, the IDEF3 is closer to a sequence diagram (OV-6c) than an OV-6a Rules Model as Popkin has labeled it. Nevertheless, the IDEF3 is certainly a dynamic model and can be a powerful tool in understanding the performance of a system. Figure 10 shows the standard symbols used by IDEF3 and Figure 11 is a simple example of an IDEF3 diagram. The example in Figure 11 shows a decision point (XOR junction) following a process labeled "evaluate proposal." If the decision is in the affirmative, the next junction, an OR junction, shows that either path can be taken prior to the last process, "Contract Award." While IDEF3 diagrams show a clear sequence of processes, the challenge for this research will be deducing the rules that guard process completion. This challenge comes to light in Chapter 3.

35

Figure 10 IDEF3 Symbols (Mayer et al, 1995:22)

Figure 11 Example of IDEF3 diagram (Mayer et al, 1995:40)

2.3 The AOC Architecture A good example of a typical DoD architecture currently under development is the Air Operations Center (AOC) Weapon System Block 10.1 Architecture. Using Popkin System Architect as the primary tool, the AOC Architecture is being developed for the

36

Air Force by the MITRE Corporation in Hampton, VA. The AOC architectural effort was started for several purposes, the principal of which was to document a coherent baseline for an AOC. According to the AV-1, "the reference material for the architecture exists as a collection of mostly unrelated heaps of PowerPoint briefings, documents, and emails. It is inconsistent, sparse in many areas, out of date, uneven in fidelity, or in some cases, non-existent." As the Air Force was considering releasing a Request for Proposal (RFP) for an AOC integration contract, they realized that there would be nowhere to start from except the above mentioned myriad of documents. Thus, the architecture is designed to support the RFP development and then "serve as a maintained, authoritative decision making tool after contract award." The use of the word "maintained" is important ­ the architecture is not just used once and discarded. After contract award, the architecture will be maintained by the contractor, but still used by the government for other purposes. Thus the AOC, as a classic example of a "system of systems," requires an architecture description primarily for the following reason, as described in the DoDAF Volume I: Acquisition program management and system development ­ e.g., Services and Agencies developing systems can use architectures to determine system concepts related to operational concepts and ensure interoperability within a family of systems/system of systems (FoS/SoS). (DoDAF, 2003:3.5). In short, the AOC architecture was created primarily to document a baseline. According to AV-1, "This architecture does not reflect any substantive analysis. Rather, it focuses on the capture of the as-is baseline architecture and will need to be analyzed in the context of a decision or focused study" (AOC Architecture, 2003:AV-1). With regards to time frame, "The architecture depicts the evolution of the weapon system from

37

its current AOC WS Block 10.0 state (2001) and certain evolutions expected to be implemented through FY06." The research in this document is based off the AOC Architecture Version 2.0 which consists of 12 products. Note that the current effort using Popkin SA took over from a previous effort using multiple tools (see "Initial Working Form" vs. "Final Working Form"). Diagram products (like OV-5) are created in the drawing space of the Popkin SA tool, sometimes using previously defined data elements. Tabular products (OV-3, for example) are generated by SA somewhat automatically, using data from the repository "encyclopedia." This appears to be an improvement over the "Initial Working Form," in which graphic and tabular data were generated independently and manually with Netviz and Excel, respectively. The current methodology allows all the data in the architecture to be stored in one place, the Popkin encyclopedia. The AOC is considered an "integrated architecture," based on the DoDAF standards. It contains the 7 essential products required, and data elements are consistent across the views. For example, the operational activities in OV-5 are mapped to SV-4 functions. This relationship is defined in the encyclopedia, so that the SV-5 matrix can be generated automatically, with some manual tweaking required. The AOC architecture is a good choice for the case study undertaken by this research. It is an integrated architecture representing a classic "system of systems." It is highly reliant on C4ISR systems, which SEAS is designed to model to a high degree of fidelity. The architecture is relatively complete, with some products containing several levels of decomposition. More about the AOC 2.0 architecture and its choice for use in the case study will be written in Chapter 3 of this document.

38

2.4 The C4I Support Plan As shown in Chapter 1 of this document, the DoD has mandated the use of integrated architectures in the acquisition process. A specific example of this is the Command, Control, Communications, Computers, and Intelligence Support Plan or C4ISP, documented in DoD 4530.8. "For all DoD Acquisition Category (ACAT) programs, C4ISP shall be used to document interoperability and supportability requirements" (DoD 4630.8, 2002:4). The C4ISP must be in place for Milestone B and "shall be maintained throughout the acquisition life cycle" (DoD 4630.8, 2002:33). The following products must be included in the C4ISP: OV-1, OV-2, OV-3, OV-6c, SV-1, SV-6, and TV-1 (DoDAF Vol I, 2003:3.6.4.2). According to Levis, "The purpose of a C4ISP is not to build architectures ­ its purpose is to use architectures to support requirements analysis and evaluation" (Levis, 2003:2-46). Various tools exist to support this analysis. The following section will outline a simulation model that is particularly relevant to C4ISR systems. 2.5 Background on DoD Modeling and Simulation Modeling and Simulation (M&S) techniques used in the DoD evolve not only with the nature of warfare, but also with the increase in computational power. Both of these factors are currently forcing a major change in the M&S community. The traditional (and most widely used) modeling technique in the DoD uses differential equations to calculate casualties and changes in the frontlines. These equations are called "Lanchester Equations" and were originally published in 1914 (Tighe, 1999:27). Many modern simulation models use variants of the Lanchester Equations to predict the attrition rates of opposed forces massed in parallel strips across the battlefront (the

39

"piston" analogy is often used here). According to a RAND study, "These models were developed when computers had much more limited capabilities, making it necessary to reduce the number of simulation entities and to use aggregation techniques" (Gonzales et al, 2001:10). More detailed information on these models can be found in DeStefano's Thesis Chapter 2. The aggregated Lanchester equation based models are beginning to fall out of favor in some DoD agencies for several reasons. Tighe cites a paper by Battilega and Grange that shows that the equations "do not accurately model many historical battles" (Tighe, 1999:28). This is surprising given that many of these battles follow the massive "force-on-force" attrition warfare formula that the equations are designed for. With modern warfare increasingly relying on C4ISR systems, the old models breakdown even further. Gonzales points out that the "legacy models" can't model individual C4ISR effects, and can only be represented by adjusting parameters. "Because of the parametric approaches employed in legacy models and because these models are largely scripted, they also cannot take into account how information is used to support command decision making processes. Thus, it is not possible to asses many IS (Information Superiority) concepts, such as force synchronization, that may be enhanced or enabled by advanced C4ISR systems in traditional legacy models" (Gonzales et al, 2001:11). In other words, what good is C4ISR when your forces' moves are scripted? These problems have been addressed by a new generation of models that attempt to model "complex adaptive systems." Forces are made up of individual "agents" that are programmed to follow a rough set of rules. "The individual agents are then responsible for making their own decisions as to how they should prosecute the battle" (Tighe, 1999:33). Hence they are "adaptive." While the rules that govern an individual

40

agent may be simple, a collection of agents will exhibit complex behavior. These "agent based" models have begun to catch on in the DoD M&S community. The Navy's Naval Simulation System (NSS) is used to model Fleet Battle Exercises (http://www.metsci.com/ssd/nss.html). The Marine Corps uses JIVES (Joint integrated visualization environment simulation) to simulate urban combat (DeStefano, 2004:2.3.3). This new generation of models is better suited for the analysis of modern concepts such as Network Enabled Warfare and Effects Based Operations (Gonzales et al, 2001 and Weber, 2003). 2.5.1 SEAS The System Effectiveness Analysis Simulation (SEAS) was developed for the Air Force by the Sparta Corporation. The Aerospace Corporation and RAND have also provided assistance with SEAS development. According to RAND, "SEAS is an entitybased, time-stepped, stochastic, multimission-level model specifically designed to help evaluate the military utility of C4ISR systems" (Gonzales et al, 2001:8). The terms "entity based" and "agent based" are often used interchangeably. Individual agents are not typically modeled in excruciating detail. This, and the lightening fast speed of modern computers allows SEAS scenarios to contain many agents yet still exhibit decent run times. The below figure shows how SEAS uses agents within the modeling hierarchy.

41

Figure 12. Agent Hierarchy (from DeStefano, 2004:2.3.3.1)

Every agent in SEAS runs a parallel execution thread and follows the "orders" that have been coded to dictate its behavior. Interactions between agents are resolved at each one minute time increment. The below figure shows the sequence of events an agent goes through each time step. This capability for adaptive behavior allows SEAS to credibly model the effects of C4ISR. SEAS is particularly well suited to model space-based platforms down to individual satellites. SEAS is also set up to model weather and geographic effects (Gonzales et al, 2001).

42

Figure 13. SEAS Top-Level Event Processing (Gonzales et al, 2001:56)

The Air Force Standard Analysis Toolkit (AFSAT) "is a collection of simulations that are accepted and supported by the Air Force analysis community, and which have AF-level oversight and investment priority" (http://afmrr.afams.af.mil). The AFSAT divides simulation into three levels: Campaign, Mission, and Engagement. SEAS is included in the toolkit as a mission level simulation. In practice, SEAS is better described as a "fully capable multi-mission model which can and has been used to perform campaign-level analysis" (Weber, 2003). SEAS accomplishes this using a "vertical slice approach," where significant entities (but not necessarily all platforms) are modeled at every level of depth on the battlefield. Currently, SEAS is being used most frequently by the Space and Missile Systems Center (SMC) and RAND to support military worth analysis of C4ISR systems.

43

2.6 Summary The DoDAF provides a well defined framework for architectural development. It is important to remember that architectures are not the "ends" but merely a means to an ends (Levis, 2003). Using the architecture to accomplish a purpose is the true goal. As the AV-1 for the AOC architecture states: architectures are "potential energy." While architecture descriptions were originally brought into the DoD to help acquire C4ISR intensive systems, agent based simulations like SEAS were created to aide in understanding the effects of C4ISR systems. It is only natural then that an interface between these two separate but related resources be investigated.

44

III. Methodology 3.1 Introduction This chapter describes the methodology used to determine the most effective way to model in agent based simulation using a DoDAF architecture. In order to understand the unique structure of agent based simulation, a review of how agents function in SEAS is presented. This is followed by a brief explanation of the SEAS scenario utilized in the case study. Using previous research as a starting point, the case study is then set up for an actual DoDAF architecture of our choosing. A mapping of DoDAF products to agent attributes in SEAS is conducted to support the case study. 3.2 Agents in SEAS Before digging into the architecture looking for data, it is important to understand how agents operate. Figure 14 provides insight into the entities that make up SEAS.

SEAS is Built Around Three Simple Entities Agents, Devices, Environment

Agent Devices

er ac tio n

n tio ac er In t

Agents Interact Through Devices

· Weapons · Sensors · Comm

Environment

Agent Devices

Interaction

Agent Devices

With each other and the Environment To Emerge Conflict Outcomes

SEAS

Figure 14 The 3 Entities in SEAS (www.teamseas.com SEAS_Agents.ppt:3)

Int

45

Recall from section 2.5 that an agent is simply an entity (such as a tank or a small special forces unit) that has the ability to make decisions. Essentially, agents use devices to interact with other agents and the environment. The environment includes factors such as terrain and weather. Typical devices include weapons, sensors, and communications equipment. These interactions occur during each one minute time step (See Figure 13 from section 2.5.1). Agents are distinguished from devices by their ability to adapt and make decisions based on pre-programmed logic. The figure below shows what makes up an agent in SEAS.

Conceptual View of a SEAS Agent

There are four key concepts that apply to agent actions and interactions in SEAS;

The local target list (LTL) The local orders list (LOL) The target interaction range (TIR) The broadcast interval (BI)

Cognitive Domain Information Domain Physical Domain Action Effect Device Environment

- Commands - Target Sightings - Broadcast Variables

I'd better surrender

SEAS Agent

Weapons

User Programmed Behaviors

· · · · · Perception Awareness Knowledge Understanding Decisions

TPL

L TP

Personnel Comm Local Orders List

·Formation Column ·Move Location ·etc.

Target

Sensors Local Target List

·TOS 1 ·TOS 2 ·etc.

TP L

Day/Night Weather Terrain

SEAS

Broadcast Variables

_______ _______ _______

Owned Sub-units

Owned Platforms

Moving

Figure 15 A Conceptual SEAS Agent (www.teamseas.com SEAS_Agents.ppt:8)

46

Note the "User Programmed Behaviors" block. This is the brain behind the agent and is what separates agent based simulation from traditional models. By modifying code in the war file, the programmer can shape the behavior of particular agents. This does not mean that programmers just add in scripted events that happen at a certain time like you might see in a traditional model. Rather, they are "orders" programmed into each agent that allows them to make decisions autonomously. For example, orders for a Surface to Air Missile Radar may say to not activate for longer than a certain amount of time to avoid detection. If no orders are written for an agent it will fall back to default behavior hard wired into SEAS. Another important part of an agent's description is how it communicates. While units with onboard sensors can find their own targets, many targets are passed along communications lines. These targets then become part of the agent's "Local Target's List." Furthermore, orders may be sent to subordinate units, and Battle Damage Assessments (BDA) may be reported up the chain of command. Each communications device has its own attributes such as reliability, susceptibility to jamming, and delay. Furthermore, each communications device may have a multiple number of channels available for use. Like targets, orders may also be passed along communications lines and become part of the agent's "local orders list." By default, orders specific to the agent take precedence over orders generated externally (www.teamseas.com SEAS_Agents.ppt:8). Information sent through communications channels in SEAS that cannot be described as targets or orders are referred to as "broadcast variables." As we can see in Figure 15, there are many other attributes of an agent that are not orders or communications devices. For the purpose of this case study, I will group these

47

together as "general attributes." These consist primarily of subordinate units, performance parameters (such as how fast it can move), and subordinate devices (such as weapons, sensors, and general information such as number of people). Other attributes that fall into this category are more obscure, such as the time it takes to deploy and take cover (dig in) if attacked. In this case study, general attributes, communications, and orders will serve as the three major categories of information needed by SEAS to model an agent. The context this agent will exist in is described below. 3.3 The Kosovo War File This case study utilized an existing SEAS war file that had been developed by SMC for other purposes. It was not prudent or necessary to create an entire war file from scratch to demonstrate the transition from an architecture to SEAS. Because certain information needed for combat simulation (like the day's weather) are scenario specific and do not belong in a weapon system's architecture, it was reasonable to simply reuse these aspects in an existing war file. The scenario chosen is a scaled down representation of the conflict in Kosovo. It essentially models Red forces conducting "ethnic cleansing" operations against the Brown civilians. The objective of the Blue forces is to stop the killing. This type of contingency was identified in a capabilities based planning study prepared for the Office of the Secretary of Defense as a likely future conflict the United States should be prepared for (Davis, 2000:20). Several different air assets, including F15s, F-16s, JSTARS, Global Hawk and Predator are available to the Blue side. To control these assets there is a rudimentary AOC agent already present in the Kosovo war file which will serve as a starting point for the case study.

48

Based on the hierarchy in Figure 12 in section 2.5.1, the AOC (referred to as the CAOC in this scenario, for "Combined" AOC) would be best described as a "unit agent." It is important that the CAOC be some form of an agent, as only agents have logical decision making ability in agent based simulation. As a unit agent, the CAOC controls "platform agents" which may also have their own decision making ability defined by orders. In SEAS, these orders are written in a particular format known as TPL code (for Tactical Programming Language). A step down from platform agents on the hierarchy is "equipment" such as sensors and weapons. Equipment, like agents, have their own parameter definitions (like "speed," for instance), but are devoid of orders ­ as they rely on their parent platform agent for this. In summary, we must model the CAOC in the format SEAS requires unit agents to be modeled (which was described previously). 3.4 Applying Previous Research to the Case Study Prior to launching the case study mentioned in Chapter 1 of this document, an analysis was conducted to determine how products in the DoDAF framework can best be used to model a system in SEAS. Conclusions drawn in this analysis could then be exercised in the case study. A starting point for this research is the work of Levis and Wagenhals, who, as we have seen, have done similar research on the creation of an executable model (albeit in Petri Net form) from architecture products (Levis and Wagenhals, 2000). The figure below shows what Levis considers to be the essential building blocks in this process.

49

Figure 16. The Synthesis Phase (Levis and Wagenhals, 2000)

Noteworthy is the requirement of the Dynamics Model ­ this will present a challenge in the selection of a suitable real-world architecture given that most architectures in the DoD do not yet have one. The building blocks outlined above should all be provided by products described in the DoDAF. Levis notes that additional information is also needed, specifically timing information and "scenarios to run simulations and collect data for the Measures of Performance and the Measures of Effectiveness" (Levis and Wagenhals, 2000:243). To validate this theory, work was done by Wagenhals, Shin, Kim and Levis to create an actual executable Colored Petri Net (CPN) based on architectural products. A CPN is a graph with two types of nodes: circles indicate places and bars indicate transitions. Tokens, which are distinguished by colors, move from place to place via directed connections ("directed" meaning you can only move in the designated direction). Wagenhals based his CPN model on the following four products: the activity model, data model, rule model, and state transition diagram (Wagenhals et al, 2000:273). These correspond with OV-5, OV-7, OV-6a, and OV-6b ­ respectively. These products provided the necessary data to create CPN based simulation that models the logical 50

behavior of the system. The qualifier "logical behavior" of the system is an important one: you may analyze how data flows through functions yet you do not have enough information to determine how long it takes. To capture the performance of the system, elements of the "physical architecture" must be added to the recipe, as shown in the above Figure 10. Wagenhals suggests that the System Performance Parameters Matrix (SV-7) and a communications description would be needed in addition to the products mentioned above. For their research, Levis, Wagenhals, et al generated their own architecture description (describing Mobile's SpeedPass "pay at the pump" system) and then used them to produce the executable model. The logical executable model they created was detailed and complete while the performance model was more of an abbreviated concept demonstration using a modified version of the logical model. The work in this thesis is partially an attempt to apply the findings of Levis and Wagenhals to a real world DoD architecture created based on the DoDAF; it is also something completely new. A performance model will be created that will not only be able to show the effects of changing individual weapons system parameters, but also the effects of changing the CONOPS that guide their operations against an enemy that has a CONOPS of his own. Furthermore, the simulation paradigm the executable model is based on is vastly different in this experiment. This research will be using the agent based, mission level combat simulation SEAS rather than CPNs, which are process oriented. Nevertheless, the findings of Levis and Wagenhals provide a strong basis upon which to proceed.

51

3.5 Fitness of the DoDAF to support Agent Based Simulation This section will briefly comment on the methods used to evaluate the DoDAF's ability to provide data to Agent Based Simulations such as SEAS. Findings from this research, backed up by results from the case study will be presented later in this document. The first step in conducting this evaluation was to familiarize ourselves with how SEAS works and what it needs in order to execute properly (see 3.2). We took advantage of AFIT graduate courses on simulation (OPER 561 Discrete Event Simulation and OPER 671 Combat Modeling, for example ­ see AFIT, 2003-2004 for description), RAND studies using SEAS (Gonzales et al, 2001 and Davis, 2002), and relevant literature (Tighe, 1997 and Cares, 2002) to get a good idea of how agent based simulation works. Furthermore, trips to Los Angeles Air Force Base to go over recent SEAS applications and discuss SEAS with modelers and developers from the Sparta Corporation proved very helpful. Knowing SEAS requirements, the next step was to find out if they could be met by the DoDAF. Knowledge of the DoDAF (then C4ISR 2.0) stems from the AFIT Systems Architecture course (SENG 640). Further education on Architectures came from countless briefings and papers (many less than a year old due to the dynamic environment surrounding architectures in the DoD ­ Dandashi, 2003; Dam, 2002; Handley, 2003; and Pawlowski et al, 2003) as well as personal interviews from architecture modelers and tool vendors (Ring, 2003; Vittori, 2003; Surer, 2003; and Popkin, 2003). As mentioned in the previous section, the information required by SEAS were categorized as General Attributes (including chain of command), Communications

52

Information, and Orders. For each of these areas, the DoDAF Volume II was reviewed for appropriate matches. The work of Levis and Wagenhals, as well as Figure 2-2 "Architectural Products By Use" in the DoDAF (Appendix B) served as a good starting point. The findings are verified by evaluating several real world architectures and choosing one for use in a case study, as explained below. 3.6 Choosing the Architecture(s) As mentioned in Chapter 2, the AOC architecture was chosen for use in the case study intended to validate hypotheses addressing architecture to agent based simulation mapping. This section provides a detailed account of why the AOC architecture was chosen, problems encountered in the early stages of modeling, and the steps taken to address these problems. As shown in Chapter 2, systems architecting in the DoD is still in its relative infancy. It should be noted that for practical and administrative reasons, this research attempted to survey only US Air Force architectures. This narrowing of candidates was not a concern, as it was assumed that the Air Force was at least on par with the other services in architecture development. Despite the fact that the use of architectures was mandated in 1998 for information systems, there are surprisingly few architecture descriptions in circulation compared with the total number of acquisition programs. The AF acquisition center for Space and Missiles (SMC) is using architectures in its new Space Based Radar System Program Office (SPO) but the architecture is in the early stages of development. This was unfortunate as SMC is the primary user of SEAS and the sponsor of this research. Little more success was encountered at the Aeronautical Systems Center (ASC) which is collocated with AFIT at Wright-Patterson Air Force Base

53

in Ohio. While systems highly reliant on C4ISR (like those at the Reconnaissance SPO) were using architectures, they were not available for academic use. An additional source for candidate architectures is the Electronic Systems Center (ESC) at Hanscom AFB near Boston. ESC is in charge of the acquisition of command & control and information systems (such as airborne platforms like the AWACS) to ground based information systems (like air traffic control). Because architectures are intended to aid in the development of C4ISR systems, it is logical that ESC would be at the forefront in this area. ESC is participating in the creation of a top level architecture for the Air Force Deputy Chief of Staff for Warfighting Integration, AF/XI. The Command and Control (C2) Constellation architecture is being created to help better define how the Air Force conducts air operations, focusing on the Theater Air Control System (TACS). In addition to more traditional uses such as aiding system development and budgeting, the C2 architecture is intended to define core data that will serve as a reference model for other architectures (C2 Constellation Architecture, 2003:AV-1). Some areas in an architecture deserve more attention and analysis than others. For this reason, a subset of activities is often modeled at a higher level of detail than the rest of the architecture. This subset is often called a "key thread." In the C2 Constellation Architecture, there is a Time Critical Targeting (TCT) key thread. The TCT thread is modeled in its own development environment (it has its own Popkin encyclopedia) but uses data elements from the C2 and AOC architectures. The TCT thread includes an OV-6a Rules Model, while the AOC and C2 architectures do not. These and other architectures from ESC, AF/XI, and other organizations were

54

evaluated for use in this research. The following criteria were used in choosing an architecture: 1) The architecture should be built using a repository based tool. This is clearly the future of systems architecting and facilitates data reuse and transfer to external programs. 2) It should describe a system that is C4ISR intensive and relevant to the combat modeling performed in SEAS. 3) It should be unclassified. 4) It should be an "integrated architecture" according to the DoDAF standard. 5) It should be as complete as possible in the total number of products and depth of detail within the products. Although our choices were limited, the AOC architecture (described previously) best met our parameters. The AOC not only had a multitude of different products, but also had an impressive amount of depth within each product. The OV-5, for example, routinely took top level internal functions down to four or five levels of decomposition. External functions, like those performed by Bombers or JSTARS, were modeled down to one level of decomposition ­ just enough for the viewer to understand where external information exchanges originated from. It was originally intended for this research to use the AOC architecture exclusively. After an initial evaluation of the architecture it was clear that more would be needed. While the OV-5 was modeled in extreme detail, it was not accompanied by any dynamics models (OV-6 products). Therefore, the viewer may know in detail the activities of the AOC, but could not decipher what triggered the transition from activity to activity nor have any idea as to their sequence. This is why Levis and Wagenhals insist that a dynamics model is essential to the creation of an executable model (shown in Figure 16).

55

To address this, the TCT thread mentioned above was used to complement the AOC architecture. The TCT thread was created by MITRE with the help of existing data from the AOC architecture encyclopedia. Activities relevant to TCT were reused in OV5 of the TCT architecture. In some cases, activities were created in TCT that represented an aggregate of detailed level AOC activities (Vittori, 2003). Essentially, the TCT and AOC architectures share much of the same data, were built by the same modeler, and describe common systems (although at different levels of detail). Therefore, it is not breaking with the spirit of the research to supplement the AOC architecture with the TCT architecture. To portray the OV-6a, Popkin chose to use IDEF3 notation ­ resulting in a product that is somewhat of a cross between a Rules Model and an Event-Trace diagram (see discussion in 2.2.1). This presented benefits and challenges, both of which will be commented on later in this document. 3.7 Setting up the Case Study The following sections describe how the data from the Popkin encyclopedia was used, to populate the SEAS war file. The processes laid out by Levis and Wagenhals provided a good basis on which to proceed, but would need to be modified and amended to account for the specific needs of SEAS, and for an architecture that did not contain all of the recommended products (SV-7, for example). Therefore, this was an iterative process; data was collected that was thought to be relevant based on Capt DeStefano's requirements as a SEAS modeler, Capt DeStefano would model what he could and request more data to fill in the gaps (DeStefano, 2004). The process would then repeat itself until the SEAS scenario was populated to a satisfactory level of detail. In the end, a

56

satisfactory method was developed as outlined below. Specifics on how the method was used in the case study are provided in Chapter 4. Based on the previous section "Structure of SEAS," the data was divided into three sections (general attributes, communications, and orders) based on the natural differences in the type of data required by SEAS. In all cases, the solution sought was the least complicated one that would extract as much of the needed data as possible. For example, if the data could be extracted from an existing product like an OV-3 or SV-6, every effort was made to modify those products rather than create a new product. The benefit of creating a new product is that it could be specifically tailored to exchange data with SEAS. The drawbacks are an increased level of complication and more potential for error. As this was a proof of concept, a decision was made to use existing products, however manual this proved to be. The process used for each section is described below. 3.7.1 Mapping the General Attributes As mentioned above, SEAS requires some general information about units being modeled. This includes mass, number of personnel, what subordinate units are assigned, and other attributes. One might think that some of this information could be found in the AV-1 because it is seen as a general overview. The AV-1 is meant to be a general overview of the architecture, not the system itself, and this information is indeed not present. The OV-4 Organizational Relationships Chart is the most likely place that this information might exist, but it does not get down to the level of individual people. In fact, there seems to be no DoDAF product that would indicate the number of people required to man a particular system. As a result, we had to look elsewhere for that information. For a mobile unit, attributes like speed and altitude are needed ­ these may be

57

found or derived from the SV-7 System Performance Parameters Matrix. Most SV-7s, however, focus on detailed level sub-system performance parameters rather than the top level parameters that SEAS is interested in. This may present a challenge in the collection of the general attributes from an architecture. Another key piece of the general attributes is the weapons and sensors assigned to the weapons system. This is much more relevant to modeling a platform agent like an F15 rather than a unit agent like an AOC. The most likely place to find this information would be in the SV-1 System Interface Description. The missiles, bombs, radar, etc that are attached to a platform should be modeled in the architecture as system nodes. Mapping the command hierarchy in SEAS was another task in the case study. The obvious source for this information would be the OV-4. For a DoD architecture, the OV-4 should provide a good overview of the chain of command, which is exactly what SEAS needs. The OV-2 also contains information that is valuable in this regard, but the OV-4 presents it in a manner that is more relevant to what SEAS expects. Figure 17 depicts the recommended method for mapping general system attributes to SEAS based on an architecture. The "External" block represents information derived from sources other than the architecture.

58

SV-7

SEAS

***** UNITS ***** Unit "USAF_CAOC" Speed 0 Unit "F15_1" Unit "F16_1" Deploy Delay ? Bodies ? Weapon "n/a"

OV-4

SV-1

External

Figure 17 Mapping General Attributes from a DoDAF Architecture to SEAS

3.7.2 Mapping the Communications Agents communicate in SEAS using specific communications devices. In a typical SEAS war file, all the communications devices are defined in a sub-section towards the end of the file. Therefore, multiple agent descriptions in the main body of the code then can refer to one or more communications devices which have already been defined. Communications devices are defined with several attributes including maximum range, delay, reliability, and available channels. There are also ways to limit the communications device to certain types of messages (like target sightings) and its ability to both send and receive. Thus, it can be seen that not only information about the communications medium is required, but also descriptions of the type of information passed. Details on the communications mediums can be found in the SV-2 Systems Communication Description. Information on the type of information passed would be stored in the OV-3 Information Exchange Matrix or SV-6 System Data Exchange Matrix.

59

These products should be combined into a spreadsheet in order to provide the simulation modeler with a single, consistent document. For example, the table below is a sample SV-6 for a generic system.

Table 2 Sample SV-6

Interface System 1 to System 3 System 3 to System 1 System 4 to System 2 ...

System Data Exchange Orders Task concurrence Status

Format Text Text Voice

Size 128k 64k n/a

Classification ... Unclass Unclass Unclass

This table may provide the necessary information about the data being transferred but does not allude to the medium of the transfer. For that, the SV-2 is required. The figure below is a sample SV-2 diagram from the DoDAF.

60

Figure 18 Sample SV-2 (DoDAF Volume II, 2003:5-13)

Using the information from the SV-2, the SV-6 can be amended as shown below.

Table 3 Modified Sample SV-6

Interface System 1 to System 3 System 3 to System 1 System 4 to System 2

System Data Exchange Orders Task concurrence Status

Format Text Text Voice

Size 128k 64k n/a

Classification Communication Unclass Unclass Unclass 2-way comm. Links 2-way comm. Links 1-way comm. Link

It is important to note that the examples above are abbreviated (the SV-6 example in the DoDAF has 33 columns). A relatively complete SV-6 combined with SV-2 should provide all the information needed to model system communication in SEAS. Because of

61

their similar nature, sometimes the OV-3 can be substituted for the SV-6. Either way, both products need to be examined when preparing this spreadsheet as they each may have relevant information. This is discussed further in Chapter 4, which gives the results of this methodology for our case study. Figure 19 summarizes the methodology for communications mapping from DoDAF products into agent based simulation.

SEAS

C O N N E C T IO N TO NOD E B

NODE A

Sy ste m 1

T w o -W ay C o m m un ic atio ns Lin k s

S y ste m 2

O n e-W ay C o m m u n ica tion s L in k

S y ste m 3

S y ste m 4

L o c a l A re a N et

S y ste m 5 E XT ER NA L C O N N E C T IO N (O U T S ID E T H E N O D E S O F IN T E R E S T )

C O N N E C T IO N TO N OD E B

C O N N E C T IO N TO NO DE C

SV-2

Check with OV-3 (if SV-6 was used) and combine

***** UNITS ***** Unit "USAF_CAOC" Comm "TacAirT" ***** COMM DEVICES ***** Comm "TacAirT" Max_Range 5000 Message_Type 1 Modes 1 External

SV-6 or OV 3

Figure 19 Mapping Communications from a DoDAF Architecture to SEAS

3.7.3 Mapping the Orders As discussed previously, the "orders" section of an agent's description give the agent its ability to make logical decisions regarding its behavior on the battlefield. This is a key concept, and is the main factor that allows SEAS and other agent based simulations to model the effects of C4ISR. This section will describe our efforts to author agent orders in SEAS based off of the AOC architecture, and specifically the Time Critical Targeting thread. Like before, this thesis will focus on the capture of data from the architecture while Capt DeStefano's thesis will describe how the data was used.

62

A similar problem was tackled by Wagenhals et al in their paper mentioned previously (Wagenhals et al, 2000). Recall that Wagenhals based their executable model on the activities in the OV-5 which became transition blocks in a Colored Petri Net. With that done, a means was needed to describe the guard conditions that allowed tokens to pass through the arcs that connect places to transitions. These guard conditions, of course, amount to the rules described in the OV-6a. Since SEAS does not inherently use transition blocks that correspond to Operational Activities like the CPN, a hard look at how the activities in the TCT thread would be represented was needed. By necessity, the modeling of operational activities in SEAS would be less structured than the modeling of activities in a CPN. As explained previously, SEAS contains default behaviors for particular types of agents like airplanes and vehicles. In some cases, these default behaviors automatically perform a desired activity. For instance, an airborn sensor in SEAS will automatically try to detect enemies within its detection range. In other cases, the default behavior of an agent might prompt activities that do not match the architectural model. To prevent this, the SEAS modeler may write TPL code in the orders section that overrides the default behavior. Giving the SEAS modeler the best architectural information to do this without overwhelming him with superfluous data is the challenge in this section. The solution lies in a combination of two important architectural products (much like the communications). By merging the OV-5 and OV-6a into a single text based document, the SEAS modeler has access to important data that is a step closer to the desired TPL code. The OV-5 can be put in this format using reporting functions built

63

into Popkin. The OV6a, in IDEF3 format (see 2.2.1), will require much more human intervention. One method to convert the OV-6a would be to try to go directly from the IDEF3 to the Tactical Programming Language (TPL code) used by SEAS. There are two major reasons why this method was not employed. First off, this method is less flexible. The results would only be relevant to SEAS. While this may work for this particular case study, the goal of the research is to be applicable to agent based simulation in general. Second, this method would require extensive manual intervention by someone who is closely familiar with IDEF3 and has widespread knowledge of TPL coding practices. Instead a more flexible and feasible method is proposed: translate the IDEF3 diagrams into an interim "pseudocode" that can be understood by any software developer. From there, the developer can translate the pseudocode into TPL code or any other language. This is the method employed in the case study. This thesis will focus on the interpretation of the IDEF3 to pseudocode while Capt DeStefano's thesis explains the translation into TPL code (DeStefano, 2004:3.3.2). To illustrate this method, the sample IDEF3 diagram from section 2.1.1 is revisited in Figure 20.

64

Figure 20 Example of IDEF3 diagram (Mayer et al, 1995:40)

Envision Figure 20 as the OV-6a Rules Model for an activity called "Select Contractor." For this activity, a new proposal would be the input and an awarded contract would be the output. Pertinent policies and regulations would act as controls to the process. In IDEF0 format (the preferred standard for OV-5 Activity Models) the activity would look like Figure 21.

Policy

Law

Proposal

Select Contractor

A1

Contract

Figure 21 Sample OV-5 diagram for "Select Contractor"

The task at hand is to put this information in a format that is closer to SEAS TPL code and more conducive to automation. To do this, the OV-5 is broken down into its parts and the OV-6 is translated into pseudocode. Figure 20 shows that after the proposal 65

is evaluated it can be accepted or rejected. For this example we assume that the criteria for rejection is that the cost is outside the budget. The lack of explicit criteria at decision junctions in an IDEF3 can be troublesome, as these are essentially the operational rules. With this assumption, the pseudocode for the first part of the diagram would read:

Evaluate Proposal IF (cost > budget) THEN Reject Proposal Else .....

Proceeding to the latter half of the diagram, the OR junction indicates that the proposal may be accepted for the core contract work or accepted for the options. Following this, the contract would be awarded. The process in pseudo code is as follows:

(Accept Proposal for Core Contract) OR (Accept Proposal for Options) OR ((Accept Proposal for Core Contract) AND (Accept Proposal for Options)) Award Contract

Figure 22 shows the two interpretations combined into one document. This is the process followed in the case study. Activity 1: Select Contractor

Description: The process used by the company to select the contractor for a new project Inputs: Proposal ­ contains the cost, schedule, and technical information as proposed by the potential contractor Outputs: Contract ­ the awarded contract Controls: Policy ­ Company contracting policy Law ­ Federal, State, and Local regulations

Pseudocode for Activity 1

Evaluate Proposal IF (cost > budget) THEN Reject Proposal Else (Accept Proposal for Core Contract) OR (Accept Proposal for Options) OR ((Accept Proposal for Core Contract) AND (Accept Proposal for Options)) Award Contract

Figure 22 Example of OV-5 Report + Pseudocode from OV-6a

66

Another area to consider when mapping activities is overall system performance. In order to accomplish this, performance parameters would be needed for the systems that performed each activity. These would essentially amount to time delays, which can be coded into the orders sections in SEAS. These individual time delays would add up and manifest themselves in the results of the simulation runs ­ overall system performance could then be analyzed. The DoDAF provides for this with SV-7, and Popkin provides a dialog for this information to be recorded for each System Element. Therefore, the SV-7 is an important product for mapping both the general attributes and the orders. One remaining issue is the role of OV-1 in the creation of the orders. Because the OV-1 is a representation of the concept of operations (CONOPS), and the orders are really a manifestation of the CONOPS, it seems natural that the OV-1 would play a key role in the creation of the order. Practically, however, there probably isn't anything to be learned from the OV-1 that can't be gathered from the above mentioned products. In fact, Levis maintains that prior to creating any OV products you must have a CONOPS in some form. Therefore, you expect an OV-1 to summarize many of the things the OV-5 and OV-6 products are saying in detail. As we know, OV-1 is a very top level document, usually ornate with graphics, that provides a broad overview of how the system will be employed. Therefore, we would recommend the OV-1 be viewed and understood as a first step in the process. The diagram below shows the process used to take architectural data to create the orders section for an agent.

67

OV-1 conops conops OV-5

A MTI/ G MTI Mon i to r T a rg et/ T ar ge t Sta t s u Trac k Dat a P ro ject ta rg et movement Target V ec tor

***** UNITS ***** Unit "USAF_CAOC"

SEAS

X

S ignif ic ant M ovem en t Y es/ No Target Moni toring

O

T arget Updat e

Mon i to r f or M o vemen t

S ta tic Target

Target Coordi nat es

OV-6a

OV-5 Report + pseudo code based off OV-6

Orders F15_tgts[0] = "T80" F15_tgts[1] = "SA6" While me_>Status == 2 Locate F15_PmassEn . . .

External

SV-7

Figure 23 Mapping Orders from a DoDAF Architecture to SEAS

3.8 Summary The methodology outlined above explicitly describes how to use data from a DoDAF based architecture to populate an agent based simulation such as SEAS. The effort was divided into three parts: mapping the general attributes, communications, and orders. The diagram below gives an overall summary of where certain DoDAF products can be used in a SEAS war file.

68

DoD Architecture Framework Version 1.0

OV-4

***** UNITS ***** Unit "USAF_CAOC" Speed 0 Unit "F15_1" Unit "F16_1" Deploy Delay ? Weapon "n/a" Orders F15_tgts[0] = "T80" F15_tgts[1] = "SA6" While me_>Status == 2 Locate F15_PmassEn....

OV-5 Report + pseudo code based off OV-6

SV-7

X Y Y X X Y Z

SV-1

OV-5

A MTI/ G MTI Mo ni tor T arg et/T a r ge t Sta t s u Trac k Dat a P rojec t ta rget mo vemen t Tar get V ec tor

X

S ignif ic ant M ovem en t Y es/ No Target Moni toring

O

T ar get U pdat e

Mo ni tor f o r M ovement

S ta tic Target

Target Coordi nat es

OV-6a

C O N N E C T IO N T O N ODE B

Comm "TacAirT" ******COMM DEVICES****** Comm "TacAirT Max_Range 5000 Message_Type 1 Modes 1

N ODE A

Sy s t em 1

T w o- Wa y C o mmunic at ions L inks

Sy s te m 2

O ne- W ay C om munic at ions L ink

Sy s t em 3

Sy s t em 4

L ocal Ar ea N et

S y s te m 5 EXTER NA L CO N N E C T IO N (O U T SI D E T H E N O D E S O F IN T E R E ST )

C O N N E CT IO N TO N O DE B

OV-1 conops conops

C O N N E C T IO N T O N O DE C

SV-2

Check with SV-6 and combine

OV-3

External Sources

Figure 24 Mapping DoDAF Products to Agent Based Simulation

69

IV. Results and Analysis

4.1 Results A key portion of this research is the evaluation of the DoDAF's ability to support agent based simulation. While some of this evaluation was conducted via a subjective review of the Framework and of SEAS, the findings were validated by the more objective case study. Because of this, the results of the case study will be presented below and general conclusions regarding the Framework will be enunciated in Chapter Five. 4.1.1 General Attributes Unlike the orders and communications, the procedure for mapping the general attributes could not be exhaustively validated by the case study. Information from the AOC architecture was not available to populate attributes like number of people in an AOC. On the other hand, information in the architecture that would normally be used in SEAS was not relevant to our particular scenario. An example of this is the SV-1 provided by the AOC architecture which contained information on individual AOC subsystems. This information was at too low a level to be of any use to a campaign level model like SEAS. The fact that the case study was unable to verify the methodology for the general attributes is much less of an issue than it would be for the orders, for example, as the orders are at the heart of agent based simulation. Furthermore, it is clear which sections of the architecture would provide this information (SV-7 for performance aspects, OV-4 for command structure). In the absence of an OV-4, the OV-2 showed that virtually all

70

systems report to the AOC. This is essentially how the command structure was modeled in the original war file, so little modification was needed. 4.1.2 Communications As previously stated, SEAS needs to know who is talking to whom, what information is being exchanged, and how it is being exchanged. The "how" includes such information as the means, frequency, capacity, and susceptibility to jamming. For this case study, two methods were evaluated to extract this information from the architecture. The first method is based off the SV-6 System Data Exchange Spreadsheet shown below. One look at this spreadsheet will reveal why it would make perfect sense to use for mapping communications. Each row represents a different system data exchange (essentially the information being exchanged ­ this is listed inside the parentheses) and they are grouped by the interface name (the "who is talking to whom"). The "how" is then enunciated by the different columns such as Media, Format, Protocols, and Size/Units.

Table 4 System Data Exchange Spreadsheet (AOC Architecture, 2003:SV-6)

INTERFACE NAME TBMCS/IRIS to DCGS/DCGS SYSTEM DATA EXCHANGE

CONTENT

MEDIA

FORMAT

PROTOCOLS

SIZE/UNITS AOC AOC

.......

TBMCS/IRIS to DCGS/DCGS (ACO) TBMCS/IRIS to DCGS/DCGS (ATO Changes)

While this would seem to be a perfect match for this case study, several problems arose while trying to use this method. The first issue is based on the fact that System Data Exchanges (SDEs) only represent data exchanged between automated systems. 71

Information Exchanges (IEXs), on the other hand, represent all information exchanged, including between people (recall that the Information Exchange Matrix is represented by OV-3). We, of course, are interested in all the information exchanged, not just those between systems. This may not be important in this particular case study as we are looking at the battlespace at a very high level where everything from Global Hawk and Fighters as well as Special Operations Forces are looked upon as "systems." In fact, there is almost a one to one mapping of Operational Nodes to System Nodes, and most of the discrepancies can be explained by aggregation (for example, the OV had a Fighter/Bomber node while the SV has a Fighter node and Bomber node). Nevertheless, it is important to make sure all information exchanges are included. Fortunately, this is made easy in the SV-6 spreadsheet provided by the AOC architecture. The spreadsheet has two tabs, one labeled SV-6(SV) (referred to previously) and SV-6(OV). The SV6(OV) is essentially the OV-3 mapped to its corresponding System Data Exchanges. One could use this spreadsheet supplemented by the extra columns of the SV-6(SV) when applicable. This method ultimately failed for two reasons. First, it was unclear where communications information is found for information exchanges that did not correspond to an SDE. Second, the information expected in the "Media, Format, Protocols, and Size/Units" columns was typically nonexistent or useless. Clearly another method was needed. Knowing that Information Exchanges were the key elements on which to base the communications mapping, we shift focus to the OV-3. The OV-3 for the AOC architecture lists all the information exchanges for a particular "Needline." For instance, the Needline "CAOC to Fighter/Bomber" contains the following information exchanges:

72

Operations Direction & Guidance, Mission Change Orders, Critical Aircrew Information, etc. As it was intended to only model the top level effects of certain areas of the Kosovo scenario, secondary needlines like "AOC to Mobility Aircraft" were not considered and were hidden in the modified spreadsheet. Each information exchange is linked with its sending and receiving node and activity, which is why the product is referred to as a matrix. The primary interest, however, is simply the "who" and "what" information provided in the first few columns. Based on this, an additional column was added for the benefit of SEAS to classify the IEX as an "Order", "Location", or "Other." Recall from section 3.2 that Orders, Locations, and Other variables are the three types of messages passed over communications lines in SEAS (for more detail, see DeStefano's section 3.3.1.1). To get the "how" information, other products would be needed. The first preference would be to check the SV-6. As we have seen in this case, however, the information is lacking. The next place to look, naturally, would be the SV-2 System Communications Interfaces Diagrams. The drawback here is that the SV-2 is solely a diagram, rather than a spreadsheet that can easily be parsed and pasted alongside Needlines in the OV-3. Therefore, at least for now, we had to fall back on a highly manual method that involved opening the SV-2 diagram corresponding to every Needline in the OV-3. Based on the diagram, three additional columns were added to aid in the transition to SEAS: 1) Comm Overview: usually drawn from a description of one of the comm nodes, such as "Theater Deployable Comm" 2) Comm Details: taken from the comm connection lines, for instance "UHF Satcom"

73

3) Bandwidth and other info: as much technical information as possible. In some cases the SV-2 will enunciate the throughput such as "128 kbs" Fortunately, an SV-2 diagram existed for every Needline relevant to the Kosovo scenario. Therefore, we were able to fill in the "Comm Details" column for every Needline. The information in the other columns was much more hit and miss. The table below is a highly abbreviated version of the table sent to the SEAS modeler to map the communications.

Table 5 Modified Information Exchange Matrix (AOC Architecture, 2003:OV-3 with SV-2)

NEED LINE CAOC to ASOC INFORMATION EXCHANGE Dynamic Targeting Execution Orders SENDING ACTIVITY CAOC-4.5.2.9Execute engagement option (Engage) CAOC4.5.2.9.2-Issue execution orders JSOTF-3Manage Joint Special Ops Assessment RECEIVING ACTIVITY ASOC-3Execute Direct Air Support Missions Orders Location Other orders Comm Overview Comm Details BW or other info

Theater Deployable Comm

SHF or EHF SATCOM

256 kbs

JSOTF toCAOC

Battle Damage Assessment

CAOC-5.3.3.2Assess weapon effectiveness CAOC5.3.3.2.1.1Perform Nonlethal Weapons Assessment (NWA) CAOC-5.3.3.1Assess battle damage CAOC-3.1.4Review BDA CAOC-3.3.5.9(see 3.3.3)

other

uhf and/or shf SATCOM

unk

This begs the following question: is it permissible to assume that every information exchange under a particular Needline uses the same communication channel? Probably not, given that some SV-2 diagrams show multiple communicationsroutes,

74

some better suited for data and some more likely to be use for voice. Unfortunately, there is nothing in the architecture to indicate which IEXs use which path. For the purpose of this case study, we used common sense and sometimes outside research to supplement the architecture in these cases. 4.1.3 Orders To model the eleven activities in the TCT thread various styles of reports were evaluated in Popkin System Architect. The report that proved most relevant to the task at hand was the Activity Model Report under the C4ISR Reports function (under the "Tools" menu). This allowed the creation of a Microsoft Word document, complete with table of contents, which listed all 11 activities, their ICOM arrows, and any accompanying text descriptions. Unfortunately, the activities did not come out in the proper order, and had to be rearranged by hand. Despite this, the report served its purpose, which really was to provide a place to interpret the rules model near its parent activity (as described in the next paragraph). Standing alone, this report would suffice to make the SEAS modeler aware of the activities conducted in the TCT thread and what information is passed between them. Still missing are the rules that dictate completion of an activity, which should be obtained from the OV-6a. Figure 25 shows the report for Activity 6: Conduct Dynamic Assessment of Target, excerpted from the full report.

75

6 Operational Activity: TCT-Determine target significance/urgency (Track) [Within OV-5 Diagram 'TCT-Level 1'] Glossary Text: Utilizing track data and other target information, C2 Warriors determine if the target/target set is threatening and/or fleeting, and estimate target availability, i.e., how long the target will remain susceptible to attack. From 2005 C2 Constellation 3.2.5.2 and CAOC-4.5.2.7 ICOM line: Air Track (J3.2) Output: going to TCT-Validate target/target set (Target) as input ICOM line: Current Intelligence - Dynamic Assessment/Target Status Input: coming from <offpage> ICOM line: Current Intelligence - Target Classification Input: coming from TCT-Define target/target set (Fix) as output ICOM line: Current Intelligence - Target Identification Input: coming from <offpage> ICOM line: Doctrine, Policy, LOAC, SROE, ROE Control: coming from <offpage> ICOM line: Dynamic Target Nomination Output: going to <offpage> ICOM line: Dynamic Targeting Execution Direction and Guidance Control: coming from <offpage> ICOM line: JMSNSTAT Input: coming from <offpage> ICOM line: Land (Ground) Point/Track (J3.5) Output: going to TCT-Validate target/target set (Target) as input ICOM line: Reattack Recommendation Output: going to TCT-Nominate engagement option (Target) as input ICOM line: TRKREP Output: going to TCT-Validate target/target set (Target) as input ICOM line: Track Management (J7.0) Input: coming from <offpage>

Figure 25 OV-5 Activity Report Generated by Popkin SA for TCT Activity 6

Translating the Operational Rules set was a much more time consuming and, unfortunately, a highly manual step. Had the rules been presented in Structured English (basically IF-THEN-ELSE pseudocode), it would be a one step process where the SEAS modeler simply would translate the given OV-6a into SEAS. Popkin, however, has chosen the IDEF3 format as its sole OV-6 diagram (and has labeled it "OV-6a

76

Operational Rules Model"). As described in Chapter 2, the IDEF3 format allows the viewer to deduce sequencing information from the diagram and is useful in this respect. Consequently, elements of the rules model (OV-6a) and the Event-Trace description (OV-6c) are shown in this diagram. This aggregation is done at the expense of both products ­ probably more so for the rules model, which is especially unfortunate for this research. The challenge, therefore, is to interpret the operational rules for each activity based on these diagrams. As a result, the translation from Popkin to SEAS would have to involve at least two steps. As discussed in Chapter 3, our solution was to use a pseudocode translation of the OV-6a as an intermediate stage. The TCT architecture has an OV-6a diagram for each of its 11 activities. This makes sense because, as we have learned, each leaf (detailed level) activity should have its own rules set. Therefore, for each activity the corresponding IDEF3 diagram was translated into pseudo code. Included here is an example of how this is done from Activity 6: Conduct Dynamic Assessment of Target. The IDEF3 diagram is show below in Figure 26 (it is chopped into two pieces to better fit the format of the page). The context of Activity 6 is immediately after a target (or target set) is found and fixed. The top diagram (first half) shows an XOR junction (depicted by an X - refer to 2.2.1 for a quick IDEF3 tutorial, if needed) indicated one of the two paths can be taken, but not both. The resulting "target update" is then put through four simultaneous analyses ­ we know all four must be conducted because of the AND junction, and we know it is simultaneous due to the parallel arrangement of the process blocks. The following XOR junction indicates another decision point ­ in this case, "Is the target time critical?" If it passes the TCT test, another decision point is presented, asking the question "Is this the initial attack

77

on the target?" If so, the target needs to be validated. If not, we proceed with the engagement.

AMTI/GMTI

Monitor Target/Target Status

Project target movement Track Data Target Vector

X

Significant Movement Yes/No Target Monitoring

O

Target Update

Monitor for Movement

Static Target

Target Coordinates

Current Intelligence

Analyz e Threat from Target

First Strike Target Threat Go To OV-6a, 7.1

O

TCT Yes Direction and Guidance Analyz e Dynamic Targeting Execution Direction and Guidance Direction and Guidance Compliance Nominate as a dynamic target (TCT) TCT Nomination Review established target lists

X

Initial Attack Yes/No Attack Decision

Target Update

&

Target Significance Analysis Determine target window of vulnerability (urgency) Synthesize Results

&

Significance/Urgency Results

X

TCT Determination Yes/No

X

Go To OV-6a, 8.1 Reattack Recommendation

Target Analysis

Target Urgency

Pass target to ATO Planners TOI Info TCT No

Monitor Target of Interest for Status Change

Determine Target Significance (Value/Effect) Target Analysis Targ et Sig nificance

Figure 26 IDEF3 diagram for "Conduct Dynamic Assessment of Target" (TCT 2005 Architecture, 2003:OV-6a)

Most of the IDEF3 diagrams are of a similar level of complexity and typically about twice as long as this one. It can be seen then that IDEF3 notation is fairly easy to understand and can convey important information regarding the dynamic nature of the system. When trying to extract precise rules from this diagram, however, much of the details are left to the imagination. The text below is the pseudo code translation of the

78

above IDEF3 diagram. To aid the SEAS modeler, it was written into the Activity Model report directly after the automatically generated text for each activity. PSUEDO CODE FOR ACTIVITY 6 ­ BASED OFF IDEF3 DIAGRAM:

IF Significant Movement of target Then Monitor Target/Target Status Project Target Movement Target Vector = . . . .. ? Else Monitor for Movement Analyze Threat from Target (is the target closing on Friendlies or Fleeing?) Analyze Dynamic Targeting Ex Direction and Guidance (does this agree with the commander's requirements?) Determine target window of vulnerability (urgency) Determine target significance ­ partly based on above findings If it is determined to be a TCT based on the above info Then IF this is the first strike attempt on this target Then Goto Activity 7 (Validate Target/Target set) Else Goto Activity 8 (Nominate engagement option) Else Pass target to ATO Planners Monitor Target of Interest for Status Change

While the IDEF3 did a good job of showing where the decision points and other junctions were, it generally lacked the information to say which path should be followed or what the entry/exit conditions were for the processes. For example, what constitutes significant movement of the target, or how do you determine target value? These are ambiguous for a reason ­ they are qualitative decisions being made by commanders on the spot and are not easily quantifiable. Another challenge was how to deal with the OR junction (not really used in the above example). If an OR junction precedes several parallel processes, at least one of these processes needs to be completed prior to moving on. Therefore, it is not known for sure which processes will happen and which will not. In a real AOC, theses paths are determined on a case by case basis based on the

79

individual situation. The rules that dictate these decisions in real life are not shown here, however, and will be difficult to speculate on. Another problem had to do with modeling overall system performance. As explained in Chapter 3, the SV-7 would be the correct source to find this information. Unfortunately, none of the information was completed in the current version of the AOC architecture. According to the lead developer for the architecture, collecting this detailed data may be the next step taken toward creating their own executable model for an AOC (Surer, 2003). Their model will be a detailed representation of the internal business processes that go on in an AOC. Our model, on the other hand, is only interested in performance at the highest level. It would have been advantageous to have some general performance numbers from SV-7 to use for this case study. 4.1.4 The Modified SEAS War File Using the process outlined above, the SEAS war file was successfully modified to incorporate data from the AOC architecture. The existing AOC function already in the war file was essentially overhauled to reflect details in the architecture, especially the 11 activities outlined in the Time Critical Targeting key thread. While successful, the process was by no means seamless or automatic. Furthermore, some of the information required by SEAS was not available in the AOC architecture. These and other issues will be laid out more specifically in the following pages. The most important changes to the war file were made in the "Orders" section for the AOC and its subordinate units. The goal was to make sure that each of the 11 activities in the TCT activity model were being properly modeled. When sequencing issues were encountered, the OV-6a diagram was referenced. When questions arose

80

regarding operational rules, the pseudocode was referenced. The original intent was to have the report created by merging elements of the OV-5 and OV-6a be a stand alone document upon which the orders in SEAS could be based. This would have been a good first step toward automation of the process. The reality is that a text report, no matter how well done, can not effectively replace the diagram it was based on. This is not to say that the pseudocode was not useful. The fact that the pseudocode was used as well as the diagrams it was based upon proves its value in the process, albeit not stand alone. Two important changes to the war file done to achieve concurrence with TCT architecture are the modeling of Activity 1, the generation of a Dynamic Target Watch List (DTWL) and Activity 4, the adjustment of theater ISR (Intelligence, Surveillance, and Reconnaissance) assets to support operations. For Activity 1, Capt DeStefano made changes to the way target priorities were assigned by the AOC. The changes better reflected the process that was modeled in the TCT architecture, which was based on the DTWL. The new method based the targeting priorities in the DTWL on specific types of targets, rather than broad categories of targets. Changes made to account for Activity 4 were also significant. The original war file did not provide for the dynamic tasking of ISR assets. ISR assets like the Global Hawk UAV would only patrol in a user specified orbit. In accordance with the architecture, changes were made that allow the Global Hawk to be tasked outside of its initial area of operations. For example, if another ISR asset (like a reconnaissance satellite) identifies a potential target with a "Probability of Identification" (PId) that is less than the criteria needed to attack, the AOC now sends the Global Hawk to that location. The Global Hawk will then attempt to make a positive identification on the

81

target for at least one orbit around the new site, after which it may be reassigned. The TPL code written to accomplish this is printed in DeStefano's thesis, along with a complete accounting of the 11 activities in the model (DeStefano, 2004:3.3.2). For each activity, the method of implementation is outlined; whether it was coded in for the purpose of the study, was already in the war file, was already inherent to SEAS, or was not implemented. In the cases where an activity could not be implemented in this case study, a solution is proposed. In the case of the communications, the existing war file already captured the information accurately. If the war file had to be created from scratch, the spreadsheet would have supplied enough information to create a communications structure from the ground up, with one exception (DeStefano, 2004:3.3.1). Capt DeStefano used the spreadsheet provided based on the OV-3 and SV-2 to verify this. The format of the spreadsheet was improved by grouping reciprocal needlines (like "CAOC to Bomber" and "Bomber to CAOC") under the same worksheet so the SEAS programmer can determine if a message is send only, receive only, or both. With this change made, the procedure proved to be a sound way to map the communications. The exception noted above refers to the fact that the architecture does not specify which communications path (if there is more than one) is used by each information exchange or system data exchange. This issue was explained at the end of section 4.1.2. This is critical information when attempting to model the effects of a communications failure. It is important to know what information exchanges would cease during the communications disruption, but there is currently no way to determine that.

82

As explained previously, very little information was used from the architecture for the general attributes. Where information could not be provided, the existing attributes in the war file were used. 4.1.5 Using the New War File With the Kosovo war file freshly modified to reflect the AOC and TCT Architectures, Capt DeStefano then performed a comparison study using the ability to reduce civilian casualties ("ethnic cleansing") as his primary Measure of Effectiveness (MOE). The purpose was to determine if the war file that was modified based on the architecture produced different results than the baseline war file provided by SMC's Transformation Directorate (SMC/TD). This is not to say that favorable results for the Blue side using the modified war file indicated success ­ our intent is not to create a "new and improved" AOC. The objective of the research is to investigate architectures as an information source for agent based simulation. The two models are both looking at the same "real world" AOC, but have been built using different sources. An AOC modeled from a different source than the baseline could more realistically depict adverse effects and actually reduce the Blue force's overall effectiveness. The reason for the study conducted below is simply to identify the effects of changes made as a result of this research. To reduce variation and provide means for a statistical comparison in the results, 100 simulation runs were conducted. The length of simulation time evaluated was one week, as early exploratory studies showed that little combat occurred after that period of time. Because most of the changes in the war file came from the TCT architecture, the results for the modified war file are labeled as "TCT" (DeStefano, 2004:4.2).

83

The results indicated a small reduction in the amount of Kosovar civilian casualties over the baseline. Figure 27 shows how this reduction is distributed among the total simulated time. There is no shift in the distribution, but the overall amount was reduced with statistical significance (using a 90% Confidence Interval).

Baseline vs. TCT: Kosovar Agents Killed

TCT_Kovovar_Cart Baseline_Kosovar_Cart

60

Count (over all replications)

50

40

TCT_Kovovar_House Baseline_Kosovar_Hou

30

20

10

Baseline_Kosovar_House TCT_Kovovar_House

0

1

6

11

16

21

26

31

36

41

46

Baseline_Kosovar_Cart

51

56

61

66

71

86

91

Figure 27 Comparison of Baseline vs. Modified War File - Kosovar Casualties (DeStefano, 2004:4.3)

A statistically significant change occurs in the number of deaths of other Red and Blue agent types, as well. This is shown by DeStefano by means of a "paired t-test," the results of which are outlined in Table 6. Note that the 100 runs are broken into 5 groups of 20 replications (DeStefano, 2004:4.3). In it's last column, the table states which war file's results would be favorable to the Blue side (for example, less Blue F-15's died in

84

96

Time(h)

76

TCT_Kovovar_Cart

81

the TCT war file, so the TCT war file is listed as favorable). If the difference was not statistically significant, "N/A" is listed.

Table 6 Paired T-Test Results for "All Agents Killed" (DeStefano, 2004:4.3)

The most significant difference between the baseline and TCT results are in the number of Blue sorties required to achieve the above results. While the number of Red casualties increased, the number of Blue sorties decreased dramatically in the TCT war file. This indicates that the changes made to the war file based on the TCT Architecture significantly improved the sortie effectiveness for the Blue side. Figure 28 shows the reduction in sorties as a function of simulation time.

85

Baseline vs. TCT Number of Sorties Flown

Number sorties over 100 replications 600 500 400 300 200 100 0 1 6 11 16 21 26 31 36 41 46 51 56 61 66 71 76 81 86 91 96 101 time (h)

Baseline TCT

TCT

Baseline

Figure 28 Sorties Flown - TCT vs. Baseline War File (DeStefano, 2004:4.3)

Accordingly, the above results lend credence to the conclusion that the methodology laid out in Figure 17, Figure 19, and Figure 23 is an effective way to create SEAS TPL code based on a DoDAF architecture. The TCT war file showed significant and meaningful improvements in Blue force effectiveness, suggesting the modifications made that were based on the architecture did make an impact. 4.2 Analysis The case study has shown that our methodology can be used for its intended purpose. Given that, we now examine some challenges that are inherent with the process, as well as compare these results to DoDAF recommendations.

86

4.2.1 Missing Pieces In this case study, the AOC and TCT architectures were unable to provide 100% of the information required by SEAS. When encountered, these "missing pieces" had to default to existing data in the war file or find other sources. In the case of the communications mapping, it was disappointing to find that information regarding bandwidth and protocols were unavailable. A column was provided for this information, and it may be filled in at a later date, however. This would be quite beneficial, as the information is crucial to modeling realistic communications in SEAS. There was also some guesswork involved in creating the orders. The inherent problem is that sometimes when a decision or judgment is to be made by a human in the AOC, the OV-6a simply states "commander's discretion," or something similar. This is of little benefit to SEAS, which really needs to reproduce the logical sequence that the commander goes through to make the decision, and then needs to make that decision itself. This is a key benefit of agent based simulation, and to fully reach its potential the details about the decision process are required. Another key shortfall was inherent to the way the IDEF3 model was created. When simultaneous processes exist, the IDEF3 does a good job of showing their parallel nature. The problem is after an OR split, where one or more of the following paths can be taken, there is no indication of the rules that dictate which path is taken in which situation. Furthermore, the entry and exit criteria for each process are loosely defined, at best. Overall, the IDEF3 does what it was intended to do by providing the viewer a good idea of the sequence of processes that go on in an AOC for Time Critical Targeting. Nevertheless, more information is needed to provide a realistic depiction in agent based simulation and minimize the guesswork.

87

Although we were unable to use data from the architecture for the general attributes, it is possible to identify potential "missing pieces." It is anticipated that top level parameters (like the ones required by SEAS) may be hard to find in a typical SV-7, which often focuses on detailed level subsystem performance. Another potential trouble area may be the OV-4, for the same reason. For example, had an AOC OV-4 focused only on the internal command structure of an AOC, rather than showing a top level command structure, it would not be of much value to SEAS. 4.2.2 Potential for Automation One of the goals of this research is to find ways to make modeling a weapon system in SEAS a more streamlined process. Even if we were to show that 100% of the information needed to do this was available in a DoD architecture, it wouldn't be worthwhile if the process took an excessive amount of time. Thankfully, the process does not take very long, especially if the modeler already knows how to interpret DoDAF products (like understanding IDEF0, for instance). The process used in this research is highly manual, however, as shown above. Two of the main reasons for this are that oftentimes diagrams need to be interpreted by the modeler and information from several products must be fused. The former reason was certainly the case for the IDEF3 diagram, which needed to be translated into pseudocode for the orders mapping. Therefore, the orders are the one area that would be virtually impossible to automate without a major change in the way Popkin depicts its OV-6a. Automation of the general attributes mapping is definitely feasible, being that most of these are simple numeric parameters. The difficulty will be matching the right parameters in the architecture to the top level parameters SEAS expects. A spot check by

88

the modeler will be essential to ensure an "apples to apples" comparison. Furthermore, it is unlikely that a typical SV-7 will contain top level parameters like "speed" or "ceiling" but will most likely be focused on the detailed parameters of subsystems. To deal with this, one may have to build a more traditional executable model to output these higher level parameters (like the EAMA work being conducted by MITRE). This would be time consuming and add another layer of complexity, however, and is probably overkill for the purposes of most studies done using SEAS. When not explicitly available in the architecture, obtaining the parameters by traditional means is probably the most efficient solution. The communications mapping also has potential for automation. The method used here, which is based on the OV-3 needline, is already in spreadsheet form. The difficult step would be transferring communications channel data from the SV-2, which is in diagram form. Extracting the data should be an easy task for someone familiar with reporting in Popkin. Like the general attributes, however, it will be important to give the results a sanity check to ensure the right communications channel is matched to the right needline. 4.2.3 Comparison to DoDAF's "Architectural Uses" Figure 2-2 Knowing now which DoDAF products are most applicable to agent based simulation, a comparison is warranted between these findings and Figure 2-2 in DoDAF Volume II "Architectural Products by Use" ­ provided here in Appendix B. The Figure breaks up the potential uses into four categories: Planning Programming Budgeting Execution System (PPBES), JCIDS, Acquisition, and Operations. Most would argue that M&S should be used in all these categories; for the purpose of this analysis, however,

89

only the Analysis of Alternatives subsection will be considered. There should be little doubt about the importance of M&S at this stage, especially top level simulation with quick turn around times like SEAS. Overall, the predicted products required for an AoA in the DoDAF figure matches up well with the findings in this research. Products like the OV-5 and OV-6, which are essential for describing agent behavior in SEAS, are listed as "highly applicable." The SV-1, a key diagram for identifying weapons and sensors attached to the platform, is also listed as "highly applicable." OV-3 and SV-2, which are each partially responsible for mapping the communications, are listed as "partially applicable." The one major discrepancy is the OV-4 which should be used to construct the command structure in M&S. The DoDAF lists the OV-4 as usually not applicable. The OV-4 may not apply to detailed level executable models that focus on the inner workings of an isolated system, but it is important for a campaign model or a mission level model like SEAS.

90

V. Conclusions and Recommendations

5.1 Conclusions When fully implemented, the products defined by the DoD Architectural Framework (DoDAF) should provide most, if not all, of the information necessary to model a weapon system in an agent based simulation. Most architectures will not contain every product defined in the DoDAF, however. This research has identified eight important products, shown in Figure 24, needed to describe a weapons system for agent based modeling. Though the process used in this case study was primarily manual, potential exists for automation. If the CADM is implemented as planned and is accepted by architectural tool vendors we can look forward to using XML as a common language between architectures and simulation. In the meantime, minor changes to the way architectures are built may make the process much friendlier to automation (see Recommendations below). Analysis using Modeling and Simulation is important at all stages of system development. Furthermore, it has been shown that agent based simulation improves the ability to model the vital effects of C4ISR systems. Architectures have been permanently adopted by the DoD to help develop and implement C4ISR intensive systems. Despite their common goals, the development of systems architectures and the modeling of the same system in simulation remain, for the most part, completely separate processes accomplished by different organizations. As a remedy, this research has shown that architectures have the potential to provide all the information needed for an agent based

91

model, and a methodology has been outlined. This should encourage a closer collaboration between the system architect and the analyst. The result will be a better simulation model that is consistent with the system's actual architecture. It should also reduce redundant work and result in faster turn around times for analysis at all stages of development and implementation. This, in turn, will allow the system architects to improve designs up front, and even allow operators to refine CONOPS prior to delivery. This cycle that feeds back into the architecture will greatly enhance the DoD's ability to acquire and utilize C4ISR systems in the future. 5.2 Recommendations While the DoDAF provides ample products to document system details, it is unclear where broad, general information on the system should be stored. We encountered this problem when looking for information like "number of people," and "deploy time." The DoDAF should provide guidance on where this information should be stored, such as in a subsection of the OV-4. This was the only real issue encountered with the Framework ­ the rest of the needed information was accounted for, and this was largely verified by the case study. As shown in Chapters 3 and 4, the case study has uncovered many ways in which the DoDAF might be used more efficiently to support agent based simulation. These recommendations are explained below. For recommendations on how the simulation world might better interface with DoDAF architectures, see DeStefano's thesis (DeStefano, 2004:5.4). 5.2.1 Recommendations for Architecture Developers Most of the architectures evaluated for this research aspire to be executable and support military worth analysis, at least as a secondary purpose. Based on the experience

92

gained from doing this in the case study, we have identified a number of things system architects can do during the development of the architecture that will greatly enhance the architecture's executability in agent based simulation. While all eight products shown in Figure 24 are important, the OV-5 and OV-6a are critical to be able to exploit the advantages of agent based simulation. These are the products that allowed us to model the logical decision making processes that occur in an AOC. This then allows key advantages like information superiority to play a role in the simulation. Therefore, we recommend having an OV-6a written in Structured English that defines the rules for each leaf activity in OV-5 as a prerequisite to making the architecture executable. Whenever possible, the architect should attempt to model the rules used by decision makers in the AOC, rather than just saying "commander's discretion." It is important to know exactly what guidelines the commander uses for his discretion when modeling in an agent based simulation. The construction of an SV-7 System Performance Parameters Matrix is often a dreaded task for a systems architect as it involves tracking down (or defining, in the case of a to-be architecture) hundreds, if not thousands, of detailed level sub-system performance specifications. While these may be useful for other reasons, SEAS is only concerned with top level performance information like speed, ceiling, average communications delay, etc. Consequently, having an abbreviated, high level SV-7 in place before attempting to map the architecture into agent based simulation will save the analyst a lot of leg work. Many architectures contain products that are only partially completed. For example, the AOC Architecture had an OV-3 and SV-6 that contained no information on

93

the frequency or protocol of the information and data exchanged. In order to accurately map communications in SEAS, this information is vital. We recommend that it be completed for every needline in the OV-3. In addition to this, the communications path (as illustrated in SV-2) taken by each system data exchange (and in some cases, information exchange) should be clearly enunciated. This is essential if the effects of communications disruptions are to be modeled. 5.2.2 Recommendations for Tool Vendors While the method Popkin chose to depict the OV-6a is beneficial in some respects, it seems to neglect the main point of the product: enunciating the operational rules for each leaf activity. We would recommend a simple text dialog that is available only for leaf activities that allows the rules to be entered in Structured English. The OV-6a could then be generated as a report, or even overlaid on the leaf activities in the OV-5 diagram. We also recommend that tool vendors pursue CADM compliance in future version of their products. Having a common data model will greatly increase the potential for versatility of a tool that aides in the transfer of data from architectures to M&S. 5.2.3 Recommendations for Further Research Due to limitations in the architectures we used for our case study, and to the scope of the Kosovo SEAS model, we were not able to validate the use of all the products in Figure 24. By repeating this case study with a different architecture and war file, information from the SV-7, OV-4, and SV-1 could be used, thus validating their role in the process. This follow on work could also focus on the automation of the process. It is highly recommended that whoever undertakes this have some background in computer

94

programming, as much of the automation would involve manipulating report generation scripts. While this research focused on the transition from architecture to M&S, it is important that changes made to the model are reflected back into the architecture. Research could be done on the transition from M&S back to architectures, or simply improving the synchronicity between the two. If the DoDAF is approved and the CADM is utilized by tool vendors, architectures in the future will be shared in XML format. Theoretically, the XML architecture would be in a much more convenient form for importing into simulation than the current standards. Future work could focus on an automated XML to SEAS interface.

95

Appendix A: DoDAF Products list (from section 2.2 of Volume II)

96

Appendix B: DoDAF's Architecture Products by Use (from section 2.2 of Volume II)

97

Bibliography Air Force Institute of Technology (AFIT). Course Catalog 2003-2004. http://www.afit.edu/information/catalogs/catalog%2003-04.pdf AOC Weapon System Block 10.1 Architecture, Version 2.0. The MITRE Corporation, Hampton, VA. 1 October 2003. Cares, Jeffrey. The Use of Agent-Based Models in Military Concept Development, 2002 Winter Simulation Conference. Dam, Steven; Willis, James. Defensible C4ISR Architectures Require the Application of Good System Engineering Practice. Systems and Proposal Engineering Company, Marshall VA, 2002. Dandashi, Fatma. DoD Architecture Framework Overview. Briefing slides, ASD (NII) DoD CIO, October 2003. Davis, Paul K. Analytic Architecture for Capabilities-Based Planning, Mission-System Analysis, and Transformation. RAND, 2000. Department of the Air Force. Air Force Policy on Enterprise Architecting. Washington: HQ USAF, 6 August 2002. Department of the Air Force. Air Force Enterprise Architecture Framework. AF Chief Architect's Office, 6 June 2003. Dickerson, Charles; Soules, Steve. Using Architecture Analysis for Mission Capability Acquisition 2000 Department of Defense. DoD Business Transformation Brief. Washington: 2003. http://dod5000.dau.mil/DOCS/V5DoDBusinessTransformationBrief.ppt. Department of Defense, USD (A&T) Memorandum: Strategic Direction for a DoD Architecture Framework. 23 February 1998. Department of Defense. C4ISR Architecture Framework Version 2.0. Washington: December 1997 Department of Defense. Procedures for Interoperability and Supportability of Information Technology (IT) and National Security Systems (NSS). DoD

Instruction 4630.8. Washington: 2 May 2002

Department of Defense. The Defense Acquisition System. DoD Directive 5000.1. Washington: May 12, 2003 98

Department of Defense. Operation of the Defense Acquisition System. DoD Directive 5000.2. Washington: May 12, 2003 DeStefano, Gregory V. Agent Based Simulation SEAS Evaluation of DoDAF Architecture. MS thesis, AFIT/GOR/ENS/04M-05. School of Engineering and Management, Air Force Institute of Technology (AU), Wright-Patterson AFB OH, March 2004 Fishwick, Paul A. Using XML for Simulation Modeling, Proceedings of the 2002 Winter Simulation Conference. Federal Information Processing Standards (FIPS) Publications. http://www.itl.nist.gov/fipspubs/by-num.htm Griffin, Allison; Lacetera, Joe; Tolk, Andreas. C4ISR/Sim Technical Reference Model Study Group Final Report, 2003 Gonzales, Dan; Moore, Lou; Pernin, Chris; Matonick, David; Dreyer, Paul. Assessing the Value of Information Superiority for Ground Forces ­ Proof of Concept, RAND, 2001 Headquarters US Air Force. Architecture Course. Class handouts. Arlington, VA, August 2003 Handley, Holly; Zaidi, Zainab; Levis, Alexander. The Use of Simulation Models in Model Driven Experimentation, System Architectures Laboratory, C3I Center, George Mason University, 2000 IEEE STD 1471-2000 Levis, Alexander; Wagenhals, Lee. C4ISR Architectures: I. Developing a Process for C4ISR Architecture Design, System Architectures Laboratory, C3I Center, MSN 4D2, George Mason University, July 2000 Levis, Alexander. Class handouts, SENG 640, Systems Architecture. Center For Systems Engineering, Air Force Institute of Technology, Wright-Patterson AFB OH, Winter Quarter, 2003. Popkin Software. Class handouts, System Architect C4ISR Course. Instructed by Preston Rogers, 9-13 September, 2003. Maier, Mark; Rechtin, Eberhardt. The Art of Systems Architecting, Second Edition, CRC Press, Boca Raton, 2002

99

Mayer, Richard; et al. Information Integration for Concurrent Engineering (IICE) IDEF3 Process Description Capture Method Report. Knowledge Based Systems Incorporated, College Station, TX, September 1995. Nimz, Arthur. The C4ISR Architecture Framework and its Impact on the System Engineering Process. Presentation Slides ­ INCOSE Delaware Valley Chapter, November 2000. Pawlowski, Tom, Paul Barr, and Steve Ring. Executable Architecture Methodology for Analysis. The MITRE Corporation, September, 2003. Rechtin, Eberhardt. Systems Architecting, Creating and Building Complex Systems, Prentice-Hall, Inc., New Jersey, 1991 Ring, Steve. Activity-Based Methodology for Integrated Architectures. Presentation Slides ­ the MITRE Corporation, September 2003 Sowell, Kathie. The C4ISR Architecture Framework: History, Status, and Plans for Evolution. The MITRE Corporation, McLean, Virginia, 2000 Surer, Scott. The MITRE Corporation, Bedford MA. Personal Correspondence. December 2003. TCT 2005 Architecture, Version 1.0. The MITRE Corporation, Hampton, VA. June 2003. Tolk, Andreas; Hieb, Michael. Building & Integrating M&S Components into C4ISR Systems for Suporting Future Military Operations, Position paper for the 2003 International Conference on Grand Challenges for Modeling and Simulation, August, 2002 Tighe, Thomas R. Strategic Effects of Airpower and Complex Adaptive Agents: An Initial Investigation. MS thesis, AFIT/GOA/ENS/99M-09. School of Engineering and Management, Air Force Institute of Technology (AU), WrightPatterson AFB OH, March 1999 Klügl, Franziska; Oechslein, Christoph; Puppe, Frank; Dornhaus, Anna. Multi-Agent Modeling in Comparison to Standard Modelling, AIS'2002, F.J Barros and N. Giambiasi (Eds.) SCS Publishing House, pp 105-110, 2002 Kuck, Inara. Warfare Simulation: Status and Issues for Space ­ An addendum, Air Force Research Laboratory Directed Energy Directorate, April 2003

100

Vittori, Jay. The MITRE Corporation, Hampton VA. Personal Correspondence. December 2003. Wagenhals, Lee; Shin, Insub; Kim, Daesik; Levis, Alexander. C4ISR Architectures: II. A Structured Analysis Approach for Architecture Design, System Architectures Laboratory, C3I Center, MSN 4D2, George Mason University, July 2000 Weber, Robert H. The Aerospace Corporation, El Segundo CA. Personal Correspondence. June 2003 ­ March 2004. Wohlstetter, Albert. Theory and Opposed-System Design, RAND D(L)-16001-1, January 1968 Zachman, John. Concepts of the Framework for Enterprise Architecture, Zachman International, 1997

101

Vita

Captain Andrew Zinn graduated from the University of Portland in 1999 with a B.S. in Mechanical Engineering. Lt Zinn participated in AFROTC and competed in the SAE Mini Baja competition while at UP. Upon his commissioning, Lt Zinn was assigned to the Aeronautical Systems Center at WPAFB as a developmental engineer. While at ASC, he oversaw the testing, delivery, and sustainment of six types of aircrew training devices (flight simulators) for the T-6A Texan II primary trainer. In August 2002, he entered the Graduate School of Engineering and Management, Air Force Institute of Technology. Upon graduation, he will be assigned to the Space and Missile Center's Transformation Directorate at Los Angeles AFB, CA.

102

Information

The Use of Integrated Architectures to Support Agent Based Simulation: An Initial Invesigation

114 pages

Report File (DMCA)

Our content is added by our users. We aim to remove reported files within 1 working day. Please use this link to notify us:

Report this file as copyright or inappropriate

365388


Notice: fwrite(): send of 202 bytes failed with errno=32 Broken pipe in /home/readbag.com/web/sphinxapi.php on line 531