The following is an interview with Mr. Scott Orosz of PMR-51 who was the driving force for funding the initiative with the Office of Naval Research and MIT Lincoln Laboratory to create Strike Force Defender (SGD), winner of the 2014 Serious Games Showcase & Challenge award for “Best Game By A Government Agency.”
You can also click here to download a PDF copy of this interview.
1) Your game, Strike Group Defender (SGD), won the “Best Game by Government Agency” at the 2014 SGS&C. Please tell us a little about the game, including the original goals that made it a “serious” game.
Strike Group Defender (SGD) came about because the Navy was investing significant resources into surface combatant ship’s self-defense system Research and Development (R&D) as a component of the Navy’s Electronic Warfare (EW) Science & Technology (S&T) portfolio. However, while those investments offered a means to significantly enhance self-defense capabilities – in particular anti-ship missile defense (ASMD) – there was no practical means for operators to become – and, especially remain – familiar with the associated tactics which were made possible by those investments. Operators were initially exposed to academic precepts and one or two training “lab” sessions and then sent to ships where the principal means of skill and process development was “discussion based”. What that translated to at the “deck-plates” was a fundamental lack of trust in the systems and concepts because they neither understood the reasons behind the new tactics nor had live training or experience on deployments to establish proficiency, confidence and, ultimately, trust. That wasn’t a “new” deficiency created by the capabilities the S&T investments were delivering to the Fleet. However; the introduction of the new technology being developed made that long-standing problem’s impact more acute for the Surface Warfare Tactical Action Officer (TAO) and their combat team to consistently and efficiently defeat anti-ship threats. Surface warriors had no simulations, simulators, or tools to provide distributed access to “play”, learn and experiment with various approaches (tactics) to “fight the ship” against potential threats. More significantly, operators had no means to fail and receive immediate, informed feedback on the “how and why” behind the failure. Nor could they examine and compare multiple courses of action; restart again; and immediately incorporate informed refinements to baseline tactics. SGD’s goal was to illustrate and explain the science behind the tactics via an easy to use, video game-like environment, which was competitive, goal-oriented and “fun”. In short, we believed (and still do) the full measure of a system’s capability is not just how capable it is – but also how capably it’s employed. That was the SGD “value metric”. Improve the systems employment at the human level to increase the return on investment and ultimately the survivability of Navy Combatants at war.
2) SGD has a rather unique origin story, blending government, academia, contractors and a traditional game company. Why did you decide upon that structure, and what have been the pros and cons of such an arrangement?
At the time, I had seen many demonstrations of what people were calling “Serious Games.” However, with respect to great people trying hard — they were really not what I was looking for. You could just tell from the look and feel that it was a traditional system simulation posing as video-game and it really wasn’t human focused. But I had seen a previous Navy game called GeoCommander (SGS&C Gov’t Game of the Year 2008) and it truly was a great game that sailors loved and it really accomplished the “serious” part. When they got done playing it (often 100s of times at sea) they knew the science, tactics and how to astutely employ the system(s) they managed. That’s what I wanted; and more importantly what the Navy needed. It had to be cutting edge, affordable and not take months or years to develop and be available for Fleet use. Which sounded very much like what GeoCommander achieved by bringing in real video game industry experience to augment Navy tactical expertise and context.
I went to MIT Lincoln Laboratory, which was the Technical R&D lead for the respective Future Naval Capabilities (FNC) project and had them engage with the original technical lead for GeoCommander, Mr. Russ Phelps. Russ happens to be a former Surface Warfare qualified Navy Officer who, after retiring, established himself in the serious games and commercial gaming industry. From that engagement a team composed of the Navy, a R&D Laboratory, a professional serious games contractor and video game industry was established. We next established a partnership with the Naval Postgraduate School’s MOVES Institute and beyond that added four PhD candidates from MIT and George Mason University. Quite fortuitously, the PhD candidates included a Naval Surface Warfare Development Center employee; an Air Force Systems Command officer; and two civilians, whose collective talents and associated doctoral theses provided a rigorous basis in Machine Learning AI, behavioral psychology and Human – Systems integration underpinning SGD’s design and inherent capabilities.
The ‘pros’ of this were immediate and clear. We got access to world class gaming talent to support the Navy and Lab to design and build an award-winning serious game which possessed a proven scientific basis AND sufficiently accurate emulations of ship self defense tactics. Adding to a winning formula, the cost of development was SIGNIFICANTLY lower; the speed of production and delivery MUCH faster; and the product quality BETTER than anything similar available to the Fleet. The ‘cons’ were (are) it is very difficult to get commercial industry clearances and to have a secure production facility. Contracting with industry is also challenging. Keep in mind, this was before DIUx and the new changes aiming to bridge these gaps. In the end we solved these programs and now have a complete ecosystem to support this level of engagement on a perpetual basis. This is also an opportunity to mention that beyond the Navy’s consistently positive reception of SGD, was the recognition of the efficiency and value provided to the government in this undertaking, by the principal SGD sub-contractor – MetaTeq being named the SBA 2015 Small Business of the Year (Region 3) and MIT LL 2015 Small Business Contractor of the Year.
3) In the four years since SGD won at the SGS&C, how has the game evolved? Who is using it now, and what are future plans for its employment?
SGD has evolved in ways, frankly, we didn’t foresee at the outset which led to the “re-branding” of the SGD “platform” to the Joint Cognitive Operations Research Environment or JCORE. The initial vision of “just one game” rapidly evolved to incorporate a very advanced data collection system, analytics, enhanced computational abilities, new geospatial data visualization methods, scenario editors, and countless other backend advancements. It quickly became clear what we had really built was essentially an “iOS” and SGD had become an “app.” From there we assessed this was a platform which could essentially be warfare content agnostic and built upon to host other “apps.” Additionally, because we built a system which can collect prolific levels of rich human user data, JCORE has become a “Go To” platform for researchers to use to generate the data they need to train advanced machine learning informed AI. Soon after our first ML/AI projects, we began to refer to the platform as JCORE while continuously incorporating relevant, resultant improvements into the baseline SGD game.
Once the Navy started seeing what we had they also saw that they could build on JCore and get completely away from having to build an entire backend architecture and just focus on delivering effective content. This saves enormous time and reduces costs dramatically. JCORE is working on nearly a dozen high profile projects that span a diverse set of users for R&D, future force modeling, wargaming, AI development, experimentation and virtual rapid prototyping.
As to who is using it, we are working with and developing solutions for OPNAV N9i on two efforts; the Office of Naval Research; Naval Surface and Mine Warfare Development Center, SPAWAR San Diego (PMW-120), 3 efforts with PMR-51 and new R&D in machine learning AI with MIT LL. JCORE continues to rapidly expand its base of users.
Future plans? We would like JCORE, which is gov’t owned, to be utilized by other services and gov’t agencies. It does not – should not – be “Navy-centric” in its usage. We would also like to see JCORE transition in whole or parts to become the basis for a permanent platform available for the Navy and DoD.
4) Are there any specific challenges in building a game form within the Department of Defense?
I don’t need to dwell on the contracting challenges, clearances and facilities – that is a well-known set of challenges. However, what we see at ONR and particularly through the lens of JCORE is that, more than ever, there is a need for new levels of creativity in the DoD. At the small and large levels, I think that lack of creativity in how we approach technology has real consequences – and costs. The gaming industry is, in particular, incredibly creative and can have real contributions to the DoD. But all too often, tasks needing their expertise are brought in-house, internal to the navy or DoD. In this sense, the challenge is mindset in the DoD and the willingness to try unconventional approaches. I think of it like this: we couldn’t expect the gaming industry to build a military satellite, so why would we reasonably expect a professional defense contractor to build a cutting-edge video game? One could say “well, they’re just games”. Perhaps. But, last year it was a $140B industry and its growing fast.
5) What do you see as the future of serious games within the DoD?
This is a great question. I see a few things trending.
How Serious Games integrate into the DoD really matters. There is increasing acceptance for serious game know-how and technology being integrated into Defense. With some exceptions, this integration is coming in pieces; meaning, solicitations that need advanced visualization, better user interfaces or user experience, virtual environments, wargaming capabilities, for instance, are more common. While this is good and may attest to serious games as a more accepted technical competency, small level integration may not be a very attractive economic motivator for industry. It’s like hiring a car manufacturing company to fix a tire, or an exhaust system. The value is the whole and a game is the composite of countless technologies, techniques and human capital. Breaking it up takes away some of the magic.
“Serious” is going to increasingly become at odds with the “notion” of a game. Another thing that I regularly see is that as a Serious game matures it becomes more “serious” or rigid and less flexible. The more “serious” a game usually means the more realistic; and, that usually means elevations in classifications, and frankly, a more controlled, inflexible environment. Serious games can easily become a mere simulation and less of a game. The preservation of flexibility, creativity, “fun” or “cool” can be overrun by the end state requirement
The intersections of systems and serious games will increase. The overall commercial and DOD trends show that distributed, data driven, virtual environments are an immutable force and the growing norm. It will become increasingly difficult for program sponsors NOT to know how their systems are being employed, or how people are learning. They will need to answer questions like how proficient are the operators; or, how proficient are ships at the individual or aggregate level? How much did an hour of training get me? How much better or worse are operators using a new upgrade – and where in the system they are failing? How well or how would this theorized prototype system be used? These are all reasonable questions that have been traditionally very hard to answer. The gaming industry are some of the most advanced users of “Business Intelligence.” Trust me; they know how to measure players and game success. Interestingly, the technology to measure a games’ entertainment performance is virtually the same as is needed to measure the performance of a DoD operator in a virtual environment doing real-world missions. In JCORE for instance, we know “down-to-the-click” what people are doing. That type of information is going to be the lifeblood in future systems, employment in the systems, AI Development, training, and education.
The increasing demand for human driven data and the tools to help make sense of it. Lastly, a logical extension of the requirement and ability to capture massive amount of human performance data is the eventual use in advanced technologies such as automation systems, and artificial intelligence and machine learning algorithms. Those at the forefront of automation or AI/ML development are increasingly looking at how to ensure their systems integrate or accommodate the behavior and performance of the human operator. If we can understand and capture (via data) the cognitive processes used by an operator, we can use that to create systems and technology that fits in with those processes- helping the warfighter instead of hindering them with clunky systems. One feature that I am really excited about is a tool we are calling “Clairvoyant.AI” and the idea behind this tool is to leverage the scientific processes used in academia and applying it within JCORE to implement the collection and analysis of human performance data. Anyone can record performance data, but not many are qualified to evaluate it. The but the goal of this tool is to capture the right type of data and the right analysis of that data to inform algorithms and answer the specific questions about how the warfighter uses a specific system.
For more information re: Past Finalists / Winners and to learn more about how to Enter Your Game in the Serious Games Showcase & Challenge, visit http://sgschallenge.com/.
News & Updates
Get information about contest deadlines and announcements.
