In an interview with DefenseScoop this week, the National Geospatial-Intelligence Agency’s new and first-ever Chief AI Officer Mark Munsell shared initial details about one ongoing pursuit to train a cutting-edge model and shed light on his approach to steering NGA’s early adoption of the still-uncertain technology.

“We could not have predicted some of these inventions with transformers, like [generative pre-trained transformer or GPT] and stuff, and because of that and the trajectory of those, I think certainly we’re going to be living in a better world. But for the first time, we’ll have to really guard against misuse,” Munsell said.

As the CAIO suggested, genAI, geoAI and associated frontier models are part of a rapidly evolving field of technologies that are not fully understood, but are pushing the boundaries of what existing AI and machine learning can accomplish. Typically, such tech can process massive volumes of data and perform increasingly “intelligent” tasks like recognizing speech or generating human-like media and code when prompted.

These capabilities hold a lot of promise to dramatically enhance how the agency’s analysts detect and make sense of objects and activities they are tracking across sprawling data sources, inside NGA’s world that revolves around capturing and deciphering geospatial intelligence about movements and happenings all around the globe.

Munsell is expressly determined in his early months as the new AI chief to set the agency on a clear path for responsibly exploring and adopting powerful frontier models in their day-to-day operations.

However, one immediate challenge he said his team is confronting has to do with the fact that the major companies developing these next-generation models to date have not prioritized geographic use cases that would impact NGA’s work.

“So, we have a big role to play, I think, there on behalf of the country — and really, potentially on behalf of the world — which is to ask these companies to focus on certain capabilities that don’t exist yet, or that the models do not do well yet,” Munsell explained. 

He provided several examples to demonstrate this issue, particularly when it comes to existing computer vision technologies.

“How well the models can identify things on a photo —  that is super important, and we want companies to do that better,” he noted.

Beyond that realm, Munsell said, modern large language models are learning to generate GEOINT assets, like graphs and graphics. But, in his view, these systems are not yet highly skilled at understanding geography and critical features, like longitude and latitude coordinates. 

“Simple things, like depiction of boundaries or understanding certain geographic locations. Today, it’s all based on maybe words in a gazetteer, or words in an encyclopedia, or words in an atlas. A lot of the training has been like that. So there’s a lot to do to turn these large models into things that are geographically aware. NGA will have a big part to play in that,” Munsell said. 

During the interview, the CAIO also offered DefenseScoop the first preview of an initial generative AI experiment that agency insiders are pursuing. 

“One of the examples we’ll have soon, early next year, is we’re doing a retrieval augmented generation — so people just use the term the RAG — implementation of a large language model that’s being trained … on every NGA report ever written,” he said.

Each day, agency officials compile detailed documents on intelligence activities around the world, such as adversary undertakings at specific locations.

“Essentially, [we’ll] be able to ask it any question on any report that’s ever been written, and it [could] have the depth of knowledge and understanding of analysts that worked on accounts for 30 years,” Munsell explained. 

The longtime technologist joined the Defense Mapping Agency (which evolved into NGA) for the first time roughly 30 years ago, as a software engineer in 1995.

Notably, he’s part of a very small percentage of senior executives in the government who have around a million lines of software code under their belts.

“For example, I wrote the system that NGA used up until last year to produce all its aeronautical information that goes into all of the DOD aircraft. That was my first assignment here at NGA,” he noted. 

After that, Munsell spent some time in the private sector. He returned to NGA in the early 2000s and has been rising through the tech ranks there ever since. In recent years, he helped launch the Data and Digital Innovation Directorate — a hub he’s continuing to lead in his now dual-hatted role as CAIO.

“It made logical sense for the agency and for the director to appoint the director of that component as the chief AI officer,” he said.

Munsell has been taking it all in as AI and machine learning have intensely evolved over the course of his career. 

Looking back on his first stint at NGA, he said he didn’t totally anticipate the emergence of generative AI. And while he believes it will improve the human experience, he also repeatedly pointed to the need to adopt these yet-to-be-fully-understood technologies responsibly.

“It’s really kind of a one-two punch. On one hand, you’re going to promote these great inventions that will do wonderful things for for the world. And on the other hand, you do really have to watch and protect people from misusing these. So, that’s … how I look at my responsibilities. It’s two gloves,” Munsell said. 

“On one hand, I’m promoting the use — the proper use — because we know it’s good and better. And on the other hand, we’re checking and we’re ensuring that people are doing right by this technology,” the CAIO told DefenseScoop. 

The post ‘One-two punch’: Inside NGA’s approach to exploring powerful next-gen AI appeared first on DefenseScoop.

During an interview with DefenseScoop on Tuesday, Munsell reflected on his motivation for taking on this nascent role, and discussed those and other key priorities on his near-term agenda as NGA’s newly named CAIO.

“I think you’ll see a lot more energy around implementing generative AI in the analytic workflow,” he said.

Historically considered America’s super-secret mapping agency, NGA is the Defense Department’s functional manager for geospatial intelligence, or GEOINT. That discipline involves the capture of imagery and data from satellites, radar, drones and other means — paired with expert analyses to visually display and monitor physical objects and geographically referenced activities on Earth.

As Munsell suggested, the maturing realm of generative AI (which broadly incorporates disruptive models that can pull patterns within existing data to generate original content across many forms) presents heaps of opportunities and risks for NGA analysts.

The emerging technology and associated, advanced frontier models were already top of mind for Munsell as NGA’s director of data and digital innovation. In addition to serving as CAIO, he’ll continue to lead that directorate — and in doing so, he’s set on “putting AI into the hands” of the agency’s users.

“And what I mean by that is a lot of these … big models being developed by the big companies like Anthropic and Google and OpenAI are still not accessible by the folks at NGA. They are still not usable by the folks in NGA. So I would consider myself successful when our analysts and the officers at NGA have access to those — the world’s best models — on a top secret network,” Munsell told DefenseScoop. 

One primary task that’s already in-play for his team related to this intent, involves revamping NGA’s guidance for how its personnel adopt artificial intelligence to include additions that better confront generative capabilities.

“A new AI strategy will be published by NGA probably after the [2024] calendar year” ends, Munsell said.

Among other responsibilities as CAIO, he’s also working closely with the agency’s first-ever Responsible AI Officer Anna Rubinstein on pursuits to establish more technology governance assets and facilitate initiatives geared at strengthening the trustworthiness and reliability of GEOINT models.

“We know that the work that we do with AI is serious work — positively identifying targets, accurately geo-locating them — it’s serious work that could be used in warfare. And we knew how important it was for us to establish that program of responsible AI implementation and to have a senior officer in charge of it,” Munsell said. 

As CAIO, he now oversees “a couple-hundred-person organization” with offices, divisions and branches. Some of that staff will also be supporting Munsell in an ongoing but complex effort to produce a full inventory of all NGA initiatives and programs on artificial intelligence. 

“We’ve been doing the inventory for a couple of years now, but it’s one of the harder things to do,” he said. 

“There are designated programs that are building and making and implementing AI, but there are hundreds of programs that could sort of suddenly be using AI. And so at some point you do want to know every program and every initiative that’s using AI, so you can have some level of audit on it, some level of security on it, and some level of safety evaluation on it. But sometimes the AI is embedded in software that you’re buying commercially off-the-shelf, and then you’re bringing [it] into the agency and you might not even know it,” Munsell explained.

For that and other reasons, NGA’s inventory will be, as he put it, “less literal,” as well as “a little more logical” and “a little bit more about investments in programs that are deliberately either building or buying or implementing AI.”

In response to questions from DefenseScoop regarding why NGA tapped him as its first CAIO at this particular point in time, Munsell noted that the move is now obligatory for federal organizations under multiple recent directions from the White House and Congress. 

But in his view, the spy agency has also been ahead of the game, especially over the last four years. He called out defense “visionaries” who impacted NGA’s path to realizing AI and enabling the Pentagon’s high-stakes Maven computer vision effort — including former and current NGA and DOD officials such as Phillip Chudoba, Sandra Auchter and Drew Cukor.

“For me, it’s an honor to be the first [CAIO]. But in a way, I’m standing on the shoulders of giants — and I think you should say that, because it’s so true,” Munsell told DefenseScoop.

The post NGA’s new artificial intelligence chief previews near-term priorities appeared first on DefenseScoop.

The move comes amid a broader push by the U.S. intelligence community and Defense Department to onboard more AI technologies. Last week, President Biden issued a new national security memorandum that prods the Defense Department and IC to go faster in adopting these types of capabilities. Munsell was at National Defense University where National Security Adviser Jake Sullivan rolled out the memo, according to a post on LinkedIn.

“NGA had an important role in developing the National Security Memo on Artificial Intelligence,” he wrote. “Congratulations to the National Security Council and the team that worked so hard to layout a framework for the promotion and safe adoption of AI for the country.”

The agency has a number of major ongoing artificial intelligence-related initiatives, including a high-profile effort known as Maven.

Last month, the organization unveiled plans to launch a $700 million data-labeling effort to advance computer vision models.

Previously, it rolled out an initiative called Accreditation of GEOINT AI Models (AGAIM) — which aims to set a new government standard for assessing the robustness and reliability of computer vision models deployed for national security purposes — and an educational tool dubbed GEOINT Responsible AI Training.

In his new role as CAIO, Munsell will be tasked with overseeing, coordinating and integrating NGA’s current and future planned geospatial intelligence AI efforts, advising leadership on applying the tech to production and operations; creating an inventory of AI initiatives and programs; recommending new initiatives; and establishing the governance and responsible use of artificial intelligence capabilities, among other responsibilities, according to a press release.

He will also serve on the intelligence community’s CAIO Council, which is responsible for helping formulate the IC’s broader AI strategy; providing oversight of investments for implementing that strategy; coordinating policy and procedural guidance on managing AI-related risks; promoting the use of enterprise purchase agreements; making recommendations for using and improving “AI-enabled” architectures; and boosting the “trustworthiness, equity, integrity and resilience” of the tech, among other efforts, per the release.

In addition to taking on the new job, he will be dual-hatted as the leader of NGA’s data and digital innovation directorate, a role which he previously held.

Prior to taking on his current positions, Munsell served as the agency’s deputy director for data and digital innovation, chief technology officer, and deputy director for the IT directorate. He is also a founder of NGA’s Moonshot Labs, according to his LinkedIn profile.

The spy agency describes Moonshot Labs as an “unclassified innovation hub and collider space for NGA, its partners and customers to meet, create, collaborate, and innovate.”

DefenseScoop reached out to an NGA spokesperson for more information about who previously held the CAIO role and where they went after exiting the position.

“We’ve had different folks filling some of the roles, but he’s the first publicly designated,” they said of Munsell.

Updated on Oct. 29. 2024, at 6:00 PM: This story has been updated to include comment from an NGA spokesperson.

The post NGA gets new chief AI officer appeared first on DefenseScoop.

“I think the blessing and curse of AI is that it’s going to think differently than us. It could make us better — but it can also confuse us, and it can also mislead us. So we really need to have ways of translating between the two, or having a lot of understanding about where it’s going to succeed and where it’s going to fail so we know where to look for problems to emerge,” NGA’s first-ever Chief of Responsible Artificial Intelligence Anna Rubinstein recently told DefenseScoop.

In her inaugural year in that nascent position, Rubinstein led the making of a new strategy and instructive platform to help guide and govern employees’ existing and future AI pursuits. That latter educational tool is called GEOINT Responsible AI Training, or GREAT.

“So you can be a ‘GREAT’ developer or a ‘GREAT’ user,” Rubinstein said.

During a joint interview, she and the NGA’s director of data and digital innovation, Mark Munsell, briefed DefenseScoop on their team’s vision and evolving approach to ensuring that the agency deploys AI in a safe, ethical and trustworthy manner.

Irresponsible AI

Geospatial intelligence, or GEOINT, encompasses the discipline via which imagery and data is captured from satellites, radar, drones and other assets — and then analyzed by experts to visually depict and assess physical features and specific geographically referenced activities on Earth.

Historically, NGA has a reputation as the United States’ secretive mapping agency.

One of its main missions now (which is closely guarded and not widely publicized) involves managing the entire AI development pipeline for Maven, the military’s prolific computer vision program.

“Prior to this role, I was the director of test and evaluation for Maven. So I got to have a lot of really cool experiences working with different types of AI technologies and applications, and figuring out how to test it at the level of the data models, systems and the human-machine teams. It was just really fun and exciting to take it to the warfighter and see how they are going to use this. We can’t just drop technology in somebody’s lap — you have to make sure the training and the tradecraft is there to support it,” Rubinstein noted.

While she was a contractor in that role, that Maven expertise is now deeply informing her approach to the new, permanent position within the federal agency that she was tapped for.

“I’m trying to leverage all that great experience that I had on Maven to figure out how we can build enterprise capabilities and processes to support NGA — in terms of training people to make sure they understand how to develop and use AI responsibly — to make sure at the program level we can identify best practices and start to distill those into guidelines that programs can use to make sure they can be interoperable and visible to each other, to make sure that we’re informing policy around how to use AI especially in high-risk use cases, and to make sure we’re bringing NGA’s expert judgment on the GEOINT front into that conversation,” Rubinstein explained. 

Inside the agency, she currently reports to Mark Munsell, an award-winning software engineer and longtime leader at NGA. 

“It’s always been NGA’s responsibility to teach, train and qualify people to do precise geo-coordinate mensuration. So this is a GEOINT tradecraft to derive a precision coordinate with imagery. That has to be practiced in a certain way so that if you do employ a precision-guided munition, you’re doing it correctly,” he told DefenseScoop.

According to Munsell, a variety of timely factors motivated the agency to hire Rubinstein and set up a new team within his directorate that focuses solely on AI assurance and workforce development.

“The White House said we should do it. The Department of Defense said we should do it. So all of the country’s leadership thinks that we should do it. I will say, too, that the recognition of both the power of what we’re seeing in tools today and trying to project the power of those tools in five or 10 years from now, says that we need to be paying attention to this now,” Munsell told DefenseScoop. 

Notably, the establishment of NGA’s AI assurance team also comes as the burgeoning field of geoAI — which encompasses methods combining AI and geospatial data and analysis technologies to advance understanding and solutions for complex, spatial problems — rapidly evolves and holds potential for drastic disruption.

“We have really good coders in the United States. They’re developing really great, powerful tools. And at any given time, those tools can be turned against us,” Munsell said. 

DefenseScoop asked both him and Rubinstein to help the audience fully visualize what “irresponsible” AI would look like from NGA’s purview. 

Munsell pointed to the techno-thriller film from 1983, WarGames.

In the movie, a young hacker accesses a U.S. military supercomputer named WOPR — or War Operation Plan Response — and inadvertently triggers a false alarm that threatens to ignite a nuclear war.

“It’s sort of the earliest mention of artificial intelligence in popular culture, even before Terminator and all that kind of stuff. And of course, WOPR decides it’s time to destroy the world and to launch all the missiles from the United States to Russia. And so it starts this countdown, and they’re trying to stop the computer, and the four-star NORAD general walks out and says, ‘Can’t you just unplug the damn thing?’ And the guy like holds a wire and says, ‘Don’t you think we’ve tried that!’” Munsell said. 

In response, Rubinstein also noted that people will often ask her who is serving as NGA’s chief of irresponsible AI, which she called “a snarky way of asking a fair question” about how to achieve and measure responsible AI adoption.

“You’re never going to know everything [with AI], but it’s about making sure you have processes in place to deal with [risk] when it happens, that you have processes for documenting issues, communicating about them and learning from them. And so, I feel like irresponsibility would be not having any of that and just chucking AI over the fence and then when something bad happens, being like ‘Oops, guess we should have [been ready for] that,’” she said.

Munsell added that in his view, “responsible AI is good AI, and it’s war-winning AI.”  

“The more that we provide quality feedback to these models, the better they’re going to be. And therefore, they will perform as designed instead of sloppy, or instead of with a bunch of mistakes and with a bunch of wrong information. And all of those things are irresponsible,” he said.

‘Just the beginning’

Almost immediately after Rubinstein joined NGA as responsible AI chief last summer, senior leadership asked her to oversee the production of a plan and training tool to direct the agency’s relevant technology pursuits.

“When one model can be used for 100 different use cases, and one use case could have 100 different models feeding into it, it’s very complicated. So, we laid out a strategy of what are all the different touchpoints to ensure that we’re building AI governance and assurance into every layer,” she said.

The strategy she and her team created is designed around four pillars. Three of those cover AI assurance at scale, program support, and policies around high-risk use cases.

“And the first is people — so that’s GREAT training,” Rubinstein told DefenseScoop.

The ultimate motivation behind the new training “is to really bring it home to AI practitioners about what AI ethics means and looks like in practice,” she added.

And the new resources her team is refining aim to help distill high-level principles down into actionable frameworks for approaching real-world problems across the AI lifecycle. 

“It’s easy to say you want AI to be transparent and unbiased, and governable and equitable. But what does that mean? And how do you do that? How do you know when you’ve actually gotten there?” Rubinstein said.

In order to adequately address different needs across the two groups, there’s two versions of the GREAT training: one for AI developers and another for AI users.

“The lessons take you somewhat linearly through the development process — like how you set requirements, how you think about data, models, systems and deployment. But then the scenario has a capstone that happens at the end, drops you into the middle of a scenario. There’s a problem, you’re on an AI red team, people have come to you to solve this issue. These are the concerns about this model. And they’re three rounds, and each round has a plot twist,” Rubinstein explained. 

“So it’s, we’re giving students a way to start to think about what that’s going to look like within their organizations and broadly, NGA — and even broader in the geospatial community,” she said. 

Multiple partners, including Penn State Applied Research Lab and In-Q-Tel Labs, have supported the making of the training so far.​​

“We got the GREAT developer course up and running in April, we got the GREAT user course up and running in May. And then beyond that, we will be thinking about how we scale this to everyone else and make sure that we can offer this beyond [our directorate] and beyond NGA,” Rubenstein said.

Her team is also beginning to discuss “what requirements need to look like around who should take it.”

Currently, everyone in NGA’s data and digital innovation directorate is required to complete GREAT. For all other staff, it’s optional.

“The closer they are to being hands-on-keyboard with the AI — either as a producer or consumer — the more we’ll prioritize getting them into classes faster,” Rubenstein noted.

Munsell chimed in: “But the training is just the beginning.”

Moving forward, he and other senior officials intend to see this fresh process formalized into an official certification.

“We want it to mean something when you say you’re a GREAT developer or a GREAT user. And then we want to be able to accredit organizations to maintain their own GEOINT AI training so that we can all be aligned on the standards of our approach to responsible GEOINT AI, but have that more distributed approach to how we offer this,” Rubinstein told DefenseScoop. “Then, beyond that, we want to look at how we can do verification and validation of tools that also support the GEOINT AI analysis mission.”

Updated on June 18, 2024, at 8:05 PM: This story has been updated to reflect a clarification from NGA about how it spells the acronym it uses for its GEOINT Responsible AI Training tool.

The post NGA launches new training to help personnel adopt AI responsibly appeared first on DefenseScoop.

Such models are at the heart of generative artificial intelligence, a buzzy, emerging technology subfield through which machine systems can be trained by humans to generate audio, code, images, text, videos and other content. 

Of those, it’s transformer models — or neural networks that can “learn” context about things by tracking relationships in sequential data, which were just invented in the last few years — that Munsell said will “be generating so much insight and so much valuable, high-quality information that we’ll be winning the wars.”

“I think five years from now we’re going to be running so fast. We are going to be developing models so fast. We’re going to be developing software so fast and capabilities so fast. I know it sounds kind of absurd to say for those of you that have been around this agency and this community for the last 20 years, but I think it’s going to be so fast that I think the companies here will be delivering software multiple times a day. I think we’ll be developing models multiple times a day — constantly testing those, through an automated fashion, and getting feedback from our analysts and feedback from our end users daily,” Munsell said on Tuesday at the annual GEOINT Symposium.

Over the last decade, NGA has adopted a cloud operating environment and applied advanced, interactive development techniques. Now, the agency is bringing in new labels and new models, with machine learning ops.

“We have positioned ourselves to lift off,” Munsell said.

Earlier at the symposium during a press briefing, he noted ongoing efforts associated with “Maven,” the Pentagon’s flagship computer vision effort.

“What’s happening with this with Maven, specifically, is that certain commands are adopting it to tackle certain workflows, and therefore it’s becoming this sort of stream of data between the different Maven instances around the commands,” he told reporters. 

That’s providing many new opportunities for the Defense Department associated with interoperable data.

“This transformer technology, while it makes an interesting piece of art, or can make a song, or generate a term paper — those are all interesting, valuable things and obviously the future is going to be leveraging that — so we are learning from those techniques. Our community is learning from those techniques and applying them to what’s called visual transformers, applying them to computer vision. And I think you’ll see some breakthroughs over the next couple of years as that technology gets used more in our domain,” Munsell said. 

The post ‘Transformer’ large language models expected to drastically advance military computer vision capabilities appeared first on DefenseScoop.