Strategic Scan insights: What fire chiefs are saying about AI - FireRescue1

For all the discussion surrounding artificial intelligence in the fire service, there has been relatively little data showing how departments are actually using it.

The CPSE Center for Innovation set out to change that with its first Strategic Scan on AI, surveying chief fire officers and leaders from accredited departments about current adoption, future plans and the challenges they see ahead. The findings show that many departments are already using AI for administrative work, while taking a more cautious approach to training and operational applications. Policy, privacy, data quality and trust remain key concerns.

FireRescue1 connected with Alex Henderson, Ph.D., vice president of the CPSE Center for Innovation Board of Directors and associate professor at Marist University, and Alex Fisher, director of the CPSE Center for Innovation, about the report’s biggest takeaways, where AI is gaining traction and why many departments remain cautiously optimistic about its future.

FireRescue1: What was the impetus for creating the Strategic Scan reports for the fire service?

Henderson: The overall desire to conduct these Strategic Scans was based on a need to generate more and better data about current trends, emerging strategies, innovations and challenges affecting the broader fire and emergency service. We might see articles or hear podcasts about important topics, but we may not get that broader perspective. So, we started developing short surveys focused on gathering input from a broader base of respondents.

We also wanted to ensure a quick turnaround and distribution. The timeframe for academic publishing can sometimes take years, which doesn’t work when you’re talking about the fire service.

How did you select AI as the focus of the Strategic Scan?

Henderson: We assembled a really wonderful technical working group to help guide the direction of these scans. It includes chiefs from Prescott, Arizona; Charleston, South Carolina; Lexington, Kentucky; El Paso, Texas; Grand Rapids, Michigan; and Miami Beach, Florida. They have lent us their expertise on the issues that are critical to the fire service right now, as well as the more granular questions we should be asking based on the challenges they have faced in their departments.

We started with three or four topics that we thought would be interesting, then asked the working group to vote. AI rose right to the top.

Fisher: It’s easy to collect anecdotal information about what a few departments are thinking about AI. But when you survey hundreds of people, you begin to see the themes that rise to the top. How is the fire service using AI? What does the data say about departments’ concerns and challenges? Those answers help us develop the actionable takeaways included in each Strategic Scan.

What are your top takeaways from the Strategic Scan findings?

Henderson: The first — and I’m not entirely surprised by this — is that the vast majority of AI adoption was on the administrative side. People easily connected AI with the relatively routine and potentially mundane work involved in reporting and administration.

The number of respondents using AI in operations was larger than I expected. Maybe that’s my lack of imagination, but some people are using interesting tools in really time-bound, high-pressure and uncertain situations. That’s neat.

I was also interested in the number using it for training. I’m an educator, so I think a lot about the nature of pedagogy and how we help people engage deeply with material and learn in ways that make them better firefighters and fire officers. It was neat to see some adoption here among people thinking about development and asking, “How does AI make our instruction better?”

Fisher: It was interesting to see that 14% of departments have a formal policy on AI use, even though 40% said their authority having jurisdiction provides access to AI tools and a majority said they were already using AI. About half of respondents also said they planned to use it in administration or operations in the future. So how can we at the CPSE Center for Innovation be helpful to them?

We know there are valid concerns about the data being accurate and correct, and what the ethical implications of using some of the data are. If you don’t have formal policy on AI use, but you plan to use it, there’s obviously some gray area. The findings assured us that there are things we can do to help in this space.

What were the top hesitations chiefs reported about AI?

Henderson: Looking across operations, administration, training and pre-operations activities, some respondents may not have adopted AI yet, may have paused their adoption or may still be evaluating it, potentially because they haven’t had much exposure to the technology.

We asked about their comfort, interest and concerns in different areas. People tend to feel pretty comfortable using AI for administrative work, such as crafting policies or writing after-action reviews. There aren’t as many concerns there. But if we ask them to use AI during an actual incident, that’s likely to create more hesitation and uncertainty.

We saw a similar pattern in the questions about effectiveness in operations and training. Many respondents were interested and saw some effectiveness, but nobody was gangbusters about it. They weren’t saying, “This is fantastic. It’s doing everything we thought it would do.” Some responses were also on the negative side. Overall, we saw a hesitant response to a new technology, which is pretty normal.

Fisher: Data quality came up as well. If you’re going to make station assignment or deployment decisions based on this data, you have to make sure it’s right. You need the expertise — or someone on your team with the expertise — to fact-check it.

I’m sure we will eventually see a situation in which someone makes a decision using inaccurate AI-generated information. Who is held accountable when that happens? The work of the fire service is important, and lives often depend on it. If you’re going to make decisions using these tools, you want to get them right.

Privacy is another concern, especially when departments are working with EMS data. If you’re using AI to help write a patient care report, what information can you submit to the tool? Is that information protected? Is it private?

What did you learn about AI use in pre-incident operations?

Henderson: For pre-incident activities, 20 of the 156 respondents said they used AI for predictive analytics and fire risk assessment, which is really interesting. That use — along with optimizing fire station locations and assessing traffic — makes sense. Departments already have data about incidents and locations. It’s easier to use AI to analyze data that has already been collected than it is to collect and analyze real-time data during an incident.

What did you learn about AI use in operations?

Henderson: Interestingly, early wildfire detection was reported by 13 of the 156 respondents — I was surprised by that one. But fewer uses for 911 call triage, predictive modeling or pre-positioning of resources, and traffic assessment.

In terms of operations during an incident, again relatively small numbers for most of the activities: incident command and strategy five respondents; fireground accountability three; and then we get back to post-incident operations like after-action reporting at 24 responses. So it gets back into a very common administrative task — using AI to help with writing a report.

What did you learn about AI use in training?

Henderson: Sixteen of the 156 respondents reported using AI-enhanced fire simulations. That’s interesting and not entirely surprising. AI has been adopted for simulations in other fields, so it makes sense that it would also be incorporated into the fire service.

Some organizations are very progressive in how they approach training — not only recruit training but also training company officers and battalion chiefs to manage incidents. They might park a truck in a bay, place a screen in front of it and have participants work through a fire, including the operational and command components. It makes sense that they would use AI to enhance those simulations.

Just behind simulations were AI-powered adaptive learning platforms. In this type of training, participants respond to questions and receive different challenges or problems based on their answers.

AI performance analysis is also interesting. I haven’t experienced those tools myself, but I think it’s fascinating to consider how AI could analyze fireground command, operations or other aspects of fire service performance.

Can you explain what performance analysis could involve?

Henderson: We’re talking about incident outcomes. For example, someone works through a particular situation, and the system looks at how that person responded to different aspects of it. Then it can help identify how those responses might be improved.

I also listened to Assistant Chief Mike Binney’s Better Every Shift podcast episode, and I think biometric data presents another opportunity. AI can help us comb through large amounts of biometric data and better understand some of the challenges represented there. Of course, that also brings us back to questions about data privacy.

Fisher: Something I found personally surprising, coming from a fire service training position, was the mixed perception of AI’s effectiveness in training. More departments than I expected said it was either not effective at all or only slightly effective.

When respondents were asked about future training uses, 70% said they did not plan to use AI in training. Going into the survey, I thought training might be one of the easier applications, whether AI was used to enhance training, evaluate performance or assist with scoring. That result surprised me.

I’d like to hear more from the people who said it was ineffective. I’ve worked on simulation projects involving AI, and the inaccuracies can be obvious. Ask it to create a firefighter avatar, for example, and it may give the firefighter a European-style helmet. If you’re teaching about health and exposures, it might show someone in dirty turnout gear. I can understand why AI would not be effective in those situations, but I’d be curious to know whether respondents were disappointed by other aspects of its use in training.

For those who said they were not planning to use AI in training, how much of that could be a knee-jerk reaction based on the idea that AI would replace live-fire or hands-on training?

Henderson: One of the things that makes me most excited about the fire service is that once you show people the value of a tool, they’re all in. If you can prove that it works, they’re likely to become advocates rather than resist it.

People often say the fire service is bound by tradition or resistant to change. But if you demonstrate the value of something, people are willing to jump in with both feet.

What did you learn about AI use in administrative work?

Henderson: This is where the tools are most available. They’re free if you choose a free version, the risk is relatively low and you can use or discard the product at your discretion.

If it’s free, only takes a little of my time and doesn’t require me to rely on the outcome in the way I would during an operation, I might as well give it a shot. So the fact that more than two-thirds of respondents use AI to proofread and revise documents makes complete sense.

The numbers were much smaller for hiring and screening, budgeting and forecasting. Unless you’re a massive department, you may not need it for budgeting and forecasting. Hiring could present more opportunities. There are, of course, governmental and civil service limitations, but AI might help departments move candidates through the system more quickly, reducing the chance that top candidates go somewhere else.

Where was planned adoption strongest among the four categories?

Henderson: About 47% said they planned to adopt AI in operations. Open-ended responses included preplanning, risk assessment, community risk reduction, 911 call-taking and routing, response-data analysis, station deployment and evaluating response patterns.

Far fewer respondents were considering applications such as on-scene personnel accountability or other advanced operational functions. The vast majority were focused on areas that are operationally important but don’t involve using AI during active incident operations: risk assessment, preplanning, data analysis, station locations and personnel deployment.

Fisher: Administration was just behind operations at 43%.

Were you surprised that 59% of chiefs and other key personnel reported being comfortable with AI, while 18% said they were uncomfortable?

Henderson: I was surprised that almost 60% reported feeling comfortable with AI. Similarly, I expected the percentage who were uncomfortable to be much higher than 18%. It’s likely that those who responded with definitive feelings of comfort or discomfort were doing to with different reference points. Those who are more comfortable may feel that way given their use of a specific tool in a routine manner, which makes sense, and those who express discomfort may not have adopted many of these tools. This points to a need for continued monitoring of AI tool type, implementation, and effectiveness.

Is there anything else you’d like to add about the Strategic Scan?

Fisher: In addition to advocating for formal AI policy as I mentioned previously, the report provides four other actionable takeaways. These revolve around piloting operational AI use cases in low-risk high-impact areas, using it for administrative tasks to free up time for field operations, training on AI use, and planning for scalable AI infrastructure. With the speed in which this technology is evolving, it will be really interesting to see how use of and comfort with AI changes over time, and what innovative ways departments will use it.