Spotlight: North Dakota Rural EMS Counts + ESO

Executive summary

State leadership in North Dakota recognized that rural EMS agencies face different challenges and settings than their urban peers. Nationwide performance metrics and standard operating procedures helped, but state EMS leaders saw that understanding incidents and data in a rural context could give agencies more actionable benchmarks, and surface both strengths and areas to improve.

Working with ESO, the Center for Rural Health at the University of North Dakota (CRH) applied for and won grant funding to develop and implement performance measures for rural EMS. The CRH, the state EMS office, the North Dakota EMS Association, and ESO together built a suite of rural EMS performance measures and an analytics toolbox for rural agencies nationwide. That award-winning Rural EMS Counts project is now used by agencies across the United States, and the first edition of the ND Rural EMS Counts Index established a statewide baseline.

Quick stats from ND Rural EMS Counts

The challenge: one size does not fit all

EMS providers in North Dakota have long known the particular circumstances of serving communities in a state where almost 72% of counties are designated frontier, meaning a population density below seven people per square mile.

“Longer response and transport times due to the vast number of miles traveled, as well as large networks of providers and facilities, are common in certain areas of the state.”

Lindsey Narloch, Project Manager, North Dakota EMS Association

Those conditions make it hard to apply performance benchmarks designed for urban settings with large call volumes, because those indicators do not account for the outside factors rural providers face.

State EMS leaders concluded that the complexity and resource constraints of rural prehospital care called for measures built for that environment. At the end of 2019, the CRH was awarded one of only four Flex EMS Supplement grants nationwide to run a demonstration project on data collection and reporting for rural-relevant EMS quality measures. The team believed those metrics could deliver more consistent care, drive protocol changes, and improve provider safety and job satisfaction.

The discovery: metrics that matter

The CRH and the North Dakota EMS Association, along with the North Dakota Department of Health Division of Emergency Medical Systems, have a long-standing relationship with ESO through the North Dakota Trauma Registry, the North Dakota State Repository, and an ESO Health Data Exchange (HDE) program covering 40 EMS agencies and eight hospitals. Every EMS agency submitting data to the state has free access to ESO Electronic Health Record (EHR) for clinical documentation and reporting.

In applying for the grant, the CRH asked ESO to partner on the project, with the nationwide incident database, commitment to data quality, expertise in improvement science, and annual EMS Index reporting in mind. ESO provided methodological guidance for the research study, and the ESO Research and Customer Success teams worked alongside the CRH, the Department of Health, and the EMS Association throughout.

“We were excited to team up with the team in North Dakota to take a deeper dive into data to better understand the challenges and strengths of rural EMS partners,” said Remle Crowe, PhD, NREMT, Director of Clinical and Operational Research at ESO.

“Measure standardization is so important for the EMS industry in telling the story, especially to be able to pool resources among low-volume agencies. We spent a lot of time creating and refining operational definitions for these key measures. It wasn’t good enough to compare apples to apples, it had to be Granny Smith apples to Granny Smith apples.”

Remle Crowe, PhD, NREMT, Director of Clinical and Operational Research, ESO

The program involved a rigorous research process: a systematic review of existing research, consensus meetings with experts in quality improvement and rural EMS, and prioritization of five focus areas.

Lindsey Narloch, Project Manager at the North Dakota EMS Association, explained that a top priority was making sure the reported data mattered to providers and helped all stakeholders without adding to their workload.

“At the end of the day, what we were collecting and why we were collecting really had to matter. We knew it had to be important to staff and field providers, along with the patients they care for. We needed to use the information we were already collecting, and it couldn’t be a further burden on agencies. And the measures had to be something to which everyone could come to consensus.”

Five areas for initial focus

Participating agencies spent about one to two hours each month running the prebuilt ND Rural EMS Counts Measures function within ESO Analytics, collecting their results, and submitting them to the CRH. Those measures show, for example, how often blood glucose was checked for stroke patients, or whether pulse, respiratory rate, blood pressure, and pulse oximetry were all documented. Measures from all five areas were built into reports in ESO Analytics so agencies could assess and compare performance.

Some measures applied to nearly every EMS encounter, such as lights and siren use. Others were low frequency and high criticality, such as stroke care. For those, standardized definitions created a foundation for data collaboratives and learning partnerships between agencies, so they could pool resources for quality improvement in low-volume settings.

“We are very familiar with a system approach for care, since many of our patients utilize a network of hospitals and providers. This is just a systems approach for Quality Improvement,” explained Narloch. “We’ve been interested in collective learning for quite a while, and this puts that concept into practice. Each agency doesn’t have to have each specific case, but we can all use the learning around it and share it with others to improve overall care across the state.”

Results: compiling and sharing

From the reported measures, ESO created the 2022 EMS Index: North Dakota Rural EMS Counts, free to download alongside other resources on the ND Rural EMS Counts website. The Index drew on data from more than 120 agencies and departments across the state, representing nearly 110,000 EMS responses between January 1 and December 31, 2021. It also compared North Dakota with national performance from the ESO Data Collaborative and included key takeaways and recommendations for local quality improvement.

Agencies can use the document to see which areas of their performance align with statewide measures and which represent opportunities for improvement, closer local monitoring, or further evaluation. By offering an objective look at aggregate data, the Index gives agencies a benchmark against peers across the state and the nation, and a framework to refine tactics, improve efficiency and outcomes, and allocate resources.

“One of the main benefits of the project was freeing up the data to agencies across the state, and promoting collective learning,” said Narloch. “Most agencies don’t have a quality improvement or data person, so we knew these results had to be easy to access and use. Even if you have limited time within your week, this benchmarking document and analytics reports help you make observations and prioritize scarce resources, since you can click directly from the report into the charts.”

Alongside the Index, the ND EMS Association hosted events to support shared learning, including virtual town hall meetings and monthly “Minutes with a Medical Director” calls discussing real deidentified cases submitted by agencies. The association also added toolkits, checklists, change worksheets for tracking quality improvement projects, and quarterly training sessions with continuing education hours to its website.

Darrell West, paramedic and manager of the West River Ambulance Service in Hettinger, North Dakota, found the project helped his team set benchmarks and improved care directly. “When I was first introduced to the Rural EMS Counts project, I was excited to be able to track how our staff was doing but dismayed when my own name appeared more than I would have liked for missed documentation and incomplete records,” West explained.

“The Rural EMS Counts has helped our agency improve documentation and patient care through better QA/QI. This is a great asset for EMS agencies in North Dakota.”

Darrell West, paramedic and manager, West River Ambulance Service, Hettinger, North Dakota

The support of the ESO Research and Customer Success team was key from start to finish, Narloch noted. “ESO shares our long-term vision of staying true to the data set, accurately benchmarking, and playing off a consistent data structure. It comes down to people and support, and having access to people that not only understand the technology, but are also experts in QI, medical direction, and even design.”

Next steps: future impact

Reception has been strongly positive and continues to build, with the project receiving the Best Research Award at the National Association of EMS State Officials meeting in June 2022. The association has also seen smaller but meaningful changes, such as greater attention to equipping basic life support services with end-tidal capnography for better patient monitoring.

The measure set includes two novel measures built on outcome data obtained directly from hospitals through Health Data Exchange: stroke assessment for emergency department-diagnosed stroke, and 12-lead ECG acquisition for patients with emergency department-diagnosed ST-segment myocardial infarction.

“These measures have already helped agencies identify areas for improvement that would have been missed without the hospital data.”

For example, analysis of stroke assessment performance showed that EMS clinicians were less likely to recognize stroke when the patient was noted to have consumed alcohol. Results from the Index have also helped agencies hold more informed discussions with their medical directors on pain management options and consistency of care.

Agencies in other states, particularly those with large rural populations, are using the document as well, and are encouraged to monitor their own rural-specific data points.

“We’ve come to realize that ‘rural-relevant EMS’ is pretty much the same as what you consider good EMS care anywhere,” explained Narloch. “There might be less frequency, but really, it’s still just good care and documentation. And in some things, rural care might even have a chance to excel since we are with those patients for a longer period of time. Using your data, you can see that there are little changes that can add up to big improvement for patient care across your state. We are looking for smart ways to make system changes, not just work harder. This Index is a new resource in the toolbox for EMS agencies everywhere to do that.”

The Rural EMS Counts project has received HRSA Flex EMS Supplement funding for two more years and will continue working with ESO toward its mission of empowering improvement for providers, patients, communities, and the future.

Explore ESO research and data