March 11, 2020 Topcoder and the CDC Work Together to Help Improve Workplace Safety

Recently, the National Institute for Occupational Safety and Health (NIOSH) — a research-oriented workplace safety administration that’s part of the Centers for Disease Control and Prevention (CDC) — and the National Aeronautics and Space Administration (NASA) approached Topcoder to help them develop a Natural Language Processing (NLP) algorithm to help automatically classify workplace-related injury data for further analysis.

Not only was Topcoder able to deliver a massive 961 submissions from 388 registrants across countries, but the winning NLP solution from Topcoder’s challenge had 90% accuracy, surpassing in-house submissions from previous competitions that NIOSH and OSHA ran internally.

The CDC recently blogged about the results, noting that this was their first-ever external crowdsourcing competition.

“The high-quality submissions achieved nearly 90% accuracy, which surpassed the 87% accuracy goal achieved during the internal competition.”

– NIOSH Science Blog

Understanding the Injury Surveillance Problem

Both The Occupational Safety and Health Administration (OSHA) and the National Institute for Occupational Safety and Health (NIOSH) are responsible for setting workplace safety standards and tracking employee safety across businesses and worksites in the United States. One of OSHA’s and NIOSH’s core responsibilities is to understand trends by tracking injuries both broadly and granularly. For OSHA, one of the main sources of this critical injury data comes in the form of “injury narratives.” these are text-heavy, open-ended records that allow employers to specify how an employee was injured (e.g., “tripped over chain”).

To transform these open-ended questions into valuable data for analysis, OSHA first has to code each narrative. Unfortunately, coding all of these “injury narratives” isn’t easy. It requires significant people hours, manual workflows, and hyper-redundant processes. To help discover how NIOSH and OSHA could maximize their existing data analysis and attribution (both OSHA and NIOSH work together to mitigate and analyze workplace injuries under the Occupational Safety and Health Act of 1970), they contracted the National Academies of Science (NAS) to conduct a study over their cost-effectiveness and surveillance systems.

The NAS report urged OSHA and NIOSH to adopt machine learning and AI-enabled solutions to help them automate some of their surveillance operations — giving them more data, fewer manual errors, and faster data collection.

NIOSH & NASA Work With Topcoder to Develop a Solution

To be clear, NIOSH didn’t start with Topcoder. First, seventeen researchers from NIOSH, OSHA, BLS, NIH, CPSC, FEMA, and the Census Bureau teamed up to host a competition for the CDC. The skilled team members at the CDC were able to create a National Language Processing (NLP) algorithm with 87% accuracy — a 6% improvement on the baseline algorithm.

But they didn’t stop there. In October 2019, NIOSH and NASA teamed up to run a challenge on the Topcoder platform. This challenge leveraged Topcoder’s immense talent pool to develop NLP algorithms that could classify those occupational injury lists that were (thus far) entirely manual.

Mike Morris — CEO at Topcoder — described how the CDC wanted to “create a way to quickly, accurately, and cost-effectively code occupational safety and health surveillance data.” He went on to say, “The resulting algorithms read the free-text injury reports and information recorded in surveillance systems, then automatically apply the appropriate classification codes.”

And it was a success!

Topcoder was able to gather 961 submissions from over 380 members of the Topcoder talent pool. In the end, the CDC received a few different algorithms that surpassed the 87% accuracy models they developed in-house — with the winning algorithm (from Raymond van Venetië in Amsterdam), achieving near-90% accuracy.

“Everyone involved benefits from faster processing times, reduced operational overhead and decreased coding errors. Process improvements will be instrumental in efforts to enhance overall worker safety and health.”

Mike Morris, CEO, Topcoder

To learn more, visit here and type “Artificial Intelligence” into the search bar at the top of the page. Or if you’re interested in running your own challenge, contact us.