catherine-roome-2017“I believe that good corporate governance requires those of us using AI in service to our clients to do an ethics review: not just because future regulatory environments will require it, but because it encompasses every aspect of the use of the tools – from deployment for employees, to impact on clients and stakeholders, to enterprise risk management. Generation R did an outstanding assessment for us which advanced our thinking around AI by years — an outstanding return on investment”

Catherine Roome, CEO, Technical Safety BC

One element of the service that I found most valuable was the effort made to restate the findings in the business language of our company. This makes the findings actionable and inspires employees and change leaders alike.

Abraham van Poortvliet, VP of Data Analytics and Decision Science, Technical Safety BC

The Client: Technical Safety BC

TechnicalSafetyBC

Technical Safety BC oversees the safety of residents in the province of British Columbia (BC, Canada). Its mandate is to ensure that a broad set of technical systems (e.g. boilers, electrical systems, elevators, etc.) found in BC’S homes, schools, and work places are properly installed, inspected, and maintained for safe operation. Technical Safety BC continuously collects data as part of its safety inspection activities, and has decided to improve the efficiency and effectiveness of their operations with data science and machine learning.

The Context: Machine Learning incorporated into a decision-support system

Technical Safety BC has an existing decision-support system that its frontline workers—safety officers—have been using to help prioritize their daily safety inspection tasks. To address shortcomings in the existing system, and to strategically assist in the allocation of Technical Safety BC’S human resources, a new decision-support system is being designed.

This new program predicts the likelihood of a technical hazard, and thus predicts where safety officers are likely to be needed most. The new machine learning system is designed to learn from Technical Safety BC’s constantly evolving data set to improve its predictions over time.

The Challenge

Vis 1

The senior management and development teams at Technical Safety BC recognized that their decision to use predictive algorithms in their operations raises new ethical risks inherent to machine learning systems. Technical Safety BC wanted to ensure these risks were addressed during the development stage of the project, so that they could plan for a successful system rollout, and ultimately increase public safety.

A key question emerged: How should Technical Safety BC identify and handle ethical risks associated with transparency, autonomy, discrimination, accountability and public reputation when using data-driven system?

Identifying and characterizing the ethical risks, and the strategies to manage them, requires a comprehensive analysis to be performed by a specialized team of experts. However, Technical Safety BC (like most organizations today) does not have an internal team of experts specializing in predictive algorithm ethics with which to accomplish such an analysis.

The Solution

Technical Safety BC hired Generation R to perform a comprehensive ethics assessment to identify, characterize, and analyze the various ethical challenges/risks uniquely posed by the new decision-support system. Technical Safety BC also asked for concrete, practical and actionable recommendations tailored to their organization, to help mitigate the challenges/risks in the development and rollout of the system.

Generation R’s Comprehensive Ethics Assessment Approach

Vis 2

To address Technical Safety BC’s needs, we applied our rigorous and exclusive Comprehensive Ethics Assessment (CEA) process that is at the core of Generation R’s expertise. Taking into account key stakeholder input, the organizational dynamics, and the technical elements of the decision-support system, we identified the primary ethical risks (i.e. barriers to success) that Technical Safety BC would likely encounter in the development and rollout of the new decision-support system.

With the ethical risks identified, we then delivered a prioritized list of recommendations in the form of practical, concrete and actionable design tasks, policy considerations and business options. This provided Technical Safety BC’s senior management and development teams with clear actionable tasks that would help to mitigate foreseeable ethical challenges/risks inherent in the decision-support system.

 

The Outcome

Technical Safety BC has enabled executives and employees with: 

  • A comprehensive understanding of the ethical risks directly relevant to their technology and business
  • A list of 31 critical business, policy and design decisions that Technical Safety BC executive and development team need to make and items to be considered to address potential ethical risks
  • Five key prioritized recommendations that Technical Safety BC teams can implement immediately

 

Generation R team’s technical expertise provided guidance into the complex topic of machine learning for inspection purposes. The Generation R team created momentum to push our organization’s data analytics projects forward by bringing different stakeholders together.

Soyean Kim, Leader, Research and Analytics, Technical Safety BC

The result was a clear overview of the choices that need to be made. During the process, blind spots became visible and complexities reduced to a half dozen dimensions on which decisions need to be made. Not necessarily easy, but clear and doable.

Abraham van Poortvliet, VP of Data Analytics and Decision Science, Technical Safety BC