The Role of Artificial Intelligence (AI) in Developing and Deploying a Decision Effectiveness Simulation Exercise
Presented at the 2023 Virtual ACSG Conference
on 16 March 2023 from 15:00 - 15:45
In a world beset by Artificial Intelligence (AI)-driven applications, Assessment Centre (AC) practitioners find themselves in a garden of forking paths. Different futures await, depending on the path(s) we take. One thing can safely be assumed, AI - in some form or another - will play a role in the future of AC science and practice.
Just as AC practitioners face diverse and daunting decisions, AC participants must similarly make decisions that have far-reaching consequences, often in the face of ambiguity and with incomplete information at their disposal. Decision effectiveness is the bedrock in transforming an organisation into a high-performance organisation, a manager into a high-impact corporate leader, and a businessperson into a high-impact entrepreneurial leader. It is a complex concept where multiple cognitive processes and different elements of reasoning must converge to produce telling decisions. Without it, individuals and business leaders will struggle to successfully steer their careers and organisations towards prosperity.
The discussion will commence with theoretical perspectives on the antecedents and determinants of decision effectiveness and then zero in on the measurement thereof in the AC context. In this regard, we will reference pioneering theories in the field of cognitive psychology proposed by Elliot Jacques, Keith Stanovich, Richard West, and Daniel Kahneman. We will then discuss the design and evolution of a contextually grounded simulation exercise for measuring the cognitive processes that combine to determine decision effectiveness. Previously a core AC exercise administered in a face-to-face context, the simulation exercise in question was automated using AI. It presents participants with a business dilemma, broadly similar to an analysis exercise. Participants must then engage a virtual resource entity in a question-and-answer conversation to uncover the root causes causing different problems at varying levels of work. Armed with contextual knowledge accumulated in this way, they must make recommendations to solve the problem(s).
At this juncture, the discussion will fork towards the role of AI in the evolution of AC technologies. We will discuss key considerations regarding AI within the AC context. Key amongst these include how AI impacts the assessment value chain. At a general level, we will discuss how AI affects the role of the assessor and the role of the participant in the AC context. At a more specific level, we will discuss how AI has affected the evolution of the decision effectiveness simulation exercise. No discussion of this nature would be complete without reference to utility. We will share research results regarding internal structure and reliability, convergent and discriminant validity, and criterion validity.
The discussion will conclude with a critical view of where we ended up in a garden of forking paths - where we are now in the lifecycle of the decision effectiveness simulation exercise, the insights we gained and the lessons we learned along the way, and the next steps of this journey.
We hope to leave delegates with a deeper understanding of the anatomy of decision effectiveness and a tentative appreciation of novel ways to measure this critical capability. Perhaps this may be useful for AC practitioners (decision-makers) faced with multiple forking paths and a daunting but exciting future.
Pieter Bronkhorst is a registered I/O Psychologist, holds a PhD (University of Cape Town) and is the founder of Organisation and Management Technologies (OMT), Evalex Talent Solutions (EICM) Odyssey Talent Management (OTM) and Evalex Leadership Evolution and Acceleration (Elea).
At an academic level, he functions as an Adjunct Professor at the University of the Western Cape and is a Lifetime Honorary member of the Society of Industrial and Organisational Psychology of South Africa (SIOPSA). He has presented numerous research studies and lectures at local and international conferences.
Pieter has developed psychological tests in the domains of cognition, personality, styles, values and interests. In addition, he has developed a cloud-based virtual assessment centre as well as a system for assessing technical competencies.
Pieter has to date developed five Artificial Intelligence driven talent assessment applications / solutions elevating the virtual cloud-based assessment centre and talent management to the leading edge of technology.
He acts as a consultant to clients on building high-performance teams and companies. He has been involved in more than 30 corporate growth and turnaround projects worldwide and has also published a book titled "The Architecture of High Performance Companies." He is a regular presenter of research papers at conferences.
Jaco de Jager holds a Master’s degree in Industrial Psychology and is a registered psychologist with the HPCSA. He is a previous ACSG committee member and served on the committees of the Society for Industrial & Organisational Psychology (SIOPSA)’s Pretoria Branch and People Assessments in Industry (PAI) Interest Group.
The greater part of his professional career centred on talent assessment within the frameworks of role fit, succession, personal growth, and team/group effectiveness. Through the course of this 30+ year career, he accumulated considerable experience in back-office (strategic product and process development) and client-facing (needs analyses, collaborative solutioning, and professional practice) roles. This included developing, deploying, and managing assessment centre technologies in general leadership/business and technical/functional contexts.
Jaco’s current role at Evalex Talent Solutions focuses on product development and enhancement, particularly in the AI domain. He is enthralled by exploring new horizons in product development, particularly at the interface of behavioural science, artificial intelligence, and business acceleration.