The purpose of this project is to establish a research program to address the challenge of organizational adaptation: In an increasingly global and dynamic world, our economic systems, financial institutions and social organizations must adapt, but how? The scientific knowledge about adaptation in social organizations is inadequate because we lack a basic understanding of the micro-processes that determine adaptive costs as well as the costs of enduring mis-adaptation. This is critical because the empirical evidence indicates an enormous scale of waste associated with both reorganizing social organizations and disposing of existing organizations. Given a lack of scientific knowledge on efficient adaptation – and the fundamental importance of this problem for the long term viability of firms, financial institutions and other social organizations in an increasingly global and dynamic world – the aim of this project is to understand the determinants of adaptive efficiency in social organizations. Following a common use in the natural sciences, we label this property of repeated efficient adaptation as plasticity. We leverage recent advances in the mathematics of learning and use them as foundation for our new organizational theory of plasticity. We expect that this project will have a major impact on theories of organizational adaptation and on the theory and practice of organization design. Our effort to advance extant theories on organizational adaptation builds on prior work and joint competence of our team, composed of: 1. the Strategic Organization Design Unit (SOD), University of Southern Denmark, led by Thorbjørn Knudsen, and 2. the Laboratory for Experimental Economics of Ca' Foscari University Venezia, led by Massimo Warglien. In 2008, Thorbjørn Knudsen defined a research program on organization design that made SOD the first FSE Research Unit in Denmark. The current project is an ambitious attempt to advance this program with an international team led by Thorbjørn Knudsen.
We believe that the planned work has the potential to decisively shift our field towards a well-founded and rigorous analysis of organizational adaptation. Our proposed project, COPE, will forcefully establish an international center of excellence in the field of organization analysis and design. It is an ambitious and coherent research program at the cutting edge of organizational science. Realization of its objectives will have significant impacts on the field of management and organization science as we will develop, test, and validate a new comprehensive theory on multi-level organizational adaptation. This new theory will provide a thorough understanding of how organizations adapt their behavior and structure, when exposed to major shifts in their task environments. Our approach is novel in terms of its theoretical foundation, vision, and mission. Our theoretical foundation is original, because we seek to enrich the literature on organizational adaptation by drawing from fundamental advances in mathematical learning theory (Cucker & Smale, 2001), information theory (Sah & Stiglitz 1986), and learning in games (Ho et al. 2007) – streams of research that are closely related and ripe for integration. Our vision is novel and contributes to scientific innovation and originality because we focus on applying the new mathematical theory of learning in function spaces (Cucker & Smale, 2001). This approach is useful because it provides a structured way to analyze the relation between multi-level action and the resultant degrees of plasticity in social organizations. Finally, our mission is novel, because it forges an international superstructure between two research groups with notable, complementary track records. Overall, the project is important because we deepen theoretical understanding and derive normative implications for organizational and institutional design.
This project has the following major objective: To develop a research program that can ultimately explain when, how, and why an organization should adapt to changes in the environment.
We situate the ability to administer change at multiple levels of organization, and refer to this property in terms of degrees of organizational plasticity. The novelty of the project is not the notion of plasticity per se, but rather the identification of its organizational determinants. We wish to understand the logic of organizational plasticity, its determinants, and implications for design of organizational structure and mechanisms. Our focus is on the individual organization, and our contribution aims at theory building and development of new experimental paradigms (Keren & Lewis, 1993) to study multi-level adaptation in changing environments. We integrate two literature streams: the organizational literature on adaptive change and the mathematical literature on function-approximation in function spaces. We identify three major objectives: 1) development of comprehensive theory on multi-level organizational adaptation, 2) examination of multi-level adaptation in a laboratory environment, and 3) drawing implications for organization design.
This project will address the grand challenge of achieving efficient adaptation in social organizations in response to radical economic and social changes. The challenge is this: Can social organizations adapt efficiently to multiple changes in their environment? If yes, how? If no, why not? These questions point to challenges of enormous scale associated with adapting social organizations in a changing world. As this challenge is also beginning to surface on the radar of European Commission – e.g. in Horizon2020 relating to area 6. Europe In a Changing World – it is a timely effort with the potential to foster a groundbreaking and trendsetting research program.
In an increasingly global and dynamic world, changes become more frequent, and the premium of successful adaptation, relative to the waste associated with disposing of existing organizations, therefore becomes higher. It is therefore of fundamental importance to understand how social organizations can repeatedly transform their resources in a way that meets changing environmental demands. Institutional and organizational rigidity is a major problem and source of cost for society. As the ongoing global transformation has emphasized, the inability of organizations to continuously adapt to new circumstances and changing rules of the game has a huge and potentially negative impact, not only on individual organizations, but also for societal welfare. Individual organizations become increasingly inefficient and unresponsive as the context changes (Hannan & Freeman, 1989) and public institutions lose legitimacy (Henisz & Zelner, 2005), which may produce large negative externalities. Understanding the logic and the bases of continued successful adaptation to changing circumstances – organizational plasticity – is therefore a critical item on the agenda of social science in general, and organizational theory in particular (Siggelkow & Rivkin, 2006).
While organization theory has traditionally been remarkably static and non-formal in characterizing the fit between organizations and their environment (Burton & Obel 1998), recent work on organizational adaptation has relied on the formal modeling of organizational dynamics. However, apart from contributions such as Siggelkow & Rivkin (2006), Knudsen & Levinthal (2007) and Carley (1997), formal work on organizational adaptation has treated the organizations as unitary actors (Denrell & March, 2001; Levinthal 1997; Rivkin 2000), thus lacking rigorous micro-foundations. This simplification is problematic, because organizations exist to aggregate the individual decisions and actions of heterogeneous agents into collective outcomes - thus these models also abstract from the role of organization design in shaping the adaptive processes.
Moreover, in these models, organizations learn from experience changing response to a stimulus by adjusting a single behavioral parameter, rather than by adaptation of multiple nested parameters. This disregards Simon's (1962) point that organizations are complex entities with multiple nested levels of dynamics, acting on different time-scales. Concentrating on just one level captures only short-term adaptation and ignores outcomes arising in the longer term and in the larger scale of interactions among organizational components. As a result, most models of organizational adaptation and learning predict asymptotic rigidity. Learning curves result in competency traps (Levinthal & March 1981), joint action generates routines (Nelson & Winter 1982), and organizational innovations crystallize in organizational inertia (Hannan & Freeman 1989).
Models of organizational adaptation include an extremely broad family of formal models, from the behavioral tradition (Cyert & March 1963; Levinthal & March 1981), evolutionary theory (Nelson & Winter 1982; Levinthal, 1997), organizational economics (Dessein & Santos 2006; Garicano, 2000), and machine learning (Marsland 2009). But a notable feature of these models is that they are either limited to a stable (static or stationary) task environment or only consider a representation of the agent, which comprises a unitary actor operating at a single level of analysis.
Figure 1 was developed on the basis of a comprehensive search of Google Scholar for articles published in international, peer-reviewed academic journals that include a formal model of organizational adaption. By visual inspection, it is immediately clear where prior research on organizational adaptation has expended most effort and where the key research gaps are. What we have is a good understanding of how unitary organizational agents adapt towards a fixed goal. What we need is a theory that explains how multi-level organizations adapt to changing environments.
Note: Map derived from Multidimensional Scaling of articles independently classified by at least two researchers.
The goal of an adaptive process is to find a suitable behavioral response to changing environmental conditions. It is a process of experiential learning, at the organizational level, where sampling has a primary role. Recent advances in the mathematical foundations of learning provide a powerful set of tools that can be used to analyze this problem (Cucker & Smale 2002). To be concrete, the crucial elements for development of our theory can be summarized in a few points:
- Adaptation is situated in function spaces. Adaptation is situated in a particular context, typically a Banach space that admits search in (a nested hierarchy of) spaces of functions.
- Tuning behavioral response is the simplest form of adaptation. Behavior can be thought of as a function that maps states of the world into a response. At the lowest level in the hierarchy, adaptation of behavior is the choice of a new response, obtained by modifying a parameter of the current response function. This approach to adaptation can be seen as tuning the parameters of a given response function. Notably, this is the approach that represents the-state-of-the art to modeling adaptation in organization science.
- Adapting the response function, rather tuning its parameters, is a more powerful form of adaptation. If we ascend one step in the organizational hierarchy, adaptation is a trajectory in a space of response functions, searching for a new function reducing the distance to the optimal (unknown) response function. This perspective, which is new to organizational science, allows multi-level tuning of nested functions - e.g., a higher level function may determine the class of functions from which response functions are picked (e.g. polynomial versus multinomial).
- The scope of adaptation can be tuned. The way an organization determines the subspace of functions within which its learning process is focused (what Smale and Cucker refer to as hypothesis space) generates fundamental trade-offs. Increasing the efficiency of the search for new response functions requires contraction of the hypothesis space – the narrower such space, the faster the adaptive process. But narrowing down the hypothesis space also increases the risk that superior response functions are located outside that space. Thus, narrow-mindedness (smaller hypothesis spaces) may promote efficient adaptation in the short run, but inhibits further learning when there are fundamental changes in the environment. Many stories of business success followed by sudden decline reflect this fundamental dilemma.
- The combination of interactive learning among multiple agents and multiple levels of adaptation provides a natural basis for transforming the field of organizational adaptation. While the study of multi-level adaptation has been prominent in complexity science (Miller & Page 2007), in machine learning (Marsland 2009), and in evolutionary biology (West-Eberhard 2003), the application of this perspective is fairly recent in organizational science (Siggelkow & Rivkin 2005, 2006, 2009). Two developments along these lines, both of which are instances of our general framework of interactive learning at multiple levels, are especially relevant for our project.
1) Information-theoretic perspective on organizational decision-making.
We build on the line of work that represents decision-making organizations as information networks (Sah & Stiglitz 1985, 1986, 1988; Marschak & Radner 1972). Prior work on decentralization of information provided robust results regarding the way organizational structures can be designed to optimize efficiency under capacity constraints (Radner 1993; Bolton & Dewatripont 1994; Van Zandt 1999). By contrast, Sah & Stiglitz and followers (e.g. Koh 1992) mainly focused on human evaluation. Our team advanced this line of work by providing a general treatment of organizations in which the network structure is given and in which decision-makers face a constant flow of information (Christensen & Knudsen 2008, 2009, 2010; Knudsen & Levinthal 2007). Taking the number of agents as a given input to the analysis, the basic result is a design that minimizes the potential for screening error by rearranging the connections amongst individual agents (see Christensen & Knudsen 2010). In this line of work, the individuals' screening function is fixed, once and for all. The advantage of this approach is that researchers can make strong predictions based on comparative statics. It is fairly easy to derive configurations that are optimal in the sense of minimizing error-rates, subject to constraints on the number of decision makers who are involved in a decision (Christensen & Knudsen 2010). This implicitly defines error-minimizing target functions in the hypothesis space bounded by the available individual screening functions and possible configurations of network structures. Marginal contributions from increasing the number of decision-makers can also be derived, and given data on project distributions and labor costs, policy recommendations can be made. The main weakness of this approach is that it assumes that the ability of decision-makers is fixed. This limits application to situations where experiential learning is absent – a serious limitation when considering organizational adaptation on realistic time-scales.
Building on a pilot study from a commercial bank, we therefore extended the classical approach to encompass experiential learning. Learning by experience occurs in two distinct ways: the network structure, that connects agents, may adapt, and agents may change their screening functions in response to feedback. This perspective introduces a systematic approach to study how organizational adaptation influences individual learning processes, and how organization structure can be designed to minimize loss from the transition phase. As such, this line of work provides a concrete starting point for development of the COPE program. It is also a good concrete example of the more abstract mathematical framework we draw on.
2) Self-tuning models of learning in games. Decisive steps in the direction of formalizing multi-level adaptation can be found in recent models of "self-tuning" learning in games (Ho et al. 2007; Marchiori & Warglien 2008, 2011). A notable feature of these models is their grounding in large empirical data sets. Models of self-tuning learning represent processes of "learning to learn" that capture the evolution in the way individuals learn in interactive contexts ("learning style"). Ho et al.'s (2007) self-tuning EWA model is based on established models of learning in games, belief learning (Fudenberg and Levine 1998) and reinforcement learning (Erev and Roth 1998). It is a model of multi-level learning in which level 0 is the level of strategies chosen by players, level I is represented by the parameters governing response to actual and foregone payoffs and the weight of past experience. Finally, level II parameters tune level I parameters by detecting surprising events – indicating shifts in a non-stationary environment – and resetting the function of the learning dynamics. Our team has developed a complementary model, based on learning in neural networks (Marchiori & Warglien 2008, 2011). Neural networks provide a canonical example of learning in function spaces. A learning neural net modifies experientially the function that it computes - in the case of games this happens in the product space of each net's function space. Our model features self-tuning properties that enable the modeling of endogenous changes in the speed of learning and in the shape of the "activation function" that characterizes how an agent responds to inputs. This model obtains a very good fit with experimental data (Marchiori & Warglien 2008, 2011). It thereby provides a second concrete starting point for development of more abstract theory.
In this project, we will significantly extend the two mentioned multi-level approaches. The theoretical investigation process will be conducted in close interaction with the development of new experiments to explore multi-level adaptation in the lab. Such experiments have to be seen as an integral part of the theory development effort, as described below. We will also use the underlying structural similarities between the two models to extract more general results and highlight their common structure as instances of our general framework - e.g. the emergence of similar tradeoffs. In fact, we expect that exploring such two models will provide a striking demonstration of the unifying power of the more abstract mathematical framework that we draw on.
We have already mentioned theories of organizational adaptation that we build on. We should also mention a large literature of a more qualitative kind (e.g. Fiol 1994; Hedberg et al. 1976) that has explored multi-level learning in organizations. A notable and widely known idea is the notion of double-loop learning developed by Argyris & Schön (1974). This family of ideas provides an important stimulus for the formal approach we wish to develop.
Selection-based theories of evolutionary change in organizational populations are natural complements to our theory (Hodgson & Knudsen 2010; Nelson & Winter 1982). We emphasize the possibility of achieving adaptive change at the level of the individual organization. In contrast, a selection-based approach is based on replacement of inefficient organizations. If the population is large enough, the outcome may well be the same, but the costs are not. Whereas the cost of selection-based evolution implies loss of resources, successful adaptation transforms, and preserves resources. Furthermore, selection and adaptation are not strict substitutes because learning alters the shape of the search space in which evolution operates (Hinton & Nowlan, 1987). Thus, organizational plasticity may emerge as a product of the evolutionary process. Further investigation of interactions between plasticity and selection is therefore a promising item on the future research agenda. However, this is beyond the scope of the current project.
Our ultimate goal is to understand how structural properties determine the organization's ability to administer the right kind of change at the right point in time. The framework we draw on provides a clear definition of organizational plasticity as well as a rigorous approach to model multi-level adaptation. This not only provides guidelines for the development of formal theory (4.1), but also for the design of a concise experimental agenda (4.2).
1. Information theoretic perspective on organizational decision making. Starting with organizations in which members in a given network structure face a stationary flow of information (Christensen & Knudsen, 2008, 2010), we add two features capturing multi-level adaptation. First, we add experiential learning at the level of agents. That is, the agent's screening adapts as a function of the information flow it is exposed to – this is formally captured by a differential equation. Second, we add changes to the network topology, capturing the organization structure. The organization is characterized as a statistical filter directing flows among agents. Shocks are changes in the project distribution agents are exposed to. The benchmark is an organization with agents whose screening functions are fixed once and for all, but otherwise operate under constraints that are identical to plastic organizations. The goal is to assess conditions and degrees of plasticity that lead to superior performance relative to the benchmark of a non-adaptive organization. Both closed form results and numerical results are extracted.
2. Self-tuning models of organizational learning. Models of self-tuning learning usually consist of systems of (non linear) difference equations whose behavior is studied through numerical methods (Ho et al., 2007). If they have free parameters, calibration methods are used to fit to the empirical data. In the current project, we build on models of learning in neural networks to study organizaional adaptation at multple time scales. These models better capture spontaneous generalization and multiple time scales of adaptation than conventional models of learning in games. We use this kind of network models to build (calibrated) artificial players in games, and the trajectories of play they generate are analyzed by simulation, as in Erev and Roth (1998), Ho et al. (2007), and Marchiori & Warglien (2008) and used to predict experimental results.
For theory development, we use analytical approaches in combination with numerical methods. For the empirical tests, we draw on laboratory experiments. Empirical validation of models is sought by testing their predictive ability against experimentally-generated data. In particular, we will conduct a comparison of the predictive capability of our multi-level models of plasticity against models with just a single level of adaptation. Experiments will be conducted according to accepted experimental economics methods (Kagel & Roth, 1997). We will run experiments at the labs of SDU in Odense and at Ca' Foscari University Venezia. Subjects will be motivated using monetary incentives related to performance in experimental tasks. All experimental data will be made accessible at request, in the respect of the subjects' anonymity.
We plan to conduct four groups of experiments. The number of subjects is set at a level where it satisfies common statistical standards. Experiments will imply multiple agents and hierarchically nested roles (often with different time scales of action) both in the context of screening flows of projects (4.1.1) and playing organizational games (4.1.2).
The use of experiments in supporting theory development is especially fruitful since it allows an iterated dialogue between hypothesis development in theory and hypothesis testing in the lab. The important advantage of the combined use of human subjects in laboratory experiments and agent-based modeling is that both approaches exploit controlled conditions as a means of isolating and decomposing the sources of aggregate phenomena (Duffy, 2006). As our goal is to develop micro-foundations of organizational adaptation, experimental methods appear as especially fit, and also provide a bridge to organizational design, creating small "wind tunnels" for design at larger scale. Our team is currently pioneering laboratory and computational methods for this purpose.
For the project “COPE: Center for Organizational Plasticity and Evolution”
Becker, M. C. 2004. Organizational routines: a review of the literature. Industrial and Corporate Change 13(4) s.643–678.
Becker, M. C., T. Knudsen, J. G. March. 2006. Schumpeter, Winter, and the Sources of Novelty. Industrial and Corporate Change 15(2) s.353–371.
Bolton, P., M. Dewatripont. 1994. The Firm as a Communication Network. The Quarterly Journal of Economics 109(4) s.809–839.
Burton, R. M., B. Obel. 1998. Strategic Organizational Diagnosis and Design: Developing Theory for Application. Dordrecht: Kluwer Academic Publishers.
Christensen, M., T. Knudsen. 2008. Entry and Exit Decisions in Flexible Teams. Journal of International Business Studies 39(8) s.1278–1292.
Christensen, M., T. Knudsen. 2009. The Architecture of Knowledge Organization. I Knowledge Governance. Processes and Perspectives. Oxford: Oxford University Press, 47–80.
Christensen, M., T. Knudsen. 2010. Design of Decision Making Organizations. Management Science 56(1) s.71–89.
Cucker, F., S. Smale. 2002. On the mathematical foundations of learning. Bulletin Amerincan Mathemathical Society 39 s.1–49.
Duffy, J. 2006. Agent-Based Models and Human Subject Experiments. I Handbook of Computational Economics (Eds. Tesfatsion, L. & Intriligator, M.D.). Stanford, CA.: Hoover Institution, 949–1011.
Erev, I., A. E. Roth. 1998. Predicting How People Play Games: Reinforcement Learning in Experimental Games with Unique, Mixed Strategy Equilibria. The American Economic Review 88(4) s.848–881.
Fang, C., J. Lee, M. Schilling. Balancing Exploration and Exploitation through Structural Design: the Isolation of Subgroups and Organizational Learning. Organization Science 21(3) s.625–642.
Fiol, C. M. 1994. Consensus, Diversity, and Learning in Organizations. Organization Science 5(3) s.403–420.
Gardenfors, P., M. Warglien. 2012. Using conceptual spaces to model actions and events. Journal of Semantics (forthcoming).
Gupta, A. K., K. G. Smith, C. E. Shalley. 2006. The Interplay between Exploration and Exploitation. The Academy of Management Journal 49(4) s.693–706.
Hannan, M. T., J. Freeman. 1989. Organizational Ecology. Cambridge, MA.: Harvard University Press.
Hedberg, B. L. T., P. C. Bystrom, W. H. Starbuck. 1976. Camping on Seesaws: Prescriptions for a Self-Designing Organization. Administrative Science Quarterly 21(1) s.41–65.
Henisz, W. J., B. A. Zelner. 2005. Legitimacy, Interest Group Pressures, and Change in Emergent Institutions: The Case of Foreign Investors and Host Country Governments. The Academy of Management Review 30(2) s.361–382.
Ho, T.-H., C. F. Camerer, J.-K. Chong. 2007. Self-tuning Experience-Weighted Attraction Learning in Games. Journal of Economic Theory 133(1) s.177–198.
Hodgson, G. M., T. Knudsen. 2010. Darwin’s Conjecture. The Search for General Principles of Social and Economic Evolution. Chicago: University of Chicago Press.
Jacobides, M. G., T. Knudsen, M. Augier. 2006. Benefitting from Innovation: Value Creation, Value Appropriation, and the Role of Industry Architecture. Research Policy 35 s.1200–1221.
Kagel, J. H., A. E. Roth. 1997. Handbook of Experimental Economics. Princeton NJ.: Princeton University Press.
Keren, G., C. Lewis. 1993. A Handbook for Data Analysis in the Behavioral Sciences. Hillsdale, NJ.: Lawrence Erlbaum Associates, Inc., Publishers.
Knudsen, T., D. A. Levinthal. 2007. Two Faces of Search: Alternative Generation and Alternative Evaluation. Organization Science 18(1) s.39–54.
Knudsen, T., R. Swedberg. 2009. Capitalist Entrepreneurship: Making Profit Through the Unmaking of Economic Orders. Capitalism and Society 4(2) s.1–26.
Koh, W. T. H. 1992. Human Fallibility and Sequential Decision Making: Hierarchy versus Polyarchy. Journal of Economic Behavior & Organization 18(3) s.317–345.
Levinthal, D. A., J. G. March. 1981. A Model of Adaptive Organizational Search. Journal of Economic Behavior & Organization 2 s.307–333.
Lounamaa, P. H., J. G. March. 1987. Adaptive Coordination of a Learning Team. Management Science 33(1) s.107–123.
Marchiori, D., M. Warglien. 2008. Predicting Human Interactive Learning by Regret-Driven Neural Networks. Science 319(5866) s.1111–1113.
Marchiori, D., M. Warglien. 2011. Neural Network Models of Learning and Categorization in Multigame Experiments. Frontiers in Neuroscience 5 s.139.
Marschak, J., R. Radner. 1972. Economic Theory of Teams. New Haven and London: Yale University Press.
Nelson, R. R., S. G. Winter. 1982. An Evolutionary Theory of Economic Change. Cambridge, MA.: Harvard University Press.
Puranam, P., M. Goetting, T. Knudsen. 2012. Organization Design: The Epistemic Interdependence Perspective. Academy of Management Review, 37(3), s. 419-440.
Radner, R. 1993. The Organization of Decentralized Information Processing. Econometrica 61(5) s.1109–1146.
Sah, R. K., J. E. Stiglitz. 1985. Human Fallibility and Economic Organization. The American Economic Review 75(2) s.292–297.
Sah, R. K., J. E. Stiglitz. 1986. The Architecture of Economic Systems: Hierarchies and Polyarchies. The American Economic Review 76(4) s.716–727.