Behavior Model Recovery in Agent-Based Environments

Complexity science is a new field of study that seeks to understand complex systems. The term complex systems refers to a system that is made up of individual components, which have non-linear interactions with each other, resulting in unpredictable, emergent properties. To better understand these systems, researchers use agent-based models that use a bottom-up approach to simulate complex environments. Observed emergent properties are recreated in agent-based models by providing agents with a set of rules and allowing them to interact with other agents and their environment.

While agent-based models have found academic recognition, they have made a negligible impact in affecting policy and operational decisions in government and industry. The author contends that ineffective validation methods result in low acceptance of agent-based model findings. Approximately 90% of agent-based simulations are either not validated or only validated using non-statistical techniques. The simulations that are validated using statistical methods only confirm that the simulation outcomes match historical data, which just proves that the model is one of potentially many that matches historical data. Therefore, there is little basis for decision-makers to have faith that the results of the simulation will accurately reflect the real-world system in previously unseen states.

The research in this dissertation relies on a validation maturity model created by Harmon and Youngblood, which codifies procedures for a convincing validation. The author asserts that it is insufficient to validate that the outcome of the model matches historical outcomes. Instead, practitioners must also empirically derive both agent classes and agent behavior models to certify that the model accurately reflects the real-world system. This dissertation examines the challenges that arise when empirically deriving agent classes and agents’ behavior models.

To demonstrate the broad applicability of agent-based models the author presents two models in different regulatory domains. The first model examines regulating High-Frequency Traders (HFT) in the commodity markets. There is an ongoing debate about the benefits of HFT traders, whose dominant presence in the markets may improve market liquidity but may also increase market volatility. The author finds that HFT’s can exasperate extreme volatility events, such as the Flash Crash of 2010. The simulation also suggests potential regulations, such as a minimum quote life, need to balance volatility mitigation and order trade time. The second model looks at the contentious topic of mass shootings and potential regulations. The simulation found that Dianne Feinstein’s proposed ban on assault weapons would have had limited effectiveness because it would not have regulated the rate-of-fire of any firearm.

Similar to many other agent-based practitioners, the author’s models have academic acceptance but have had little effect on decision-makers, which question the validity of the models. To improve agent-based model validation methods, the author explores the challenges in empirically deriving agent classes. Size-First Hierarchical clustering is introduced, which is a modification to traditional agglomerative hierarchical clustering. Experiments demonstrate that Size-First Hierarchical clustering outperforms several conventional unsupervised clustering methods on integer clusters with loosely defined nuclei.

There are times when it is insufficient to model individuals as a cluster of agents, and thus an explicit behavior model is required. The author presents his M.S. thesis work as a case study in behavior model recovery. The work is presented because it lays out the foundational issues in behavior model recovery. Using the University of Virginia McIntire Hedge Fund Tournament, the author shows that Classification and Regression Trees (CART) can recover trading strategies with a high degree of accuracy. An action feature representation is shown to increase the accuracy of classification trees. Additionally, stepwise linear regression was integrated with traditional regression tree models. However, it was found to be unstable outside of the training dataset.

To model realistic systems, behavior models will need to be recovered over a large state space, where there is only partial observation of real-world actors. Previous research in this area assumes either a large observation window or that the test and training state space are identical. The dissertation compares Neural Networks, Classification and Regression Trees, and Inverse Reinforcement Learning in terms of their ability to recover behavior models in partially observed Markov environments. CART was found to outperform the other two models, which contradicts several publish findings.

The author's research demonstrates that agent-based modes are effective at assessing potential regulations. To improve the acceptance of agent-based models' findings, researchers should validate both their agent classes and agents’ behavior model against their real-world counterparts. Recovering behavior models through observations of experts is a method to both build the agents’ behavior model and validate them simultaneously. Additionally, the dissertation findings suggest CART is a fast, robust method for recovering behavior models from observations.