Classification of generic system dynamics model outputs via supervised time series pattern discovery

Abstract

System dynamics (SD) is a simulation-based approach for analyzing feedback-rich systems. An ideal SD modeling cycle requires evaluating the qualitative pattern characteristics of a large set of time series model output for testing, validation, scenario analysis, and policy analysis purposes. This traditionally requires expert judgement, which limits the extent of experimentation due to time constraints. Although time series recognition approaches can help to automate such an evaluation, utilization of them has been limited to a hidden Markov model classifier, namely the Indirect Structure Testing Software (ISTS) algorithm. Despite being used within several automated model-analysis tools, ISTS has several shortcomings. In that respect, we propose an interpretable time series classification algorithm for the SD field, which also addresses the shortcomings of ISTS. Our approach, which can highlight the regions of a certain time series that are influential in the class assignment, is an extension of the symbolic multivariate time series approach with the use of a local importance measure. We compare the performance of the proposed approach against both ISTS and nearest-neighbor (NN) classifiers. Our experiments on a SD-specific application show that the proposed approach outperforms ISTS as well as conventional NN classifiers on both noisy and nonnoisy datasets. Additionally, its class assignments are interpretable as opposed to the other approaches considered in the experiments.