Active Learning for event detection in large-scale information networks
MoRE2020 Fellow Qing Zhao, incoming mobility from Cornell University, USA, to Chalmers University of Technology
The problem of detecting rare events of interest in massive data streams and large complex networks is ubiquitous. The rare events may represent opportunities with exceptional returns or anomalies associated with high costs or potential catastrophic consequences. This project addresses the problem of detecting rare events as quickly and as reliably as possible when the total number of hypotheses is large, the observations are noisy, and the prior knowledge on the rare events may be as little as "they are different from the nominal." We aim to establish fundamental theory, performance limits, and adaptive algorithms with scalable computation complexity and guaranteed performance. The scientific methodologies of this research lie in the intersection of active inference, machine learning, graph theory, and information theory. This project is a systematic study of a class of problems most relevant in the era of increasing network size and abundance of data. Specific applications include anomaly prevision in radio access networks of next-generation wireless communication systems, Internet traffic monitoring and engineering, anomaly detection in vehicle fleet data and multi-contingency analysis in energy systems.
Collaborating end-users: Ericsson AB, Volvo Car Corporation
Summary of Project Results
The objective of this research is to develop general design methodologies for rare event detection in large-scale information and infrastructure networks. We aim to develop active learning and inference strategies that offer optimal sample complexity in terms of the required detection accuracy and the network size, all under unknown models of rare events. Building upon our prior results, we have successfully addressed the issue of switching cost and the impact of nonlinearity of the cost functions on the design and the performance of optimal learning strategies. The issue of unknown models is also addressed under a parametric framework.
In terms of career plan, a monograph on "Multi-Armed Bandit: Theory and Application to Online Learning in Networks" under the support of this MoRE2020 project has been completed and will be published soon.
Collaborations with the end user Volvo Car Corporation was established through a seminar and discussion session held with researchers at Volvo Car Corporation, which provided valuable feedback on research issues most relevant to the end user to guide the research directions of this project. A short course on "Active Learning and Inference" was offered at Chalmers, which was attended by Chalmers Ph.D. students, faculty member, and researchers from local industries.