Learning with Small Data

Live Q & A

11:00am – 3:00am EST 8/23 | 8:00am – 12:00pm PST 8/23 |11:00pm 8/23 – 3:00am 8/24 Beijing

Join Zoom Meeting

Abstract

In the era of big data, data-driven methods have become increasingly popular in various applications, such as image recognition, traffic signal control, fake news detection. The superior performance of these data-driven approaches relies on large-scale labeled training data, which are probably inaccessible in real-world applications, i.e., “small (labeled) data” challenge. Examples include predicting emergent events in a city, detecting emerging fake news, and forecasting the progression of conditions for rare diseases. In most scenarios, people care about these small data cases most and thus improving the learning effectiveness of machine learning algorithms with small labeled data has been a popular research topic. In this tutorial, we will review the trending state-of-the-art machine learning techniques for learning with small (labeled) data. These techniques are organized from two aspects: (1) providing a comprehensive review of recent studies about knowledge generalization, transfer, and sharing, where transfer learning, multi-task learning, and meta-learning are discussed. Particularly, we will focus more on meta-learning, which improves the model generalization ability and has been proven to be an effective approach recently; (2) introducing the cutting-edge techniques which focus on incorporating domain knowledge into machine learning models. Different from model-based knowledge transfer techniques, in real-world applications, domain knowledge (e.g., physical laws) provides us with a new angle to deal with the small data challenge. Specifically, domain knowledge can be used to optimize learning strategies and/or guide the model design. In data mining field, we believe that learning with small data is a trending topic with important social impact, which will attract both researchers and practitioners from academia and industry.

Outline

- Introduction
- Part I: Auxiliary Task
  - Transfer Learning
    - Supervised transfer learning
    - Unsupervised domain adaptation
  - Multi-task learning
    - Hard parameter sharing
    - Soft parameter sharing
    - Adapted parameter sharing
  - Meta-learning
    - Meta-learning algorithms
      - Gradient-based meta-learning
      - Metric-based meta-learning
    - Meta-learning challenges
      - Task heterogeneity
      - Meta-overfitting
  - Applications
- Part II: Domain Knowledge
  - Loss function design
    - Residual modelling
    - Regularization
  - Model initialization
  - Model design
    - Encoding intermediate domain variables
    - Encoding other domain-specific prior knowledge
- Conclusion and Discussion
  - Conclusion
  - Q & A

Materials

Slides

PDF: https://tinyurl.com/kdd-smalldata-slides

Video

Part I: https://tinyurl.com/kdd-smalldata-video-part1

Part II: https://tinyurl.com/kdd-smalldata-video-part2

Part III: https://tinyurl.com/kdd-smalldata-video-part3

Part IV: https://tinyurl.com/kdd-smalldata-video-part4

YouTube

Bilibili

Presenters

Huaxiu Yao is currently a Ph.D. candidate of the College of Information Sciences and Technology at the Pennsylvania State University. He got his B.Eng. degree from the University of Electronic Science and Technology of China. His current research goal is to improve the generalization capability of machine learning algorithms through the combination of knowledge transfer and relation learning. He is also passionate about applying these methods for solving social problems (i.e., AI for social good). He has published over 10 papers on top conferences and journals such as ICML, ICLR, KDD, AAAI, WWW and TIST. He has served as a program committee member in major machine learning and data mining conferences such as ICML, ICLR, NeurIPS, KDD, IJCAI, AAAI.

Dr. Xiaowei Jia is an assistant professor of computer science at the University of Pittsburgh starting this Fall. His research interests include knowledge-guided data science, spatio-temporal data mining, social network analysis, and other data science problems of great societal impacts. His work is highly interdisciplinary and has been published in major data mining conferences and journals as well as scientific journals in hydrology, agriculture and remote sensing. He has served as a program committee member in AI conferences such as IJCAI and AAAI.

Dr. Guanjie Zheng is an assistant professor at Shanghai Jiao Tong University. His research interest lies in building machine learning and data mining techniques to solve real-world problems, especially spatial-temporal problems. He is particularly interested in developing intelligent data-driven decision-making techniques. He has developed innovative models to tackle the challenges in data-driven decision making and apply the models in multiple domains, including urban traffic control, news recommendation, and environmental protection. He has published more than 20 papers in top venues, including KDD, AAAI, WWW, CIKM, ICDE, etc.

Dr. Vipin Kumar is a Regents Professor and William Norris Chair in Large Scale Computing in the Department of Computer Science and Engineering at the University of Minnesota. Kumar’s research interests include data mining, high-performance computing, and their applications in climate/ecosystems and biomedical domains. Kumar received his Ph.D. in CS from University of Maryland. Kumar has been elected a Fellow of the American Association for Advancement for Science (AAAS), Association for Computing Machinery (ACM), Institute of Electrical and Electronics Engineers (IEEE), and Society for Industrial and Applied Mathematics (SIAM). He received the Distinguished Alumnus Award from the Indian Institute of Technology (IIT) Roorkee (2013), the Distinguished Alumnus Award from the Computer Science Department, University of Maryland College Park (2009), and IEEE Computer Society’s Technical Achievement Award (2005).

Dr. Zhenhui Li is an associate professor of Information Sciences and Technology at Pennsylvania State University. Before joining Penn State, she received her Ph.D. degree in Computer Science from University of Illinois Urbana-Champaign in 2012. Her research has been focused on mining spatial-temporal data with applications in transportation, ecology, environment, social science, and urban computing. She is a passionate interdisciplinary researcher and has been actively collaborating with cross-domain researchers. She has served as organizing committee or senior program committee of many conferences including KDD, ICDM, SDM, CIKM, and SIGSPATIAL. She has been regularly offering classes on data organizing and data mining since 2012. Her classes have constantly received high student ratings. She has received NSF CAREER award, junior faculty excellence in research, and George J. McMurtry junior faculty excellence in teaching and learning award.