AIRS in the AIR

• 

  查宏远

  查宏远，香港中文大学（深圳）校长学勤讲座教授、AIRS机器学习与应用中心主任

  执行主席、圆桌环节嘉宾

  嘉宾介绍

  查宏远现为香港中文大学（深圳）校长学勤讲座教授、数据科学学院副院长（科研），深圳市人工智能与机器人研究院机器学习与应用中心主任。

  查宏远教授1984年毕业于复旦大学数学系，并于1993年获得斯坦福大学科学计算专业博士学位。查教授于2006年至2020年任职于佐治亚理工学院计算机学院，1992年至2006年任职于宾州州立大学计算机科学与工程系，他也曾于1999年至2001年任职于Inktomi公司。他目前的研究方向是机器学习及应用。

  查教授在计算机等相关领域的主流科技期刊和顶级学术会议上发表300多篇论文，据谷歌学术统计，截止2021年04月，谷歌H-index 79，总引用率超25100次。曾荣获多项重要学术奖项，如 Institute of Mathematics and Applications（IMA）授予的“莱斯利福克斯奖（Leslie Fox Prize）”二等奖（1991年），第34届ACM SIGIR国际信息检索会议（SIGIR 2011）最佳学生论文奖（指导教授）（2011年），第26届NeurIPS“最佳论文奖” (2013年)。

  个人简介
• 

  徐菁

  徐菁，哈尔滨工业大学（深圳）博士生

  通过移除任务分布中心的投影的方法缓解小样本中的样本选择偏差的问题

  嘉宾介绍

  Jing Xu is currently a Ph.D. student in the School of Harbin Institute of Technology, Shenzhen, China. She received the M.Sc. degree in the School of Computer
  Science and Engineering, University of Electronic Science and Technology of China, China, in 2017. Her research interests include few-shot learning and representation learning.

  个人简介

  Few-shot learning (FSL) targets at generalization of vision models towards unseen tasks without sufficient annotations. Despite the emergence of a number of few-shot learning methods, the sample selection bias problem, i.e., the sensitivity to the limited amount of support data, has not been well understood. In this paper, we find that this problem usually occurs when the positions of support samples are in the vicinity of task centroid---the mean of all class centroids in the task. This motivates us to propose an extremely simple feature transformation to alleviate this problem, dubbed Task Centroid Projection Removing(TCPR). TCPR is applied directly to all image features in a given task, aiming at removing the dimension of features along the direction of the task centroid. While the exact task centroid cannot be accurately obtained from limited data, we estimate it using base features that are each similar to one of the support features. Our method effectively prevents features from being too close to the task centroid. Extensive experiments over ten datasets from different domains show that TCPR can rebly improve classification accuracy across various feature extractors, training algorithms and datasets.

• 

  陈泓瑞

  香港中文大学（深圳）数据科学博士生

  BackdoorBench: A Comprehensive Benchmark of Backdoor Learning

  嘉宾介绍

  陈泓瑞，数据科学在读博⼠⽣，师从吴保元教授。本科在⾹港中⽂⼤学（深圳）获得应⽤数学学⼠学位，硕⼠在⾹港中⽂⼤学（深圳）获得数据科学硕⼠学位。

  个人简介

  随着深度神经⽹络（DNNs）在许多场景中的⼴泛运⽤，DNN的安全问题已经引起了越来越多的关注。如果⽤户从第三⽅平台下载未经验证的数据集/checkpoint来训练/微调⾃定义的模型，甚⾄将模型训练过程直接外包给第三⽅平台，后⻔攻击会对这类⽤户产⽣极⼤的威胁。当后⻔模型输⼊正常样本时，会预测出正确的结果；但是⼀旦后⻔模型遇到被故意篡改的样本时，则会输出恶意的结果。

  虽然后⻔学习在机器学习领域是⼀个新型的研究⽅向，但其发展速度惊⼈，并呈现出快速军备竞赛的态势。然⽽，我们注意到很多新⽅法的评估往往不够充分，通常其论⽂中只会对⽐⼀⼩部分⽅法/模型/数据集。如果没有完整的评估和公平⽐较，则很难验证和评估新⽅法的真实性能，并且会阻碍对后⻔学习的内在原理的探索。

  为了缓解这种困境，我们建⽴了⼀个全⾯的后⻔学习基准，BackdoorBench。

• 

  杨念祖

  上海交通大学计算机系博士生

  Learning Substructure Invariance for Out-of-Distribution Molecular Representations

  嘉宾介绍

  上海交通大学计算机系博士生，目前是直博二年级。2021年毕业于上海交通大学IEEE试点班计算机专业，取得学士学位。他的研究兴趣包括图神经网络，生成模型，OoD泛化，以及AI制药等方向。

  个人简介

  Molecule representation learning (MRL) has been extensively studied and current methods have shown promising power for various tasks, e.g., molecular property prediction and target  identification. However, a common hypothesis of existing methods is that either the model development or experimental evaluation is mostly based on i.i.d. data across training and testing. Such a hypothesis can be violated in real-world applications where testing molecules could come from new environments, bringing about serious performance degradation or unexpected prediction. We propose a new representation learning framework entitled MoleOOD to enhance the robustness of MRL models against such distribution shifts, motivated by an observation that the (bio)chemical properties of molecules are usually invariantly associated with certain privileged molecular substructures across different environments (e.g., scaffolds, sizes, etc.). Specifically, We introduce an environment inference model to identify the latent factors that impact data generation from different distributions in a fully data-driven manner. We also propose a new learning objective to guide the molecule encoder to leverage environment-invariant substructures that more stably relate with the labels across environments. Extensive experiments on ten real-world datasets demonstrate that our model has a stronger generalization ability than existing methods under various out-of-distribution (OOD) settings, despite the absence of manual specifications of environments.

• 

  郭丹丹

  香港中文大学（深圳）博士后研究员

  针对不平衡分类的最优传输（OT）自动加权方

  嘉宾介绍

  郭丹丹2020年博士毕业于西安电子科技大学，此后在香港中文大学（深圳）机器人与智能制造研究院（IRIM）、数据科学学院进行博士后研究，师从我校数据科学学院执行院长、机器学习著名学者查宏远教授。她的主要研究方向是模式识别机器学习，包括概率模型构建与统计推断，以数据为中心的机器学习算法，最优传输理论。所涉及的应用有图像生成及分类、文本分析、自然语言生成等。目前，她专注于现实应用中小样本分类、小样本生成、训练数据分布有偏等问题。她的科研成果发表在机器学习国际顶级会议、期刊上，如NeurIPS,ICML,ICLR, IJCV, TNNLS等。 她也是多个国际会议的程序委员会委员和期刊审稿人，如ICML，NeurIPS，ICLR，JMLR, TSP等。

  个人简介

  Deep neural networks (DNNs) have achieved remarkable success in various applications, which is undoubtedly inseparable from the high-quality large-scale datasets. Imbalanced data pose challenges for deep learning based classification models. This paper introduces a novel automatic re-weighting method for imbalance classification based on optimal transport (OT). This method presents the imbalanced training set as a to-be-learned distribution over its training examples, each of which is associated with a probability weight. Similarly, our method views another balanced meta set as a balanced distribution over the examples. By minimizing the OT distance between the two distributions in terms of the defined cost function, the learning of weight vector is formulated as a distribution approximation problem. Our proposed re-weighting method bypasses the commonly-used classification loss on the meta set and uses OT to learn the weights, disengaging the dependence of the weight learning on the concerned classifier at each iteration. This is an approach different from most of the existing re-weighting methods and may provide new thoughts for future work. Experimental results on a variety of imbalanced datasets of both images and texts validate the effectiveness and flexibility of our proposed method.

• 

  Andre Milzarek

  香港中文大学（深圳）数据科学学院助理教授

  A Unified Convergence Theorem for Stochastic Optimization Methods

  嘉宾介绍

  Andre Milzarek现为香港中文大学（深圳）数据科学学院助理教授。他曾在北京大学北京国际数学研究中心担任博士后研究员。2013年和2016年，Andre Milzarek教授在德国慕尼黑工业大学Michael Ulbrich教授的指导下，以优异的成绩获得硕士学位和博士学位。

  Andre Milzarek教授主要的研究方向和兴趣包括非光滑优化、大规模随机优化、二阶方法和理论。他曾受邀在工业和应用数学学会（AISM）、数学编程国际研讨会（ISMP）等国际会议上发表报告，并在多份期刊上发表论文。Andre Milzarek教授从2010年至2012年获德国巴伐利亚州Max-Weber项目资助，2017年获北京大学博雅博士后奖学金。

  个人简介

  论文针对一般的随机优化方法提出了一个统一的收敛性定理的证明框架，其中收敛性主要指迭代最终步的梯度范数依期望收敛或者几乎必然收敛到0。本项工作的意义主要在于两点：

  （1）不同于已有的复杂度类型的结果，我们讨论的方法与实际应用更为贴合，例如迭代步长的选取不依赖于迭代总步数，探讨的收敛性针对最后一迭代的梯度而非所有迭代在某种平均下的梯度或者所有迭代的梯度的最小值，这一点尤其重要，因为随机算法在迭代过程中并不会计算全梯度，并且算法通常会返回最后一次迭代，所以，只有估计最后一次迭代的梯度范数的表现才是最有意义的。

  （2）当下已有的一些类似的收敛性结果通常只针对某类特殊的算法，我们对于一般的随机优化算法提出了通用的证明框架，使用者只需要验证算法是否满足若干典型条件即可。我们运用这一收敛框架在更弱的假设下重现了stochastic gradient method (SGD) 和random reshuffling (RR) 的收敛性证明，并且对更复杂的stochastic proximal gradient method (prox-SGD)  和stochastic model-based methods (SMM) 两种算法也得到了依期望收敛和几乎必然收敛的结果。

  该文章中所提出的统一收敛框架有望被广泛应用于其他随机算法，例如Momentum SGD、STORM等。

• 

  卞亚涛

  腾讯AI Lab机器学习中心高级研究员

  Learning Neural Set Functions Under the Optimal Subset Oracle

  嘉宾介绍

  卞亚涛博士是腾讯人工智能实验室（Tencent AI Lab）机器学习中心高级研究员， 深圳市政府认定海外高层次人才。 2019年在瑞士苏黎世联邦理工学院（ETH Zurich）计算机系机器学习研究所获得博士学位， 本科毕业于上海交通大学。自 2015 年到2019年，他也是马克斯普朗克 ETH 学习系统中心（Max Planck ETH Center for Learning Systems）的联合研究员。 他现在致力于图表示学习、AI辅助药物发现、可信AI等方面的研究。 曾获2011-2012年AMD中国加速计算大赛全国冠军。 在NeurIPS、ICML、ICLR、 IJCAI、AISTATS、T-PAMI等机器学习顶会期刊发表多篇论文，曾担任ICML、NeurIPS、ICLR、AISTATS、CVPR、AAAI、AISTATS, STOC等会议及JMLR， T-PAMI期刊的程序委员和审稿人。个人主页： https://yataobian.com/

  个人简介

  Learning neural set functions becomes increasingly more important in many applications like product recommendation and compound selection in AI-aided drug discovery. The majority of existing works study methodologies of set function learning under the function value oracle, which, however, requires expensive supervision signals. This renders it impractical for applications with only weak supervisions under the Optimal Subset (OS) oracle, the study of which is surprisingly overlooked. In this work, we present a principled yet practical maximum likelihood learning framework, termed as EquiVSet, that simultaneously meets the following desiderata of learning set functions under the OS oracle: i) permutation invariance of the set mass function being modeled; ii) permission of varying ground set; iii) minimum prior; and iv) scalability. The main components of our framework involve: an energy-based treatment of the set mass function, DeepSet-style architectures to handle permutation invariance, mean-field variational inference, and its amortized variants. Thanks to the elegant combination of these advanced architectures, empirical studies on three real-world applications (including Amazon product recommendation, set anomaly detection, and compound selection for virtual screening) demonstrate that EquiVSet outperforms the baselines by a large margin.

• 

  王本友

  香港中文大学（深圳）数据科学学院助理教授

  On the parameter compression of pre-trained language models

  嘉宾介绍

  Benyou Wang is an assistant professor in the School of Data Science, The Chinese University of Hong Kong, Shenzhen.  He was a Marie Curie Researcher at the European Union and got his Ph.D. degree from the University of Padua, Italy in 2022. He is committed to building explainable,  robust, and efficient natural language processing systems that are with both technical rationality and linguistic motivations. So far, he and his collaborators have won the Best Paper Nomination Award in  SIGIR 2017 and  Best Explainable NLP Paper in NAACL 2019, and Best paper in NLPCC 2022.

  个人简介

  Pre-trained Language Models (PLMs) have shown great success in natural language processing and information retrieval. Recent studies (like GPT 3) show that super large-scaled PLMs have better potential in natural language processing, while it also raises some concerns from an environmental and economical point of view since it is expensive to train/inference and causes large carbon emissions. We propose to compress pre-trained language models with 1/50 size and 3 times speedup without a big performance drop, this may help the deployment of PLMs for small or middle enterprises or universities. Further work also tests it in edge devices and also compensate for the compression with linguistic knowledge. Related papers were published in ICLR 2022 (https://openreview.net/forum?id=RftryyYyjiG), EMNLP 2022, and NeurIPS 2022 (https://openreview.net/pdf?id=7ilJhkpm1H).

• 

  唐晓莹

  香港中文大学（深圳）理工学院助理教授、AIRS群体智能中心副研究员

  圆桌环节主持人

  嘉宾介绍

  唐晓莹，香港中文大学（深圳）理工学院助理教授、深圳市人工智能与机器人研究院副研究员。于2020年入选 IEEE PES中国区电动汽车技术委员会会员，并担任电动汽车与能源交通系统融合技术分委会常务理事（任期：2020-2023 ）。

  她的研究方向包括计算机、信息学与新能源的交叉领域，特别针对电动车联网、能源互联网的信息系统优化与智能算法研究。在中科院一区期刊如IEEE Transactions on Smart Grid (TSG)、IEEE Transactions on Power Systems (TPS)、IEEE Transactions on Industrial Informatics (TII)、IEEE Communication Magazine和学科旗舰会议如 IEEE 48th Annual Conference on Information Sciences and Systems (CISS), IEEE SmartGridComm, ICML2021等发表多篇论文，其中以第一作者获得IEEE SmarGridComm 2013年唯一最佳论文奖，并以第一作者在国际知名出版社Springer出版科研专著“Optimal charging control of electric vehicles in smart grids”。

  个人简介
• 

  徐增林

  哈尔滨工业大学（深圳）教授

  圆桌环节嘉宾

  嘉宾介绍

  徐增林，哈尔滨工业大学（深圳）教授、博士生导师。担任国际神经网络学会（INNS）理事、副主席（教育）。2009年毕业于香港中文大学计算机科学与工程专业并获博士学位，先后在密西根州立大学、德国马普计算机研究所萨尔大学、普渡大学、电子科技大学等从事研究工作。主要研究兴趣为机器学习及应用，他在包括IEEE TPAMI, IEEE TNNLS，IEEE TKDE，NeurIPS, ICML, IJCAI, AAAI，CVPR，ACL等重要会议和刊物发表论文150多篇，发表专著1部，获得2015年AAAI大会最佳学生论文奖提名、ACML2016最佳学生论文奖亚军，2016年APNNS青年学者奖。多次担任人工智能领域的主要国际会议如AAAIIJCAI等会议的领域主席和程序委员会成员。目前担任Neural Networks 和Neurocomputing的编委。

  个人简介

视频回顾