Monday Lunch Talk: Efficient memory-usage techniques in deep neural networks
Please join us with your bag lunch in Room 336 Boardman Hall on Monday January 27 for a scintillating presentation by our very own Dr. Salimeh Yasaei Sekeh. Hope to see you then!
Title: Efficient memory-usage techniques in deep neural networks via a graph-based approach
Abstract:
Memory consumption is an important aspect of artificial neural networks that needs to be carefully considered to make them applicable to real-world problems. In deep networks, the total number of parameters for a given network and the stored activations for large datasets use a massive amount of memory. There are a number of different approaches used to compress or reduce the memory consumed, including streaming feature selection, network quantization, tensor decomposition, knowledge distillation, and network pruning. In this talk, two new efficient deep memory usage techniques based on the geometric dependency criterion are introduced: (1) Online Streaming Deep Feature Selection which is a new technique based on a novel supervised streaming setting. This technique measures deep feature relevance and maintains a minimal deep feature subset with relatively high classification performance and less memory requirement. (2) Geometric Dependency-based Neuron Trimming which is a data-driven pruning method that evaluates the relationship between nodes in consecutive layers. In this approach, a new dependency-based pruning score removes neurons with least importance, and then the network is fine-tuned to retain its predictive power. Both methods are evaluated on several datasets with multiple CNN models and demonstrated to achieve significant memory compression compared to the baselines.
Bio:
Salimeh Yasaei Sekeh is an Assistant Professor of Computer Science at the University of Maine. Prior to UMaine, she was a postdoctoral research fellow in the Electrical Engineering and Computer Science Department at the University of Michigan, Ann Arbor. She has also held appointments in Brazil and Italy. Salimeh’s research focuses on designing and analyzing machine learning algorithms, deep learning techniques, applications of machine learning approaches in real-time problems, data mining, pattern recognition, and network structure learning with applications in biology.