I’m broadly interested in exploring the following topics:
Safe ML
Existing machine learning models are vulnerable to adversarial attacks and distributional shifts. In the real world, underlying data distributions are hardly stationary, and simply optimizing for accuracy on the training set will result in models that are brittle and unreliable. I aim to build models that are robust to these deviations, and can generalize to unseen data. On this topic, I’ve worked on:
- Scaling up zeroth-order optimization for larger models, a powerful tool for black-box adversarial attacks and contrastive explanations.
Efficient ML
Deep learning models are becoming increasingly large and computationally expensive. This makes them difficult to deploy on edge devices, which have limited computational resources. I aim to build models that are smaller and faster, while maintaining high performance metrics. On this topic, I’ve worked on:
- Knowledge distillation for compressing large language models into smaller ones.
- Tensor compression for reducing the number of parameters in a model.
- Zeroth-order training of spiking neural networks for neuromorphic hardware.