About me
I am a 5th year PhD student at the University Of Chicago in the department of Computer Science. I am being advised by Professor Michael Maire.My research is focused on advancing the capabilities and efficiency of LLMs. I've worked on architectural innovations like designing sparse augmentations that introduce dynamic, token-level specialization within standard transformers to improve model performance. I'm also working on developing techniques for doing structured reasoning and context pruning for reasoning models.
Publications

StructMoE: Structured Mixture of Experts Using Low Rank Experts
Zain Sarwar, Ashwinee Panda, Benjamin Thérien, Stephen Rawls, Anirban Das, Kartik Balasubramaniam, Berkcan Kapusuzoglu, Shixiong Zhang, Sambit Sahu, Milind Naphade, Supriyo Chakraborty
NeurIPS EMNSLP 2024
PDF
Zain Sarwar, Ashwinee Panda, Benjamin Thérien, Stephen Rawls, Anirban Das, Kartik Balasubramaniam, Berkcan Kapusuzoglu, Shixiong Zhang, Sambit Sahu, Milind Naphade, Supriyo Chakraborty
NeurIPS EMNSLP 2024

Dense Backpropagation Improves Training for Sparse Mixture-of-Experts
Ashwinee Panda, Vatsal Baherwani, Zain Sarwar, Benjamin Thérien, Sambit Sahu, Tom Goldstein, Supriyo Chakraborty
NeurIPS 2025
PDF
Ashwinee Panda, Vatsal Baherwani, Zain Sarwar, Benjamin Thérien, Sambit Sahu, Tom Goldstein, Supriyo Chakraborty
NeurIPS 2025

Continual Pre-training of MoEs: How robust is your router?
Benjamin Thérien, Charles-Étienne Joseph, Zain Sarwar, Ashwinee Panda, Anirban Das, Shi-Xiong Zhang, Stephen Rawls, Sambit Sahu, Eugene Belilovsky, Irina Rish
TMLR 2025
PDF
Benjamin Thérien, Charles-Étienne Joseph, Zain Sarwar, Ashwinee Panda, Anirban Das, Shi-Xiong Zhang, Stephen Rawls, Sambit Sahu, Eugene Belilovsky, Irina Rish
TMLR 2025

MYCROFT: Towards Effective and Efficient External Data Augmentation
Zain Sarwar, Van Tran, Arjun Nitin Bhagoji, Nick Feamster, Ben Y Zhao, Supriyo Chakraborty
NeurIPS MlforSYS 2024
PDF
Zain Sarwar, Van Tran, Arjun Nitin Bhagoji, Nick Feamster, Ben Y Zhao, Supriyo Chakraborty
NeurIPS MlforSYS 2024

Deepfake Text Detection: Limitations and Opportunities
Jiameng Pu*, Zain Sarwar*, Sifat Muhammad Abdullah, Abdullah Rehman, Yoonjin Kim, Parantapa Bhattacharya, Mobin Javed, and Bimal Viswanath
IEEE S&P (Oakland) 2023
PDF
Jiameng Pu*, Zain Sarwar*, Sifat Muhammad Abdullah, Abdullah Rehman, Yoonjin Kim, Parantapa Bhattacharya, Mobin Javed, and Bimal Viswanath
IEEE S&P (Oakland) 2023

Can Virtual Reality Protect Users from Keystroke Inference Attacks?
Zhuolin Yang, Zain Sarwar, Iris Hwang, Ronik Bhaskar, Ben Y. Zhao, Haitao Zheng
USENIX 2024
PDF
Zhuolin Yang, Zain Sarwar, Iris Hwang, Ronik Bhaskar, Ben Y. Zhao, Haitao Zheng
USENIX 2024

Towards a General Video-based Keystroke Inference Attack
Zhuolin Yang, Yuxin Chen, Zain Sarwar, Hadleigh Schwartz, Ben Y. Zhao, Haitao Zheng
USENIX 2023
PDF
Zhuolin Yang, Yuxin Chen, Zain Sarwar, Hadleigh Schwartz, Ben Y. Zhao, Haitao Zheng
USENIX 2023
