Zain Sarwar

Applied Researcher, LLM Core & Agentic AI · Capital One

prof_pic.jpg

I’m an Applied Researcher on the LLM Core team at Capital One. I recently completed my PhD in Computer Science at the University of Chicago, advised by Michael Maire.

My research is about making large language models more capable and more efficient. During my PhD I worked on Mixture-of-Experts (sparse and low-rank experts, dense routing gradients, continual pre-training), memory, and efficient reasoning (structured thoughts and context pruning for reasoning models). Earlier in my PhD I worked on security and machine learning, including deepfake text detection and keystroke inference attacks.

news

Oct 25, 2026 Launched my new website!

selected publications

  1. arXiv
    Structured Thoughts For Improved Reasoning And Context Pruning
    Zain Sarwar, Supriyo Chakraborty, Berkcan Kapusuzoglu, and 4 more authors
    2026
  2. NeurIPS
    origrouter.png
    Dense Backpropagation Improves Training for Sparse Mixture-of-Experts
    Ashwinee Panda, Vatsal Baherwani, Zain Sarwar, and 4 more authors
    In Advances in Neural Information Processing Systems (NeurIPS), 2025
  3. ICML
    Critique-Guided Distillation for Robust Reasoning via Refinement
    Berkcan Kapusuzoglu, Supriyo Chakraborty, Zain Sarwar, and 2 more authors
    2026
  4. TMLR
    cpt.png
    Continual Pre-training of MoEs: How robust is your router?
    Benjamin Thérien, Charles-Étienne Joseph, Zain Sarwar, and 7 more authors
    Transactions on Machine Learning Research (TMLR), 2025
  5. Preprint
    Low Rank Experts Enable Specialization In Dense Transformers
    Zain Sarwar, Ashwinee Panda, Benjamin Thérien, and 3 more authors
  6. NeurIPS Workshop
    structmoe.png
    StructMoE: Structured Mixture of Experts Using Low Rank Experts
    Zain Sarwar, Ashwinee Panda, Benjamin Thérien, and 8 more authors
    In NeurIPS Efficient Natural Language and Speech Processing Workshop (ENLSP), 2024
  7. NeurIPS Workshop
    Dense Backpropagation Improves Routing for Sparsely-Gated Mixture-of-Experts
    Ashwinee Panda, Vatsal Baherwani, Zain Sarwar, and 4 more authors
    In Workshop on Machine Learning and Compression, NeurIPS, 2024
  8. NeurIPS Workshop
    sharing_mechanism.png
    MYCROFT: Towards Effective and Efficient External Data Augmentation
    Zain Sarwar, Van Tran, Arjun Nitin Bhagoji, and 3 more authors
    In NeurIPS Workshop on Machine Learning for Systems (MLforSys), 2024
  9. USENIX Security
    vr_keystroke.png
    Can Virtual Reality Protect Users from Keystroke Inference Attacks?
    Zhuolin Yang, Zain Sarwar, Iris Hwang, and 3 more authors
    In 33rd USENIX Security Symposium (USENIX Security 24)Press: New Scientist , 2024
  10. USENIX Security
    keystroke.png
    Towards a General Video-based Keystroke Inference Attack
    Zhuolin Yang, Yuxin Chen, Zain Sarwar, and 3 more authors
    In 32nd USENIX Security Symposium (USENIX Security 23), 2023
  11. IEEE S&P
    deepfake.png
    Deepfake Text Detection: Limitations and Opportunities
    Jiameng Pu*, Zain Sarwar*, Sifat Muhammad Abdullah, and 5 more authors
    In IEEE Symposium on Security and Privacy (S&P)Press: CACM, TechXplore , 2023