research

2025

1. Deep Learning

   Beyond Top-k: A Comparative Study for Dynamic Routing Mechanisms in Mixture of Experts Models

   Steven Su. Tarunyaa Sivakumar. Areeb Alam

   CIS 6200: Advanced Topics in Deep Learning Final Project, 2025

   Abs PDF

   Mixture of Experts (MoE) models offer a scalable and compute-efficient alternative to dense transformer architectures by activating only a small, relevant subset of expert subnetworks for each input token. This sparsity allows MoE models to maintain high model capacity while significantly reducing computation per token. However, the effectiveness of these models critically depends on the routing strategy used to determine which experts are activated during inference. In this paper, we present a comprehensive comparative study of expert routing mechanisms within MoE-augmented GPT-2 models. We specifically evaluate fixed Top-k routing alongside three dynamic strategies: Top-p, Top-Any, and Entropy-Adaptive routing. Each method introduces varying levels of adaptability and complexity in how experts are selected for a given token. Our experiments, conducted on the SlimPajama dataset, assess the performance of these routing strategies across multiple metrics, including cross-entropy loss, perplexity, inference latency, memory usage, floating-point operations (FLOPs), and expert utilization. The results demonstrate that dynamic routing strategies such as Top-p and Top-Any significantly outperform both the dense baseline and the fixed Top-k routing in terms of accuracy and expert load balancing. These methods also present meaningful trade-offs in terms of compute cost and memory efficiency. To better understand the behavior of these routing strategies, we analyze the interaction between input complexity and expert selection patterns through visualizations and correlation analyses. These insights reveal that adaptive routing not only improves model expressivity but also leads to more efficient and effective expert utilization. Overall, our findings highlight the importance of dynamic routing in the design of sparse architectures. By enabling better performance and resource usage, adaptive strategies pave the way for the development of next-generation Mixture of Experts models that are both powerful and efficient.

2024

1. Neuroscience

   Autism Spectrum Disorder (ASD): Interaction Between Sex and Epigenetic Factors on the Neuronal Transcriptome

   Hanqi (Steven) Su. Professor Erica Korb

   Recipient of the 2024 Ernest M. Brown, Jr. College Alumni Society Undergraduate Research Grant, 2024

   Abs PDF

   Autism Spectrum Disorder (ASD) affects millions worldwide, with males being more prone to diagnosis than females, at a ratio of 4.5:1. The cellular basis for this gender disparity remains unclear. Recent studies have identified several chromatin-modifying genes associated with ASD, which regulate gene expression by controlling genome accessibility. This study investigates the interactions between these chromatin modifications and the sex-specific neuronal transcriptome. E16 cortical neurons were cultured and infected with lentiviral shRNA targeting chromatin-modifying genes, followed by RNA-seq analysis. Our findings reveal that Gpr153 is expressed 15% higher in male neurons and shows neuron-specific expression independent of hormonal differences. Additionally, the loss of Med13l and Chd8 results in a distinct, sex-specific neuronal transcriptomic response, impacting neurodevelopmental pathways. These results suggest that interactions between chromatin modifiers and the sex-specific transcriptome may contribute to the observed gender differences in ASD.

2. Comp Bio

   MEAnalysis: An R-package for Efficient Analysis & Visualization of Neuron Multi-Electrode Array Experiment Data

   Hanqi (Steven) Su. Professor Erica Korb

   2024

   Abs

   The MEAnalysis R package is an open-source tool designed for the efficient and automated analysis of neuron multielectrode array (MEA) experiment data, developed primarily for use at the Penn Epigenetics Institute. This package addresses the institute’s need for a powerful and user-friendly tool to analyze and visualize data from their newly adopted MEA assays, overcoming the limitations of existing methods. The MEAnalysis package is built with object-oriented programming principles, emphasizing ease of use, scientific accuracy, scalability, and maintainability. The package comprises three main classes: MEAnalysis, BatchMEAnalysis, and ElectrodeBursts. These classes provide comprehensive tools for single-file analysis, batch processing of multiple files, and visualization of electrode burst data, respectively. Users can generate detailed plots and significance tables, facilitating a deeper understanding of the experimental results. The package supports both single and batch analyses, allowing researchers to streamline their workflows and focus on interpreting the results. It offers a flexible and scalable solution to meet the growing demands of high-throughput MEA experiments in neuroscience research.

3. Comp Bio

   Histone variant H2BE controls activity-dependent gene expression and homeostatic scaling

   Emily F. Alekh P. Steven S.et al.

   In Preprint, 2024

   Abs PDF

   A cell’s ability to respond and adapt to environmental stimuli relies in part on transcriptional programs controlled by histone proteins. Histones affect transcription through numerous mechanisms including through replacement with variant forms that carry out specific functions. We recently identified the first widely expressed H2B histone variant, H2BE and found that it promotes transcription and is critical for neuronal function and long-term memory. However, how H2BE is regulated by extracellular stimuli and whether it controls activity-dependent transcription and cellular plasticity remain unknown. We used CUT&Tag and RNA-sequencing of primary neurons, single-nucleus sequencing of cortical tissue, and multielectrode array recordings to interrogate the expression of H2BE in response to stimuli and the role of H2BE in activity-dependent gene expression and plasticity. We find that unlike Further, we show that neurons lacking H2BE are unable to mount proper long-term activity-dependent transcriptional responses both in cultured neurons and in animal models. Lastly, we demonstrate that H2BE knockout neurons fail to undergo the electrophysiological changes associated with homeostatic plasticity in neurons after long-term stimulation. In summary, these data demonstrate that H2BE expression is inversely correlated to activity and necessary for long-term activity-dependent responses, revealing the first instance of a histone variant involved in the homeostatic plasticity response in neurons.

2023

1. Protein Biochemistry

   Sedmentation Assay Screening of TRIM Proteins on Polyglutamated ATXN1

   Hanqi (Steven) Su. Professor Xiaolu Yang

   Recipient of the 2023 Louis H Castor, M.D., C’48 Undergraduate Research Grant, 2023

   Abs PDF

   When their linear amino acid sequence is translated and folded into the correct three-dimensional conformation, proteins play crucial roles in various biological processes. However, proteins can undergo misfolding for various reasons, including genetic mutations and translational errors. One prevalent consequence of misfolded proteins is insoluble aggregates. The intracellular propagation of such aggregates underly pathogenesis in numerous neurodegenerative diseases. To counteract the detrimental effects of protein misfolding, organisms across diverse kingdoms have evolved sophisticated protein quality control systems that eliminate such aggregates. Recent studies have suggested that the tripartite motif proteins (TRIM), may possess the ability to fulfill this function. Spinocerebellar Ataxia Type 1 (SCA1) is a neurodegenerative disease caused by the aggregation of the polyglutamine- expanded ATXN1 protein. In this study, we undertake a systematic analysis of virtually all known human TRIM proteins and their potential to mitigate Atxn1 82Q aggregation in cultured HEK293T cells. When co-transfected in HEK293T cells, TRIM10, 11, 55, 58 are potent disaggregases of GFP-Atxn1 82Q. Moreover, these four TRIM proteins do not reduce Atxn1 82Q mRNA level and interact with GFP-Atxn1 82Q on a protein-protein level. These results indicate these four TRIM proteins are potentially guardians against polyglutamine-expanded Atxn1 protein aggregation and SCA1 pathogenesis.