Integrating Structure-Based Genetic Algorithms and Reinforcement Learning for the De Novo Discovery of Novel BACE1 Inhibitors

Alzheimer’s disease (AD) remains one of the most pressing neurodegenerative challenges, with β-secretase 1 (BACE1) representing a key therapeutic target for amyloid-β reduction. Despite extensive discovery efforts, the development of clinically viable BACE1 inhibitors remains hindered by poor selectivity, off-target toxicity, and limited blood–brain barrier penetration; to overcome these challenges, we introduce an integrated generative framework that synergistically couples ligand-based reinforcement learning (RL) with structure-based genetic algorithm (GA) design to explore novel chemical space for potent, drug-like inhibitors. The ligand-based pipeline employs a fine-tuned LSTM model guided by a QSAR ensemble reward function (R² = 0.67, RMSE = 0.65), generating 3,000 chemically valid molecules with 100% novelty (Tanimoto < 0.4), while the complementary AutoGrow4-based pipeline integrated a Size-Independent Ligand Efficiency (SILE) scoring function to balance affinity and molecular size, producing 5,020 pocket-compatible compounds. Sequential docking, ADME screening, molecular dynamics, and MM/GBSA analyses revealed stable, high-affinity leads (ΔGbind = –39.50 kcal/mol) exhibiting non-canonical binding through the flap region and hydrophobic sub-pockets, establishing a scalable, dual-pronged strategy that bridges data-driven and structure-guided discovery for complex enzymatic targets.