Patients with non-small cell lung cancer (NSCLC) with epidermal growth factor receptor (EGFR) mutations exhibit an unfavorable response to PD-1 inhibitor through unclear mechanisms. Hypothesizing that EGFR mutations alter tumor-immune interactions, we compare tumor-infiltrating lymphocytes between EGFR mutant (EGFR-MT) and wild type (EGFR-WT) tumors through single-cell transcriptomic analysis. We find that B cells, CXCL13-producing follicular helper CD4+ T (TFH)-like cells, and tissue-resident memory CD8+ T (TRM)-like cells decreased in EGFR-MT tumors. The NOTCH-RBPJ regulatory network, which is vital for persistence of TRM state, is perturbed, and the interactions between TFH and B cells through the CXCL13-CXCR5 axis disappear in EGFR-MT tumors. Notably, the proportion of TRM-like cells is predictive for anti-PD-1 response in NSCLC. Our findings suggest that the impairment of TFH-B-TRM cooperation in tertiary lymphoid structure formation, accompanied by the dysregulation of TRM homeostasis and the loss of TFH-B crosstalk, underlies unfavorable anti-PD-1 response in EGFR-MT lung tumors.