This study used a conceptual model to examine the factors influencing physical, mental, and overall health-related quality of life (HRQoL) in women and men aged 45 and older with knee osteoarthritis (KOA) in Saudi Arabia. In this multicenter cross-sectional study, we randomly included 356 individuals aged 45 years or above with doctor-confirmed KOA from the orthopedic and physiotherapy departments of the 5 tertiary hospitals in Riyadh, Saudi Arabia, between March 2016 and March 2017. We split all participants into men (n = 146) and women (n = 210) based on gender. A conceptual model was developed using the HRQoL influential potential factors, such as age, sex, education, occupation, and way of eating (sociodemographic), and clinical factors, such as osteoarthritis knee and its severity, duration, pain, and body mass index. The 36-item short form health survey and its subscales of the physical composite scale and mental composite scale were used to evaluate overall HRQoL, physical, and mental health, respectively. We used unadjusted multiple linear regression analyses to investigate the associations between gender-specific potential factors and HRQoL outcomes. Women and men aged between 60 and 64 years were more strongly associated significantly with less physical composite scale score by -3.17, (standard error [SE] = 1.71, P = .021) and -3.18 (SE = 1.69, P = .023) respectively, followed by the primary school or less education by -3.40 (SE = 1.27, P = .0002), severe KOA of -8.94 (SE = 0.99, P < .001), eating on the floor bending the knee of -3.93 (SE = 1.63, P = .042), and pain of -2.39 (SE = 0.26, P < .0001). Women and men with primary school or less education significantly had low mental composite scale and 36-item short form health survey scores of -3.07 (SE = 1.22, P = .041) and -3.23 (SE = 0.99, P = .018), respectively, followed by severe KOA of -4.07 (SE = 1.22, P = .001) and -6.50 (SE = 0.83, P < .0001) and eating on the floor, extending the knee at -3.35 (SE = 1.74, P = .043). Risk factors like age, education, pain, body mass index, and severe KOA are linked to poor physical, mental, and overall HRQoL among women and men in Saudi Arabia.