Hearing loss is associated with cognitive decline and impairment in daily living activities. Access to hearing health care has broad implications for healthy aging of the U.S. population. This qualitative study investigated factors related to the socio-ecological domains of hearing health in a U.S.-Mexico border community experiencing disparities in access to care. A multidisciplinary research team partnered with community health workers (CHWs) from a Federally Qualified Health Center (FQHC) in designing the study. CHWs conducted interviews with people with hearing loss (n = 20) and focus groups with their family/friends (n = 27) and with members of the community-at-large (n = 47). The research team conducted interviews with FQHC providers and staff (n = 12). Individuals experienced depression, sadness, and social isolation, as well as frustration and even anger regarding communication. Family members experienced negative impacts of deteriorating communication, but expressed few coping strategies. There was general agreement across data sources that hearing loss was not routinely addressed within primary care and assistive hearing technology was generally unaffordable. Community members described stigma related to hearing loss and a need for greater access to hearing health care and broader community education. Findings confirm the causal sequence of hearing impairment on quality of life aggravated by socioeconomic conditions and lack of access to hearing health care. Hearing loss requires a comprehensive and innovative public health response across the socio-ecological framework that includes both individual communication intervention and greater access to hearing health resources. CHWs can be effective in tailoring intervention strategies to community characteristics.