This study explored the associations between personality traits and cognitive failure (including minor lapses and prospective and retrospective memory failure) among middle-aged and older adults living in Japan. The participants were 373 adults, aged 40–84 (167 men and 206 women). The 15-item Japanese version of the Short Inventory of Minor Lapses was used to evaluate minor lapses, and the 16-item Japanese version of the Prospective and Retrospective Memory Questionnaire was used to assess prospective and retrospective memory failure. The participants’ variables evaluated for their association with cognitive failure were gender, age, education, paid work, social network, chronic disease, sleep quality, and the Big Five personality traits (i.e., neuroticism, extraversion, openness, agreeableness, and conscientiousness). Multivariable regression analyses demonstrated that sleep quality (β = −0.232), neuroticism (β = 0.163), and conscientiousness (β = −0.295) were related to minor lapses; age (β = 0.152), sleep quality (β = −0.168), and conscientiousness (β = −0.290) were associated with prospective memory failure; and age (β = 0.268), sleep quality (β = −0.146), and conscientiousness (β = −0.221) were associated with retrospective memory failure. These findings may facilitate the development of efficient strategies for the prevention of cognitive dysfunction and its adverse consequences for personal health