When we talk about people who "have difficulty" with women, we might say that they are sexist, or we might say that they are misogynist. The two things are connected, and are often found together, but the distinction between them is a useful one. Sexism is public, structural, and may be institutional. Misogyny is personal, psychological, perhaps pathological. Misogyny may underlie institutional sexism, but I at least would hesitate to call an institution misogynistic per se. Similarly, it's possible to say that somebody is being sexist without necessarily accusing them of misogyny as well.
In the area of race, it all gets a bit blurrier. "Racist" does double service, both as a description of what people and institutions say and do, and also of their motivation. Nevertheless, there is a word (if a rather specialized one) that does some of the same semantic work as misogyny - namely "xenophobia". Like "misogyny", "xenophobia" is a diagnosis of motivation rather than a description of action. (There's also the problem that "xenophobia" refers to nationality or culture, rather than to race as such - although to racists this distinction may be less apparent.)
Then there's "homophobia" (and indeed "transphobia", which was coined by analogy). Again you've got that pathology-suggesting suffix, and it's easy to see "homophobia" as doing the same kind of job as "misogyny" and "xenophobia". But where is the equivalent of "sexism" and "racism"? What is the word to use when you want to point out that somebody has just said or done something oppressive or othering or heteronormative or otherwise shitty, but you don't necessarily want to accuse them of being motivated by fear or hatred of homosexuals? How, in short, do you avoid this kind of derailing comment:
Dictionary.com defines homophobia as an “intense hatred or fear of homosexuals or homosexuality.” This to me is probably the biggest and most offensive stereotype in your essay. [...] I can give you my word that I do not hate a single person on this earth, and I am certainly not afraid of homosexuals. In fact, I have a cousin who is a lesbian, and I give her a big hug every time I see her and love her just as much as I love any of my cousins. Just because I believe that her actions are immoral definitely does not mean that I love her any differently. [For the context of this comment, see here].
I've read on several blogs about racism that one of the things people tend to do when told that they have said something racist is to take offence, or be over-apologetic, or in some other way make it all about them, rather than understanding it as a simple correction from which they can learn and move on. However rare the latter reaction may in fact be it is at least possible, because in saying that an expression is racist you're not necessarily saying that the speaker is a xenophobe through and through and motivated by pathological fear and hatred. But "homophobic" has that accusation built in - especially for those of an etymological cast of mind.
What to do? Do we invent a new word that will have the same relation to homophobia that racism does to xenophobia, and sexism does to misogyny? You can invent till you're blue in the gums, but who will use or even learn it? Or do we continue to let "homophobia" be stretched thinly over a large semantic territory, and trust that usage will sort things out? But how do we avoid the kind of derailing mentioned above while we're waiting?
Where is Ludwig Zamenhof when you need him?