And it can be. There's an expression in Yiddish, "goyishe kop," that literally means "gentile head," but should be translated as "moron head," or "big dummy head," or something similarly childish. As far as I can tell, the phrase isn't an especially mean-spirited one -- when Jews want to call someone else an idiot, they have far more scabrous Yiddish words they can turn to.
But goy mostly just means "gentile," because when you're a Jew, you need a word to describe people who aren't Jews. The origin of the word is innocuous enough: It's the Hebrew word for nation. In Isaiah 2:4, when the prophet declares that "Nation will not lift up sword against nation, And never again will they learn war," it's goys he's talking about. That's the word he uses.
Some people still rankle. I presume it is similar to the fact that people nowadays respond badly to the word "cis," which is used as the opposite of "trans" when referring to gender. There's nothing per se wrong with the word -- it's a perfectly honorable Latin prefix meaning "on the same side of." It works well enough of the opposite of "trans," which, after all, means "across or beyond."
But people don't like it. As I recall, people didn't like the word "straight" when it was first used as another word for heterosexual either. They also didn't like the word "gay," as it suggested that people who were not gay were somehow sad.
I suppose it's just that the people who are the majority don't like it when people who are not start naming things. Most of the time, when a Jew uses the word "goy," it's innocuous, but it can be a bit irritating to be reminded that there are other cultures out there, and they consider you an outsider, and they have their own name for you, and they never even bothered to ask you if you like it.
"Goy" is often used with disinterest. There are gentiles in the world, yes, but we need not think about them too hard. This was dramatized in 2009: There's a long segment in "A Serious Man," the Coen Brothers film set in my childhood neighborhood of St. Louis Park. The segment is called "The Goy's Teeth" and tells the story of a dentist who discovers Hebrew writing carved into the back of one of his patient's incisors. The teeth seem to be communicating a message, but the meaning is obscure, and the dentist starts to have a nervous breakdown, not knowing why the message appeared or what it meant. Finally, he decides he can't ever know and just drops the subject altogether. "But what about the goy?" the film's main character asks his rabbi. "What happened to the goy?"
"The goy?" the rabbi responds. "Who cares?
Yiddish actually does have some insulting words that mean "non-Jews." A "shaygetz" is a non-Jewish man, and it also comes from Hebrew, but not from a neutral word, but instead from "sheketz," which means "blemish" or "abomination." There's a female form of "shaygetz" as well, which you've probably heard: "shiksa." Both can be used mildly -- sometimes a shiksa is even an object of desire -- but the word itself is a pretty unkind one. At best, shaygetz means "rascal," and one supposed a Jewish man might tousle the hair of a non-Jewish friend and call him a rascal, but even that seems a bit condescending.
Yiddish has a feminine form of goy too: "Goye." This is never meant disparagingly, except by people for whom anything non-Jewish is frowned upon. In the terrific Joan Micklin Silver movie "Hester Street," Carole Kane keeps referring to non-Jewish women as "goyes," and you can tell she doesn't think too highly of them.
But Jews need goys, especially religious Jews. For them, the Sabbath is mostly a day of not being able to do the sorts of things you sort of need to do. You can't turn lights on and off, for example. And before it was possible to get timers to turn the lights on and off for you, you got yourself a shabbos goy.
This was often a kid who lived in the neighborhood who would swing by the house and help out, and you'd throw him a little money, but it could also be somebody who was on hand to help out anyway -- if you lived in a high rise with a door man, he might swing by to turn your oven off, or whatever was needed. Nobody has ever used the word shabbos goy as an insult, and a lot of famous people have helped out their neighbors by working as a shabbos goy.
Elvis Presley was a shabbos goy. He helped out a local rabbi named Fruchter who happened to live in the upstairs apartment in Memphis. Interestingly, technically Elvis was Jewish -- his mother's mother was Jewish. In Judaism, if your mother is Jewish, or your mother's mother, you're Jewish. He certainly didn't consider himself to be a Jew, but the rabbis don't really take Elvis' opinion into account, so every time Elvis turned on his lights on Saturday he was acting as his own shabbos goy.
Here are a few quotes that show how the word "goy" is used in context:
Lenny Bruce: "Kool-Aid is goyish. All Drake’s cakes are goyish. Pumpernickel is Jewish, and, as you know, white bread is very goyish. Instant potatoes–goyish. Black cherry soda’s very Jewish. Macaroons are very Jewish–very Jewish cake. Fruit salad is Jewish. Lime jello is goyish. Lime soda is very goyish."
Schmoozing: The Private Conversations of American Jews, Joshua Halberstam: "Myron converts to Christianity. Next morning he gets up, dons his prayer shawl and phylacteries, and begins to recite the traditional morning prayers. His baffled wife reminds him, 'Myron, Don't you remember that last night you converted to Christianity?' Myron slaps his forehead in dismay, 'Goyishe kop.'"
Three for the Price of One, Anna Tzelniker: "There is an old saying among Jewish actors 'If the "Goy" says it's Kosher, the Jew will eat it,' meaning that a Jewish actor becomes greater in the eyes of his Yiddish audience only after he has been acclaimed by Gentiles."