Does saying magic words make more sense? Does saying the magic words do anything by itself, And if yes, who needs to hear it? If a wizard says a spell in a forest and no one is around to hear it, does it make an explosion?
I can understand gestures, because that would help control an energy flow, or something.
The way Equestrian magic is described in the show, it relies heavily on emotion. So I guess a spell requires having to feel a certain emotion and picture a certain image, or for more complex spells several at the same time or in a set order. Maybe speaking words could help to invoke the emotion or image, but they are not necessary.