How do you handle Q and A?  The traditional way is to take the first 45 minutes of an hour to talk, and then stop, saying, "I'd be happy to answer your questions now." 

What happens?  You handle a few good questions, and then the session starts to run out of gas.  Finally, there's one dumb or irrelevant last question, and the hour is mercifully over. 

Think about it — the last thing the audience remembers hearing is therefore not your brilliant words, but that last dumb question. 

Instead, take questions at 20 minutes, and 40 minutes.  (Those timings happen to coincide with our natural attention span, which is about 20 minutes, give or take.)  Then, take one last question at 55 minutes and wrap up with the stirring words (on message) that you've been saving for the end.  Result?  Your listeners come away with your brilliant final thoughts echoing in their ears. 

Much more effective.  But how do you handle those questions at 20 minutes and 40?  What if somebody in the 3rd row raises a hand and asks a question that, you're afraid, will take you a little off subject and put finishing on time in jeopardy.  What do you do?

Start by remembering why you're there to give a speech.  Not to hear yourself talk.  You could give a speech in the privacy of your own bathroom for that.  The point of public speaking is to communicate with a group of people.  So you haven't succeeded in that endeavor unless someone has heard and understood you.

The audience is all-important.  And when you think of it like that, why wouldn't you take the time to answer the question? 

So don't worry too much about your agenda.  Do worry about how the speech is coming across, and what the audience is getting out of it.  If someone asks a question, answer it.  You should know your speech and your content so thoroughly that you can easily adjust on the fly to take into account your audience's feedback.

That said, you do have the right to sort through the questions and pass on the rude, the irrelevant, and the idiotic.  But never let on that you think a question is idiotic.  Just deal with it quickly and painlessly and move on. 

Back in my teaching days at Princeton University, I was showing a videotape of Martin Luther King's I have a dream speech as an example of great rhetoric, brilliantly delivered.  The discussion moved on to Patrick Henry's Give me liberty or give me death speech.  A student raised his hand and asked, "Do you have any videotape of Patrick Henry?" 

For a split second, I honestly didn't know what to say.  The guffaws of fellow students quickly tipped the hapless junior off, and he blushed bright red as he realized his error.  We moved on.  That student probably got a lifetime's education in a couple of seconds right then and there. 

There are stupid questions, and you don't have to answer them all.  But you are there for the audience, and mostly it's your job to respect their reactions to your talk and respond accordingly.