As a teacher, the most important is to reach your student and provide him the information he wants and requires in a way he can understand. You have to see how he likes to work and adjust to him accordingly.
However, I can give you general advice as well.
Time settings and at what point to explain
The teaching process should start prior to the game. Talk to your student before playing even a single move. Is there anything he wants to focus on? Or is he unsure what his weaknesses are? Keep the information gained here in mind.
It is possible, though pretty rare, to teach during the game: This is a matter of style, but I like to pause the game after important mistakes or turning points and explain them. Alternatively and more common, wait until the end before you start explaining. Then again, how do you actually define "end"? There is no need to play until resignation or scoring, you are also free to hold the game at any point and start the review (I never had a student who minded this practice).
If you're giving the student enough time and explain during the game, chances are you're forced to cut the game off at some point because it would take too long. Actually, I finish only a tiny fraction of my teaching games. Most of them are stopped in middle game or even earlier, but thanks to my teaching style explained later in this answer the student will still get told enough to make his head spin.
So, if possible, give him a lot of time. Blitz games are unsuitable for teaching. I like to play without time or with 5x60 byoyomi. If he lacks thinking time, you could not expect him to play his best moves and show his full strength. Teaching is not about stupid blunders, it is about subtle mistakes. On the other hand, if you only want to offer him a chance at playing a stronger player and possibly encounter and learn new shapes, a blitz game is fine, just tell him to review the game himself later.
How to explain
So, now that for whatever reason you're in explanation mode:
First, let's talk about situations that came up in the actual game. During the game you should have noticed what exactly the student did wrong in this situation. Did he misread? That's easy, show him the right way, possibly showing the the tesuji he missed. Did he play bad shape? Show him the right shape and how it is superior. Did he play a slow move, or a too wide extension? Explain the reasons and show him the right move.
Next, there are theory questions the student may be curious about, either because they relate to the game or because he's just interested. What is the idea of the Chinese opening? What is its intention in comparison to sanrensei? Show him examples, tell him about the basic characteristics. What's the joseki move in this corner shape? Or earlier, what is the right joseki to chose on this board? Show him the right moves, show him a few common mistakes and explain why they are bad.
The theoretical knowledge required is probably present when you're used to giving and receiving teaching games. However, I met high dan players that had basically no clue about fuseki (they compensated by insanely strong shape sense and reading ability). In case that applies to you, grabbing a good book on the subject will certainly help very efficiently (and make you a even stronger, too).
Go for the root
One particular thing I love to do, but which requires the student to cooperate, is asking him questions. "Look at your move. Is it a good move?". Don't just show them, make them show you! Many teachers disregard this point and simply show the correct moves, but I believe it is rather crucial to remove the roots of the wrong moves.
Make him think about the move, make him critically question his motives and intentions. If he comes up with the right answer, splendid. If not, even better, you just found a flaw in his reasoning. Those mistakes are the most important, and correcting them is the best thing a teacher can possibly achieve.
Note that you can do this for both wrong and correct moves. Is he sure that his correct move is correct? Or was he unsure? You're basically trying to make him be confident about any move he plays. He should be able to tell you his reasons for any move, no matter wrong or right. "I don't know why", "I played there because it's joseki", "I was vaguely afraid so I added a move just to be sure" are typical examples of answers you should focus on and make him understand the situation better.
You should let him know in advance that you'll ask trap questions to avoid confusion. "Do you really think your move here is correct?" - "Yes I am!" - "Excellent, you're right!" will occur often in the review.
This style of review takes a lot of time, but I strongly believe it is the right way to teach in the West since players won't be able to study for themselves as much as they can in the East with tons of strong players and knowledge around.
Note though that you need to take care not to demand too much of your students. If you notice that they reached their limits of understanding, e.g. because they simply cannot read a tsumego well enough, it would be counter-productive to force them to try. In their mind, the situation is simply too complicated, and rightfully so. You should simply move on, maybe a few moves later the situation became easier to grasp, and they can evaluate it more accurately.
I mentioned that I like to play without time. This fits perfectly in this scheme: If he's been thinking for more than a minute, ask him: What are you contemplating? What moves are you considering? What are your intentions (life, get sente, get a vital point, find largest point on board, whatever)? Do you understand the situation (you're dead if you don't play the right move - are you thinking about this or mistakenly about something else)?
Make sure to apply this to the white moves as well. Especially ask him about your mistakes in case you noticed them. Give him confidence to try and find weaknesses in your moves, make him build up kiai.
If you're unsure at any point, say so. Never try to convince him of anything without understanding the situation yourself (understand it at least enough to be able to convincingly explain it to someone of your strength).
It is very useful to have some example (pro) games in mind that highlight the situation you're focusing on. If you have trouble memorizing pro fuseki (like I do), instead try to come up with reasonable similar situations. This helps a lot to understand why certain joseki suck on certain boards and are great on a slightly different board, especially with weaker students that lack the visualization ability yet.
And obviously, knowing the standard content of teaching games is most useful in any situation. Tell your student about the table shape, tell him about the L+2 shape in the corner, tell him about monkey jumps and the wall+1 rule. This stuff can be extracted from books or learned from good teachers.
When I was a kyu player and even now, I got regular lessons by a great teacher who, interestingly, isn't "strong" in the classic sense (low-mid dan). In practice that never mattered and thanks to his particular style I learned very quickly and always found his lessons most interesting and understandable. The teaching style described in this answer is a continuation of his ideas.