I usually just leave the disc in the tray until I get ready to play a different game or watch a movie. The way I see it, if you remove a disc from the disc tray every time you finish playing a game or watching a movie, it puts more wear and tear on the mechanisms that control the tray. If you can minimize the opening and closing, you can avoid wearing out your disc tray prematurely.
From experience, you should remove the game every time you power off the console (especially if you have your console standing upright). You stand a chance of "burning a ring" that makes a perfect circle scratch on your game. This happens if the system is bumped or moved while a dics is still in the system. Basically, if the disc is not seated perfectly in the tray and the system is powered on, the disc will spin unevenly. The last thing you want to do is ruin a $60 investment.
I always try to take them out, more so because I'm organized and like to have all of my games in cases but it's so easy to forget! I wasn't even aware that it could cause damage so I'm glad I know that now. Thankfully I've never had a problem with leaving a disc in there but I'll have to be more careful now since money doesn't grow on trees, sadly.
Having small kids, I try to take MY games out (Skyrim, Black Ops, etc), and leave theirs in (Lego anything). If they start the system without me, they won't fudge up my games, and if they do play, it's acceptable. Having said that, I did have my copy of Fallout 3 moved while running, scraping a ring in it, becoming unusable. I should probably start taking the games out altogether, putting them back inside their protective cases. One simple step, right?
My xbox is laying long ways, not upright, up on a shelf and it never gets moved so I don't worry too much about removing the games. I tend to play the same ones over and over so I guess I agree southpark about wearing out the mechanism that opens and closes the drawer.
I try not to do this with any of my devices. My leaving a game in my first Xbox 360 is what got it stuck in there. And no matter what we did, it wouldn't get out. So now, I have a new system of course. But when I'm done with a game, I take it out of the game system. Because if it happens again, I won't be buying a 3rd 360.
I always remove them, I'm not the only one using the 360 so I have to take the game out when I finish playing it. I would suggest you guys take it out if you can remember, it's a precaution to prevent scratches on the CD.