When playing on consoles I always have the game music playing. I tried listening to music while playing, but it just felt odd.
As for handhelds, iOS or PC gaming, I do sometimes have music playing. These are usually games like Pokemon (with lots of grinding involved), in which music can help break the repetitiveness, or online PC games, in which there aren't any cutscenes or anything that would be ruined by music.
As a general rule, RPGs are a no-go for music.