Help with Windows Media Player?

For some reason, I can't tuly close WMP. When I press the X on, the music continues to play, and I have to open it again, and pause the music for it to stop, but WMP is still running when I exit out. How can I change this?