I faced your problem too before. But now my listening skill is fine. I will share you my experiences
1. Listen from simple topics to hard topics
2. At first, you listen without seeing tapescript. Try go answer all questions. After that, open the tapescript then listen again. Check your answer again.
3. You should repeat the talk or dialog until you fully understand.
4. Don't listen to one or two topics. You should listen to various topics with different readers.
5. Listen to English MP3 any where any time. Remember, the more time you listen to English, the more kills you will get.
Those are my experiences.
Good luck!