I have no idea what might cause those problems with the refreshrates. Another possible fix would be to use a program like powerstrip, which in theory, can force a screen to a certain resolution/refresh rate. It dosn't work for me though, but it might for you.
Concerning Dosbox however, your PC should be able to any game with it pretty fine. Did you try adjusting the clock cycles with ctrl+F11/F12?
btw. newest Direct X is 9.0c, you might want to update that for newer games.