Interesting thoughts. I agree 100% about the necessity of knowing the basics about number systems, because you just can't avoid coming across that when programming, and understanding it helps. But yes, the JPEG standard always involves the DCT-2 transform, and that's something not normally taught in schools here in UK (although I appreciate the situation may be different on the continent), so it's down to what the hobbyist is trying to achieve. In addition, some of the graphics stuff from SVG to 3D polygons require a high level of competence in Euclidean geometry. I take the view that when discussing what's advisable for a complete newbie to programming, although it's "safe" to just dispense the tough answers and say: "learn for yourself what I was taught", it's nice to bear in mind that not everyone has a higher education background in maths/physics, and it's easy to put people off this exciting hobby for life.
I just want to be nice and help people. A really nice forum for people thinking about starting off in game programming is:
http://www.gamedev.net/community/forums