No, thats the problem. It is in one way or another associated with your head set/compression bolt. It sound like it may not be installed correctly. There should be NO wobble and the steer tube should rotate freely. If its not destroyed allready it will be soon so stop riding it untill you can get this resolved. A bike shop will probably tune it up for about $10.