Analogies between the measurable world and pure math are wonderfully useful, but the abstract “game” of mathematics is not the same as the measurable world.
			When we represent the world, abstract math and our measurement procedures and physical laws aren’t the same.  Implicitly, there has to be an interface, and if we’ve had troubles at that interface, it is fair game to define that interface and what goes on in arithmetical representations there.
		We can do so by checking for consistency at the level of symbols and at the level of physical experiment.