Let this be known: I have never seen run-of-the-mill guys in their undershirts, standing under street-lamps, scratching themselves and muttering: "World don't matter! World don't matter! World don't matter!". Should this behavior begin to occur, we are finished as a species - because only the world ultimately matters.
Those (loudly) issuing screw-ball claims: who (1) deny what exists in the universe (non-human mate preferences for sex traits and fundamental, physical constants) - yet who (2) affirm that imaginary experiments exist and that thermostats "have representational states" register a uniquely high frequency of distribution in this nauseating barf-cult. The "world-don't-matter ideas" - constantly on parade here - could not have been entertained (for very long) by our ancient ancestors - due to the presence of environmental feedback correcting them, their ideas and their behaviors in their own habitats.
Suppose that - 150K years ago - one of this cult's "world-don't-matter ideas" arose in a human. If a lion crouched behind a tree and an ancient human closed in on, pointing at the lion - screaming: "You lion with your sharp canine teeth do not matter!", there would have been an enormous probability that he would have gone extinct. The conditions (the lion's jaws) in the environment would have corrected (ate) him. Burp!
What we seem to be drifting-into is the fundamental taxonomy of humans: those adapted-to-reality and those not-adapted-to-reality. Most ancient and modern humans fall-into the former class. Some politicians, some religious kooks and all the celebrants in this Myspace, dislocation-of-the-mind cult fall-into the latter.