The Curse of Dimensionality

by Kim Pederson…….

Sometimes you run across a word or phrase that just cries out for attention. This happened recently as I edited a paper on how used-good markets affect digital game sales. As part of the discourse, the author mentioned “the curse of dimensionality” in reference to analyzing the data collected. On the scientific side, the term refers to, well, I’ll let Wikipedia explain it:

The curse of dimensionality refers to various phenomena that arise when analyzing and organizing data in high-dimensional spaces(often with hundreds or thousands of dimensions)…when the dimensionality increases, the volume of the space increases so fast that the available data become sparse…[which is] problematic for any method that requires statistical significance.

And I will stop right there because I can see your eyes glazing over but more so because the phrase has me thinking about unstatistical things. As you may remember from high school physics (or Carl Sagan or Neil deGrasse Tyson), we view and relate to our world in four dimensions. Three of them involve space (length and width [2D]) and depth or volume [3D]) and the fourth is time [4D].

8-cell-simple
I’ve used this before but who can resist it? Don’t stare too long. I did and forgot what my name is, where I was, and why for three days.*

There’s all sorts of math that “explains” these concepts but that’s not important (or comprehensible) to me. No, the definitions of “dimension” that interest me more are these: 1) “a lifelike or realistic quality,” and 2) “one of the elements or factors making up a complete personality or entity.” Physically and psychologically, humans are complex organisms, multidimensional as it were. (Finally a chance to use one of my favorite British expressions!)

The problem in politics (yes, we’re back to that again) is that candidates fear and struggle mightily to avoid their particular “curse of dimensionality.” If asked, they would deny this vehemently, of course, but they all do it. This is clearly evidenced by those (still) running for president here in the United States. These individuals are likely as complex and multifaceted in their ideas and beliefs as anyone but you would never know it from the way they present themselves. They could campaign just as successfully (and get much more rest) by having a lifesize cardboard cutout of themselves stand in at all of their events. The cutouts would have to be modified a bit, sprouting dialog balloons that mouth all pertinent talking points for that person, but that’s it. What more do we need to see (and not hear)?

And finally it is easy to understand why they behave this way, probably in many cases (one hopes) against their better nature. As Meghan Trainor might put it, “It’s all about the base.” And the base, which you might know from Merriam-Webster, is “the bottom or lowest part of something.”

*3D projection of a tesseract undergoing a simple (ha!) rotation in four-dimensional space. “8-cell-simple” by JasonHise at English Wikipedia – Transferred from en.wikipedia to Commons.. Licensed under Public Domain via Commons.

2 thoughts on “The Curse of Dimensionality

  1. Kim, Perhaps the best example of what you so beautifully talk about here happened recently with that greatest of all the greatest of the great Americans in the greatest country of all time, Donald Trump. Did you see him meeting with about 20 or so evangelical leaders? Did you see when they all did their touching thing and one of them put his hand on Donald’s forehead as they lowered their eyes in prayer … for about a minute! And Donald, getting down with it all. If he were allowed to be a Republican candidate with a mind of his own, he’d be saying, “fellas, I’m really not too religious, I mean, y’know, God and all that, y’know, you’re all nice guys, but, can you believe it, y’know, church and all that, I can’t remember the last time, maybe my communion, c’mon … “

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.