Imagine two companies with competing products.
One company develops their product by employing the brightest minds from the nation’s best universities. Millions are invested in marketing, advertising, and copyright protection. Through the selling of licenses of their product the company is able to fund their enterprise and make a handsome profit. Meanwhile a second company develops their product with minimal capital investment. They make the product’s design openly available and let anyone use and modify it. They do not charge for the use of their product. Its free.
What company should produce the best product? Obviously, if I am taking the time to write this, it must mean that the second company has counterintuitively produced the best product.
I spent most of my PHD years learning SPSS. The strength of SPSS is its ease of use. It is incredibly user friendly and is ideal for someone like me who has to work hard to wrap my head around things like standard deviations and alpha scores. But over the last few years, I have migrated over to R. R is, by contrast, not as user friendly as SPSS. But if one goes through the trouble of learning how to use the software, it becomes clear fairly early that it is far and away more powerful, more customizable, and more flexible than its counterpart. It is simply a higher quality software.
What makes R so interesting is not that it is better, but how and why it is better. Let me explain using the ideas of a few classical sociologists. Max Weber and Karl Marx represent two-thirds of the Holy Sociological Trinity (the other being Emile Durkheim). They formulated their ideas during a time when the social patterns of the world were changing rapidly, from rural and agrarian to urban and industrial – not coincidentally this was the time when sociology as a discipline was born. I’m going to cherry-pick a few of their ideas to help me explain why R is beating SPSS.
Weber’s Iron Cage
Max Weber argued that as large organizations became dominant in society, people find they cannot escape the de-individualizing and dehumanizing rules they inevitably oblige us to follow. In an effort to organize large bodies of people towards a common goal, organizations put numerous rules, regulations, and procedures in place. These rules are bars of a metaphorical iron cage, making sure people do exactly what they should do when they should do it.
As an example, consider people using public transportation. If only a few people come to the bus stop each day, then little is needed to organize the task of picking up bus passengers. Maybe just the stop and the time. People hop on and hop off just as they please. Problems can be handled at one central office by one administrator.
Now imagine if the city is large, and hundreds of thousands of people catch the bus each day. The bust stop is constantly crowded, with the maximum number of people jostling to get on the bus at each stop. As the task gets larger, the organization that is charged with completing this task also gets larger. That organization will need to put rules in place in order to prevent chaos and foster the efficient use of the bus:
1. There may need to be a sign or some type of physical structure in place to compel people to stand in a line
2. There may need to be a some rule as to how many bags or personal belonging someone can bring onto the bus
3. Passengers may be asked to seat themselves in the back of the bus first, making it easier for oncoming passengers to find a seat.
4. Because of so many people, noise, eating, and other rules of decorum will become evident
5. If you have a problem with rules 1 – 4, you must contact the local bus administrator between the hours 10:30 and 2:15.
6. If your problem with rules 1 – 4 are deemed worthy, then you can complete a form, and the county administrator will then hear your case at a future time.
These rules ignore the unique situations of each bus rider. Riders are not people per se, but abstract “units” that must be serviced. As an intended consequence, originality and spontaneity of thought are stifled. A rider who may have a better way of boarding the bus is discouraged because he is breaking a rule. A bus driver who may be able to help a customer in a unique way, will be stifled because he must follow rule 4 from section B. Any idea that is formulated in spite of the rules must still make it up a byzantine chain of command – with each level having its own set of stifling rules! There is little creative wiggle room in the iron cage.
Most people do not want to go to work everyday. Some people downright hate their work. Others find their jobs okay, but could do without it if they could help it. For many of us, we have no control over anything at our work – rules come down from above, we are told to push this key for that reason, pull that lever at this time, or say this phrase to that customer when they do so and so. The job has no meaning. Many of us handle the meaningless of work by shifting our attention to our coworkers and if we are lucky forming lasting bonds. Some of us handle this meaningless by resigning ourselves to our fates, doing only what we are supposed to do and bolting as soon as the iron cage is open (i.e. the boss says we can go).
But what rarely happens is that we try to do our jobs better. The only way we do our jobs better is when the company incentivizes better work by compensating us for our performance. Indeed the whole reason why we are there is money, without which we cannot survive. We exchange our hours, our energies, and our talents for dollars. We then exchange those dollars for the latest item on the menu, gadget, or piece of clothing.
Most of us could care less for our jobs. Paradoxically, as Marx notes, we like working! People are happy when they can choose their labor and then enjoy the fruits of it: picture the person who slaves over a stove all day and then relishes the meal afterwards, or the person who spends hours in the hot sun building something so they can then sit in their house and stare at it with admiration. Furthermore, as social beings, we like working with others. We are happy when we are interacting with friends and family. Marx calls the state of working in meaningless jobs – being disconnected from the work process and from friends and family – alienation.
Marx was wrong about a lot of things, but not this one. The natural and healthy state of human existence is to do work that you enjoy, do it with other people, and see the fruits of your labor. Most people simply do not like working in meaningless jobs, and only do so because they have to. Because we must be compelled to perform tasks in an unnatural state, we do not perform at our highest potential (unless money is thrown at us).
R in the Digital Environment
OK. So now I have to tie all of this together – R, SPSS, the Iron Cage, and Alienation.
IBM is a large organization that organizes all the tasks involved in producing SPSS – including licensing, distribution, marketing, and all other tasks I am not familiar with. In order for these tasks to be completed, rules must be put in place. ID cards are issued. Employee handbooks are printed. The ubiquitous organizational chart is memorized. And so the creativity stifling iron cage is built. Moreover, people may not love doing the work that IBM wants (although some will), but the carrot for doing that work is so large and sweet that it is hard to pass up. It makes alienation barely perceptible – but it is there. People work until the the whistle blows. Ideas are only introduced when bonuses are offered or termination is threatened (actually, I have no idea how it works…but it sounds accurate doesn’t it?)
If IBM’s competition was also another large industrial organization, or if this was a time before computer networking, then IBM would produce the better product by dangling bigger and sweeter carrots to capable employees and strangle R by spending more on marketing and advertising. But that is not the current state of affairs.
R works a bit differently. It is collaborative, peer-to-peer, and its production process is open. The “contributers” page for the R website reads:
“R was initially written by Robert Gentleman and Ross Ihaka—also known as ‘R & R’ of the Statistics Department of the University of Auckland. Since mid-1997 there has been a core group with write access to the R source, currently consisting of…”
And then one sees a list of 20 or so consistent contributers. There is no hierarchy here. Just people working together.
There are a lot more people who contribute – albeit not as consistently. This is because the process is open to all (no ID cards needed to help out here). People write add-ons to R, called packages, that increase the functionality of the core program. Code is written, reviewed, and rewritten in numerous online forums where coders get feedback. Moreover, countless online websites and communities have sprung up that provide assistance in learning R. For example, help manuals and information about packages can be found on the Comprehensive R Archive Network (CRAN). There is the R Google+ community where people post the latest information and links to latest releases. There is a website called R-Bloggers where people post the code they used to complete practical statistical tasks. And so on. All this hustle and bustle works with minimal organization and by people who do the work willingly. For no pay. After their day jobs. Through this collaborative and voluntary process, R currently sports 5755 available packages, tailored to whatever statistical task one has.
The Iron Cage and Alienation – Applying Some Sociological Ideas to Success in the Digital Environment
The R program for statistical computing is such a wellspring of creativity because it avoids the pitfalls of Weber’s iron cage and Marx’s alienation. It avoids these pitfalls because it leverages the properties of the digital environment. There is no need to have a top down bureaucratic structure propped up with countless rules. There are rules for sure, but minimal. And so people can contribute when they want and how they want. Weber noted that these rules make work more efficient, but stifle original thought and creativity. The production of R in the digital environment avoids this. People are willing to expend their creative energies for free because, as Marx pointed out, the natural state of affairs for people is to do work they like with others and see the fruits of their labor. They do not want to be alienated from the work process, and want to do meaningful work. Because we are connected by computer networks, the pool of potential contributers is global. Of the billions of networked computer users there will be hundreds of thousands of people who will find joy in coding just for the fun of it.