A physical analogy for the creative thought process

As a software engineer, I tend to be very methodical in my thinking. I approach every problem by breaking it down into discrete pieces, analyzing each one in a logical fashion, and following a series of coherent steps to a solution. It's easy for me to picture my conscious, problem-solving mind as a some sort of geared device, a Rube Goldberg machine, or even a computer applying an algorithm.

But what about creative thinking? What about moments of inspiration? You’re sitting on the can and WHAM, out of nowhere, an idea hits you. There was no conscious thought process, no gears turning, nothing visible happening. It's as if creativity happens behind a curtain and when the process completes, your unconscious hands you an answer. What physical analogy could explain what happens behind this "curtain"?

When it comes to creativity, the analogy of gears and machines seems largely unsatisfying, failing to provide an explanation for both (a) the random and unpredictable aspects of creative thinking, such as how answers and ideas can just appear at any time or place and (b) the more deterministic trends, such as some people consistently being more creative than others and some situations fostering more creativity than others. Rather appropriately, a possible solution to this problem came to me in a moment of inspiration: seemingly out of nowhere, I started to think about coin flips, index cards and an interesting analogy came to mind.

Creative thinking as a deck of index cards

Imagine that you have a deck of index cards and that each card has a word or phrase on it. These cards represent the ideas and thoughts that are floating in your head. The creative thought process can then be seen as:

1. Shuffle the index cards.
2. Drop them on the floor. 
3. Scan over the cards and see what sentences have formed from the random arrangement of words.

 Most of the time, the random permutations of words will be meaningless. You just have to pick up the cards and return to step 1. However, every now and then, a meaningful sentence or thought will emerge. Sometimes this will be a full solution to a problem - the "aha!" moment; other times, this will be a mere stepping stone from which you gather enough info to add or remove index cards from your deck before returning to step 1.

What I like about this analogy

1. It cleanly explains the unpredictability of the creative thought process without resorting to any magical or divine reasoning. Sometimes the cards fall the way you want, sometimes they don't.
2. The particular choice of cards in your deck and the way you adjust them with each iteration is a great model for why your creativity may "click" for some problems and not for others. For example, consider "linguistic equations" or "DITLOIDS" style puzzles (warning: these links are highly addictive). Most people are able to solve a few very quickly, but get stuck on the rest for a long time. This could be seen as the natural outcome of which index cards you pull into your deck as you read each problem. For example, when you see the number 26, there are very few concepts - and consequently index cards - that you would associate with that number. The size of the deck is small and it only takes a few trials before a meaningful answer emerges. On the other hand, a number like 3 has MANY possibilities, resulting in many more cards in the deck and consequently many more iterations before you stumble upon the proper permutation. 
3. We can also explain why, even when faced with the same types of problems, you creativity may fluctuate by considering the possibility that not all the possible "index cards" of your mind are equally accessible. Depending on what you've been doing and thinking about, some words/phrases will be top of mind and readily pulled into your deck; other cards will represent concepts you haven't considered in a long time, stashed in the deep recesses of your memory, and will rarely get pulled into a deck.
4. I've always find that increased constraints lead to increased creativity. If I'm trying to solve too big or vague of a problem, I actually find it harder to come up with ideas than when faced with a smaller and more clearly defined problem. Again, the card analogy works here: broad and vague problems would be pulling in cards from all over your mind. These would represent totally unrelated thoughts and ideas and result in permutations that are completely meaningless. On the other hand, more constrained problems would result in pulling in cards that are closely related and consequently, much more likely to form coherent thoughts when they land next to each other.
5. Finally, the index card system has several tuning points that could account for the creativity differences between people. This could result from varying values in the total number of cards available, the number of iterations per unit time the person can do, the ability to scan cards after each iteration and recognize that a permutation has meaning or value, the skill of picking which cards to put into the deck initially and the ability to adjust the cards in the deck after each iteration.

Vegas?

If this analogy were true, it would suggest that the brain uses a Las Vegas Algorithm for creative thinking. This may also help explain why brainstorming sessions can be an effective way to get creativity going: they are a conscious way to pull in ideas and try all sorts of permutations until, hopefully, a useful one emerges. Einstein may have refused the idea that God would play dice, but perhaps, behind the curtain of our unconscious, we are just playing cards.

I finally understand open source software

What does Google stand to gain from having so many open source projects? What about Twitter or Facebook? Why would companies freely give away software that cost them time, money and may help their competitors? Why is Github growing at an absurd rate, with over 2 million repositories? Why are developers world-wide giving their time and work away for free?

I've used a TON of open source software (e.g. see the "what technologies were used section" of the Resume Builder) and am a very strong believer in using open libraries and standards whenever possible. However, until just recently, the full motivation behind open source software - why so many individuals and companies contribute - never really clicked in my head. As soon as it did, I created my first open source Github project.

I realized that open source isn't about doing the world a favor, sharing, or acting charitable. It's not about freedom, choice, human rights, standardization, or any of that. Sure, all of these play a role, but none of them are enough to explain how the open source movement got to where it is today. What I think really drives open source are three major benefits to the project creator: free labor, cleaner code and portfolios.

Free labor

The benefits of open source software to an end-user are obvious: you get to use amazing libraries, operating systems, standards, and tools, for free. You can take advantage of projects that have been built and tested by hundreds or thousands of developers, learn from the source code, customize it for your needs and build bigger, better things in less time. You get to stand on the shoulders of giants.

What wasn't as obvious to me was just how much the project owner benefited from me using it. Every time I ran the code, found a bug, or tried out a benchmark, I was performing QA and performance testing - for free. Every time I asked questions online or posted a tutorial, I was writing documentation - for free. Every time I used the project in my codebase and told others about it, I was advertising the project - for free. If I created a patch, or added a new feature, or made suggestions for improvements, I was helping to design and develop the project - all for free.

In other words, the open source community using your projects is, quite literary, a totally free and incredibly effective workforce. Google open sourcing snappy may help everyone in the community do fast compression, but if they can get enough people interested in the project, it helps Google even more when that community finds bugs, fixes them, builds new features and contributes it all back to snappy. The cost of hiring a few hundred developers and QA to work on a project like snappy would be prohibitively high, even for a big company; for a lone developer, totally impossible. But open source it, and you get a huge pool of labor for free.

Cleaner code

It turns out that knowing that other people will scrutinize your code, tear apart your design, and use it in ways that you didn't expect is a superb motivation to keep things clean. The very act of taking some code and making it a "project" will encourage you to make things more modular and reusable, write documentation, use source control, track bugs, all the good stuff. It's just human nature to clean the apartment more for guests than yourself; as such, open source projects tend to be cleaner than proprietary ones.

Portfolios

Open source projects are the best portfolio a software developer or company can have. It's hard to learn much from just seeing the end product (if it's even publicly visible); interviews are sadly not too revealing either (a topic for another blog post); resumes and "about me" pages are all but useless. But when I can see every line of code, the design decisions, and the technologies involved, I can get a very good idea of the type of person or company I'm dealing with. It's the ultimate branding play: show, don't tell.

When it comes to hiring, I'll take a Github commit log over a resume any day. - John Resig

I'm a believer

I've been an open source end user for a long time. It's about time I actively start contributing. Not because it's good for the world or because I want to better humanity - it is, and I do, but that hasn't been enough motivation before. No, I'm going to contribute to open source because I finally see how it'll directly benefit me. No reason I can't be selfish and save the world at the same time.

This is where I work

LinkedIn just had a "pimp your row" competition where each row of cubes got money to spend on decorations. The results were astounding. As I walked around, I had to remind myself this was an office. This is where I work.

The Platform Team

Core team

The LinkedIn Zoo

Gotta love the aquarium

Funneling red bull?

Corporate IT totally blew me away

They turned their office space into a night club

With beer on tap

Awesome shirts

Crazy lights

The photos do it no justice, so definitely check out this video

Our university recruiting team, making college students feel right at home

Content and community

Enterprise hiring solutions

Hooka bar

(My apologies for the quality of the photos: I only had my iPhone on me).

Java's String.split()

Just ran into a "gotcha" in Java that had me scratching my head. I had a string of the form "abc.def.ghi" and I wanted to split it on the dot (".") character. So I had the following code:

String test = "abc.def.ghi";
String[] parts = test.split(".");

Pop quiz: what does the parts array contain?

• Did you say {"abc", "def", "ghi"}? Sorry, no. 
• Maybe {"abc.def.ghi"}? Nope. 
• "Oh, I got it!" you think, and happily announce the solution is {"a", "b", "c", "d", "e", "f", "g", "h", "i"}. Bad news, friend: you are still wrong.

The real solution? parts = {}. That's right, parts is an empty array.

Huh?

Let's start by looking at the API Docs for String's split function. The first thing to realize is that the parameter it takes is a regular expression, and in regex syntax, the dot matches any non-whitespace character. So instead of matching the dots like I wanted, I was matching everything in the string. To actually match a dot character, I needed to escape it with backslashes: test.split("\\."). 

That's a simple enough error, but why would it result in an empty array? If the dot matches every character, shouldn't I have gotten an array where each character is a separate entry? That certainly feels like the right answer for anyone used to matching things with regex, but we have to remember the regex plays a slightly different role in the split function: it's the delimiter. It's the value on which to split the String and is NOT included in the resulting array. 

So my delightful code snippet ended up matching every character in the input ... and then throwing it away. A fun example of simple code that looks right, but isn't.