I can't stop thinking about algorithms. I am obsessed, and I want to tell you why.
Let's be clear: I am not a data scientist. I am a guy who finds technology and applications of technology fascinating. I am not writing this for technology nerds. I am writing this for professionals who want a working knowledge of technology.
If you are reading this, then you understand computers. A computer is nothing more than rules programmed by a human. Those rules are then executed and create an output.
But algorithms are so much more; they are breathtaking. An algorithm is a computer writing its own rules and then creating output from those rules.
It's easy to focus on the scary part of algorithms. In the Avengers movie, a super algorithm results in a machine - Ultron - bent on destroying the world. I will leave those scenarios to the Elon Musks of the world.
Algorithms can do so much good
Think about any repetitive task you do. An algorithm can be created to solve that task. Some algorithms are used for fun. For example, Facebook uses algorithms to suggest friends for you to connect with. Google Photos uses an algorithm to identify faces and group pictures of the same person together (which can lead to terrible results).
Algorithms are already being used in the insurance industry. Take a look at CoverHound or PolicyGenius; the algorithms behind these applications quote personal lines of insurance based on your needs.
How algorithms work (and why they are awesome)
Again, I am not a data scientist, but here is my simple explanation of how most if not all algorithms are created:
1. Create a seed set.
First, you identify a seed set, which is the core learning that is taught to the algorithm. Yes, that's right, even a computer algorithm has to be taught something from a human! For example, with the Facebook algorithm, I'm almost certain that the algorithm was first fed a giant spreadsheet that contained information about individuals and how they were connected (you do know your data created Facebook, Google and every other big data company you can think of, right?).
2. Feed the seed set to the algorithm.
The algorithm then reads all of the information it is fed and starts making its own rules. For example, the Facebook algorithm may determine: "Oh, I see, Jimmy likes Teenage Mutant Ninja Turtles, and he is connected with Bobby from the same city, and Bobby also likes Teenage Mutant Ninja Turtles. I bet Jimmy also knows Steve from the same city who also has a love for Donatello. They should connect."
3. A human reviews the results.
A human (see, you are still needed!) then reviews the output of the application of the algorithm rules. In the Facebook example, a human might determine if Jimmy and Steve should actually connect on Facebook. Maybe they are part of rival gangs, and the algorithm didn't recognize this. The human would then add this data to the spreadsheet and feed it back to the algorithm.
4. The algorithm rules are improved based on new input.
The algorithm creates rules to account for the new information. "Don't connect rival gang members even if they live in the same city and like the Teenage Mutant Ninja Turtles."
5. Steps three and four continue indefinitely.
Now stop for a second and think about all the rules that are built up in your head about people you connect with. Maybe you prefer to hang out with people who brew beer or read Harry Potter. There are literally hundreds of millions of personal preferences that human beings use to associate with people.
What if you could store all of those preferences and use them to connect people?
That's Facebook.
Algorithms are good for insurance workers
Now think about your work and all the stuff you know and all of the stuff your colleagues know. What if all of that information could be fed into an algorithm and used to create rules. You could then use those rules to more quickly do your work.
I can hear you thinking "But then I will be out of a job." Therein lies the rub, one that has been discussed ad nauseum (more than 9 million results for a Google search on "technology will destroy jobs"). Fatalists argue that algorithms and the advanced software programs they create will destroy jobs. Famous technologist and investor Marc Andreessen expressed as much when he proclaimed in 2011 that "software is eating the world."
But what happens if software starts doing repetitive tasks previously done by humans? I believe humans find new ways to be productive. And, I believe history supports my theory. But that's a blog post for another day.
I will leave you with two questions.
What repetitive tasks do you despise?
Wouldn't it be great if you could offload these tasks to a computer?
