Unity in ActIon
A summary of the book: “The Master Algorithm” by Pedro Domingos
Written by: Cameron Hernandez
The Problem
Society has a knack of segregating the masses – making distinctions about individuals and placing them in boxes labeled with respect to who they are expected to be. Such segregation leaves little room for variety and diversity. For example, in the government most politicians identify as either Democrat or Republican. Yes, there are many other parties, but the majority of politicians pick one of the two aforementioned. Once a side is selected, an individual is expected to think and act a certain way, in unity with the selected party – no matter what. This has resulted in votes against competing parties, not necessarily because an individual disagreed with what was being voted on, but simply because agreeing would mean siding with the “enemy.” In fact, according to a poll conducted by Pew Research Center, negative views of opposing parties accounted for over 70% of why people join an opposing party.
The same, sadly, has been found true of race, religion, worldview, geographical location and even your favorite sports team. Rarely do we stop and think that perhaps there is a gray area – a common and rational intersection – between competing patterns of thought which is logical, feasible and simply more efficient. Planet Earth contains the smartest and most capable human beings alive. Yet, sadly, we find our smartest and brightest allies tirelessly arguing and debating at podiums around the globe rather than joining forces to solve the world’s most complex and convoluted issues. Bottom line – humanity works better when we work together as opposed to, well, opposed to each other. This is where Pedro Domingos, in his book The Master Algorithm, takes his stand – with the crazy hunch that maybe if we all got along we could make the world a much better place.
The Question
Every discipline has a “Holy Grail.” For musicians, this is discovering a chord never heard before. For architects, this is building the tallest or most impressive structure ever built. For Machine Learning Engineers this is developing the Master Algorithm. However, the field of machine learning (ML) is comprised of many different approaches and methods, and the leaders of each of these methods are confident that their method is the key to the Master Algorithm. Domingos, in his book, highlights the five leading ML “tribes,” representing the five dominating schools of thought and their respective core algorithms – symbolists (inverse deduction), connectionists (backpropagation), evolutionaries (genetic programming), Bayesians (probabilistic inference) and analogizers (support vector machine).
It was at this junction that I began to question – why the labels? It is understandable to draw a dividing line between varying methods handling separate use cases. After all, it’s nonsensical to categorize competing frameworks as equivalent. Oftentimes, in any field, there are various approaches which are all valid in their own right, but vary from each other based primarily on their individual strengths and weaknesses. However, this distinction is more than separating methodologies – this is separating researchers, professors and dreamers alike. Though a technique is labeled, its users can learn to appreciate competing techniques which circumvent its shortcomings and weaknesses. Labeled techniques can get along quite well. Labeled humans, on the other hand, find it nearly impossible to do so. Once the line is drawn, those on the other side appear nothing like them – aliens in a common land. In other words, you can either draw a line in paper, or on the floor. For the sake of our future, I beseech you to adopt the former.
The Solution
Pedro Domingos, despite not necessarily agreeing with all of the methods prescribed by each ML “tribe,” understands that if there’s a Master Algorithm to be discovered it would take the collective wisdom of the most decorated ML engineers and researchers on the planet to realize it. In his book, Domingos provides a helpful example introducing the need for unity amongst ML “tribes.” He explains the need for each methodology in the fight against cancer:
“Symbolists know how to combine … knowledge with data from DNA sequencers, gene expression microarrays, and so on, to produce results that you couldn’t get with either alone.
But the knowledge we obtain by inverse deduction [from the symbolists] is purely qualitative; We need to learn not just who interacts with whom, but how much, and backpropagation [from the connectionists] can do that.
Nevertheless, both … would be lost in space without some basic structure on which to hang the interactions and parameters they find, and genetic programming [from the evolutionaries] can discover it …
But in reality the information we have is always very incomplete, and even incorrect in places; we need to make headway despite of that, and that’s what probabilistic inference [from the bayesians] is for.
In the hardest cases, the patient’s cancer looks very different from previous ones, and all our learned knowledge fails. Similarity-based algorithms can save the day by seeing analogies [from the analogizers] between superficially very different situations, zeroing in on their essential similarities and ignoring the rest.
Pedro Domingos, The Master Algorithm, pages 53-54
The solution is cooperation on an intimate level – not just joining forces, but intertwining opposing approaches such that all associated fragility is compensated by the collective strengths of a dedicated community of research. Got data you don’t know what to do with? No problem… Symbolists can help turn that data into symbols used in the same manner as puzzle pieces – filling in the blanks one-by-one and learning as they progress. Got a finished puzzle but you don’t know what it quantitatively means? Never fear… Connectionists can take the puzzle and explain exactly what you’re looking at with the precision necessary for it to be proved useful, in addition to pointing out which puzzle pieces tell more important parts of the story. Got an understanding of what a finished puzzle means but then realize that there are infinitely many puzzles to understand? Look no further… Bayesians can provide increasingly-accurate probabilities of being confronted with any common puzzle in existence. Got the likelihood of seeing various puzzles but still don’t feel prepared for every possible case? You’re in good hands… Analogizers can help spot similarities between common and never-before-seen puzzle instances and use what is known to develop an educated, precise and efficient result for any given scenario, ignoring all true unknowns (which should be few).
The Struggle
Now all we have to do is put differences aside, shake hands, share research and make progress! Insert awkward pause here… You might not have realized it by just reading through, but that pause lasted approximately five and a half minutes. As mentioned before, when lines are drawn and people are segregated into “tribes,” we find ourselves on unstable ground. Domingos makes it rather clear in his book – competing viewpoints don’t get along nicely. Just consider the statements below.
“Connectionists, in particular, are highly critical of symbolist learning. According to them, concepts you can define with logical rules are only the tip of the iceberg … You can’t just build a disembodied automated scientist and hope he’ll do something meaningful – you have to first endow him with something like a real brain … By reverse engineering the [brain].”
Pedro Domingos, The Master Algorithm, page 91
“Some connectionists have been overheard claiming that backprop is the Master Algorithm and we just need to scale it up. But symbolists pour scorn on this notion”
Pedro Domingos, The Master Algorithm, page 118
“Evolutionaries and connectionists have something important in common: they both design learning algorithms inspired by nature. But then they part ways.
Pedro Domingos, The Master Algorithm, page 137
“Symbolists don’t like probabilities and tell jokes like “How many Bayesians does it take to change a lightbulb? They’re not sure. Come to think of it, they’re not sure the lightbulb is burned out.”
Pedro Domingos, The Master Algorithm, page 173
“This is an instance of a tension that runs throughout much of science and philosophy: the split between descriptive and normative theories, between ‘this is how it is’ and ‘this is how it should be.'”
Pedro Domingos, The Master Algorithm, page 141
Time and time again, we see Domingos put the tension between “tribes” on display. However, there is a consistent theme – the weaknesses of each “tribe” can be circumvented by the strengths of others. The keyword, of course, is “can.” There is an unspoken conditional statement in that last sentence. If ML engineers and researchers from each “tribe” refuse to cooperate with each other, then their algorithms are destined for a lifetime of solving narrow AI problems.
Come Together… Right Now
Beatles reference intended, for the sake of the future of ML and the hunt for the Master Algorithm, it is time for all ML “tribes” to put down their proverbial fists and let go of their hubris. Only through recognizing that no solitary method is superior or capable of Artificial General Intelligence (AGI); and only through admitting that a combination of methods would be the most rational means of advancing ML ever so closer to the elusive Master Algorithm – only then will paradigm-shifting progress be made. Apart from teamwork, we may have more successes like self-driving vehicles, Jeopardy-playing robots, chess-champion robots, etc., but I wouldn’t be expecting a single algorithm capable of all of these things and more.
The question then becomes, do we need a Master Algorithm? Perhaps the world can live happily, and humanity just as efficiently, without an algorithm doing most of, or even all of the thinking and working for us. Maybe a series of single-process narrow AI algorithms can, working in parallel, achieve all truly essential procedures and operations in everyday life; or maybe we simply can’t live with ourselves knowing that there is a higher peak to reach. Whatever the case, if the Master Algorithm is necessary to attain, it will never come to fruition without the joint sagacity of the ML community at large.
The Conclusion
As Pedro Domingos makes plainly evident in his book The Master Algorithm, the search for a Master Algorithm capable of solving all current and future ML use cases rests in the hands of individuals with an important decision to make. Do they follow society’s tendency to segregate groups into factions, or refuse to draw lines in the sand and work together? Their answer to this question represents the viability of the evasive, yet infinitely rewarding Master Algorithm. Unity is the only key to success, and even though this key may still in the end not open the lock, it is no doubt a risk worth taking.
[…] posts, such as the ones by Kevin Anderson, Cameron Hernandez, and Zuber […]
FAU CAP6635 Artificial Inteligence 
[…] posts, such as the ones by Kevin Anderson and Cameron […]