Musings of a Software Designer

Tuesday, March 21, 2006

If there's one thing that I admire most about the major scientific and mathematical discoveries of the 20th century, it's how humble they are. Compare the last century to say... the Scientific Revolution. Mathematicians and scientists had opened up an entire treasure chest of physical and mathematic laws, and they were certainly convinced that these laws were absolute in their truth. According to the likes of Isaac Newton, Karl Gauss and the other great minds of the classical era, the universe could be entirely described by adhering to a few simple laws and rules. Their acheivements can not be understated and the relevance of their work is just as influential today as it was 100s of years ago. What is remarkable about the 20th century is just how quickly the methods of the times past were abandoned to make way for a new era of uncertainty.

Take mathematics for instance. There was little doubt in the minds of mathematicians in the 19th century that all there was to know about mathematics would be unified under one single infallible system. Of course many to this day still do regard mathematics as being a rigid, indisputible field. The drive to capture math as a whole and then dissect it had reached its peak with mathematicians such as Russell and Whitehead publishing the beast that is Principia Mathematica. It was only shortly after that their dream has crumbled and Kurt Godel, one of the greatest logicians to ever live, illustrated the impossibility of fully comprehending mathematics or logic itself by any finite means.

It was a century that saw Albert Einstein, a man who changed how it is we think of space and time and a man so determined to unify the laws that governed nature, watch as his ideal of a grand unified theory fall before the mysterious and peculiar quantum mechanics.

And finally, it was a century where people convinced that computers could at least in theory be able to mechanically solve any problem, come to terms with the fact that computers are also restricted in their capabilities.

Whereas the classical era was about what human beings could know and come to understand about the universe... the modern era focused on what science and mathematics are unable to teach us about the universe. That math is subject to just as much uncertainty, fuzziness, and imprecision as art. That science is full of incredibly strange and even contradictory natural phenomenon. That computation can never compare itself to the ingenuity held by our imaginations.

Where does that leave us today? Perhaps the greatest marvel to come out of the last 100 years, and also the one least known, is Information Theory. What is so special about Information Theory is how in such a subtle way, it combines all the ideas of computation, science, and mathematics. It is my firm belief that the greatest advancements that await us in this century will be a direct result of the application of Information Theory.

Information theory, spawned by Claude Shannon in 1948, was at the time nothing more than an attempt to transmit signals efficiently over a physical medium, such as a telephone cable. But what makes it so great is that for the first time in human history, someone was able to take such an abstract and formless concept, and quantify it. Information theory is a way to measure, in precise terms, just how much information content there exists in a pattern of symbols.

It's remarkable... for example, one can take a newspaper, a novel, or heck, even take this very blog, and actually quantify into a single number just how much information it contains. Furthermore, by measuring how much information is contained in a pattern, and then measuring how many symbols are being used in a pattern, one can determine how much redundancy there is in the pattern.

The implications of this have been well understood for practical purposes. For example, anytime you listen to an MP3 or watch a movie on your computer, information theory was right at the core of making sure that the sound file or video file contained only as many symbols (binary digits) as is needed to communicate the content of the sounds or images. It's only recently, however, that information theory has begun to play a major role in mathematics, and theoretical science.

Now that scientists and mathematicians have matured to acknowledging that there are limits to what we can know, information theory has played an influential role in quantifying whether the solution to a problem is knowable or not. If one seeks to determine if the solution to a problem can be solved more efficiently, one can simply ask "Is the amount of information contained in this problem less than the amount of information contained in this solution to the problem."

Since information is at the core of every single problem or idea, any single problem or idea can be rephrased so as to analyze its information content. It's because of this, that information theory will prove to be very powerful tool this century. Now that we have a better grasp of what's knowable and what is not, what is needed is a way to understand the very root of knowledge itself and to study it in its most purest of forms: raw information.

Monday, March 20, 2006

Let's face it, no one studies philosophy thinking they're going to get rich off of it. The one thing I can admire most about those studying philosophy is that they do so out of genuine interest and passion for the subject, for the breadth of ideas that philosophy exposes to them. It may simply be my own personal experience at the University of Toronto, but students who study computer science are the exact opposite.

Computers were, especially during the 80s and 90s, a great way to get rich. They were also very intimidating at that time too though, so only those who were really brave and certain they wanted to involve themselves completely in the subject would dare to study them. The problem I find that exists today, is that now almost any student who gets accepted into a University to study computer science feels a sense of entitlement to the degree which they hope to get issued after 4 years. Worst of all, they then think they're entitled to get rich because of it. This has seriously had a negative impact on the quality of education. Rather than students getting exposed to an entire world of ideas, expressiveness and inspiration that computer science ought to have been about, students seem to only care about learning strictly what is required to pass their assignments or tests or inch closer to the day when someone hands them the piece of paper with the University's stamp on it.

Computer science, underneath all of the technical jargon and funny looking symbols, is an art. It's a way of taking ideas you have that exist in this fuzzy realm known as your brain, and giving them some physical meaning. Just as a painter takes an image in their head or an emotion in their heart and can export it onto a canvass, so too does a computer scientist. Now being an artist isn't exactly known to turn someone into a millionaire... but lets suppose that it did. Now imagine how absurd it would be if all of a sudden a huge influx of students decided to take up learning how to paint, not because they're really passionate deep down about painting and love it, but simply because painting is where all the money's at.

Just imagine how severely art education would suffer as a result of that. With the culture of entitlement that students have, commonly complaining or nitpicking over marks and numbers and fairness, eventually professors would have to give in and lower their standard of what they grade as being good art.

That is the problem with computer science today. It has gone from being such a great subject matter in its own right, to being a means for which people feel they can make money. Those who are truly passionate about computer science are getting drowned out by those who aren't and as far as virtually all my undergraduate studies at the University of Toronto goes... this drowning out has led to some very serious degradation in the standard and quality of computer science education.

Not that I just came to this realization... but I have a tendency to talk to myself a heck of a lot. And I don't mean the casual "What am I going to eat tonight?" kind of self chatter... I mean I sometimes get into full blown debates with myself; political debates, ethical debates, philosophical debates, you name it.

The problem really is that I like to debate very controversial issues... I like controversy, I like finding issues that most people would rather keep hush hush about and just go ahead and dig right into it. Unfortunately, by it's very nature, that means there will be very few people around to debate these issues with... which is quite likely why I have to debate them with myself.

This blog... let's say its a way for people to get in on the debates I have with myself. As the title suggests, I am very passionate about creating software, learning computer science, and so you'll read a lot of my struggles in tackling what I consider one of the most facinating subject matter.

And if I make a comment or state an opinion so outrageous that you feel you have to jump in there and correct me... it would only make me so happy for you to do so. Believe me... I definitely do not enjoy debating with myself, it's just that I find I sometimes have no choice.