Thursday, July 19, 2007

ANN Tutorial - Forward Propagation

So far, if you've been keeping track of my blog, you'll have seen lots of stuff on Artificial Neural Networks (ANNs) and what they can do, but you might not know how they work. This is what this post is going to be about; today, I'll be talking exclusively of forward propagation, or, in layman's terms the act of getting the ANN to 'think'.

If you know much about anatomy, you'll know that the brain is made up of neurons interconnected together - It is believed that the strength of these connections is what forms a memory. Now, I'm not going to get into too much biology because it'll simply bore you out of your mind, instead, let's just tackle the real thing and I'll throw-in biological references from time to time... How's that?

First of all, one question that probably didn't pop into your head is; "what does an ANN look like" - You were probably expecting it to be some absurdly abstract concept, but it isn't quite that way, it is actually feasible, and very common might I say; to represent them using flowcharts. They say a picture means a thousand words, so here goes:

First of all, I'd like to direct you to the different 'layers' of this ANN; the input, hidden and output... This is rather simple; the input layer represents input data, the hidden layer is the one that does the processing and the output layer just outputs the result as either a 1 or a 0.

Secondly, you should take note of the 'interconnections' that connect all the nodes together. The numbers you see for every red and black connector are called 'weights' and they're used to modify output from the previous node (more on that later). You need to note that the green and blue connectors also have weights, but they were omitted from the diagram as they couldn't fit in. Typically, to begin with, each weight of every connection is initialized to a random value. Personally, I like to keep it in the range of -10 to 10.

As you can see from the diagram, in this case, the input layer is made up of 3 nodes and each of them has a value of either 1 or 0 (binary). If we were to use this ANN to identify a pattern on a 3*1 bitmap of black and white pixels (black being 0 and white being 1), each input node would hold the value of one of the pixels. Then, each of these values would be processed, this is where the weights come in; each output from the input nodes (1 or 0) is distorted in some way by the weight of each of the connectors; this is done via a simple multiplication.

Let's assume that the first pixel in our 3*1 bitmap was white, looking back at our diagram, we can say that the red input node will be responsible for it. Let's go through this process shall we; first of all, since the first pixel is white, the input node will output a 1. This value will then be propagated to each of the nodes in the hidden layer via different connections (the red ones).
The top hidden node, for example, will receive a value of -5.2 because the weight of the connector (-5.2) multiplied by the input (1) gives -5.2.

This process is done for each of the input nodes, and, once this is completed, each hidden node will evaluate the sum of all the 'weighed' values which they have received from each of the input nodes. If that sum is greater than a specified threshold (in most cases something like 0.1); then that hidden node will output a 1, otherwise, it will output a 0.

Once that's done, this output from each of the hidden nodes (1 or 0) will again be weighed by the connection between the hidden and output layer. Again, this is just a simple multiplications... Once this is completed, the output node will find the sum of all the weighed values and if it is above the threshold, then that output node will output a 1 (true) or 0 (false).

In the examples which I posted before, I designed an artificial neural network to associate certain drawings with 1 and others with 0; since pretty much anything can be represented by binary, you might want to play around with that.

For those of you interested in biology; as we've heard, the brain is made up of neurons interconnected together. Well, it is believed that the strength of each of these connections is what determines the strength of each memory unit that it holds; that's probably why scientists say that the brain is a muscle; if your neurons are buff in some areas, then these areas are going to represent the most prominent aspect of your intelligence. If only we knew what each of these memory units actually were we would probably be able to better understand, and thus exercise much more control over our ANNs.

In this case, I have showed you what is known as a step ANN because the outputs of each node is only either a 1 or 0. There is another type of ANN which performs sigmoid calculations... You might want to do some research on that. Also, you can be pretty creative with your ANNs; each layer can have variable amounts of nodes; you don't have to have the same number of hidden nodes as input nodes and you could have more than one output node if you liked, also, you could go as far as having more than one hidden layer... Think big!

Ok, this concludes the tutorial on forward propagation. Next time, I'll be talking about back-error propagation which will teach you a way to teach your ANN - Just to get you thinking; it's done by altering the weights of each interconnection by comparing the desired output with the actual final output... As the name of the algorithm suggests, it involves starting at the output node and 'correcting' the weights of each successive connector as you make your way back through the layers.

Wednesday, July 11, 2007

Java IntelligentBitmapAnalyser Released!

Hi, I have released a Java version of my ANN.
It's basically a Java implementation of my ActionScript source code.

Here are some source files and an executable JAR file:

IntelligentBitmapAnalyser

You need the Java runtime environment to run
P.S. It is not an applet.

Monday, July 2, 2007

Safari Browser for Windows

To kick-off my blog, I'd like to share my thoughts on the *new* Safari 3 Browser (Beta) for Windows.
As you might have heard, the Safari browser has finally made its way to the Windows operating system and to be honest, so far, I have very mixed feelings about it.

After downloading it, I was excited to try it out; judging from its excellent reputation on the Mac, I was expecting something that would completely annihilate Firefox. To my surprise, however, what I saw was rather displeasing.

What annoys me above all is the way it renders text. If you've tried IE7 before, you might have noticed that the text is a bit peculiar; this is all due to Microsoft's text-rendering technology 'ClearType'. If you've got a fine eye for detail like me, this new technology would have been more of a downfall than a benefit. Actually, my first Google search after downloading IE7 went something along the lines of "IE7 turning off blurry text." Anyway, as I started up Safari for the first time, I was shocked. Until that point, I could have sworn that font-rendering was just a one-off fad by Microsoft, I had never thought that it would actually be replicated ever again. Not only was Safari's font a replica of the ClearType disaster, but it was even worse. The text in Safari 3 Beta for Windows literally looked blurry, so much so that I'm still puzzled by why a huge company like Apple would integrate it in their very first browser for the Windows platform. Not only is it ugly, but, to my surprise, it can't be turned off!!! That was perhaps the biggest downfall... I mean, first of all; they fail to make a decent text-rendering system, then, to top it all, they will not allow me to turn it off! Why did they even release this Beta? They're just turning away potential users even before they get the real version. To me, it's just money down the drain.

Another problem that I found with Safari is the graphical user interface... Hello!!! I'm using Windows, I don't want a Mac interface! Not only is the look out of place with Windows, but it's pretty plain and boring. That being said, I like how the browser displays buttons, check-boxes and the like; Mac style does have a place in a Windows environment but it needs to have boundaries. I find that the top bar in Safari just takes it too far; it doesn't feel like Windows and on a semi-conscious level, it reminds me of the Netscape browser and that's not a very nice image.

One major aspect of this browser that I really enjoy, though, is the loading. Seriously, it's a beauty; it's like a perfect compromise between Firefox's progressive loading and IE's all-at-once approach. I find that in Firefox, for many pages, pretty much as soon as you press 'Enter' in the address bar, the content of that page starts to load, in fact, it sets-off so quickly that sometimes the content doesn't even get the time to format itself properly. IE, on the other hand is on the other side of the equation; it takes much longer to load, but at least, when it's done, the content is pretty much all ready. In brief, Safari is a bit closer to IE, but the way it loads content is more free-flowing and it's MUCH faster. I really never envisaged that it would make such a great difference in speed!

I will be looking forward to the complete version of Safari for Windows, hoping that they'll have fixed-up at least some of the major problems which I highlighted in this post.