What is a feed­for­ward neural network (FNN)?

Feed­for­ward neural networks (FNNs) are simple neural networks that pass in­for­ma­tion one layer at a time, moving from one layer to the next in a se­quen­tial order. They can be cat­e­go­rized into single-layer FFNs and multi-layer FFNs, each of which serve different purposes in deep learning.

What is a feed­for­ward neural network (FNN)?

A feed­for­ward neural network (FNN) is an ar­ti­fi­cial neural network that operates without any feedback loops. This network type is con­sid­ered one of the simplest forms of ar­ti­fi­cial neural networks because it only works in a forward-moving direction. Deep feed­for­ward neural networks play an essential role in creating models in the fields of deep learning and ar­ti­fi­cial in­tel­li­gence. Depending on the number of layers used, FNNs can be clas­si­fied as single-layer or multi-layer networks.

How does a feed­for­ward neural network work?

Like all ar­ti­fi­cial neural networks, feed­for­ward neural networks are inspired by the human brain, which processes in­for­ma­tion through a network of neurons. FNNs have a minimum of two layers: an input layer and an output layer. Between these layers, there can also be ad­di­tion­al layers, known as hidden layers. Each layer is only connected to the layer directly above it. The con­nec­tions between the layers are formed through edges, a term taken from graph theory.

• Input Layer: The input layer receives all the input data that is fed into the network. Each neuron in this layer cor­re­sponds to a feature in the incoming data.
• Hidden Layers: Between the input and output layers, there can also be hidden layers. Each hidden layer consists of multiple neurons that connect the input and output layers.
• Output Layer: The output layer of the network produces the final result.

In a feed­for­ward neural network, in­for­ma­tion only flows in one direction, from the input layer to the output layer. A set of inputs is in­tro­duced into the input layer, and then the neurons in this layer take the data and apply weights to them. In a single-layer FFN, the neurons pass the in­for­ma­tion onto the output layer. In a multi-layer FFN, however, the in­for­ma­tion is first passed to the hidden layers. Since this process can’t be seen, these layers are referred to as “hidden.” Once the data reaches the hidden layers, it’s reweight­ed. The processed in­for­ma­tion is then delivered as the final result by the neurons in the output layer.

Close

Through­out the process, the weighted in­for­ma­tion in each step is added up. Then, a threshold value is applied to determine whether a neuron should pass the in­for­ma­tion along or not. This threshold is typically set to zero. In a feed­for­ward neural network, there are no backward con­nec­tions between layers, meaning each edge only links to the layer that directly follows it.

What are the most important use cases for FNNs?

There are many potential uses for feed­for­ward neural networks. These types of networks are par­tic­u­lar­ly useful for pro­cess­ing and linking large volumes of un­struc­tured data. Below are some examples of when FNNs are used:

• Speech recog­ni­tion and pro­cess­ing: Feed­for­ward neural networks can be used to convert text into speech or for con­vert­ing spoken language into written text.
• Image recog­ni­tion and pro­cess­ing: FFNs can analyze images and identify certain features. This can be used to digitize hand­writ­ten notes, for example.
• Clas­si­fi­ca­tion: A feed­for­ward neural network can classify data based on pre­de­fined pa­ra­me­ters.
• Fore­cast­ing: Feed­for­ward neural networks are also great for making pre­dic­tions, for example, for events or trends. They can be used in weather pre­dic­tion, early warning systems in disaster man­age­ment, space ex­plo­ration and defense.
• Fraud detection: These networks can play an important role in iden­ti­fy­ing fraud­u­lent ac­tiv­i­ties or patterns.

What’s the dif­fer­ence between a feed­for­ward neural network and a recurrent neural network (RNN)?

While both networks use neurons to process in­for­ma­tion and pass it from an input layer to an output layer, a recurrent neural network (RNN) can also send in­for­ma­tion backwards. An RNN has con­nec­tions that allow in­for­ma­tion to travel back and forth through the layers, giving the network feedback loops where in­for­ma­tion can be stored.

Such networks are es­pe­cial­ly useful for de­ter­min­ing results when context is important, such as in text pro­cess­ing. Take the word “bank”. This could refer to a financial in­sti­tu­tion or the area bordering a river. To determine the correct meaning, it’s necessary to know what the context is. Unlike RNNs, feed­for­ward neural networks don’t have a mechanism to store this type of in­for­ma­tion.