Have you ever used stencils to draw something? It makes it very easy to draw even complex objects. Now, imagine instead of using a stencil to draw with, you use one to pass over a drawing to see if you can find that object.

So if I had a stencil of a cat, I would pass that stencil over every part of the drawing to see if at any time the stencil passes over the shape of a cat, in other words, I would try to make this stencil "fit" over any cats drawn in the image.

Now, imagine the stencil can grow a little or shrink a little on any of its borders, so if a cat was a little smaller, or wider, or taller, etc... we could still "fit" over any cats.

To me, that is one of the most impressive mechanisms in neural networks, their ability to fit on top of complex patterns with a ability to change that fit using "bias".

But what if I need to fit over cats, ducks, horse and dogs? Things get complex quickly and the stencil may need to become much more complex. The activation function can let you create much more complex stencils by allowing the stencils to do things like add another dimension for your stencil so it can fit over much more complex spaces.

There are other useful mechanisms, such as, imagine a huge number of people, and each one had a statement and a number that rated how confident they were that the statement was true. Now Imagine that there are rows of these people, and each one in every row knew what every other person in the rows next to them statements and confidence were too. The speed and efficency of being able to cross check and consolidate information from person to all people in the row before and after them is a huge feat and benefit.

I like to think of things in simpler terms, in ways I can truly understand and relate to, and these types of analogies are helpful to me. So the people (neurons) line up in rows (layers), share information with each other (connections) and can make simple changes to a stencil (model's fit) so it fits over things easier. If the shapes the stencil needs to fit over are too complex for a simple 2d stencil, then it can use another trick (activation method) to allow the stencil to be much more flexible and complex, like adding another dimension, to the stencil.

The best book I ever read for having true insight into neural networks was "Building neural networks from scratch in Python". It is totally worth it even though it's a thick book.