Complete Glossary of Keras Optimizers and When to Use Them (With Code)

Read the full article here: https://analyticsarora.com/complete-glossary-of-keras-optimizers-and-when-to-use-them-with-code/

Introduction

When a deep neural network ends up going through a training batch, where it propagates the inputs through the layers, it needs a mechanism to decide how it will use the predicted results against the known values to adjust the parameters of the neural network. These parameters are commonly known as the weights and biases of the nodes within the hidden layers.

This above-mentioned mechanism is where the optimizers kick in. Optimizers are the algorithms deciding how the learning parameters are adjusted. These optimizers, along with the loss functions, are the backbone of all deep neural networks.

Throughout this guide, we'll go through a detailed explanation of how the optimizers work and the different types of optimizers that Keras provides us, along with instantiation examples. Moreover, we'll also be taking a look at the situations where certain optimizers work better than others.

Article Overview

• How Do Optimizers Work?
• How To Use Optimizers in Keras?
• SGD Optimizer
• Adagrad Optimizer
• RMS Optimizer
• Adadelta Optimizer
• Adam Optimizer
• Summary