Keras Initializers

Updated March 16, 2023

Definition of Keras Initializers

Keras initializers are the keyword arguments which is used for passing initializers for layers that depend on the layer. Usually, we can say it is a bias initializer and kernel initializer. The keras initialization will define the way for setting initial random weights onto the layers of keras. Training the neural network completely depended on the types of parameters which was used in initializing.

Key Takeaways

• Keras initializers is generating tensors by using values that were constant. The argument value is nothing but a list of values of constant values for the dtype.
• The resulting tensor is populated by using a type of dtype values, this was specified into a value that was desired in the new tensor.

What are Keras Initializers?

Suppose we have done initialization parameters correctly then the result of the optimization is achieved in a minimal time and suppose we have not initialized, then it creates problems. The performance of the neural network depends on how the parameter is initialized at the time of starting the train. The neural network performance depends on how the parameter is initialized and starts while training the model.

Suppose we are training the neural network by using random weights, then the output will be reproducible. Suppose we train the neural network by using constant weights, then it will consume a lot of time to converge. We will be comparing the accuracy of the keras initializers, and we are looking at visualization. Snippets code is displayed while visualization is done in wandb. By adding the model which was trained by using 20 epochs and the dataset which was chosen in MNIST dataset.

Keras Initializers Constant

If the value is least, then we can say that length is less than or equal to the number of the elements which was implied in the desired shape or tensor. In some cases where a total number of values is less than or greater which is required in tensor shape for initializing or raising the value error.

There are three types of arguments we need to use while working with keras initializers constant as follows:

• Value
• Dtype
• Verify_shape

The below steps show how we can use the keras initializer constant. We are importing the tensorflow module as follows.

1. In the first step we are importing the numpy and tensorflow modules by using the import keyword as follows.

Code:

import numpy as np
import tensorflow as tf

Output:

2. After importing the module now in this step, we are defining the value and declaring the constant initializer as follows.

Code:

val = [10, 21, 32, 43, 54, 65, 76, 87]
con = tf.constant_initializer (val)
print (con)

Output:

3. After defining the constant initializer now in this step, we are printing values. The below example shows the standalone usage of keras constant initializer as follows.

Code:

ini = tf.keras.initializers.Constant(3.)
val = ini(shape=(2, 2))
print(val)

Output:

4. After printing the standalone usage value, we can also print the usage of the keras layer.

Code:

init = tf.keras.initializers.Constant(3.)
lay = tf.keras.layers.Dense(3, kernel_initializer = ini)
print (lay)

Output:

Keras Initializers Layer

Keras initializer layer requires the input shape for understanding the structure of input data. To define the keras initializer layer we need to import the below modules.

Code:

from keras.models import Sequential
….
from keras import constraints

Output:

Basically, in machine learning, the weight will be assigned to the input data. Keras initializers provide different functions. The below example generates 0 from all the input data.

Code:

tf.keras.initializers.Zeros ()
ini = tf.keras.initializers.Zeros()
val = ini(shape=(2, 2))

Output:

In the below example we are generating one value with all input values by using keras initializers layers.

Code:

tf.keras.initializers.Ones ()
ini = tf.keras.initializers.Ones()
val = ini(shape=(2, 2))

Output:

In the below example, we are generating constant values with all input values which were provided by the user using keras initializers layers as follows.

Code:

tf.keras.initializers.Constant(value=0)
ini = tf.keras.initializers.Constant(3)
values = ini (shape=(2, 2))

Output:

In the below example, we are generating values by using normal distribution in the keras initializer’s layers.

Code:

tf.keras.initializers.RandomNormal (mean=1.0, stddev=0.5, seed=None)
ini = tf.keras.initializers.RandomNormal (mean=0, stddev=1)
val = ini (shape=(2, 2))

Output:

In the below example, we are generating values by using uniform distribution in the keras initializer’s layers.

Code:

tf.keras.initializers.RandomUniform(minval=-0.07, maxval=0.09, seed=None)
ini = tf.keras.initializers.RandomUniform(minval=0, maxval=1)
val = ini(shape=(2, 2))

Output:

In the below example, we are generating values by the truncated normal distribution in the keras initializer’s layers.

Code:

tf.keras.initializers.TruncatedNormal(mean=0.0, stddev=0.07, seed=None)
ini = tf.keras.initializers.TruncatedNormal(mean=0., stddev=1.)
val = ini(shape=(2, 2))

Output:

Keras Initializers Method

All the methods return the object or the specified initializer. Below is the method of tf.keras.initializers.constant._call_. This method returns the tensor object which was specified by initializers.

The below example shows how tf.keras.initializers.constant._call_ method is defined.

Code:

ini = tf.initializers.Constant.__call__(shape,
…
verify_shape = None)

Output:

The tf.keras.initializers.Constant.from_config method is used to instantiate the initializer from the initialization directory.

Code:

tf.keras.initializers.Constant.from_config (cls,
config)

Output:

The below example shows how keras initializer method works. We are defining the random uniform method.

Code:

tf.keras.initializers.RandomUniform(minval=-0.07, maxval=0.09, seed=None)
ini = tf.keras.initializers.RandomUniform(minval=0, maxval=1)
val = ini(shape=(2, 2))

Output:

Examples of Keras Initializers

Given below are the examples mentioned:

Example #1

In the below example, we are defining the he_normal method.

Code:

from keras.models import Sequential
from keras import initializers
from keras.layers import Activation, Dense
tf.keras.initializers.HeNormal (seed=None)
ini = tf.keras.initializers.HeNormal()
val = ini(shape=(2, 2))

Output:

Example #2

In the below example, we are using the constant method.

Code:

from keras.models import Sequential
from keras import initializers
from keras.layers import Activation, Dense
tf.keras.initializers.Constant(value=0)
ini = tf.keras.initializers.Constant(3)
values = ini (shape=(2, 2))

Output:

Example #3

In the below example, we are using the random normal method.

Code:

from keras.models import Sequential
from keras import initializers
from keras.layers import Activation, Dense
tf.keras.initializers.RandomNormal(mean=0.0, stddev=0.05, seed=None)
ini = tf.keras.initializers.RandomNormal(mean=0., stddev=1.)
val = ini(shape=(2, 2))

Output:

Example #4

In the below example, we are using the random uniform method.

Code:

from keras.models import Sequential
from keras import initializers
from keras.layers import Activation, Dense
tf.keras.initializers.RandomUniform (minval=-0.07, maxval=0.09, seed=None)
ini = tf.keras.initializers.RandomUniform(minval=0, maxval=1)
val = ini(shape=(2, 2))

Output:

FAQ

Given below are the FAQs mentioned:

Q1. What is the use of keras initializer?

Answer: Basically keras initializer is used to pass the initializer from the specified layer which we have defined in our code.

Q2. Which dependencies do we need to import while using the keras initializer?

Answer: We need to import the keras, numpy, and tensorflow dependency at the time of working with the keras initializer.

Q3. What is weight initialization in keras neural net?

Answer: By using keras we are initializing weight as per perception models of multi-layer. We need to initialize weight carefully at the time of working with the keras initializer.

Conclusion

Suppose we have done initialization parameters correctly then the result of the optimization is achieved in a minimal time and if we have not initialized, then it will create a problem. Keras initializers are the keyword arguments which was used for passing initializers for layers that depend on the layer.

Recommended Articles

This is a guide to Keras Initializers. Here we discuss the introduction, keras initializers constant, layer and method, and examples. You may also have a look at the following articles to learn more –

1. Keras Tuner
2. What is Keras GPU?
3. Keras Embedding
4. Keras predict