Inference mode

Inference mode used for Dropout and Batch Normalization.

This is an introduction to the “inference mode”. Dropout and Batch Normalization , and some probablistic method has the inference mode and learning mode.
For example, you have not to use the dropout when you predict the data after training.

Required Libraries

  • numpy 1.12.1
In [1]:
import numpy as np
import renom as rm

What is inference mode?

There are some functions which may exhibit different behavior between training and testing. For example, dropout and batch normalization .

The dropout function randomly sets some of the data to zero, during training. During testing however, nothing is dropped-out.

To control this, we have to set the “inference” mode flag in our model.

How to set the model to inference mode

In ReNom, there is a flag that switches-on the inference mode.

The following code shows how to switch the mode:

In [2]:
x = np.random.rand(2, 3)
model = rm.Sequential([
    rm.Dropout(dropout_ratio=0.5),
])

model.set_models(inference=False)
# If Model is set to the "training mode",
# then the Dropout function drops some of the data.
print("Training mode. Some data are dropped.")
print(model(x))
print()

model.set_models(inference=True)
#  If Model is set to the "inference mode",
#  then the Dropout function doesn't drop part of data.
print("Inference mode. Any data are not dropped.")
print(model(x))
Training mode. Some data are dropped.
[[ 0.          0.15597625  1.60988951]
 [ 0.          0.52721781  0.62134868]]

Inference mode. Any data are not dropped.
[[ 0.43925468  0.07798813  0.80494478]
 [ 0.34552573  0.26360891  0.31067435]]