Dynamic mini batch creation during training
One of the ways to only load the data corresponding to a mini-batch is to dynamically create mini-batches by processing images randomly from their location. The number of images to be processed in a mini-batch would be equal to the mini-batch size we specify. Of course there would be some bottleneck in the training process because of the dynamic mini-batch creation during training time but that bottleneck is negligible. Specially packages such as keras have efficient dynamic batch creation mechanism. We would be leveraging flow_from_directory functionality in keras to dynamically create mini-batches during training to reduce the memory requirements of the training process. We will still continue to use ImageDataGenerator for image augmentation. The train generator and the validation generator can be defined as follows.
The image preprocessing step of subtracting the mean images pixel intensities from the three channels is done by feeding the pre_process function as input to the preprocessing_function of the ImageDataGenerator:
def pre_process(img):
img[:,:,0] = img[:,:,0] - 103.939
img[:,:,1] = img[:,:,0] - 116.779
img[:,:,2] = img[:,:,0] - 123.68
return img
train_file_names = glob.glob(f'{train_dir}/*/*')
val_file_names = glob.glob(f'{val_dir}/*/*')
train_steps_per_epoch = len(train_file_names)/float(batch_size)
val_steps_per_epoch = len(val_file_names)/float(batch_size)
train_datagen =
ImageDataGenerator(horizontal_flip =
True,vertical_flip =
True,width_shift_range =
0.1,height_shift_range = 0.1,
channel_shift_range=0,zoom_range = 0.2,
rotation_range = 20,
preprocessing_function=pre_process)
val_datagen =
ImageDataGenerator(preprocessing_function=pre_process)
train_generator =
train_datagen.flow_from_directory(train_dir,
target_size=(dim,dim),
batch_size=batch_size,
class_mode='categorical')
val_generator =
val_datagen.flow_from_directory(val_dir,
target_size=(dim,dim),
batch_size=batch_size,
class_mode='categorical')
print(train_generator.class_indices)
joblib.dump(train_generator.class_indices,
f'{self.outdir}/class_indices.pkl')
The flow_from_directory function takes in an image directory as and input and expects a folder pertaining to a class within the image directory. It then infers the class labels from the folder names. If an image directory has the following structure for the image directory then the classes are inferred as 0, 1, 2, 3, 4, pertaining to the class folders 'class0','class1','class2','class3', and 'class4'.
The other important inputs to the flow_from_directory function is the batch_size, target_size and class_mode. The target_size is to specify the dimension of the image to be fed to the neural network while class_mode is to specify the nature of the problem. For binary classification class_mode is set to binary while for multi class classification the same is set to categorical.
We will next train the same model by creating dynamic batches instead of loading all the data to memory at once. We just need to create a generator using flow_from_directory option and tie it to the data augmentation object. The data generator object can be generated as follows:
# Pre processing for channel wise mean pixel subtraction
def pre_process(img):
img[:,:,0] = img[:,:,0] - 103.939
img[:,:,1] = img[:,:,0] - 116.779
img[:,:,2] = img[:,:,0] - 123.68
return img
# Add the pre_process function at the end of the ImageDataGenerator,
#rest all of the data augmentation options
# remain the same.
train_datagen =
ImageDataGenerator(horizontal_flip = True,vertical_flip = True,
width_shift_range = 0.1,height_shift_range = 0.1,
channel_shift_range=0,zoom_range =
0.2,rotation_range = 20,
preprocessing_function=pre_process)
# For validation no data augmentation on image mean subtraction preprocessing
val_datagen = ImageDataGenerator(preprocessing_function=pre_process)
# We build the train generator using flow_from_directory
train_generator = train_datagen.flow_from_directory(train_dir,
target_size=(dim,dim),
batch_size=batch_size,
class_mode='categorical')
# We build the validation generator using flow_from_directory
val_generator = val_datagen.flow_from_directory(val_dir,
target_size=(dim,dim),
batch_size=batch_size,
class_mode='categorical')
In the preceding code we pass the ImageDataGenerator an additional task of performing the mean pixel subtraction since we don't have any control of loading the image data in memory and passing it through the pre_process function. In the preprocessing_function option, we can pass any desired custom function for any specific preprocessing task.
Through train_dir, and val_dir we pass the training and validation directories to the train and validation generator that we created with flow_with_directory option. The generators identifies the number of classes by looking at the number of class folders within the training data directory (here train_dir)passed. During training time based on the target_size the images are read into memory based on the specified batch_size
The class_mode helps the generator identify whether its a binary classification or a multi class('categotical') one.
The detailed implementation is laid down in the TransferLearning_ffd.py folder on GitHub at https://github.com/PacktPublishing/Python-Artificial-Intelligence-Projects/tree/master/Chapter02.
The Python script TransferLearning_ffd.py can be invoked as follows:
python TransferLearning_ffd.py --path '/media/santanu/9eb9b6dc-b380-486e-b4fd-c424a325b976/book AI/Diabetic Retinopathy/Extra/assignment2_train_dataset/' --class_folders '["class0","class1","class2","class3","class4"]' --dim 224 --lr 1e-4 --batch_size 32 --epochs 50 --initial_layers_to_freeze 10 --model InceptionV3 --outdir '/home/santanu/ML_DS_Catalog-/Transfer_Learning_DR/'
The end of the output log from the job run is as follows:
Validation results saved at : /home/santanu/ML_DS_Catalog-/Transfer_Learning_DR/val_results.csv
[0 0 0 ... 4 2 2]
[0 0 0 ... 4 4 4]
Validation Accuracy: 0.5183708345200797
Validation Quadratic Kappa Score: 0.44422008110380984
As we can see by reusing an existing network and performing transfer learning on the same we are able to achieve a decent Quadratic Kappa of 0.44.