# Build A Custom Deep Learning Model Using Transfer Learning

# Introduction

Transfer learning is used in machine learning and is a method in which already-trained or pre-trained neural networks are present and these pre-trained neural networks are trained using millions of data points.

This technique is currently most famous for training deep neural networks because its performance is great when training deep neural networks with fewer data. Actually, it's instrumental in the data science field because most of the real-world data typically do not have millions of data points for training a robust deep learning model.

Numerous models are already trained with millions of data points and can be used for training complex deep-learning neural networks with maximum accuracy.

Ahead in this tutorial, you'll learn to train deep neural networks using the transfer learning technique.

# Keras applications for transfer learning

Before building or training a deep neural network, you must know what are options available for transfer learning and which one you must use to train a complex deep neural network for your project.

Keras applications are deep learning models that have pre-trained weights that are used for predictions, feature extraction, and fine-tuning. There are numerous models present in the **Keras** library and some of the popular models are

*   **Xception**
    
*   **VGG16 and VGG19**
    
*   **ResNet Series**
    
*   **MobileNet**
    

%[https://keras.io/api/applications/] 

In this article, you'll see the use of the [MobileNet](https://keras.io/api/applications/mobilenet/) model for transfer learning.

# Training a deep learning model

In this section, you'll learn to build a custom deep-learning model for image recognition in a few steps without writing any series of layers of convolution neural networks (CNN), you just need to fine-tune the pre-trained model and your model will be ready to train on the training data.

In this article, the deep learning model will recognize the images of the **hand sign language digits**. Let's start building a custom deep-learning model.

## Grab the dataset

To start the process of building a deep learning model, you will need the data first and you can easily choose the right dataset from millions of datasets by visiting a website called Kaggle. Though there are other websites also that provide datasets for building deep learning or machine learning model.

But the dataset that will be used in this article is taken from the **Kaggle** named [American Sign Language Digit Dataset](https://www.kaggle.com/datasets/rayeed045/american-sign-language-digit-dataset).

## Data preprocessing

After downloading and saving the dataset into the local storage, now it's time to perform some preprocessing on the dataset like preparing the data, splitting the data into train, valid, and test directories, defining their path, and creating batches for training purposes.

### Preparing the data

When you download the dataset, it contains directories from **0** to **9** in which there are three sub-folders **input images**, **output images,** and a **CSV** folder for each directory.

Go ahead and remove the **output images** and **CSV** folder from each directory and move the content of the **input images** to the **main directory** and then remove the **input image** folder.

Each main directory of the dataset now have 500 images, it's your choice to keep all the image but for demonstration purposes, only **200** images will be used in each directory in this article.

The structure of the dataset will look as shown in the image below.

![Dataset folder structure](https://cdn.hashnode.com/res/hashnode/image/upload/v1671371207097/-PB652m8V.png align="center")

### Splitting dataset

Now let's start by splitting the dataset into the train, valid, and test directories.

The `train` directory will contain training data which will be the input data that we feed into the model for learning the pattern and irregularities.

The `valid` directory will contain the validation data that will be infused into the model and will be the first unseen data that the model has seen and it will help in getting maximum accuracy.

The `test` directory will contain the testing data that will be used for testing the model.

Import the libraries that will be used further in the code.

%[https://gist.github.com/Sachin-crypto/fa3e0067274cff0378f559d79ea3cb37] 

Here's the code that will make the required directories and move the data into the specific directories.

%[https://gist.github.com/Sachin-crypto/14ff4f35172f98fedc8e0b7455c57fb1] 

In the above code, first, we changed the directory where the dataset is present in the local storage, and then we checked if the `train/0` directory is already present if not then we created the **train**, **valid,** and **test** directories.

Then we created the sub-directories `0 to 9` and moved all the data inside the `train` directory and along with it created the sub-directories `0 to 9` for `valid` and `test` directories also.

Then we iterated over the sub-directories `0 to 9` inside the `train` directory and taking the **90** image data randomly from each sub-directory and moving them to the corresponding sub-directories inside the `valid` directory.

And the same was done for the `test` directory also.

%[https://geekpython.in/shutil-module-in-python] 

### Defining the path to the directories

After creating the required directories, now the path to the `train`, `valid` and `test` directories need to be defined.

%[https://gist.github.com/Sachin-crypto/9bfbb6c6bede27066d3b134995fab0cd] 

### Applying preprocessing

Pre-trained deep learning model needs some preprocessed data that would fit perfectly for training. So, the data needs to be in the format that the pre-trained model wants it.

Before applying any preprocessing, let's import the TensorFlow and its utilities that will be used further in the code.

%[https://gist.github.com/Sachin-crypto/64450034314f51314da59cc6ecdcc274] 

%[https://gist.github.com/Sachin-crypto/9ac6649fbf32e30effad73f8d5888cd9] 

We used the `ImageDatagenerator` that takes an argument `preprocessing_function` in which we applied preprocessing to the images which are provided by the **MobileNet** model.

Then using the `flow_from_directory` function, we provided the path of the directories and dimensions in which our image should be trained because the **MobileNet** model was trained for the images having dimensions `224x224`.

Then defined the batch size and it can be defined as how many images can go in one **iteration** and then we **shuffled the images**. Image shuffling for test data was not applied because testing data will not be going for training.

After running the above cell in Jupyter notebook or Google Colab, you will see the result like the following.

![Output after running the above cell](https://cdn.hashnode.com/res/hashnode/image/upload/v1671192336046/UKJNHS2jf.png align="center")

The general use case of the `ImageDataGenerator` is for augmenting the data and below is the guide for performing data augmentation using Keras `ImageDataGenerator`.

%[https://geekpython.in/data-augmentation-in-deep-learning] 

## Creating model

Before fitting the training and validation data into the model, the deep learning model **MobileNet** needs to be fine-tuned by adding the output layer, removing the unnecessary layers, and making some layers non-trainable for better accuracy.

The following code will download the **MobileNet** model from Keras and store it in the `mobile` variable. You need to connect to the internet the first time running the following cell.

%[https://gist.github.com/Sachin-crypto/ae56f9dba5baf5d20f9c041ae0fccd4d] 

If you run the following code, then you'll see the summary of the model in which you can see the series of layers of neural networks.

%[https://gist.github.com/Sachin-crypto/6097143df1d913b6f623a7c7aba8b013] 

Now, we'll add the dense output layer with a unit of **10** into the model because there will be 10 outputs **from 0 to 9**. Additionally, we removed the last six layers from the **MobileNet** model.

%[https://gist.github.com/Sachin-crypto/fb436c5587fcdda7351490a9bc944bfa] 

Then we added all the input and output layers into the model.

%[https://gist.github.com/Sachin-crypto/2ebaf792f643bb9a14bbf798d8d846e2] 

Now, we are making the last **23 layers** to be non-trainable and it's an arbitrary number. This specific number has been achieved through many trials and errors. The sole purpose of this code is to better the accuracy by making some layers non-trainable.

%[https://gist.github.com/Sachin-crypto/83cf1bb333459e5599e53bffc4027c17] 

If you see the summary of the fine-tuned model, then you'll notice some differences in the number of non-trainable params and layers as compared to the original summary that you've seen earlier.

%[https://gist.github.com/Sachin-crypto/6d394f877a3df5197ebfc10b9b385ac1] 

Now we are compiling the optimizer named `Adam` with learning rate `0.0001` along with **loss function** and **metrics** to measure the accuracy of the model.

%[https://gist.github.com/Sachin-crypto/fba3798dbb103c893861917eb691da40] 

It's time when the model is ready to train on the training and validation data. In the following code, we fed the training and validation data along with the number of epochs. The `verbose` is nothing but to display the accuracy progress and you can specify the numbers 0, 1, and 2.

%[https://gist.github.com/Sachin-crypto/941424697890009f153d3094d98a5064] 

If you will run the above cell, then you'll see every step of the epoch with training data's loss and accuracy. You'll see the same for validation data.

![Steps of the epochs showing accuracy and validation accuracy](https://cdn.hashnode.com/res/hashnode/image/upload/v1671193927325/hobqaey7v.png align="center")

## Saving the model

The model is now ready with an accuracy score of **99 percentile**. Now keep one thing in mind this model is probably overfitted and might not perform well for images other than images of the dataset.

%[https://gist.github.com/Sachin-crypto/5b8b3f8eaac4c63f3554ba52abb11b8b] 

The above code will check if there is already a copy of the model and if not then the `save` function will save the model in the specified path.

## Testing the model

The model is trained and ready for recognizing images. This segment will cover **loading the model** and making functions for **image preparation**, **predicting results,** and **displaying and printing the predicted result**.

Before writing any code, some libraries that will be used further in the code need to be imported.

%[https://gist.github.com/Sachin-crypto/478647ab721954844bde86a327e5c26a] 

### Loading the custom model

The prediction on images will be made using the model that was created above using the transfer learning technique, so first, it needs to be loaded for usage.

%[https://gist.github.com/Sachin-crypto/1574130b75af249fc805cf3c4beb9543] 

Using the `load_model` function, the model has been loaded from the specified path and stored inside the `my_model` variable for further use ahead in the code.

### Preparing input image

Before giving any image for predicting or recognizing to the model, it needs to be in the format that the model wants it.

%[https://gist.github.com/Sachin-crypto/bf20ad28ef5806b51cf58053d1035e21] 

First, we defined a function `preprocess_img` that takes the path of the image and then we loaded that image using the `load_img` function from the `image` utility and set the target size of `224x224`, then converted that image into an array and divided that array by `255.0` which converted the pixels of the image between `0 and 1` and then reshaped that image array into the shape (`224, 224, 3`) and finally returned that reshaped image.

### Predicting function

%[https://gist.github.com/Sachin-crypto/c114ccc2c0dc2b9af8250ef01eab8b93] 

Here, we defined a function `predict_result` that takes predict argument which is basically a preprocessed image and then used the `predict` function to predict the result and then returned the maximum value from the predicted result.

%[https://geekpython.in/numpy-argmax-function-in-python] 

### Display and predict image

First, there'll be a function that will take a path of the image and then display the image and the predicted result.

%[https://gist.github.com/Sachin-crypto/5b96685b2bc4681b77525abae6590122] 

The above function `display_and_predict` takes the path of an image and used the `Image.open` function from the `PIL` library to open the image and then used the `matplotlib` library to display the image, and then for printing the predicted result the image is passed to the `preprocess_img` function and then used the `predict_result` function to obtain the result and finally printed it.

%[https://gist.github.com/Sachin-crypto/20ba0958eb425bd7eb0fb1182242f339] 

If you will run the above cell and enter the path of the image from the dataset, then you'll get the desired output.

![Prediction result](https://cdn.hashnode.com/res/hashnode/image/upload/v1671367956994/sLVlDy94G.png align="center")

The model is successfully created using the transfer learning technique without writing any series of layers of neural networks.

The model can be used for making the web app for recognizing images and here's how you can implement the model into the Flask app.

%[https://geekpython.in/flask-app-for-image-recognition] 

## Conclusion

The article covers the making of a custom deep learning model using the pre-trained model or transfer learning.

You've learned each step involved in creating a full-fledged deep-learning model. The steps you've seen are:

*   **Preparing the dataset**
    
*   **Preprocessing the data**
    
*   **Creating the model**
    
*   **Saving the custom model**
    
*   **Testing the custom model**
    

Get the complete source code on [**GitHub**](https://github.com/Sachin-crypto/Image-Recognition-DL-Model).

* * *

🏆**Other article you might like if you like this article**

✅[Upload and display static and dynamic images using Flask](https://geekpython.in/render-images-from-flask).

✅[NumPy and TensorFlow have a common function](https://geekpython.in/argmax-function-in-numpy-and-tensorflow).

✅[Create a dynamic contact form that will save a message in database and will send you a mail](https://geekpython.in/contact-form-for-your-blog-get-emails-and-save-in-db-dynamically).

✅[Build a Covid-19 EDA and Visualization Streamlit app](https://geekpython.in/python-web-app-under-100-lines-of-code-using-streamlit).

✅[Use asynchronous programming in Python using asyncio module](https://geekpython.in/asyncio-how-to-use-asyncawait-in-python).

* * *

**That's all for now**

**Keep Coding✌✌**