text:
## Overview

• Supervised deep learning is when the output we want to predict is labelled in the data that we’re using for training.
• This article dives deeper into the nuances and differences between different supervised deep learning architectures.

## Fully connected networks

• A Fully Connected Network (FCN) consists of a series of fully connected layers that connect every neuron in one layer to every neuron in the other layer.
• The major advantage of fully connected networks is that they are “structure agnostic”, i.e., there are no special assumptions that are needed to be made about the input.
• While being structure agnostic makes fully connected networks very broadly applicable, such networks do tend to have weaker performance than special-purpose networks tuned to the structure of a problem space.
• The following diagram shows a Multilayer Deep Fully Connected Network:

## Convolutional Neural Networks

• A Convolutional Neural Network (CNN) is a multilayer neural network architecture which is primarily used in image-processing applications. CNN architectures make the explicit assumption that the input has a spatial dimension (along with an optional depth dimension) such as images, which allows encoding certain properties into the model architecture.
• Yann LeCun created the first CNN where the architecture focused on recognizing handwritten characters.
• Let’s break down the technical details on a computer vision model using CNN:
• Input to the model:
• The input for a CNN model is usually an image, represented as a grid of pixels (grid of positive whole numbers where each number is assigned a color).
• Output of the model:
• The output of the model depends on what it was trying to predict, the examples below represent a few common tasks:
• A simple CNN is a sequence of layers, and every layer of a CNN transforms one volume of activations to another through a differentiable function. Three main types of layers are used to build CNN a architecture for feature extraction: Convolutional Layer, Non-linearity, and Pooling Layer. Finally, we utilize a Fully-Connected Layer to perform classification. The following diagram shows the different parts of a CNN layer:
• Convolution: The convolution filters scan the image and while convolving the pixels in the input image with the kernel using multiplication and addition. A CNN tries to learn the weights in the convolutional filters to be able to predict the desired output.
• Nonlinearity: This is an equation used on the convolutional filter to enable modeling of non-linear relationships in data.
• Pooling: Also known as “max pooling”, it is just choosing the biggest number out of a range of numbers. It helps in reducing the size of the representation and reduces the amount of computations the CNN has to make and thus, helps with efficiency.
• These aforementioned three operations combined make a fully convolutional network for feature extraction.

### Use-cases of CNNs

• CNNs are commonly used in solving problems related to spatial data, such as images (2D CNNs) and audio (1D CNNs).
• Use cases for CNNs include facial recognition, medical analysis, and classification.

### Pros of CNNs compared to FCNs

• Parameter sharing/Computational tractability: Since CNNs utilize parameter sharing, the number of weights for a CNN vs. FCN architecture typically have a difference of several orders of magnitude.
• For a fully connected layer, you have an input of shape $$(H_{in} \times W_{in} \times C_{in})$$ and the output of shape $$(H_{out} \times W_{out} \times C_{out})$$. This means, that each color of the pixel of the output feature map is connected to every color of the pixel from the input feature map. There is a separate learnable parameter for each pixel in the input image and the output. Hence, one gets this huge number of parameters : $$(H_{in} \times H_{out} \times W_{in} \times W_{out} \times C_{in} \times C_{out})$$
• In the convolutional layer, the input is the image of shape $$(H_{in}, W_{in}, C_{in})$$ and the weights account for the neighborhood of the given pixel, say of size $$K \times K$$. The output is obtained as a weighted sum of the given pixel and its neighborhood. There is a separate kernel for each pair of the input and output channel $$(C_{in}, C_{out})$$, but the weights of the kernel (a tensor of shape $$(K, K, C_{in}, C_{out})$$ are independent of the location. Actually, this layer can accept images of any resolution, whereas, the fully connected can work only with a fixed resolution. Finally one has $$(K, K, C_{in}, C_{out})$$ parameters, which for the kernel size $$K$$ much smaller, than the input resolution result into significant drop in the number of variables.
• The sole fact that since AlexNet won the ImageNet competition, every neural network that has won it has used CNNs should be enough to convince you that CNNs are better for image data!
• You most likely won’t be able to find any meaningful comparison, since CNNs are able to handle image data that is infeasible using only FC layers. Here’s why:
• The number of weights in FC layer with 1000 neurons for $$224 \times 224 \times 3$$ image is something like 150M. That’s 150M for only one layer.
• If you’re not convinced this is a huge number, then note that modern CNN architectures that have 50-100 layers while having overall couple dozen million parameters (for example ResNet50 has 23M params, InceptionV3 has 21M parameters).
• Mathematically, let’s compare the number of weights between CNNs and FCNs (with 100 hidden units) for an input image of shape $$500 \times 500 \times 3$$:
• FC layer = Wx = $$100 \times (500 \times 500 \times 3) = 100 \times 750000 = 75M$$.
• CNN layer = ((shape of width of the filter * shape of height of the filter * number of filters in the previous layer+1)*number of filters) (note that the +1 is for bias) = $$(F_w \times Fh \times D + 1 ) \times F = (5 \times 5 \times 3 + 1 )*2 = 152$$.
• Translation invariance: Invariance refers to the ability to remember an object as an object even though its object place changes. This is usually a positive thing because it maintains the object’s identity (category). Note that translation here has a specific meaning in vision, borrowed from geometry. The following figure shows The the same object in different locations (note that a CNN would recognize both as cats owing to translation invariance).

## Recurrent Neural Networks

• RNNs are one of the foundational network architectures that are leveraged in building other deep learning architectures.
• However, one key different is that unlike normal feed-forward networks, RNN can have connections that feed back to its prior or same layer. In a way, RNNs have “memory” over previous computations and use this information in processing during the current time-step.

• The term “recurrent” applies as the network performs the same task over each instance of the sequence such that the output is dependent on the previous computations and results.
• RNNs are naturally suited for many NLP tasks such as language modeling.
• They are able to capture the difference in meaning between “dog” and “hot dog” and thus, RNNs are tailor-made for modeling such context dependencies in language and similar sequence modeling tasks.
• This has resulted to be a strong motivation for researchers to use RNNs over CNNs in these areas.
• Another advantage of RNNs is that the model size does not increase with the size of input, thus enabling the possibility of processing inputs of any length.
• Additionally, unlike CNNs, RNNs have flexible computational steps that provide better modeling capability and create the possibility to capture unbounded context since it takes into account historical information and its weights are shared across time.
• However, RNNs suffer from a vanishing gradient problem. The gradient becomes small and thus, makes the updated weights for backpropogation very small.
• Another pitfall we with RNNs is that they are slow to train (due to sequential processing of each token) and sometimes difficult to converge (due to vanishing/exploding gradients).
• Below we will see architecture illustrations of RNNs used for various applications provided by Stanford’s CS230 Deep Learning: Recurrent Neural Networks cheatsheet by Afshine Amidi and Shervine Amidi.

• Note that a CNN has a different architecture from an RNN. CNNs are a type of feed-forward neural networks that use filters and pooling layers, whereas RNNs feed results back into the network (in an auto-regressive manner).

### Use-cases of RNNs

• RNNs are better suited to analyzing temporal, sequential data such as text or videos.
• Use cases for RNNs include text translation, natural language processing, sentiment analysis and speech analysis.

### Pros of RNNs vs. CNNs

• In CNNs, the size of the input and the resulting output are fixed. That is, a CNN receives images of fixed size and outputs them to the appropriate level, along with the confidence level of its prediction. In contrast, with RNNs, the model size does not increase with the size of input, thus enabling the possibility of processing inputs of any length. In other words, the size of the input and the resulting output may vary (this feature lends itself to applications where variable size input and output is needed, say in generating text).

• Both Gated Recurrent Units (GRU) and Long Short-Term Memory units (LSTM) provide a solution to the vanishing gradient problem that RNNs experience.
• Let’s delve into each.

#### Gated Recurrent Units (GRU)

• GRUs have two gates, update and reset (which are essentially two vectors), to decide what information should be passed to the output.
• Note that they’re missing the output gate present in LSTMs.

• Reset gate: helps the model to decide how much of the past information (previous time steps) can be forgotten.
• Update gate: helps the model in determining how much of the past information needs to be passed along to the future and how much of the new information needs to be saved in memory.
##### Pros of GRUs compared to LSTMs and RNNs
• Just like RNNs, they are able to keep information for a long time and capture longer range dependencies than RNNs.
• However, unlike LSTMs, GRUs are simpler and can be trained more quickly.
##### Cons of GRUs compared to LSTMs and RNNs
• More parameters compared to RNNs owing to two gates (vs. none in RNNs).
• Typically cannot capture longer range dependencies than LSTMs.

#### Long-Short Term Memory (LSTM) Cells

• Long-Short Term Memory (LSTM) cells are a special kind of RNN that make it easier for RNNs to preserve information over many timestamps by learning long term dependencies.
• Below we can see a visual representation of the LSTM architecture provided from CS224N’s lectures.

• What makes LSTM such a powerhouse is that it departed from the typical neuron-based architecture and instead employed the concept of a memory cell.
• This memory cell retains its value for a short or long time as a function of its input. This allows the cell to remember what’s important and not just the last computed value.
• The LSTM memory cell contains three gates that control the information flow in or out of its cell.
• Input gate:
• The input gate controls when information can flow into the memory.
• Forget gate:
• The forget gate is responsible for keeping track of which information can be “forgotten”, making room for the cell to remember new data.

• Output gate:
• The output gate decides when the information that the cell has stored inside, can be used as the output of the cell.

##### Pros of LSTMs compared to GRUs and RNNs
• Can learn longer term dependencies compared to GRUs and especially RNNs.
##### Cons of LSTMs compared to GRUs and RNNs
• More parameters compared to RNNs and GRUs owing to three gates (vs. two in GRUs and none in RNNs).

## Transformers

• The Transformer architecture was proposed in the paper “Attention is All You Need.
• As shown below, Transformers are big encoder-decoder models able to process a whole sequence with a sophisticated attention mechanism.

• As is the case in NLP applications in general, we begin by turning each input word into a vector using an embedding algorithm.
• The embedding only happens in the bottom-most encoder. The abstraction that is common to all the encoders is that they receive a list of vectors each of the size 512 –- in the bottom encoder that would be the word embeddings, but in other encoders, it would be the output of the encoder that’s directly below. The size of this list is a hyperparameter we can set – basically it would be the length of the longest sentence in our training dataset.
• Attention:
• Attention provides a solution to the bottleneck problem.
• Bottleneck problem: The context vector turned out to be a bottleneck for these types of models. It made it challenging for the models to deal with long sentences.
• Attention allows the model to focus on the relevant parts of the input sequence as needed.
• Attention treats each words representation as a query to access and incorporate information from a set of values.
• Per usual, in the transformer architecture, the encoder is able to pass all the hidden states to the decoder.
• However, before producing its output, with attention, the decoder does an extra step. The decoder multiplies each hidden state by its softmax score, thus amplifying hidden states with higher scores and drowning out the rest.
• This allows the model to focus on the parts of the input that are relevant to the output.
• Self attention:
• Self attention is in the encoder. The calculation process for self-attention is delineated as part of the Attention section in the primer on the Transformer architecture.
• The dimension of the $$Q$$, $$K$$, $$V$$ matrices is 64 while the embeddings and encoder input/output vectors have dimension of 512
• The image below, take from Jay Alammar’s Illustrated Transformer is the best visualization for this.

• What are query, key and value vectors?
• They are abstractions that are useful for calculating attention.
• Cross attention:
• Cross attention is in the decoder. Except for inputs, cross-attention calculation is the same as self-attention.
• Cross-attention combines asymmetrically two separate embedding sequences of same dimension, in contrast self-attention input is a single embedding sequence.
• In order to talk about Transformer, we also have to talk about, two pretrained models which are, BERT and GPT as it led to its success.
• GPT:
• Pretrained decoder with 12 layers.
• 768 dimensional hidden states, 3072 dimensional feed forward hidden layers
• Byte pair encoding with 40,000 merges
• Use case:
• Natural language generation: generating prose/poetry.
• Natural language inference: label pairs of sentences as entailing or contradictory or neutral.
• BERT:
• Pretrained Encoder that uses masked language modeling which replaces some fraction of words in the input with a special [MASK] token and then tries to predict these words.
• Thus, you only need to take loss on the masked words that were predicted.
• Comes in two flavors: BERT-base and BERT-large.
• Both BERT model sizes have a large number of encoder layers (which the paper calls Transformer Blocks) – twelve for the Base version, and twenty four for the Large version. These also have larger feedforward-networks (768 and 1024 hidden units respectively), and more attention heads (12 and 16 respectively) than the default configuration in the reference implementation of the Transformer in the initial paper (6 encoder layers, 512 hidden units, and 8 attention heads).
• Very easy to finetune and is common to do on a single GPU.
• Use case:
• In NLP for translation, especially low-resource language translations.
• A few disadvantages of Transformers are that they have quadratic compute time in self attention whereas, as we saw earlier, RNNs only grew linearly.
• This is a very brief overview of transformers, for an in-depth discourse on Transformers, it is highly recommend to check out the primer on the Transformer architecture.

### Use-cases of Transformers

#### Language

• In a vanilla language model, for example, nearby words would first get grouped together. The transformer, by contrast, runs processes so that every element in the input data connects, or pays attention, to every other element. This is referred to as “self-attention.” This means that as soon as it starts training, the transformer can see traces of the entire data set.
• Before transformers came along, progress on AI language tasks largely lagged behind developments in other areas. Infact, in this deep learning revolution that happened in the past 10 years or so, natural language processing was a latecomer and NLP was, in a sense, behind computer vision, per the computer scientist Anna Rumshisky of the University of Massachusetts, Lowell.
• However, with the arrival of Transformers, the field of NLP has received a much-needed push and has churned model after model that have beat the state-of-the-art in various NLP tasks.
• As an example, to understand the difference between vanilla language models (based on say, a recurrent architecture such as RNNs, LSTMs or GRUs) vs. transformers, consider these sentences: “The owl spied a squirrel. It tried to grab it with its talons but only got the end of its tail.” The structure of the second sentence is confusing: What do those “it”s refer to? A vanilla language model that focuses only on the words immediately around the “it”s would struggle, but a transformer connecting every word to every other word could discern that the owl did the grabbing, and the squirrel lost part of its tail.

#### Vision

In CNNs, you start off being very local and slowly get a global perspective. A CNN recognizes an image pixel by pixel, identifying features like corners or lines by building its way up from the local to the global. But in transformers, with self-attention, even the very first layer of information processing makes connections between distant image locations (just as with language). If a CNN’s approach is like starting at a single pixel and zooming out, a transformer slowly brings the whole fuzzy image into focus.

• CNNs work by repeatedly applying filters on local patches of the input data, generating local feature representations (or “feature maps”) and incrementally increase their receptive field and build up to global feature representations. It is because of convolutions that photo apps can organize your library by faces or tell an avocado apart from a cloud. Prior to the transformer architecture, CNNs were thus considered indispensable to vision tasks.
• With the Vision Transformer (ViT), the architecture of the model is nearly identical to that of the first transformer proposed in 2017, with only minor changes allowing it to analyze images instead of words. Since language tends to be discrete, a lot of adaptations were to discretize the input image to make transformers work with visual input. Exactly mimicing the language approach and performing self-attention on every pixel would be prohibitively expensive in computing time. Instead, ViT divides the larger image into square units, or patches (akin to tokens in NLP). The size is arbitrary, as the tokens could be made larger or smaller depending on the resolution of the original image (the default is 16x16 pixels). But by processing pixels in groups, and applying self-attention to each, the ViT could quickly churn through enormous training data sets, spitting out increasingly accurate classifications.
• In Do Vision Transformers See Like Convolutional Neural Networks?, Raghu et al. sought to understand how self-attention powers transformers in vision-based tasks.

• Compared to the Transformer, other architectures are “one trick ponies” while multimodal learning requires handling of modalities with different patterns within a streamlined architecture with a reasonably high relational inductive bias to even remotely reach human-like intelligence. In other words, we needs a single versatile architecture that seamlessly transitions between senses like reading/seeing, speaking, and listening.
• The potential to offer a universal architecture that can be adopted for multimodal tasks (that requires simultaneously handling multiple types of data, such as raw images, video and language) is something that makes the transformer architecture unique and popular.
• Because of the siloed approach with earlier architectures where each type of data had its own specialized model, this was a difficult task to accomplish. However, transformers offer an easy way to combine multiple input sources. For example, multimodal networks might power a system that reads a person’s lips in addition to listening to their voice using rich representations of both language and image information.
• With cross-attention where the query, key and value vectors are derived from different sources, transformers are able to lend themselves as a powerful tool for multimodal learning.
• The transformer thus offers be a big step toward achieving a kind of “convergence” for neural net architectures, resulting in a universal approach to processing data from multiple modalities.

### Benefits of Transformers compared to RNNs/GRUs/LSTMs

• The Transformer can learn longer-range dependencies than RNNs and its variants such as GRUs and LSTMs.
• The biggest benefit, however, comes from how the Transformer lends itself to parallelization. Unlike an RNN which processes a word at each time step, a key property of the Transformer is that the word at each position flows through its own path in the encoder. There are dependencies between these paths in the self-attention layer (since the self-attention layer computes how important each other word in the input sequence is to this word). However, once the self-attention output is generated, the feed-forward layer does not have those dependencies, and thus the various paths can be executed in parallel while flowing through the feed-forward layer. This is an especially useful trait in case of the Transformer encoder which can process each input word in parallel with other words after the self-attention layer. This feature, is however, not of great importance for the decoder since it generates one word at a time and thus does not utilize parallel word paths.

### Drawbacks of Transformers compared to RNNs/GRUs/LSTMs

• The runtime of Transformer architecture is quadratic in the length of the input sequence, which means it can be slow when processing long documents or taking characters as inputs. In other words, computing all pairs of interactions during self-attention means our computation grows quadratically with the sequence length, i.e., $$O(T^2 d)$$, where $$T$$ is the sequence length, and $$d$$ is the dimensionality. Note that for recurrent models, it only grew linearly!
• Say, $$d = 1000$$. So, for a single (shortish) sentence, $$T \leq 30 \Rightarrow T^{2} \leq 900 \Rightarrow T^2 d \approx 900K$$. Note that in practice, we set a bound such as $$T=512$$. Imagine working on long documents with $$T \geq 10,000$$!?
• Wouldn’t it be nice for Transformers if we didn’t have to compute pair-wise interactions between each word pair in the sentence? Recent studies such as:
• Compared to CNNs, the data appetite of transformers is obscenely high. CNNs are still sample efficient, which makes them great candidates for low-resource tasks. This is especially true for image/video generation tasks where an exceptionally large amount of data is needed, even for CNN architectures (and thus implies that Transformer architectures would have a ridiculously high data requirement). For example, the recent CLIP architecture by Radford et al. was trained with CNN-based ResNets as vision backbones (and not a ViT-like transformer architecture). While transformers do offer accuracy bumps once their data requirement is satisfied, CNNs offer a way to deliver decent accuracy performance in tasks where the amount of data available is not exceptionally high. Both architectures thus have their usecases.
• The runtime of the Transformer architecture is quadratic in the length of the input sequence. Computing attention over all word-pairs requires the number of edges in the graph to scale quadratically with the number of nodes, i.e., in an $$n$$ word sentence, a Transformer would be doing computations over $$n^{2}$$ pairs of words. This implies a large parameter count (implying high memory footprint) and thereby high computational complexity.
• High compute requirements has a negative impact on power and battery life requirements, especially for portable device targets.
• Overall, a transformer requires higher computational power, more data, power/battery life, and memory footprint, for it to offer better performance (in terms of say, accuracy) compared to its conventional competitors.

## Inductive bias

• Every ML algorithm used in practice, from nearest neighbors to gradient boosting machines, comes with its own set of inductive biases about what classifications (or functions, in the case of a regression problem) are easier to learn. Nearly all learning algorithms are biased to learn that items similar to each other (“close” to one another in some feature space) are more likely to be in the same class. Linear models, such as logistic regression, additionally assume that the classes can be separated by a linear boundary (this is a “hard” bias, since the model can’t learn anything else). For regularized regression, which is the type almost always used in ML, there’s an additional bias towards learning a boundary that involves few features, with low feature weights (this is a “soft” bias because the model can learn class boundaries involving lots of features with high weights, it’s just harder/requires more data).

• Even deep learning models have inductive biases, and these biases are part of what makes them so effective. For example, an LSTM (long short-term memory) neural network is effective for natural language processing tasks because it is biased towards preserving contextual information over long sequences.

• As an ML practitioner, knowledge about the domain and difficulty of your problem can help you choose the right algorithm to apply. As an example, let’s take the problem of identifying whether a patient has been diagnosed with metastatic cancer by extracting relevant terms from clinical notes. In this case, logistic regression performs well because there are many independently informative terms (‘mets,’ ‘chemo,’ etc.). For other problems, like extracting the result of a genetic test from a complex PDF report, we get better performance using LSTMs that can account for the long-range context of each word.

• Once a base algorithm has been chosen, understanding its biases can also help you perform feature engineering, the process of choosing which information to input into the learning algorithm.

### Inductive biases of standard neural architectures

• Every model architecture has an inherent inductive bias which helps understand patterns in data and thus enables learning. For instance, CNNs exhibit spatial parameter sharing, translational/spatial invariance, while RNNs exhibit temporal parameter sharing.
• Relational inductive biases, deep learning, and graph networks by Battaglia et al. (2018) from DeepMind/Google, MIT and the University of Edinburgh offers a great overview of the relational inductive biases of various neural net architectures, summarized in the table below from the paper.

## Citation

If you found our work useful, please cite it as:

@article{Chadha2020DistilledDLAlgorithmsCompAnalysis,
title   = {Deep Learning Architectures Comparative Analysis},
journal = {Distilled AI},
year    = {2020},
note    = {\url{https://aman.ai}}
}


text:
# Object Classification with CNNs Using the Keras Deep Learning Library

Last Updated on August 7, 2022

Keras is a Python library for deep learning that wraps the powerful numerical libraries Theano and TensorFlow.

A difficult problem where traditional neural networks fall down is called object recognition. It is where a model is able to identify the objects in images.

In this post, you will discover how to develop and evaluate deep learning models for object recognition in Keras. After completing this tutorial, you will know:

• About the CIFAR-10 object classification dataset and how to load and use it in Keras
• How to create a simple Convolutional Neural Network for object recognition
• How to lift performance by creating deeper Convolutional Neural Networks

Kick-start your project with my new book Deep Learning With Python, including step-by-step tutorials and the Python source code files for all examples.

Let’s get started.

• Jul/2016: First published
• Update Oct/2016: Updated for Keras 1.1.0 and TensorFlow 0.10.0.
• Update Mar/2017: Updated for Keras 2.0.2, TensorFlow 1.0.1 and Theano 0.9.0.
• Update Sep/2019: Updated or Keras 2.2.5 API.
• Update Jul/2022: Updated for TensorFlow 2.x API

For an extended tutorial on developing a CNN for CIFAR-10, see the post:

## The CIFAR-10 Problem Description

The problem of automatically classifying photographs of objects is difficult because of the nearly infinite number of permutations of objects, positions, lighting, and so on. It’s a tough problem.

This is a well-studied problem in computer vision and, more recently, an important demonstration of the capability of deep learning. A standard computer vision and deep learning dataset for this problem was developed by the Canadian Institute for Advanced Research (CIFAR).

The CIFAR-10 dataset consists of 60,000 photos divided into 10 classes (hence the name CIFAR-10). Classes include common objects such as airplanes, automobiles, birds, cats, and so on. The dataset is split in a standard way, where 50,000 images are used for training a model and the remaining 10,000 for evaluating its performance.

The photos are in color with red, green, and blue components but are small, measuring 32 by 32 pixel squares.

State-of-the-art results are achieved using very large convolutional neural networks. You can learn about state-of-the-art results on CIFAR-10 on Rodrigo Benenson’s webpage. Model performance is reported in classification accuracy, with very good performance above 90%, with human performance on the problem at 94% and state-of-the-art results at 96% at the time of writing.

There is a Kaggle competition that makes use of the CIFAR-10 dataset. It is a good place to join the discussion of developing new models for the problem and picking up models and scripts as a starting point.

### Need help with Deep Learning in Python?

Take my free 2-week email course and discover MLPs, CNNs and LSTMs (with code).

Click to sign-up now and also get a free PDF Ebook version of the course.

The CIFAR-10 dataset can easily be loaded in Keras.

Keras has the facility to automatically download standard datasets like CIFAR-10 and store them in the ~/.keras/datasets directory using the cifar10.load_data() function. This dataset is large at 163 megabytes, so it may take a few minutes to download.

The dataset is stored as pickled training and test sets, ready for use in Keras. Each image is represented as a three-dimensional matrix, with dimensions for red, green, blue, width, and height. We can plot images directly using matplotlib.

Running the code creates a 3×3 plot of photographs. The images have been scaled up from their small 32×32 size, but you can clearly see trucks, horses, and cars. You can also see some distortion in some images that have been forced to the square aspect ratio.

Small sample of CIFAR-10 images

## Simple Convolutional Neural Network for CIFAR-10

The CIFAR-10 problem is best solved using a convolutional neural network (CNN).

You can quickly start by defining all the classes and functions you will need in this example.

Next, you can load the CIFAR-10 dataset.

The pixel values range from 0 to 255 for each of the red, green, and blue channels.

It is good practice to work with normalized data. Because the input values are well understood, you can easily normalize to the range 0 to 1 by dividing each value by the maximum observation, which is 255.

Note that the data is loaded as integers, so you must cast it to floating point values in order to perform the division.

The output variables are defined as a vector of integers from 0 to 1 for each class.

You can use a one-hot encoding to transform them into a binary matrix to best model the classification problem. There are ten classes for this problem, so you can expect the binary matrix to have a width of 10.

Let’s start by defining a simple CNN structure as a baseline and evaluate how well it performs on the problem.

You will use a structure with two convolutional layers followed by max pooling and a flattening out of the network to fully connected layers to make predictions.

The baseline network structure can be summarized as follows:

1. Convolutional input layer, 32 feature maps with a size of 3×3, a rectifier activation function, and a weight constraint of max norm set to 3
2. Dropout set to 20%
3. Convolutional layer, 32 feature maps with a size of 3×3, a rectifier activation function, and a weight constraint of max norm set to 3
4. Max Pool layer with size 2×2
5. Flatten layer
6. Fully connected layer with 512 units and a rectifier activation function
7. Dropout set to 50%
8. Fully connected output layer with 10 units and a softmax activation function

A logarithmic loss function is used with the stochastic gradient descent optimization algorithm configured with a large momentum and weight decay start with a learning rate of 0.01.

You can fit this model with 25 epochs and a batch size of 32.

A small number of epochs was chosen to help keep this tutorial moving. Usually, the number of epochs would be one or two orders of magnitude larger for this problem.

Once the model is fit, you evaluate it on the test dataset and print out the classification accuracy.

Tying this all together, the complete example is listed below.

Running this example provides the results below. First, the network structure is summarized, which confirms the design was implemented correctly.

The classification accuracy and loss are printed after each epoch on both the training and test datasets.

Note: Your results may vary given the stochastic nature of the algorithm or evaluation procedure, or differences in numerical precision. Consider running the example a few times and compare the average outcome.

The model is evaluated on the test set and achieves an accuracy of 70.5%, which is not excellent.

You can improve the accuracy significantly by creating a much deeper network. This is what you will look at in the next section.

## Larger Convolutional Neural Network for CIFAR-10

You have seen that a simple CNN performs poorly on this complex problem. In this section, you will look at scaling up the size and complexity of your model.

Let’s design a deep version of the simple CNN above. You can introduce an additional round of convolutions with many more feature maps. You will use the same pattern of Convolutional, Dropout, Convolutional, and Max Pooling layers.

This pattern will be repeated three times with 32, 64, and 128 feature maps. The effect is an increasing number of feature maps with a smaller and smaller size given the max pooling layers. Finally, an additional and larger Dense layer will be used at the output end of the network in an attempt to better translate the large number of feature maps to class values.

A summary of the new network architecture is as follows:

• Convolutional input layer, 32 feature maps with a size of 3×3, and a rectifier activation function
• Dropout layer at 20%
• Convolutional layer, 32 feature maps with a size of 3×3, and a rectifier activation function
• Max Pool layer with size 2×2
• Convolutional layer, 64 feature maps with a size of 3×3, and a rectifier activation function
• Dropout layer at 20%.
• Convolutional layer, 64 feature maps with a size of 3×3, and a rectifier activation function
• Max Pool layer with size 2×2
• Convolutional layer, 128 feature maps with a size of 3×3, and a rectifier activation function
• Dropout layer at 20%
• Convolutional layer,128 feature maps with a size of 3×3, and a rectifier activation function
• Max Pool layer with size 2×2
• Flatten layer
• Dropout layer at 20%
• Fully connected layer with 1024 units and a rectifier activation function
• Dropout layer at 20%
• Fully connected layer with 512 units and a rectifier activation function
• Dropout layer at 20%
• Fully connected output layer with 10 units and a softmax activation function

You can very easily define this network topology in Keras as follows:

You can fit and evaluate this model using the same procedure from above and the same number of epochs but a larger batch size of 64, found through some minor experimentation.

Tying this all together, the complete example is listed below.

Running this example prints the classification accuracy and loss on the training and test datasets for each epoch.

Note: Your results may vary given the stochastic nature of the algorithm or evaluation procedure, or differences in numerical precision. Consider running the example a few times and compare the average outcome.

The estimate of classification accuracy for the final model is 79.5% which is nine points better than our simpler model.

## Extensions to Improve Model Performance

You have achieved good results on this very difficult problem, but you are still a good way from achieving world-class results.

Below are some ideas that you can try to extend upon the models and improve model performance.

• Train for More Epochs. Each model was trained for a very small number of epochs, 25. It is common to train large convolutional neural networks for hundreds or thousands of epochs. You should expect performance gains can be achieved by significantly raising the number of training epochs.
• Image Data Augmentation. The objects in the image vary in their position. Another boost in model performance can likely be achieved by using some data augmentation. Methods such as standardization, random shifts, or horizontal image flips may be beneficial.
• Deeper Network Topology. The larger network presented is deep, but larger networks could be designed for the problem. This may involve more feature maps closer to the input and perhaps less aggressive pooling. Additionally, standard convolutional network topologies that have been shown useful may be adopted and evaluated on the problem.

## Summary

In this post, you discovered how to create deep learning models in Keras for object recognition in photographs.

After working through this tutorial, you learned:

• About the CIFAR-10 dataset and how to load it in Keras and plot ad hoc examples from the dataset
• How to train and evaluate a simple Convolutional Neural Network on the problem
• How to expand a simple Convolutional Neural Network into a deep Convolutional Neural Network in order to boost performance on the difficult problem
• How to use data augmentation to get a further boost on the difficult object recognition problem

### 188 Responses to Object Classification with CNNs Using the Keras Deep Learning Library

1. Aakash Nain July 24, 2016 at 9:53 pm #

Hello Jason,
What is the use of maxnorm in context of deep learning ?

2. Aqsa July 31, 2016 at 4:46 pm #

Hi Jason
I am doing detection of road signs in real time . The size of my images is 800*1360. The size of the road sign varies from 16*16 to 256*256. How can I use convolutional neural network for this puppose to get good detection accuracy in real time

• Jason Brownlee August 1, 2016 at 6:28 am #

Consider how you frame the problem Aqsa. Two options are:

1) You could rescale all images to the same size.
2) You could zero-pad all images.

As for the specifics of the network for this problem, you will have to design and test different structures. Perhaps you can leverage an existing well performing structure like VGG or inception.

3. Jack September 1, 2016 at 12:54 pm #

What should be the input dimensions for 3D dataset of pcd format or off format.?

• Jason Brownlee September 2, 2016 at 8:04 am #

Sorry, I don’t know what those formats are Jack.

• Jack September 2, 2016 at 2:48 pm #

Point cloud data(PCD) contains the x,y and z coordinates of the object….. I want to build a neural network for 3D object classification… The problem I am facing is I don’t know what shd be the input to my network… For a neural network that classifies images you pass the pixel values (0-255), but a pcd file just has the coordinates…Is it wise to pass the coordinates as the inputs?..
I can extract some features of the object ( from pcd file)… Can I pass those features as input ??
I am new to this field, so I having difficulty understanding things…

• Jason Brownlee September 3, 2016 at 6:55 am #

I wonder if you can rescale the coordinates to all have the range 0-to-1. Then provide them directly.

From there, you will have a baseline and can start to explore other transforms of your coords, such as perhaps projections into 2D.

• Sanketh February 18, 2018 at 4:41 am #

Did you figure out a way to do that. I am currently facing a similar situation. Can you tell me how you went on to solve that problem

4. shudhan September 2, 2016 at 4:49 pm #

May i know how to extract features from the images?

• Jason Brownlee September 3, 2016 at 6:58 am #

We no longer need to extract features when using deep learning methods as we are performing automatic feature learning. A great benefit of the approach.

5. Walid Ahmed September 10, 2016 at 4:52 am #

Thanks a lot.

I have one question

In Keras, How can I extract the exact location of the detected object (or objects) within image that includes a background?
I assume it uses sliding window for object detection

• Jason Brownlee September 10, 2016 at 7:12 am #

Great question Walid.

This is called object identification in an image. I do not have an example at the moment, but I will prepare one in the future.

6. Vinay September 12, 2016 at 5:04 am #

Hi…could you please give same example for prima diabetes or airline passenger data set. My question is to apply CNN for direct numeric features. You could give any simple example

7. Walid Ahmed September 14, 2016 at 3:22 am #

Thanks Jason, I will look forward to your example.

8. NotMikeJones September 29, 2016 at 11:31 am #

Where in Keras are you specifying the input dimension for your first convolution layer? I’d like to try a convolution NN with time series for event detection, and am having issues with keras 1d convolution working. Let’s say each of my samples are a timeseries represented by a 1 x 100 vector, and within the vector, I expect three types of events to occur somewhere in those time frames (unclear what the length of the event would be, but lets say roughly 10 time points across). Would I use three feature maps, and then use a ‘window’ of, say 10 time points, that map onto a single neuron in the convolution later? So I would have a convolution layer of 3 x 10?

Thanks!

• Jason Brownlee September 30, 2016 at 7:47 am #

LSTM is the network for dealing with sequences rather than CNN.

CNN is good at spatial structure such as in images or text.

This tutorial on time series with LSTMs might be what you’re looking for:
https://machinelearningmastery.com/time-series-prediction-lstm-recurrent-neural-networks-python-keras/

• NotMikeJones October 1, 2016 at 12:12 am #

Thanks for that! I actually think I misspoke – I don’t want to forecast values at future time points, but instead identify if a current set of timepoints fit a pattern that indicate a certain event.

For example, let’s say we have fitness tracker data with various types of sensors (heart rate, pedometer, accelerometer), and we know a person does three types of activity: yoga, running, cooking. I want to train a model to identify these activities based on sensor data, and then be able to pull real-time data and classify what they are currently doing.

I was thinking a CNN with a windowed-approach would be the best bet, but I might be completely off-base.

• Jason Brownlee October 1, 2016 at 8:02 am #

It does sound like an anomaly detection or change detection problem, you may have benefit in framing the problem this way.

9. Rafi October 18, 2016 at 6:34 pm #

Hi Jason,

I’m running the exact same code in this page which produced a 71.82% accuracy on test data.

only difference is I’m using a validation data set split by 70-30%. I’m getting only less than 11% validation and test accuracy. What could be my mistake? Have you faced such

Thanks
Rafi

35000/35000 [==============================] – 94s – loss: 2.2974 – acc: 0.1158 – val_loss: 2.3033 – val_acc: 0.0991
Epoch 24/25
35000/35000 [==============================] – 94s – loss: 2.2961 – acc: 0.1170 – val_loss: 2.3035 – val_acc: 0.1022
Epoch 25/25
35000/35000 [==============================] – 93s – loss: 2.2954 – acc: 0.1213 – val_loss: 2.3036 – val_acc: 0.0987

10. Walid Ahmed November 2, 2016 at 2:30 am #

Dear Jason

I appreciate if you can illustrate In Keras : How can I extract the exact location of the detected object (or objects) within image that includes a background?

• Jason Brownlee November 2, 2016 at 9:08 am #

Great question Walid,

Sorry, I don’t have an example of object localization with Keras yet. It is on the TODO list though.

11. Augusto Aguirre November 2, 2016 at 2:28 pm #

Hello! Very good explanation!
I am using your model to classify images containing either 0 or 1.
To resolve this, I am able to pleanteas using initially with a variant in the last hidden layer:

model.add (Dense (1, activation = ‘sigmoid’))

For me return a result between 0 or 1.

When I want to adjust the model gives me the following error:
‘Error when checking input model: convolution2d_input_20 expected to have 4 dimensions, but got array With shape (8000, 3072)’
My dataset are 32×32 RGB images. Therefore, contains 3072 columns, but one with a 0 or 1.
Regards!

• Jason Brownlee November 3, 2016 at 7:50 am #

Hi Augusto, sorry to hear about the error on your own data.

It is not clear what the cause could be, sorry. Perhaps you are able to experiment and discover the root cause. Try simplifying your example to the minimum required and see if that helps to flush it out.

12. Walid Ahmed November 11, 2016 at 12:49 am #

Why would I need to apply a dropout layer before a convolutional layer?

It just make sense for me when applied to input layer or any other layers in the fully connected layers.
Request.

• Jason Brownlee November 11, 2016 at 10:03 am #

Hi Walid,

It’s all about putting pressure on the network to force it to generalize. It may or may not be a good pressure point to force this type of learning. Try and see on your problem.

13. William Amador November 11, 2016 at 3:51 am #

Hi,Jason I have a question, how is the procedure for training if in my initial layer I do not handle a single image but a sequence of 30 images. For a specific case a sequence of movements of a person; Do you have any examples of training for a CNN for this case

Thank you

• Jason Brownlee November 11, 2016 at 10:05 am #

Hi William, great question. Sorry, I don’t have any worked examples of working with sequences of images at the moment.

14. Walid Ahmed November 15, 2016 at 2:47 am #

Hi Jason

When I removed the dropout layers before any convolutional layer, my results improved.
especially when the size of dataset is large.

• Jason Brownlee November 15, 2016 at 7:57 am #

Thanks for the feedback Walid. Nice work!

15. Walid Ahmed November 16, 2016 at 6:06 am #

Hi Jason.
Another question , I know that keras comes with different optimizers, in your code you used sgd,others may use another optimizer like adam.
https://keras.io/optimizers/

• Jason Brownlee November 16, 2016 at 9:34 am #

Hi Walid,

I find optimizers generally make minor differences to the results – move the needle less than the network topology.

SGD is well understood and a great place to start. ADAM is fast and gives good results and I often use it in practice. I don’t really have much opinions beyond that.

16. Bharath Paturi November 16, 2016 at 5:54 pm #

Hi Jason,

I have very large images to analyze. Probably each image size will be around 500 MB to 1 GB.
I wanted to apply segmentation to the images. Can we use convolution NN to do unsupervised learning.

• Jason Brownlee November 17, 2016 at 9:52 am #

Ouch Bharath, they are massive images.

It is possible, but you’re going to run out of memory really fast!

I don’t have good advice, sorry. I have not researched this specific problem.

17. Michael November 17, 2016 at 8:40 am #

Hi,

I saw the line where you add a layer has a typo. It should read

model.add(Convolution2D(32, 3, 3, input_shape=(32, 32, 3), border_mode=’same’, activation=’relu’,

• Jason Brownlee November 17, 2016 at 9:58 am #

Are you sure Michael? It all looks good to me and the example runs with Theano and TensorFlow backends.

Maybe I’m missing something?

• Sean July 4, 2017 at 8:35 am #

I think Michael is right. It throws me an error with input_shape=(3,32,32). input_shape=(32,32,3) should be the correct one

• Maxim December 19, 2016 at 1:44 am #

Michael, for solving your problem put the line: K.set_image_dim_ordering(‘th’)

ABOVE the

18. Shristi Baral November 21, 2016 at 10:15 pm #

How do I deal with the error? Got this while running script under “Loading The CIFAR-10 Dataset in Keras”. I tried altering the script of cifar10.py to figure out what the error is. But I couldnot.
—————————————————————————
UnicodeDecodeError Traceback (most recent call last)
in ()
4 from scipy.misc import toimage
—-> 6 (X_train, y_train), (X_test, y_test) = cifar10.load_data()
7 # create a grid of 3×3 images
8 for i in range(0, 9):

18 for i in range(1, 6):
19 fpath = os.path.join(path, ‘data_batch_’ + str(i))
—> 20 data, labels = load_batch(fpath)
21 X_train[(i-1)*10000:i*10000, :, :, :] = data
22 y_train[(i-1)*10000:i*10000] = labels

11 else:
—> 12 d = cPickle.load(f, encoding=”bytes”)
13 # decode utf8
14 for k, v in d.items():

UnicodeDecodeError: ‘ascii’ codec can’t decode byte 0x80 in position 3031: ordinal not in range(128)

• Jason Brownlee November 22, 2016 at 7:05 am #

I have not seen this error before, perhaps post to stack overflow or the Keras list?

19. Aquib Javed Khan November 22, 2016 at 8:35 pm #

Hi, Thanks for the awesome tutorial, you’r codes and explanations helps me lot in understanding the classification task.

I actually want to feed new images want to get the return the label matches to it, How can I do it like I am doing like this:

import keras
from keras.models import Sequential
import cv2
import numpy as np
from keras.preprocessing.image import ImageDataGenerator, array_to_img, img_to_array, load_img
model = Sequential()

model.compile(loss=’binary_crossentropy’,
optimizer=’rmsprop’,
metrics=[‘accuracy’])

img = cv2.resize(img,(150,150))

classes = model.predict_classes(img, batch_size=32)
print classes

I’m getting error:
Exception: Error when checking : expected convolution2d_input_1 to have 4 dimensions, but got array with shape (150, 150)

How to fix it?

• Jason Brownlee November 23, 2016 at 8:58 am #

My suggestion would be to ensure that your loaded data matches the expected input dimensions exactly. You may have to resize or pad new data to make it match.

20. s December 1, 2016 at 1:23 pm #

This is NOT objection detection, it’s classification

21. Walid December 2, 2016 at 4:40 am #

Dear Jason

I am so eager for an example of object localization with Keras yet, I hope you can come with one soon

• Jason Brownlee December 2, 2016 at 8:18 am #

I would like to prepare one soon Walid, hopefully in the new year.

22. Ahmed Desoky December 7, 2016 at 4:55 am #

Hello Jason,

I am working on a classification problem using Keras on kitti dataset. I found that kitti is not supported yet in https://keras.io/datasets/ .

Is there any advice or an introductory point to work out with my problem ?

Thanks for help

• Jason Brownlee December 7, 2016 at 8:56 am #

Sorry, I have not looked at that dataset yet.

23. Thanawin December 7, 2016 at 1:31 pm #

HI Jason

I have seen many tutorials, but this is the best.
I am wondering if “scores = model.evaluate(X_test, y_test, verbose=0)” will take the last model from “model.fit()”. I would really appreciate if you can suggest me how can I evaluate my test data using the best model from “model.fit()”

24. TSchecker December 14, 2016 at 7:35 pm #

Hi Jason,

just in case facing the error

“AttributeError: ‘module’ object has no attribute ‘control_flow_ops'”

import tensorflow as tf
tf.python.control_flow_ops = tf

• Jason Brownlee December 15, 2016 at 8:28 am #

Thanks for sharing TSchecker.

25. yask December 15, 2016 at 3:04 am #

Hi Jason
I would like to extract objects such as ice, water etc. from photograph.
is this library useful for that? How can I define training areas? Do I need to provide sample image of ice, water as a training area to the classifier? Which classifier servers best here? how about Artificial Neural Network ?

26. pranoy December 19, 2016 at 7:58 pm #

which function is used for predicting..

if i give an image of a horse and i want to predict the output. which function i have to use

• Jason Brownlee December 20, 2016 at 7:30 am #

You can use model.predict() to make a prediction on new data.

The new data must have the same shape as the data used to train the network.

• Philip L. December 3, 2019 at 8:30 am #

Hello, I’m a pure newbie to all this but I’m learning all thanks to your amazing tutorials. But I still don’t understand how to use model.predict() especially on this example. Do you have an example of this project with model.predict()?? I really don’t have any idea yet how am I able to code that will load my own dataset to be predicted, rescale it to 32×32 dimension (is this the size of data?? not sure though) and then use that image as X_test. there’s so much I don’t know yet..
Any response will be of so much significance to me.. Thank you

• Jason Brownlee December 3, 2019 at 1:32 pm #

Thanks!

You can predict for a single image by calling model.predict(), it will take an array with the shape [samples, rows, cols, channels], where you probably have 1 sample, 32×32 rows and cols and 3 channels.

Does that help?

27. Milos December 29, 2016 at 10:34 pm #

Hi Jason,

great article. I have a question. In Dense part you have specified 512 neurons. Can you tell me how did you determine number of neurons?

tnx

• Jason Brownlee December 30, 2016 at 5:50 am #

Hi Milos, I used trial and error. Selecting the number and size of layer is an art – test a lot of configs.

• Milos December 30, 2016 at 7:07 pm #

Great and really useful articles I have found on your site :).

28. Deepak January 30, 2017 at 3:56 pm #

TypeError Traceback (most recent call last)
in ()
—-> 1 model.predict(tmp)

/usr/local/lib/python2.7/dist-packages/keras/models.pyc in predict(self, x, batch_size, verbose)
722 if self.model is None:
723 self.build()
–> 724 return self.model.predict(x, batch_size=batch_size, verbose=verbose)
725
726 def predict_on_batch(self, x):

/usr/local/lib/python2.7/dist-packages/keras/engine/training.pyc in predict(self, x, batch_size, verbose)
1266 f = self.predict_function
1267 return self._predict_loop(f, ins,
-> 1268 batch_size=batch_size, verbose=verbose)
1269
1270 def train_on_batch(self, x, y,

/usr/local/lib/python2.7/dist-packages/keras/engine/training.pyc in _predict_loop(self, f, ins, batch_size, verbose)
944 ins_batch = slice_X(ins, batch_ids)
945
–> 946 batch_outs = f(ins_batch)
947 if not isinstance(batch_outs, list):
948 batch_outs = [batch_outs]

/usr/local/lib/python2.7/dist-packages/keras/backend/theano_backend.pyc in __call__(self, inputs)
957 def __call__(self, inputs):
958 assert isinstance(inputs, (list, tuple))
–> 959 return self.function(*inputs)
960
961

/usr/local/lib/python2.7/dist-packages/Theano-0.9.0.dev5-py2.7.egg/theano/compile/function_module.pyc in __call__(self, *args, **kwargs)
786 s.storage[0] = s.type.filter(
787 arg, strict=s.strict,
–> 788 allow_downcast=s.allow_downcast)
789
790 except Exception as e:

/usr/local/lib/python2.7/dist-packages/Theano-0.9.0.dev5-py2.7.egg/theano/tensor/type.pyc in filter(self, data, strict, allow_downcast)
115 if allow_downcast:
116 # Convert to self.dtype, regardless of the type of data
–> 117 data = theano._asarray(data, dtype=self.dtype)
118 # TODO: consider to pad shape with ones to make it consistent
119 # with self.broadcastable… like vector->row type thing

/usr/local/lib/python2.7/dist-packages/Theano-0.9.0.dev5-py2.7.egg/theano/misc/safe_asarray.pyc in _asarray(a, dtype, order)
32 dtype = theano.config.floatX
33 dtype = numpy.dtype(dtype) # Convert into dtype object.
—> 34 rval = numpy.asarray(a, dtype=dtype, order=order)
35 # Note that dtype comparison must be done by comparing their num
36 # attribute. One cannot assume that two identical data types are pointers

/home/yashwanth/.local/lib/python2.7/site-packages/numpy/core/numeric.pyc in asarray(a, dtype, order)
529
530 “””
–> 531 return array(a, dtype, copy=False, order=order)
532
533

TypeError: Bad input argument to theano function with name “/usr/local/lib/python2.7/dist-packages/keras/backend/theano_backend.py:955” at index 0 (0-based).
Backtrace when that variable is created:

File “/usr/local/lib/python2.7/dist-packages/IPython/core/interactiveshell.py”, line 2821, in run_ast_nodes
if self.run_code(code, result):
File “/usr/local/lib/python2.7/dist-packages/IPython/core/interactiveshell.py”, line 2881, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File “”, line 2, in
model.add(Convolution1D(64, 2, input_shape=[1,4], border_mode=’same’, activation=’relu’, W_constraint=maxnorm(3)))
File “/usr/local/lib/python2.7/dist-packages/keras/models.py”, line 299, in add
layer.create_input_layer(batch_input_shape, input_dtype)
File “/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py”, line 397, in create_input_layer
dtype=input_dtype, name=name)
File “/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py”, line 1198, in Input
input_tensor=tensor)
File “/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py”, line 1116, in __init__
name=self.name)
File “/usr/local/lib/python2.7/dist-packages/keras/backend/theano_backend.py”, line 110, in placeholder
float() argument must be a string or a number

• Jason Brownlee February 1, 2017 at 10:21 am #

I’m sorry to hear that.

The cause is not obvious to me, the stack trace is hard to read.

Perhaps you could try posting to stackoverflow or the Keras google group?

29. Rajesh February 6, 2017 at 4:08 am #

Hi Jason,

Any idea about the following error? Everything looked good till the model.summary point. But when I tried to fit the model, I am seeing the following error.

ValueError Traceback (most recent call last)
in ()
1 # Fit the model
—-> 2 model.fit(X_train, y_train, validation_data=(X_test, y_test), nb_epoch=epochs, batch_size=32)
3 # Final evaluation of the model
4 scores = model.evaluate(X_test, y_test, verbose=0)
5 print(“Accuracy: %.2f%%” % (scores[1]*100))

/home/rajesh/anaconda2/lib/python2.7/site-packages/keras/models.pyc in fit(self, x, y, batch_size, nb_epoch, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, **kwargs)
670 class_weight=class_weight,
671 sample_weight=sample_weight,
–> 672 initial_epoch=initial_epoch)
673
674 def evaluate(self, x, y, batch_size=32, verbose=1,

/home/rajesh/anaconda2/lib/python2.7/site-packages/keras/engine/training.pyc in fit(self, x, y, batch_size, nb_epoch, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch)
1115 class_weight=class_weight,
1116 check_batch_axis=False,
-> 1117 batch_size=batch_size)
1118 # prepare validation data
1119 if validation_data:

/home/rajesh/anaconda2/lib/python2.7/site-packages/keras/engine/training.pyc in _standardize_user_data(self, x, y, sample_weight, class_weight, check_batch_axis, batch_size)
1028 self.internal_input_shapes,
1029 check_batch_axis=False,
-> 1030 exception_prefix=’model input’)
1031 y = standardize_input_data(y, self.output_names,
1032 output_shapes,

/home/rajesh/anaconda2/lib/python2.7/site-packages/keras/engine/training.pyc in standardize_input_data(data, names, shapes, check_batch_axis, exception_prefix)
122 ‘ to have shape ‘ + str(shapes[i]) +
123 ‘ but got array with shape ‘ +
–> 124 str(array.shape))
125 return arrays
126

ValueError: Error when checking model input: expected convolution2d_input_6 to have shape (None, 3, 32, 32) but got array with shape (50000, 32, 32, 3)

• Rajesh February 6, 2017 at 4:22 am #

I understood the issue. I was mistaking at the input_shape step.

It is a very nice tutorial. very well written. Thanks

• Rajesh February 6, 2017 at 8:59 am #

Sorry to spam..but I am still seeing the error 🙁

• Jason Brownlee February 6, 2017 at 9:45 am #

Hi Rajesh, you may want to confirm that you are not missing any lines of code.

Also, confirm your version of Keras, TensorFlow/Theano and Python.

30. Sam February 6, 2017 at 9:11 pm #

Hello,
Thank you for your example. I tried to run your code. But our network server is blocked and the code could not get data from the below url.

Can you let me know if there is another way to make training data set without using download wrapper method ?

• Sam February 6, 2017 at 9:37 pm #

Oh, I found a solution through googling. Thanks anyway.

31. John February 7, 2017 at 2:10 pm #

Hello,

How I can recognize bike in video. It would be great if you can give example.

• Jason Brownlee February 8, 2017 at 9:32 am #

Great question John, it is an area I’d like to cover in the future.

32. Ikhsan March 17, 2017 at 2:22 pm #

Hi Jason,

Thanks for the tutorial. I have a question about the random.seed(seed). Why do we need to seed it first and where is the random number generator used in the rest of the code?

Thanks.

33. Ali April 21, 2017 at 5:43 pm #

Hi jason

Thanks for this great tutorial, I am trying to run the code but I got the following error any suggestion to fix

I am using keras with theona 2.7 back end

model = Sequential()
# Compile model
epochs = 25
lrate = 0.01
decay = lrate/epochs
sgd = SGD(lr=lrate, momentum=0.9, decay=decay, nesterov=False)
model.compile(loss=’categorical_crossentropy’, optimizer=sgd, metrics=[‘accuracy’])
print(model.summary())

Traceback (most recent call last):

File “”, line 2, in

TypeError: __init__() takes at least 4 arguments (4 given)

• Jason Brownlee April 22, 2017 at 9:24 am #

I’m not sure Ali, I have not seen this error before.

Perhaps confirm that you have copied the code exactly?

Consider removing arguments to help zoom in on the cause of the fault.

34. AIZEN May 1, 2017 at 12:12 am #

Hi, i already manage to detect object in the still images. What am i supposed to do to detect objects in a video input? i intend to use the same Keras CNN.

• Jason Brownlee May 1, 2017 at 5:57 am #

You could process each frame of the video as an image with a CNN and use an LSTM to handle sequences of data from the CNN.

35. Supriya May 2, 2017 at 2:24 am #

Hi, in this example we have created CNN model , but how to test it..

• Jason Brownlee May 2, 2017 at 6:02 am #

You can use a train/test split or k-fold cross validation.

36. Chao May 18, 2017 at 3:55 pm #

What does “kernel_constraint=maxnorm(3)” mean?

Thanks a lot!

37. Mohamed Mnete June 1, 2017 at 5:22 pm #

Hi, say I had an image I would like the model you just made to predict what it has on it. I save it in the same directory as the python file. How can I load this to put into he prediction function and how can I write the prediction function. I would also like to try and use my own images on your model. I was also stuck on the technique of changing the images to numpy.arrays. I would also like the output to be either -1 or 1 . How can I code this up. Please help….

38. Natthaphon June 8, 2017 at 1:25 pm #

So can I insert annotation in each images

39. Daniel June 18, 2017 at 5:33 am #

Hi,

I’m trying to do an image classifier that determines if something should be given a specific hashtag or not. My problem is that the accuracy of the classifier after each epoch remains constant, and is essentially assigning the same class to all images. This makes no sense as the image classes are fairly distinct (#gym and #foraging). I basically copied the smaller CNN you used:

model = Sequential()
model.add(Convolution2D(32, 3, 3, input_shape=(3, 100, 100), activation=’relu’))

epochs = 10
lrate = 0.001
decay = lrate/epochs
sgd = SGD(lr=lrate, momentum=0.9, decay=decay, nesterov=False)
print(‘compiling model’)
model.compile(loss=’binary_crossentropy’, optimizer=sgd, metrics=[‘accuracy’])

print(“fitting model”)
model.fit(X_training, Y_training, nb_epoch=epochs, batch_size=100)

But every time, I get these results:

fitting model
Train on 908 samples, validate on 908 samples
Epoch 1/10
908/908 [==============================] – 104s – loss: 7.9712 – acc: 0.5000
Epoch 2/10
908/908 [==============================] – 104s – loss: 7.9712 – acc: 0.5000
Epoch 3/10
908/908 [==============================] – 104s – loss: 7.9712 – acc: 0.5000
Epoch 4/10
908/908 [==============================] – 104s – loss: 7.9712 – acc: 0.5000
Epoch 5/10
908/908 [==============================] – 104s – loss: 7.9712 – acc: 0.5000
Epoch 6/10
908/908 [==============================] – 104s – loss: 7.9712 – acc: 0.5000
Epoch 7/10
908/908 [==============================] – 104s – loss: 7.9712 – acc: 0.5000
Epoch 8/10
908/908 [==============================] – 104s – loss: 7.9712 – acc: 0.5000
Epoch 9/10
908/908 [==============================] – 104s – loss: 7.9712 – acc: 0.5000
Epoch 10/10
908/908 [==============================] – 104s – loss: 7.9712 – acc: 0.5000

What am I doing wrong?

40. Darlington Akogo June 28, 2017 at 10:42 am #

Hello Jason, thanks for your great tutorials! I need some help please, I’m working on an a medical image recognition(diagnostics) project based on your tutorials. Over here, you import the Cifar10 dataset provided via Keras with the load_data() function, but since am downloading the dataset from a diff source, that wouldn’t be possible. So from some research, I came across Keras’ flow_from_directory() function for Image data processing, which is amazing, I could just separate the images into folders and it’d consider them as classes. However, medical images are in the “DICOM” image format, and Keras image functions don’t seem to support it, so with further research, I came across pydicom module, for processing DICOM images in python, however, now, I can’t use flow_from_directory(), can you PLEASE offer some help as to how I can train my ConvNet model with DICOM images and be able to use it to classify(with predict() function) any new DICOM image?

• Jason Brownlee June 29, 2017 at 6:28 am #

I don’t know about that format.

Perhaps you can covert the images?
Perhaps you can put together your own DICOM-comapitable flow from dir function?

• shila mosammami December 10, 2017 at 9:21 am #

Dear Friend,
try this:

41. Bruce Wind June 29, 2017 at 11:30 am #

Hi, Jason, thanks for sharing. I test the code you provided, but my machine does not support CUDA, so it runs very slowly( half an hour per epoch). Since you have such a powerful computer, could you please show the results after hundreds or thousands epoches later? Thanks.

42. Nunu July 23, 2017 at 10:00 pm #

Dear Jason,
Really it is a very nice tutorial :). If you will plot acc vs acc_val there is a gap between the graph of acc and the graph of acc_val does this mean an overfitting ?! and also what I ( correct me if iam wrong) the accuracy graph should become after a certain number of epochs asymptotic( that is the acuracy will not increase anymore) !!
Noura

• Jason Brownlee July 24, 2017 at 6:54 am #

If acc is less than val_acc than it may mean that the model is underfitting and that perhaps a larger model or a model fit for longer would do better on the validation set.

• Nunu July 24, 2017 at 6:55 pm #

yes it is true but also if the acc_val is more than the acc then there is overfitting and I noticed this in your both results above ! what could be the reason of the overfitting ? and what we can do to get rid form it.

regards
Nunu

• Jason Brownlee July 25, 2017 at 9:40 am #

Perhaps train less, perhaps train a smaller model, perhaps add some regularization like dropout.

I hope that helps as a start.

43. Nunu July 25, 2017 at 7:40 pm #

Yes I added dropouts and I added one more fully connected layer and i guess it worked.

Thanks a lot Jason 🙂
Best regards,
Nunu

44. Luna August 27, 2017 at 6:46 pm #

Hi Jason,

Normally CNN for image classification will result in a vector contented probabilities of possible class. Is there a function in this library to extract the vector? Thank you!

• Jason Brownlee August 28, 2017 at 6:49 am #

You can use a softmax activation function on the output layer to get probabilities for multiple classes or sigmoid activation for binary class probabilities.

45. Walid Ahmed September 2, 2017 at 4:55 am #

Hello,
is there is a way to apply the keras.predict function in a vectorized manner so that it can handle more than one input(image) simultaneously?

Thanks

• Jason Brownlee September 2, 2017 at 6:17 am #

Yes, I believe predict() can take a list of samples (X) and return a list of predictions (yhat).

46. zoda September 5, 2017 at 3:32 am #

hello,

what can i do if i have one image which consistent of 6 objects in the image how can i detect and measure the size of the object in the image. it would be helpful if you gave me some tips
thank you .

• Jason Brownlee September 7, 2017 at 12:37 pm #

You can use object localization.

Sorry, I don’t have an example.

47. Sunil A Patel September 24, 2017 at 4:10 pm #

Sir,

I want to detect hand in a image. my problem is the size of hand is tiny in a image and its vary image to image(my image size is 1920*1080),. An image contain hand, cup ,face, table, etc…..

My problem is
1. should we train CNN for various hand shape and fixed size for example 50X50 or else
2. how to find localization of hand.

• Jason Brownlee September 25, 2017 at 5:37 am #

Sorry, I don’t have examples of object localization. I hope to develop examples in the future.

48. vinay October 17, 2017 at 12:00 am #

How to run this code on GPU enabled machine?

49. Miles October 18, 2017 at 1:28 am #

Hi Jason,

In the first example network, you use a maxnorm kernel constraint in all hidden layers (conv and fully connected). I understand the advantages of this, particularly when used in combination with dropout. I’m wondering if there was a reason you removed the kernel constraint from the conv. layers in the second iteration of your model? If so, would you mind explaining the motivation for doing so? Thanks!

• Jason Brownlee October 18, 2017 at 5:39 am #

It was few years ago now, I don’t remember. Perhaps the model achieved better results without the constraint?

50. Tin Tran October 18, 2017 at 6:44 pm #

Hi Jason,

When we completed training the model, how do we use that model with a new picture?

51. Gabriele Minniti October 28, 2017 at 8:10 pm #

Hi Jason! My name is Gabriele, and I’m a young data scientis! I’m current working with the CNNs and i’ve few question for you.

1. I’ve seen that you are using images 32 x 32, my question is about which is the best shape for an high-resolution image of 1070 – 720? It’s necessary that the images are squared?

2. How significatly could be the improvement of the number of epoch in ‘accuracy metrics’? Maybe 100 epoch? 1000? Could them return an high perfomarce model?

Thanks a lot!
PS. Sorry for my “not perfect english” i hope that you understand my questions!
GM

52. Kumar November 7, 2017 at 6:44 pm #

Dear Jason,

I am trying to identify multiple objects in an image and count the number of objects for each class in each image. Can you help me on this.

• Jason Brownlee November 8, 2017 at 9:21 am #

Sounds like a great problem, but I don’t have material on this type of problem, sorry.

53. Arun November 15, 2017 at 12:10 am #

hey

Traceback (most recent call last):

File “”, line 1, in
b=model.fit(X, Y,validation_split=0.33, epochs=150, batch_size=10,verbose=0)

File “/Users/arun/anaconda/lib/python3.6/site-packages/keras/models.py”, line 893, in fit
initial_epoch=initial_epoch)

File “/Users/arun/anaconda/lib/python3.6/site-packages/keras/engine/training.py”, line 1555, in fit
batch_size=batch_size)

File “/Users/arun/anaconda/lib/python3.6/site-packages/keras/engine/training.py”, line 1409, in _standardize_user_data
exception_prefix=’input’)

File “/Users/arun/anaconda/lib/python3.6/site-packages/keras/engine/training.py”, line 126, in _standardize_input_data
array = data[i]

UnboundLocalError: local variable ‘arrays’ referenced before assignment

I am getting this error when trying to fit the model. Looks like its an error in Keras.

54. Himmat Ram Bairwa November 16, 2017 at 6:37 pm #

hi
i have give my own .jpg image then i want to predict the class but i am getting wrong result.

img = cv2.resize(img,(32,32))
img = numpy.array(img)
img = numpy.reshape(img,[1,3,32,32])
preds = model.predict_classes(img)

• Jason Brownlee November 17, 2017 at 9:23 am #

Perhaps the model is not good enough for your example?

Try more examples, try training the model with more examples like the example you tried, try augmenting during training, and so on.

55. Sheila M December 10, 2017 at 9:38 am #

Dear Jason,
First thanks a million for the tutorial you have provided.
I am trying to work on liver segmentation with ConvNN, and my data set is Sliver07 which includes meta images. there are two folders named scan which includes the oroginal meta images of CTscan, 20 on total and the second folder named segment which includes 20 meta images of segmented livers. Each instance has 2 formats .mhd and .raw, they can be easily viewed by the itksnap. I can load data and get the numpy array but the numpy array i get from the output for the original images and segmented images are as below:
[[[-1007 -997 -1007 …, -973 -969 -1007]
[-1002 -1000 -1007 …, -979 -999 -1015]
[-1000 -993 -1003 …, -999 -1009 -1007]
…,
[ -894 -877 -883 …, -893 -895 -906]
[ -872 -878 -894 …, -896 -892 -894]
[ -873 -882 -893 …, -898 -909 -896]]

[[-1005 -999 -1006 …, -972 -964 -1008]
[-1008 -995 -1004 …, -966 -995 -1022]
[-1004 -991 -1001 …, -990 -1015 -1008]
…,
[ -905 -882 -897 …, -889 -878 -895]
[ -879 -884 -910 …, -885 -880 -891]
[ -874 -893 -907 …, -882 -887 -897]]

[[-1001 -1010 -1009 …, -991 -967 -1000]
[-1000 -1004 -1006 …, -977 -989 -1019]
[ -993 -993 -1001 …, -988 -1013 -1001]
…,
[ -902 -918 -911 …, -899 -888 -892]
[ -905 -909 -911 …, -899 -887 -888]
[ -909 -911 -908 …, -900 -901 -896]]

…,
[[-1001 -1002 -1002 …, -1007 -998 -1004]
[-1004 -1004 -1005 …, -1003 -1009 -1000]
[ -996 -1002 -1005 …, -996 -1000 -988]
…,
[ -888 -896 -897 …, -883 -888 -898]
[ -879 -875 -866 …, -886 -881 -894]
[ -878 -866 -867 …, -895 -891 -896]]

[[ -986 -990 -997 …, -1010 -1004 -1008]
[ -999 -994 -995 …, -1003 -1007 -1002]
[-1006 -1011 -1006 …, -1000 -996 -980]
…,
[ -887 -894 -900 …, -888 -893 -903]
[ -882 -885 -879 …, -891 -883 -892]
[ -880 -876 -876 …, -896 -884 -882]]

[[-1000 -1004 -1003 …, -1002 -1003 -997]
[-1007 -1002 -1004 …, -1002 -997 -998]
[-1013 -1011 -997 …, -998 -995 -993]
…,
[ -887 -894 -891 …, -886 -891 -900]
[ -892 -892 -885 …, -892 -885 -893]
[ -895 -893 -885 …, -902 -893 -889]]]
segmented numpy:
[[[0 0 0 …, 0 0 0]
[0 0 0 …, 0 0 0]
[0 0 0 …, 0 0 0]
…,
[0 0 0 …, 0 0 0]
[0 0 0 …, 0 0 0]
[0 0 0 …, 0 0 0]]

[[0 0 0 …, 0 0 0]
[0 0 0 …, 0 0 0]
[0 0 0 …, 0 0 0]
…,
[0 0 0 …, 0 0 0]
[0 0 0 …, 0 0 0]
[0 0 0 …, 0 0 0]]

[[0 0 0 …, 0 0 0]
[0 0 0 …, 0 0 0]
[0 0 0 …, 0 0 0]
…,
[0 0 0 …, 0 0 0]
[0 0 0 …, 0 0 0]
[0 0 0 …, 0 0 0]]

…,
[[0 0 0 …, 0 0 0]
[0 0 0 …, 0 0 0]
[0 0 0 …, 0 0 0]
…,
[0 0 0 …, 0 0 0]
[0 0 0 …, 0 0 0]
[0 0 0 …, 0 0 0]]

[[0 0 0 …, 0 0 0]
[0 0 0 …, 0 0 0]
[0 0 0 …, 0 0 0]
…,
[0 0 0 …, 0 0 0]
[0 0 0 …, 0 0 0]
[0 0 0 …, 0 0 0]]

[[0 0 0 …, 0 0 0]
[0 0 0 …, 0 0 0]
[0 0 0 …, 0 0 0]
…,
[0 0 0 …, 0 0 0]
[0 0 0 …, 0 0 0]
[0 0 0 …, 0 0 0]]]
1) I have no idea how i can build a model for this.
2)The numbers seem to be so strange for me, why all negative in original and all zero in segmented ones?
3) I expected to have 20 array for each folder, which I cannot make sense with these results.
Best Regards

• Jason Brownlee December 11, 2017 at 5:20 am #

56. Fabrizio January 18, 2018 at 11:29 pm #

Hi Jason,
I’m currently working on a real-time object detection. I have some constrains I need to keep in mind: this program should work under an ordinary PC desktop and the main idea is that it should be able to detect objects taking a video as input. So it shouldn’t be expensive when it comes to computational cost.
Is a CNN a good approach to solve this problem? Googling, I’ve seen a pretrained DNN on OpenCV (I should use OpenCV as much as possible) but it runs under Caffe and I have no idea about Caffe.
That’s why I am here and I think a CNN would be a nice approach. What do you think? Do you have some extra advices/tips?

• Jason Brownlee January 19, 2018 at 6:29 am #

I think running a CNN is straight forward, training is very slow.

57. ali February 3, 2018 at 6:37 am #

hi.i did not see any image identification!

58. Pratip March 11, 2018 at 12:35 am #

Hello Sir ,

Can you please tell me what is the error , i have just copied the above code but it displays some error .

ValueError: Error when checking input: expected conv2d_26_input to have shape (3, 32, 32) but got array with shape (32, 32, 3)

I have tried changing the input shape but error still exists .

• Jason Brownlee March 11, 2018 at 6:29 am #

It looks like the model expects the data to be one size and the data is another. You can change the model or change the data.

59. Satendra Ramesh Varma March 26, 2018 at 7:56 am #

It takes too long for CIFAR dataset to download (EST 4 HRS+). Is there a way we can load the data directly using the zip file from their website ?

• Satendra Ramesh Varma March 26, 2018 at 8:29 am #

Never mind. I found the solution.
Place the zip file in keras–>datasets folder.
The zip file contains the batches folder.
Now use 7zip on windows to compress the file first as tar and then as zip using Add to Archive option in right click menu
And run the code again.
Saves a lot of time and no code changes.

• Jason Brownlee March 26, 2018 at 10:04 am #

I’m not sure off hand, sorry.

60. Meenakshi Choudhary April 7, 2018 at 4:15 pm #

Could You please share the code for fusion of multiple convolutional neural networks in python/R/Matlab.

61. Satyajit Pattnaik April 9, 2018 at 3:15 pm #

Need a direction on how to proceed with Crack detection, I have certain images of some machines out of which some images have cracks on it, need to identify the images having cracks, how this problem can be solved?

Tried using Canny edge detection, but after finding the edges, a bit stuck how to proceed, if there are any other ways to achieve, please reply, i shall try working on it..

• Jason Brownlee April 10, 2018 at 6:13 am #

Sounds like a great problem. Perhaps you can use transfer learning with a pre-trained model like the VGG to get started with model development?

62. Raj April 12, 2018 at 11:14 pm #

Dear Jason,

I have some images of scatterplots (matplotlib) and idl file containing the coordinates for the bounding boxes of the tick marks, tick values and points, and I want to do the object detection task now can you please suggest me some pretrained models with which I can do object detection for the tick marks, tick values and points?

• Jason Brownlee April 13, 2018 at 6:41 am #

That sounds like a great project and using a pre-trained model is a great way to accelerate progress.

https://keras.io/applications/

• Raj April 15, 2018 at 2:03 am #

Dear Jason,
Thank you for your reply . Actually I have one more question, in object detection task if I am giving the images and the (Xmin,Ymin,Xmax,Ymax) i.e. bounding boxes in the training of the model then what will I give in the testing of the model only the images ?

• Jason Brownlee April 15, 2018 at 6:35 am #

I’m not sure I follow, what do you mean exactly?

63. Raj April 15, 2018 at 8:26 am #

I mean if I have training and testing images and I have the bounding boxes for the training as well as testing images and I am giving the images and the bounding boxes as the labelled data to my model for training then what will I give during the testing of my model only the testing images or images with the bounding boxes (labelled data ) ?

64. Kartik May 10, 2018 at 4:23 pm #

Hi, I want to detect an object as small as 30×40 pixels in an image that is 6000×4000 pixels. Would this approach be correct ?

• Jason Brownlee May 11, 2018 at 6:32 am #

Wow, it might be challenging.

Try it and see.

65. Ekaterina May 17, 2018 at 10:49 am #

Hi everyone, I have a problem with importing cifar10 dataset:

(x_train, y_train), (x_test, y_test) = cifar10.load_data()

AttributeError Traceback (most recent call last)
in ()
3
4 # load the pre-shuffled train and test data
—-> 5 (x_train, y_train), (x_test, y_test) = cifar10.load_data()

18 for i in range(1, 6):
19 fpath = os.path.join(path, ‘data_batch_’ + str(i))
—> 20 data, labels = load_batch(fpath)
21 X_train[(i-1)*10000:i*10000, :, :, :] = data
22 y_train[(i-1)*10000:i*10000] = labels

14 for k, v in d.items():
15 del(d[k])
—> 16 d[k.decode(“utf8”)] = v
17 f.close()
18 data = d[“data”]

AttributeError: ‘str’ object has no attribute ‘decode’

Did anyone have this problem? Any solution?

• Ekaterina May 17, 2018 at 1:06 pm #

Solved by reinstalling Keras

• Jason Brownlee May 17, 2018 at 3:11 pm #

I’m sorry to hear that. Perhaps try posting your code and error to stackoverflow?

66. Julia June 7, 2018 at 1:48 am #

Hi, thanks for this tutorial, it is very helpful! I used the same architecture as your deep CNN but it is taking over 10 minutes to do one epoch, but on yours it looks like it only took about 30 seconds. As far as I can tell, my code is the same as yours and my computer has never been this slow before. Any ideas on what might be causing this?

67. Anindya July 19, 2018 at 10:36 pm #

Hi Jason,

Great tutorial, very helpful for my project. 🙂
It will be very helpful if you can publish any post on real time object detection and localization, like YOLO or SSD. If you already have any post on that can you please share the link here. Thanks 🙂

68. Inder August 5, 2018 at 4:53 pm #

Jason, what if our testing image have other object which is not from this dataset. or simply plain image without class object.
This will still classify and match something from our class set, how we can handle this problem.
One solution i found is create a class which have only negative images (not related to existing classes), then in this world there are two many different negative images which is hard to collect. please explain how we can make this more perfect.

69. Vishwas August 7, 2018 at 9:02 pm #

Hi Jason,

How do we decide on the number of Convolutional layers for an image,Is it Trial-error method or is there a rule behind it

70. Suprasad Kamath August 9, 2018 at 4:26 pm #

Thanks a lot, Jason, for all your posts. It has given me the confidence to write simple deep learning models using keras. This post really helped me create my first CNN model, something which I have been banging my head around for a few days. I just have few questions.

1) How did you derive the number of neurons for each hidden layer? I know we can do hyperparameter tuning using GridSearchCV from scikit-learn as detailed out in your other blog post. I am able to tune the neurons for the input layer but I get an error when I try to tune the subsequent layers.

2) How can I check if my model is overfitting or underfitting? I know you have used Dropout to hep reduce overfitting. But is there a way I can be sure if my model is overfitting or underfitting?

3) Why have you added two Conv2D layers followed by just one MaxPooling2D layer? I was under the impression that max pooling has to be done after every convolution layer. Can you please put some more light on this?

4) I am using tensorflow-gpu as my backend. In order for my gpu to take the processing load I have to increase the batch_size. I am just curious to know how batch_size affects accuracy while there is a significant boost to processing speed.

71. Suprasad Kamath August 11, 2018 at 1:35 pm #

Thank you, sir. I will go through these links too.

I do have one small suggestion. The example you have used did not have many steps to prepare the training and test datasets. What if someone wanted to use their own image files to create a CNN model instead of the standard CIFAR-10? This is where I had a lot of problems, something which is not as straightforward as other supervised ML codes. I finally figured it out after a lot of trial and error. It would really help beginners like me who visit your blog posts if an example was provided (if you have not already written a post on this) where own data is used to build a model. This is just my personal opinion as only I know how hard it was for me to write an end to end program. I am assuming there would be many more like me who are new to deep learning.

Thanks again for all your blog posts.

72. Deepanshu Agarwal August 13, 2018 at 3:05 pm #

Hi Jason,

I am working on a similar project, in which i have to determine whether handwritten signature is present or not in a scanned image.

Could you please suggest some approach solve this problem, to get yes or no as output for signature presence.

Thanks

• Jason Brownlee August 14, 2018 at 6:14 am #

You might be able to solve it with classical computer vision techniques>, maybe you don’t need a learned model?

E.g. if white space on document is not empty, then there is a signature.

73. Dhiraj August 27, 2018 at 7:47 pm #

Hi Jason,

Your article helped me a lot to understand the implementation of the convolutional neural net.
Thank you.

I want to know can I implement the same concept to differentiate between different types of coin using CNN ?

74. YASH MEHTA October 19, 2018 at 8:44 pm #

Hi Jason,

I am doing detecting the water leakage on raod using CNN with kears library. How to decide the no of convolution layer to be applied and the values that are there in the conv2d function. From where i can get to understang this basics of this things. The dataset of water leakage is not available so i myself downloading the images creating the dataset so how much data i need to get better result. Can i train model using 400 images because i m not able to find more images

• Jason Brownlee October 20, 2018 at 5:55 am #

I recommend testing a suite of different model configurations in order to discover what works best for your specific problem.

75. Arjun November 10, 2018 at 1:54 pm #

Hi Jason…this is a great post on CNN and thanks for this awsome post.
I would be glad if u could tell me how can we implement object localisation on this…..Thanks in advance😁

76. prisilla January 8, 2019 at 11:16 pm #

Hi Jason,

I trying evaluate the total accuracy

my piece of code is as follows
training_set = train_datagen.flow_from_directory(‘dataset/training_set’,
target_size = (64, 64),
batch_size = batch_size,
class_mode = ‘binary’)
test_set = test_datagen.flow_from_directory(‘dataset/test_set’,
target_size = (64, 64),
batch_size = batch_size,
class_mode = ‘binary’)
H = classifier.fit_generator(training_set,
steps_per_epoch = 2000,
epochs = 2,
validation_data = None)

# Making Predictions
import numpy as np
from keras.preprocessing import image
test_image = image.load_img(r’dataset\dog.4028.jpg’, target_size = (64,64))
test_image = image.img_to_array(test_image)
test_image = np.expand_dims(test_image, axis = 0)
acc = classifier.evaluate(training_set,test_image, verbose = 0)
print(‘Test accuracy:’, acc)

Two statements:

scores = model.evaluate(X_test, y_test, verbose=0)
print(“Accuracy: %.2f%%” % (scores[1]*100))

are replaced by following two statement

acc = classifier.evaluate(training_set,test_image, verbose = 0)
print(‘Test accuracy:’, acc)

but i am not getting the accuracy it is giving error. what is wrong in the syntax

• Jason Brownlee January 9, 2019 at 8:46 am #

Perhaps try posting your code and error to stackoverflow?

77. ajith February 6, 2019 at 3:11 pm #

Hy, strated with deeplearing
Cats and dogs
But my accuracy is stuck @50%
I tried droupout, regulations and data ammune

78. Amit February 8, 2019 at 3:19 am #

Hi Jason,
I am new to Machine Learning & wanted to understand the first layer weight dimension calculation for 3 channels. Modified the first layer like
—————–
—————–
and displayed below weight matrix dimension.
—————–
len(model.layers[0].get_weights()[0]),
len(model.layers[0].get_weights()[0][0]),
len(model.layers[0].get_weights()[0][0][0]),
len(model.layers[0].get_weights()[0][0][0][0])
—————–

The result is
—————
(5, 5, 3, 8)
—————

what have learned so far theoritically , 5*5 filter will convolve to 32*32 having stride 1*1 & padding same, it will result into 28*28 for each channel.

addition of such result matrix(28*28) will happen for all three channels, still the result matrix will be 28*28.

finally for all kernels (8) total weight dimension will be 8*28*28

But how it is getting calculated as (5, 5, 3, 8)?

Regards

• Jason Brownlee February 8, 2019 at 7:56 am #

The filters are the weights, e.g. 8 5×5 filters with 3 channels.

79. Vatsal March 8, 2019 at 9:39 pm #

Hi Json,
can you tell or send me a link on how to save the model and use it to predict some new images,?

80. Sakthi Dasan Sekar May 14, 2019 at 8:40 pm #

This is an image classification problem and not object recognition/detection.

• Jason Brownlee May 15, 2019 at 8:12 am #

It is the classification of photos of objects.

Indeed, it is not object recognition.

81. Mbunga Blaise July 8, 2019 at 7:50 pm #

Hi Jason, It ‘s a pleasure to read all your tutorials concerning Machine Learning and deep learning. I would like to thank you for all your tutorials and the responses you bring to us. I have created two blocks of CNN using Keras functionnal API to extract best representations of Smiles (drugs sequence) and Proteines sequences, then I concanated the two outputs from the 2 blocs, dropped out and gave it to dense layers. When I printed the summary of the model. it was fine, however when I try to fit the model, the screen displayed the following error message:

ValueError: Error when checking input: expected input_8 to have shape (103, 64) but got array with shape (31824, 72)
Below is my short code

Regarding the error message, could you be so kind to tell me what is wrong

82. Peter August 30, 2019 at 2:16 am #

Hello Jason
I was wondering if loading all the images would at some point make the memory collapse.
So .. is there any way in Keras for reading the images direcly from a folder not loading all of them at once into memory?
Regards

83. tiennguyen February 23, 2021 at 12:14 pm #

Thank for your sharing, Jason Brownlee.
I have a question: Why this tutorial, we do not use Reshape() function for data like example https://machinelearningmastery.com/handwritten-digit-recognition-using-convolutional-neural-networks-python-keras/. More detail:
# flatten 28*28 images to a 784 vector for each image
num_pixels = X_train.shape[1] * X_train.shape[2]
X_train = X_train.reshape((X_train.shape[0], num_pixels)).astype(‘float32’)
X_test = X_test.reshape((X_test.shape[0], num_pixels)).astype(‘float32’)

• Jason Brownlee February 23, 2021 at 1:25 pm #

Because the input data already has color channels and is ready to model directly.

84. tiennguyen February 23, 2021 at 12:17 pm #

Sorry, I post wrong details. Correctly detail which I want to show with you:

format: plain


text:
1. ## Identity Proof!

tan( pi/4 + x/2 ) = sec x + tan x

I have worked on this for over two hours now.
Help would be greatly appreciated

Thanks!

2. Originally Posted by dxscorp
tan( pi/4 + x/2 ) = sec x + tan x

I have worked on this for over two hours now.
Help would be greatly appreciated

Thanks!
using this
$\tan (a+b) = \frac{\tan a + \tan b}{1 - \tan a \tan b}$

$\tan \left(\frac{\pi}{4} + \frac{x}{2}\right)=\frac{1+\tan x/2}{1 - \tan x/2 }$

$\tan \left(\frac{\pi}{4} + \frac{x}{2}\right)=\frac{1+\tan x/2}{1 - \tan x/2 } \left(\frac{1+\tan x/2}{1+\tan x/2} \right)$

can you continue?, you must know

$\cos 2x = \cos ^2 x - \sin ^2 x \Longrightarrow \cos x = \cos ^2 \left(\frac{x}{2}\right) - \sin ^2 \left(\frac{x}{2}\right)$

$\sin 2x = 2\sin x \cos x \Longrightarrow \sin x = 2 \sin \left(\frac{x}{2}\right)\cos \left(\frac{x}{2}\right)$

3. Hello, dxscorp!

$\tan\left(\frac{\pi}{4}+ \frac{x}{2}\right) \:=\: \sec x + \tan x$

$\tan\left(\frac{\pi}{4} + \frac{x}{2}\right) \;=\;\frac{\tan\frac{\pi}{4} + \tan\frac{x}{2}}{1 - \tan\frac{\pi}{4}\tan\frac{x}{2}} \;=\;\frac{1 + \tan\frac{x}{2}}{1 - \tan\frac{x}{2}}$ . $=\;\frac{1 + \dfrac{\sin\frac{x}{2}}{\cos\frac{x}{2}}} {1 - \dfrac{\sin\frac{x}{2}}{\cos\frac{x}{2}}} \;=\;\frac{\cos\frac{x}{2} + \sin\frac{x}{2}} {\cos\frac{x}{2} - \sin\frac{x}{2}}$

Multiply by $\frac{\cos\frac{x}{2} + \sin\frac{x}{2}}{\cos\frac{x}{2} + \sin\frac{x}{2}}\!:$

. . $\frac{\cos\frac{x}{2} + \sin\frac{x}{2}}{\cos\frac{x}{2} - \sin\frac{x}{2}} \cdot {\color{blue}\frac{\cos\frac{x}{2} + \sin\frac{x}{2}}{\cos\frac{x}{2} + \sin\frac{x}{2}}}$

. . . . . . $=\;\;\frac{\overbrace{(\cos^2\tfrac{x}{2} + \sin^2\tfrac{x}{2})}^{\text{This is 1}} \;+\; \overbrace{2\sin\tfrac{x}{2}\cos\tfrac{x}{2}}^{\te xt{This is }\sin x}} {\underbrace{\cos^2\tfrac{x}{2} - \sin^2\tfrac{x}{2}}_{\text{This is }\cos x}}$

. . . . . . $=\;\;\frac{1 + \sin x}{\cos x} \;\;=\;\;\frac{1}{\cos x} + \frac{\sin x}{\cos x} \;\;=\;\; \sec x + \tan x$


text:
# DISCRETE 2018

26-30 November 2018
Europe/Vienna timezone

## Tests of discrete symmetries in positronium decays with the J-PET detector

27 Nov 2018, 11:15
35m
Festsaal

#### Festsaal

Austrian Academy of Sciences, Dr.-Ignaz-Seipel-Platz 2, 1010 Vienna, AUSTRIA
Invited Talk [2] Emergence of symmetries from entanglement

### Speaker

Dr Eryk Czerwiński (Institute of Physics, Jagiellonian University)

### Description

Positronium is the lightest purely leptonic object decaying into photons. As an atom bound by a central potential, it is a parity eigenstate, and as an atom built out of an electron and an anti-electron, it is an eigenstate of the charge conjugation operator. Therefore, the positronium is a unique laboratory to study discrete symmetries whose precision is limited, in principle, only by the effects due to the weak interactions expected at the level of $10^{-14}$ and photon- photon interactions expected at the level of $10^{-9}$. Violation of T or CP invariance in purely leptonic systems has never been seen thus far. The experimental limits on CP and CPT symmetry violation in the decays of positronium are set at the level of $10^{-3}$ and litmits on charge conjugation violation are set at the level of $10^{-7}$. Thus, there is still a range of six orders of magnitude as regards T and CP, and two order of magnitude as regards the C symmetry, where the phenomena beyond the Standard Model can be sought for by improving the experimental precision in investigations of decays of positronium atoms.

The newly constructed Jagiellonian Positron Emission Tomograph (J-PET) is a first PET tomograph built from plastic scintillators. As a detector optimized for the registration of photons from the electron-positron annihilations, it also enables tests of discrete symmetries in decays of positronium atoms via the determination of the expectation values of the discrete-symmetries-odd operators, which may be constructed from the spin of ortho-positronium atom and the momenta and polarization vectors of photons originating from its annihilation. J-PET is also a unique facility to study the entanglement of photons originating from positronium annihilations.

In the talk we will present the capability of the J-PET detector to improve the current precision of testing CP, T and CPT symmetries in the decays of positronium atoms and report on results from the first data-taking campaigns. With respect to the previous experiments performed with crystal based detectors, J-PET built of plastic scintillators provides superior time resolution, higher granularity, lower pile-ups, and opportunity of determining photon's polarization through the registration of primary and secondary Compton scatterings in the detector. These features makes J-PET capable of improving present experimental limits in tests of discrete symmetries in decays of positronium atom (a purely leptonic system).

Content of the contribution Experiment

### Primary authors

Dr Eryk Czerwiński (Institute of Physics, Jagiellonian University) on behalf of the J-PET Collaboration

Paper files: