What is TensorFlow?

It’s an open source library made by google for being used in Machine Learning (ML) problems, specially problems that is related to Neural Networks. It’s written in C++ from the ground, designed to be very efficient in computations.

You can think of TensorFlow as a tool that transforms your ML code into a GPU running code, so that saves you a lot of time as a developer also, it uses its magic to make things run very efficient and uses the resources on your machine in a very good way.

So, When Can I say that Tensorflow is my best option for my project?

Basically, TensorFlow shines more in the area of using neural networks -with the most of its types- especially in the Deep Learning area that can be touched through most of the apps like:

  • image processing and learning tasks, like when the Facebook suggest to tag someone in an image or detecting words on an image via your mobile camera and convert it into a text,
  • machine game players, like Googles Alpha Go,
  • coloring white and black picture.

TensorFlow is great for both research and product side; it has a flexible architecture which allows you to deploy computation to one or more CPUs or GPUs in a desktop, server, or mobile device with a single API.

Tensorflow also provide a very useful tool called TensorBoard, it makes you explor visually the models made by others and debug them.

Let’s start discovering this awesome tool

import tensorflow as tf 

Next, let’s play with it, but before playing let’s define some basic parts of TensorFlow. The main idea here is to express our numeric computation as a graph which has:

  • nodes that represent operations that take zero or more Tensors, performs some computation and produces zero or more Tensors
  • tensors are multi-dimensional array which flow between nodes

x * y

so x and y are tensors and the multiplication is the operation

let’s take a closer look

x = tf.constant([[5.5, 5.5]])
y = tf.constant([[2.],[2.]])

product = tf.matmul(x, y)

Tensor("MatMul:0", shape=(1, 1), dtype=float32)

so what’s happend?

First we defined x and y as constant matrices with predefined values, then we defined our operation as Matrix Multiplication and finally we tried to print our variables, but as you see, nothing actually happened, WHY?

The reason is TensorFlow operations are different from basic python operations, in python operations made line by line but in TensorFlow the whole operation runs as a unit, which called a Session. So, we need to define our session and run our operation via it like this:

sess = tf.Session()
result = sess.run(product)

[[ 22.]]

we can also make it this way (looks similar to open and close a file)

with tf.Session() as sess:
    result = sess.run([product])

[array([[ 22.]], dtype=float32)]

This was a very basic intro to what is TensorFlow and how we can start dealing with this computation library, you can design your graph with your custom operations.

Now let’s jump into a more complex example which we will be using RNN (LSTM) with the MNIST dataset. In the land of neural networks the most “classic” classical problem is the MNIST handwritten digit classification.

First, What is RNN (Recurrent Neural Network)?

RNNs are Neural Nets with loops , that makes it able with sequential information unlike the normal neural nets.

Normal NN


RNNs makes more sense when we dealing with data changes its states by time, it’s ability of feedbacking the info from state to another makes it able to deal with data like video frames which each frame should be connected to the one before to make a good sense of what happening and also data like conversations, like you ask your google assistant or Siri or even Cortana about the time in England, then you ask it “How is the Weather there!”, it should connect this with the last info related to understanding the meaning by “there”.

So let’s get into the business

In this tutorial, we will deal with the MNIST Data set, which is a collection of handwritten digit images, we will make a model that learns to classify these images and tells us what digit is written in this one!


import tensorflow as tf
from tensorflow.python.ops import rnn, rnn_cell
import numpy as np

# Import MNIST data
from tensorflow.examples.tutorials.mnist import input_data
mnist = input_data.read_data_sets("/tmp/data/", one_hot=True)

Extracting /tmp/data/train-images-idx3-ubyte.gz
Extracting /tmp/data/train-labels-idx1-ubyte.gz
Extracting /tmp/data/t10k-images-idx3-ubyte.gz
Extracting /tmp/data/t10k-labels-idx1-ubyte.gz

First, we import our dependencies, then load the MNIST dataset from TensorFlow lib, it might take minutes till it downloads the dataset from the internet.

Just to look at the data, I can grab the first array of image data and reshape it back into an 28×28 pixel image.

import matplotlib.pyplot as plt
import matplotlib.cm as cm
tmp = mnist.train.images[2]
tmp = tmp.reshape((28,28))
plt.imshow(tmp, cmap = cm.Greys)

learning_rate = 0.001
training_iters = 100000
batch_size = 128
display_step = 100

Here we define our parameters that we will be using in the learning process, it really depends on your design, try to change these parameters and see what the difference is…

# here  we start building our RNN
# First
# Network Parameters
n_input = 28 # MNIST data input (img shape: 28*28)
n_steps = 28 # timesteps, remember inputs from up to 28 time steps in the past
n_hidden = 128 # hidden layer num of features
n_classes = 10 # MNIST total classes (0-9 digits)

# tf Graph input
x = tf.placeholder("float", [None, n_steps, n_input])
y = tf.placeholder("float", [None, n_classes])

# Define weights
weights = {
    'out': tf.Variable(tf.random_normal([n_hidden, n_classes]))
biases = {
    'out': tf.Variable(tf.random_normal([n_classes]))

Here we start building our RNN. Our dataset contains images with size 28*28 pixels, so we define our parameters based on this.

Then define our input data(the images) as x and target(the labels like 1,2,…9) as y.

Then we define weights and biases for our model.

def RNN(x, weights, biases):

    # Prepare data shape to match `rnn` function requirements
    # Current data input shape: (batch_size, n_steps, n_input)
    # Required shape: 'n_steps' tensors list of shape (batch_size, n_input)
    # Permuting batch_size and n_steps
    x = tf.transpose(x, [1, 0, 2])
    # Reshaping to (n_steps*batch_size, n_input)
    x = tf.reshape(x, [-1, n_input])
    # Split to get a list of 'n_steps' tensors of shape (batch_size, n_input)
    x = tf.split(0, n_steps, x)

    # Define a lstm cell with tensorflow
    lstm_cell = rnn_cell.BasicLSTMCell(n_hidden, forget_bias=1.0)

    # Get lstm cell output
    outputs, states = rnn.rnn(lstm_cell, x, dtype=tf.float32)

    # Linear activation, using rnn inner loop last output
    return tf.matmul(outputs[-1], weights['out']) + biases['out']

pred = RNN(x, weights, biases)

# Define loss and optimizer
cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(pred, y))
optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(cost)

# Evaluate model
correct_pred = tf.equal(tf.argmax(pred,1), tf.argmax(y,1))
accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.float32))

# Initializing the variables
init = tf.initialize_all_variables()

Here we just adjust our data to use it in building our RNN by using the rnn and rnn_cell modules that we just imported it makes us able to make a basic RNN with specs we define.

So we make the input more flatten the list of pixels that are 28*28 for each input pic.

# Launch the graph
with tf.Session() as sess:
    step = 1
    # Keep training until reach max iterations
    while step * batch_size < training_iters:
        batch_x, batch_y = mnist.train.next_batch(batch_size)
        # Reshape data to get 28 seq of 28 elements
        batch_x = batch_x.reshape((batch_size, n_steps, n_input))
        # Run optimization op (backprop)
        sess.run(optimizer, feed_dict={x: batch_x, y: batch_y})
        if step % display_step == 0:
            # Calculate batch accuracy
            acc = sess.run(accuracy, feed_dict={x: batch_x, y: batch_y})
            # Calculate batch loss
            loss = sess.run(cost, feed_dict={x: batch_x, y: batch_y})
            print ("Iter " + str(step*batch_size) + ", Minibatch Loss= " + \
                  "{:.6f}".format(loss) + ", Training Accuracy= " + \
        step += 1
    print("Optimization Finished!")

    # Calculate accuracy for 128 mnist test images
    test_len = 128
    test_data = mnist.test.images[:test_len].reshape((-1, n_steps, n_input))
    test_label = mnist.test.labels[:test_len]
    print ("Testing Accuracy:", \
        sess.run(accuracy, feed_dict={x: test_data, y: test_label}))

Iter 12800, Minibatch Loss= 0.305791, Training Accuracy= 0.90625
Iter 25600, Minibatch Loss= 0.252651, Training Accuracy= 0.92969
Iter 38400, Minibatch Loss= 0.203888, Training Accuracy= 0.93750
Iter 51200, Minibatch Loss= 0.097179, Training Accuracy= 0.96094
Iter 64000, Minibatch Loss= 0.091221, Training Accuracy= 0.97656
Iter 76800, Minibatch Loss= 0.176814, Training Accuracy= 0.95312
Iter 89600, Minibatch Loss= 0.110684, Training Accuracy= 0.96875
Optimization Finished!
Testing Accuracy: 0.992188

After we defined the data inputs and our RNN, It’s time to begin TRAINING: first, we make our session to use it to make our computations happens, we run through our data in the sequential manner, we get them into pieces every piece is sized by the batch size that we defined, then we take every piece and feed it to our optimizer and calculate our accuracy and error and repeat it by feeding new chunks, and so on in this process our accuracy getting better the more we feed it the more our accuracy getting better.

Testing accuracy 99.2%! Over 97% the result is very acceptable. The best models can get to over 99.7% accuracy! (For more information, have a look at this list of results).


After we finished our job we close the session. Goodnight TensorFlow!

Are you interested in object classification? Give a look to my post Image recognition tutorial in Python/MXNet using deep convolutional.
To learn about neural networks in general, Michael Nielsen’s free online book is an excellent resource.

Posted by lorenzo

Full-time engineer. I like to write about data science and artificial intelligence.

Vuoi commentare?