Seed random variables (Tensorflow)
Overview
In this tutorial I would quickly show few examples on how to use Tensorflow random seed.
Tensorflow random seed
If you would like to keep repeatability on your machine learning model results Tensorflow random seed should be a
must, but you have to be careful on how to use it.
Operations that rely on a random seed actually derive it from two seeds: the graph-level and operation-level seeds.
example 1: operation seed is set
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
import tensorflow as tf
mu = 0
sigma = 0.3
# variables with/without function seed `seed=1`
fc1_W = tf.Variable(tf.truncated_normal(shape=(2, 2), mean=mu, stddev=sigma, seed=1))
fc2_W = tf.Variable(tf.truncated_normal(shape=(2, 2), mean=mu, stddev=sigma))
# initialize session
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
print('\n variables with/without function seed')
print('round 1.0 ')
print(fc1_W.eval(sess))
print(fc2_W.eval(sess))
# new session
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
print('round 1.1')
print(fc1_W.eval(sess))
print(fc2_W.eval(sess))
Results:
example 2: graph-level seed is set tf.set_random_seed(1)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
# using global seed
tf.set_random_seed(1)
fc1_W = tf.Variable(tf.truncated_normal(shape=(2, 2), mean=mu, stddev=sigma))
fc2_W = tf.Variable(tf.truncated_normal(shape=(2, 2), mean=mu, stddev=sigma))
# initialize session
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
print('\n variables with global seed ')
print('round 2.0')
print(fc1_W.eval(sess))
print(fc2_W.eval(sess))
# new session
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
print('round 2.1')
print(fc1_W.eval(sess))
print(fc2_W.eval(sess))
Results:
example 3: graph-level seed is set againtf.set_random_seed(1)
NOTE: This is a common mistake, if you compare the output of this code with the above you would find that output is different even though both are using the same graph-level seed
tf.set_random_seed(1). Make sure you only initialize one graph-level seed ones, other wise you would get different results.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
# new global variables
tf.set_random_seed(1)
fc1_W = tf.Variable(tf.truncated_normal(shape=(2, 2), mean=mu, stddev=sigma, name='fc1_W'))
fc2_W = tf.Variable(tf.truncated_normal(shape=(2, 2), mean=mu, stddev=sigma, name='fc2_W'))
# initialize session
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
print('\n re-initialize variables')
print('round 4.0')
print(fc1_W.eval(sess))
print(fc2_W.eval(sess))
# new session
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
print('round 4.1')
print(fc1_W.eval(sess))
print(fc2_W.eval(sess))
Results:
example 4: changing the order of your variables declaration should not make a difference
as long as you name them.
1
2
3
4
5
6
7
8
9
10
# changing order of variables declaration with global seed
fc2_W = tf.Variable(tf.truncated_normal(shape=(2, 2), mean=mu, stddev=sigma, name='fc2_W'))
fc1_W = tf.Variable(tf.truncated_normal(shape=(2, 2), mean=mu, stddev=sigma, name='fc1_W'))
# initialize session
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
print('\n changing order of variables declaration with global seed')
print('round 5.0')
print(fc1_W.eval(sess))
print(fc2_W.eval(sess))
Results:
Reference
