您好,登錄后才能下訂單哦!
這篇文章主要介紹了TensorFlow中Checkpoint為模型添加檢查點(diǎn)的示例分析,具有一定借鑒價(jià)值,感興趣的朋友可以參考下,希望大家閱讀完這篇文章之后大有收獲,下面讓小編帶著大家一起了解一下。
1.檢查點(diǎn)
保存模型并不限于在訓(xùn)練模型后,在訓(xùn)練模型之中也需要保存,因?yàn)門ensorFlow訓(xùn)練模型時(shí)難免會出現(xiàn)中斷的情況,我們自然希望能夠?qū)⒂?xùn)練得到的參數(shù)保存下來,否則下次又要重新訓(xùn)練。
這種在訓(xùn)練中保存模型,習(xí)慣上稱之為保存檢查點(diǎn)。
2.添加保存點(diǎn)
通過添加檢查點(diǎn),可以生成載入檢查點(diǎn)文件,并能夠指定生成檢查文件的個(gè)數(shù),例如使用saver的另一個(gè)參數(shù)——max_to_keep=1,表明最多只保存一個(gè)檢查點(diǎn)文件,在保存時(shí)使用如下的代碼傳入迭代次數(shù)。
import tensorflow as tf import numpy as np import matplotlib.pyplot as plt import os train_x = np.linspace(-5, 3, 50) train_y = train_x * 5 + 10 + np.random.random(50) * 10 - 5 plt.plot(train_x, train_y, 'r.') plt.grid(True) plt.show() tf.reset_default_graph() X = tf.placeholder(dtype=tf.float32) Y = tf.placeholder(dtype=tf.float32) w = tf.Variable(tf.random.truncated_normal([1]), name='Weight') b = tf.Variable(tf.random.truncated_normal([1]), name='bias') z = tf.multiply(X, w) + b cost = tf.reduce_mean(tf.square(Y - z)) learning_rate = 0.01 optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(cost) init = tf.global_variables_initializer() training_epochs = 20 display_step = 2 saver = tf.train.Saver(max_to_keep=15) savedir = "model/" if __name__ == '__main__': with tf.Session() as sess: sess.run(init) loss_list = [] for epoch in range(training_epochs): for (x, y) in zip(train_x, train_y): sess.run(optimizer, feed_dict={X: x, Y: y}) if epoch % display_step == 0: loss = sess.run(cost, feed_dict={X: x, Y: y}) loss_list.append(loss) print('Iter: ', epoch, ' Loss: ', loss) w_, b_ = sess.run([w, b], feed_dict={X: x, Y: y}) saver.save(sess, savedir + "linear.cpkt", global_step=epoch) print(" Finished ") print("W: ", w_, " b: ", b_, " loss: ", loss) plt.plot(train_x, train_x * w_ + b_, 'g-', train_x, train_y, 'r.') plt.grid(True) plt.show() load_epoch = 10 with tf.Session() as sess2: sess2.run(tf.global_variables_initializer()) saver.restore(sess2, savedir + "linear.cpkt-" + str(load_epoch)) print(sess2.run([w, b], feed_dict={X: train_x, Y: train_y}))
在上述的代碼中,我們使用saver.save(sess, savedir + "linear.cpkt", global_step=epoch)將訓(xùn)練的參數(shù)傳入檢查點(diǎn)進(jìn)行保存,saver = tf.train.Saver(max_to_keep=1)表示只保存一個(gè)文件,這樣在訓(xùn)練過程中得到的新的模型就會覆蓋以前的模型。
cpkt = tf.train.get_checkpoint_state(savedir) if cpkt and cpkt.model_checkpoint_path: saver.restore(sess2, cpkt.model_checkpoint_path) kpt = tf.train.latest_checkpoint(savedir) saver.restore(sess2, kpt)
上述的兩種方法也可以對checkpoint文件進(jìn)行加載,tf.train.latest_checkpoint(savedir)為加載最后的檢查點(diǎn)文件。這種方式,我們可以通過保存指定訓(xùn)練次數(shù)的檢查點(diǎn),比如保存5的倍數(shù)次保存一下檢查點(diǎn)。
3.簡便保存檢查點(diǎn)
我們還可以用更加簡單的方法進(jìn)行檢查點(diǎn)的保存,tf.train.MonitoredTrainingSession()函數(shù),該函數(shù)可以直接實(shí)現(xiàn)保存載入檢查點(diǎn)模型的文件,與前面的方法不同的是,它是按照訓(xùn)練時(shí)間來保存檢查點(diǎn)的,可以通過指定save_checkpoint_secs參數(shù)的具體秒數(shù),設(shè)置多久保存一次檢查點(diǎn)。
import tensorflow as tf import numpy as np import matplotlib.pyplot as plt import os train_x = np.linspace(-5, 3, 50) train_y = train_x * 5 + 10 + np.random.random(50) * 10 - 5 # plt.plot(train_x, train_y, 'r.') # plt.grid(True) # plt.show() tf.reset_default_graph() X = tf.placeholder(dtype=tf.float32) Y = tf.placeholder(dtype=tf.float32) w = tf.Variable(tf.random.truncated_normal([1]), name='Weight') b = tf.Variable(tf.random.truncated_normal([1]), name='bias') z = tf.multiply(X, w) + b cost = tf.reduce_mean(tf.square(Y - z)) learning_rate = 0.01 optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(cost) init = tf.global_variables_initializer() training_epochs = 30 display_step = 2 global_step = tf.train.get_or_create_global_step() step = tf.assign_add(global_step, 1) saver = tf.train.Saver() savedir = "check-point/" if __name__ == '__main__': with tf.train.MonitoredTrainingSession(checkpoint_dir=savedir + 'linear.cpkt', save_checkpoint_secs=5) as sess: sess.run(init) loss_list = [] for epoch in range(training_epochs): sess.run(global_step) for (x, y) in zip(train_x, train_y): sess.run(optimizer, feed_dict={X: x, Y: y}) if epoch % display_step == 0: loss = sess.run(cost, feed_dict={X: x, Y: y}) loss_list.append(loss) print('Iter: ', epoch, ' Loss: ', loss) w_, b_ = sess.run([w, b], feed_dict={X: x, Y: y}) sess.run(step) print(" Finished ") print("W: ", w_, " b: ", b_, " loss: ", loss) plt.plot(train_x, train_x * w_ + b_, 'g-', train_x, train_y, 'r.') plt.grid(True) plt.show() load_epoch = 10 with tf.Session() as sess2: sess2.run(tf.global_variables_initializer()) # saver.restore(sess2, savedir + 'linear.cpkt-' + str(load_epoch)) # cpkt = tf.train.get_checkpoint_state(savedir) # if cpkt and cpkt.model_checkpoint_path: # saver.restore(sess2, cpkt.model_checkpoint_path) # kpt = tf.train.latest_checkpoint(savedir + 'linear.cpkt') saver.restore(sess2, kpt) print(sess2.run([w, b], feed_dict={X: train_x, Y: train_y}))
上述的代碼中,我們設(shè)置了沒訓(xùn)練了5秒中之后,就保存一次檢查點(diǎn),它默認(rèn)的保存時(shí)間間隔是10分鐘,這種按照時(shí)間的保存模式更適合使用大型數(shù)據(jù)集訓(xùn)練復(fù)雜模型的情況,注意在使用上述的方法時(shí),要定義global_step變量,在訓(xùn)練完一個(gè)批次或者一個(gè)樣本之后,要將其進(jìn)行加1的操作,否則將會報(bào)錯。
感謝你能夠認(rèn)真閱讀完這篇文章,希望小編分享的“TensorFlow中Checkpoint為模型添加檢查點(diǎn)的示例分析”這篇文章對大家有幫助,同時(shí)也希望大家多多支持億速云,關(guān)注億速云行業(yè)資訊頻道,更多相關(guān)知識等著你來學(xué)習(xí)!
免責(zé)聲明:本站發(fā)布的內(nèi)容(圖片、視頻和文字)以原創(chuàng)、轉(zhuǎn)載和分享為主,文章觀點(diǎn)不代表本網(wǎng)站立場,如果涉及侵權(quán)請聯(lián)系站長郵箱:is@yisu.com進(jìn)行舉報(bào),并提供相關(guān)證據(jù),一經(jīng)查實(shí),將立刻刪除涉嫌侵權(quán)內(nèi)容。