用TensorFlow实现弹性网络回归算法

用TensorFlow实现弹性网络回归算法

弹性网络回归算法（Elastic Net Regression）是综合lasso回归和岭回归的一种回归算法，通过在损失函数中增加L1和L2正则项。

本文使用多线性回归的方法实现弹性网络回归算法，以iris数据集为训练数据，用花瓣长度、花瓣宽度和花萼宽度三个特征预测花萼长度。

# Elastic Net Regression# 弹性网络回归#----------------------------------## This function shows how to use TensorFlow to# solve elastic net regression.# y = Ax + b## We will use the iris data, specifically:# y = Sepal Length# x = Pedal Length, Petal Width, Sepal Widthimport matplotlib.pyplot as pltimport numpy as npimport tensorflow as tffrom sklearn import datasetsfrom tensorflow.python.framework import ops#### Set up for TensorFlow###ops.reset_default_graph()# Create graphsess = tf.Session()#### Obtain data#### Load the data# iris.data = [(Sepal Length, Sepal Width, Petal Length, Petal Width)]iris = datasets.load_iris()x_vals = np.array([[x[1], x[2], x[3]] for x in iris.data])y_vals = np.array([y[0] for y in iris.data])#### Setup model#### make results reproducibleseed = 13np.random.seed(seed)tf.set_random_seed(seed)# Declare batch sizebatch_size = 50# Initialize placeholdersx_data = tf.placeholder(shape=[None, 3], dtype=tf.float32)y_target = tf.placeholder(shape=[None, 1], dtype=tf.float32)# Create variables for linear regressionA = tf.Variable(tf.random_normal(shape=[3,1]))b = tf.Variable(tf.random_normal(shape=[1,1]))# Declare model operationsmodel_output = tf.add(tf.matmul(x_data, A), b)# Declare the elastic net loss function# 对于弹性网络回归算法，损失函数包含斜率的L1正则和L2正则。# 创建L1和L2正则项，然后加入到损失函数中elastic_param1 = tf.constant(1.)elastic_param2 = tf.constant(1.)l1_a_loss = tf.reduce_mean(tf.abs(A))l2_a_loss = tf.reduce_mean(tf.square(A))e1_term = tf.multiply(elastic_param1, l1_a_loss)e2_term = tf.multiply(elastic_param2, l2_a_loss)loss = tf.expand_dims(tf.add(tf.add(tf.reduce_mean(tf.square(y_target - model_output)), e1_term), e2_term), 0)# Declare optimizermy_opt = tf.train.GradientDescentOptimizer(0.0001)train_step = my_opt.minimize(loss)#### Train model#### Initialize variablesinit = tf.global_variables_initializer()sess.run(init)# Training looploss_vec = []for i in range(1000): rand_index = np.random.choice(len(x_vals), size=batch_size) rand_x = x_vals[rand_index] rand_y = np.transpose([y_vals[rand_index]]) sess.run(train_step, feed_dict={x_data: rand_x, y_target: rand_y}) temp_loss = sess.run(loss, feed_dict={x_data: rand_x, y_target: rand_y}) loss_vec.append(temp_loss[0]) if (i+1)%250==0: print('Step #' + str(i+1) + ' A = ' + str(sess.run(A)) + ' b = ' + str(sess.run(b))) print('Loss = ' + str(temp_loss))#### Extract model results#### Get the optimal coefficients[[sw_coef], [pl_coef], [pw_ceof]] = sess.run(A)[y_intercept] = sess.run(b)#### Plot results#### Plot loss over timeplt.plot(loss_vec, 'k-')plt.title('Loss per Generation')plt.xlabel('Generation')plt.ylabel('Loss')plt.show()

结果：

Step #250 A = [[ 0.93870646] [-0.37139279] [ 0.27290201]] b = [[-0.36246276]]Loss = [ 23.83867645]Step #500 A = [[ 1.26683569] [ 0.14753909] [ 0.42754883]] b = [[-0.2424359]]Loss = [ 2.98364353]Step #750 A = [[ 1.33217096] [ 0.28339788] [ 0.45837855]] b = [[-0.20850581]]Loss = [ 1.82120061]Step #1000 A = [[ 1.33412337] [ 0.32563502] [ 0.45811999]] b = [[-0.19569117]]Loss = [ 1.64923525]

版权声明：本文内容由网络用户投稿，版权归原作者所有，本站不拥有其著作权，亦不承担相应法律责任。如果您发现本站中有涉嫌抄袭或描述失实的内容，请联系我们jiasou666@gmail.com 处理，核实后本网站将在24小时内删除侵权内容。