tensorflow 计算两个序列的co-attention矩阵

tensorflow 计算两个序列的co-attention矩阵

序列1的每个元素和序列2的每个元素​​feature dim计算​​最终排成一个矩阵∈Rn1×n2

import tensorflow as tfimport numpy as npn1 = tf.ones([10,3],dtype=tf.int32)*2 # 序列1,长度10, feature dim 3n2 = tf.ones([2,3],dtype=tf.int32)*3 # 序列2,长度2, feature dim 3ta1 = tf.TensorArray(tf.int32, tf.constant(2))n1_tile = tf.tile(tf.expand_dims(n1,0),[2,1,1]) # 2个n1n2_tile = tf.tile(tf.expand_dims(n2,0),[10,1,1]) # 10个n2final = []for i in range(2): final.append([]) for j in range(10): element = n1_tile[i][j]*n2_tile[j][i] final[i].append(element)final = tf.stack(final)sess=tf.Session()# feature dim 按位相乘，feature dim求和后才是co-attention matrixprint(sess.run(final))print(np.shape(sess.run(final)))# feature dim 按位相乘，feature dim求和后才是co-attention matrixfinal_compare = n1_tile*tf.transpose(n2_tile,[1,0,2])print(sess.run(final_compare))print(np.shape(sess.run(final_compare)))#直接求co-attention matrix，feature dim 按位相乘再求和，∈Rn1×n2final_compare2 = tf.matmul(n1,n2,adjoint_b=True)print(sess.run(final_compare2))print(np.shape(sess.run(final_compare2)))

PyTorch实现的更多参考​​https://github.com/allenai/allennlp/blob/master/allennlp/modules/matrix_attention.py​​

版权声明：本文内容由网络用户投稿，版权归原作者所有，本站不拥有其著作权，亦不承担相应法律责任。如果您发现本站中有涉嫌抄袭或描述失实的内容，请联系我们jiasou666@gmail.com 处理，核实后本网站将在24小时内删除侵权内容。