大概两点：
optimizer = tf.keras.optimizers.SGD (learning_rate=5e-4)
在 tensorflow 的程序示例中 learning_rate 和下文中 “ 在这里，我们使用了前文的方式计算了损失函数关于参数的偏导数。同时，使用 tf.keras.optimizers.SGD (learning_rate=1e-3) 声明了一个梯度下降 优化器 （Optimizer），其学习率为 1e-3。” 的 learning_rate 不一致
谢谢大佬，我以为自己的基础没打牢，看来可以自信一点点了，感谢感谢
已修正，感谢提醒~
IDE 设置的文档链接是：https://tf.wiki/zh/basic/installation.html#ide
配置好了，但是每次运行的时候总是出现以下错误代码，是什么原因呢：
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py:516: FutureWarning: Passing (type, 1) or ‘1type’ as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / ‘(1,) type’.
_np_qint8 = np.dtype ([(“qint8”, np.int8, 1)])
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py:517: FutureWarning: Passing (type, 1) or ‘1type’ as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / ‘(1,) type’.
_np_quint8 = np.dtype ([(“quint8”, np.uint8, 1)])
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py:518: FutureWarning: Passing (type, 1) or ‘1type’ as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / ‘(1,) type’.
_np_qint16 = np.dtype ([(“qint16”, np.int16, 1)])
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py:519: FutureWarning: Passing (type, 1) or ‘1type’ as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / ‘(1,) type’.
_np_quint16 = np.dtype ([(“quint16”, np.uint16, 1)])
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py:520: FutureWarning: Passing (type, 1) or ‘1type’ as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / ‘(1,) type’.
_np_qint32 = np.dtype ([(“qint32”, np.int32, 1)])
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or ‘1type’ as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / ‘(1,) type’.
np_resource = np.dtype ([(“resource”, np.ubyte, 1)])
/home/fish/.local/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:541: FutureWarning: Passing (type, 1) or ‘1type’ as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / ‘(1,) type’.
_np_qint8 = np.dtype ([(“qint8”, np.int8, 1)])
/home/fish/.local/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:542: FutureWarning: Passing (type, 1) or ‘1type’ as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / ‘(1,) type’.
_np_quint8 = np.dtype ([(“quint8”, np.uint8, 1)])
/home/fish/.local/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:543: FutureWarning: Passing (type, 1) or ‘1type’ as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / ‘(1,) type’.
_np_qint16 = np.dtype ([(“qint16”, np.int16, 1)])
/home/fish/.local/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:544: FutureWarning: Passing (type, 1) or ‘1type’ as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / ‘(1,) type’.
_np_quint16 = np.dtype ([(“quint16”, np.uint16, 1)])
/home/fish/.local/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:545: FutureWarning: Passing (type, 1) or ‘1type’ as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / ‘(1,) type’.
_np_qint32 = np.dtype ([(“qint32”, np.int32, 1)])
/home/fish/.local/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:550: FutureWarning: Passing (type, 1) or ‘1type’ as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / ‘(1,) type’.
np_resource = np.dtype ([(“resource”, np.ubyte, 1)])
2020-04-25 11:25:56.201023: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Could not dlopen library ‘libcuda.so.1’; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory
2020-04-25 11:25:56.201044: E tensorflow/stream_executor/cuda/cuda_driver.cc:318] failed call to cuInit: UNKNOWN ERROR (303)
2020-04-25 11:25:56.201061: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (fish-desktop): /proc/driver/nvidia/version does not exist
2020-04-25 11:25:56.201256: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2020-04-25 11:25:56.235142: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 3591575000 Hz
2020-04-25 11:25:56.235842: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x5ba5910 executing computations on platform Host. Devices:
2020-04-25 11:25:56.235864: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device
大佬你好, 我想请问一下我遇到的这个问题怎么解决:
InvalidArgumentError Traceback (most recent call last)
<ipython-input-24-dfbd31710005> in <module>
11 for e in range (num_epoch):
12 with tf.GradientTape () as tape:
---> 13 y_pred = a * X + b
14 loss = tf.reduce_sum (tf.square (y - y_pred))
15 grads = tape.gradient (loss, variables)
InvalidArgumentError: cannot compute Mul as input #1 (zero-based) was expected to be a float tensor but is a double tensor [Op:Mul] name: mul/
看上去像是数据类型的问题, 应该怎么解决呢?
2 replies
import tensorflow as tf
X = tf.constant ([[1., 2.], [3., 4.]])
y = tf.constant ([[1.], [2.]])
w = tf.Variable (initial_value=[[1.], [2.]])
b = tf.Variable (initial_value=1.)
with tf.GradientTape () as tape:
L = tf.reduce_sum (tf.square (tf.matmul (X, w) + b - y))
w_grad, b_grad = tape.gradient (L, [w, b]) # 计算 L (w, b) 关于 w, b 的偏导数
print (L, w_grad, b_grad)
上面这段执行完后报错如下，请赐教如何修改，谢谢。
2020-04-26 16:31:11.896090: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
Traceback (most recent call last):
File "E:/[RemoteSync]/Shared/PycharmProjects/test/src/03.partial_derivative.py", line 8, in <module>
L = tf.reduce_sum (tf.square (tf.matmul (X, w) + b - y))
File "C:\Users\demo\AppData\Local\conda\conda\envs\tf2\lib\site-packages\tensorflow_core\python\util\dispatch.py", line 180, in wrapper
return target (*args, **kwargs)
File "C:\Users\demo\AppData\Local\conda\conda\envs\tf2\lib\site-packages\tensorflow_core\python\ops\math_ops.py", line 1595, in reduce_sum
_ReductionDims (input_tensor, axis))
File "C:\Users\demo\AppData\Local\conda\conda\envs\tf2\lib\site-packages\tensorflow_core\python\ops\math_ops.py", line 1473, in _ReductionDims
return constant_op.constant (np.arange (rank, dtype=np.int32))
File "C:\Users\demo\AppData\Local\conda\conda\envs\tf2\lib\site-packages\tensorflow_core\python\framework\constant_op.py", line 258, in constant
allow_broadcast=True)
File "C:\Users\demo\AppData\Local\conda\conda\envs\tf2\lib\site-packages\tensorflow_core\python\framework\constant_op.py", line 266, in _constant_impl
t = convert_to_eager_tensor (value, ctx, dtype)
File "C:\Users\demo\AppData\Local\conda\conda\envs\tf2\lib\site-packages\tensorflow_core\python\framework\constant_op.py", line 96, in convert_to_eager_tensor
return ops.EagerTensor (value, ctx.device_name, dtype)
ValueError: Attempt to convert a value (0) with an unsupported type (<class 'numpy.int32'>) to a Tensor.
Process finished with exit code 1请检查一下你的X变量的数据类型，如果是float64的话转为float32。
X_raw = np.array ([2013, 2014, 2015, 2016, 2017], dtype=np.float32)
y_raw = np.array ([12000, 14000, 15000, 16500, 17500], dtype=np.float32)
X = (X_raw - X_raw.min ()) / (X_raw.max () - X_raw.min ())
y = (y_raw - y_raw.min ()) / (y_raw.max () - y_raw.min ())
注意代码中的 dtype
2 replies
感谢回复! 上面的问题确实是数据类型的问题, 已经解决了. 可是在下一条语句又遇到了问题
<ipython-input-9-4a00192d6d84> in <module>
8 y = tf.constant (y)
9
---> 10 a = tf.Variable (initial_value=0.)
11 b = tf.Variable (initial_value=0.)
12 variables = [a, b]
InvalidArgumentError: assertion failed: [0] [Op:Assert] name: EagerVariableNameReuse
这里又出现了问题, 能麻烦大佬再帮忙解答一下吗
1 reply
建议在 IDE 里新开一个.py文件，将要运行的代码写好，然后直接运行代码。在没有上下文的情况下很难确定 ipython 环境里的报错是怎么回事。
这里面并没有真正意义上的异常报错，基本都是提示信息。
想问问这个 X 和 y 计算出的是什么，没搞懂
感谢大佬。
import tensorflow as tf
x = tf.constant ([1.0, 2.0, 3.0], [4.0, 5.0, 6.0])
y = tf.constant ([10.0], [20.0])
class Linear (tf.keras.Model):
def init(self):
super ().init()
self.dense = tf.keras.layers.Dense (
units=1,
activation=None,
kernel_initializer=tf.zeros_initializer (),
bias_initializer=tf.zeros_initializer ()
)
def call (self, input):
output = self.dense (input)
return output
model = Linear ()
optimizer = tf.keras.optimizers.SGD (learning_rate=0.01)
for i in range (100):
with tf.GradientTape () as tape:
y_pred = model (x)
loss = tf.reduce_mean (tf.square (y_pred - y))
grads = tape.gradient (loss, model.variables)
optimizer.apply_gradients (grads_and_vars=zip (grads, model.variables))
print (model.variables)
错误信息：
D:\ProgramData\Anaconda3\python.exe E:/develop/python/tensorflow/pyqt.py
Traceback (most recent call last):
File “D:\ProgramData\Anaconda3\lib\site-packages\tensorflow_core\python\framework\constant_op.py”, line 92, in convert_to_eager_tensor
dtype = dtype.as_datatype_enum
AttributeError: ‘list’ object has no attribute ‘as_datatype_enum’
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File “E:/develop/python/tensorflow/pyqt.py”, line 3, in
x = tf.constant ([1.0, 2.0, 3.0], [4.0, 5.0, 6.0])
File “D:\ProgramData\Anaconda3\lib\site-packages\tensorflow_core\python\framework\constant_op.py”, line 258, in constant
allow_broadcast=True)
File “D:\ProgramData\Anaconda3\lib\site-packages\tensorflow_core\python\framework\constant_op.py”, line 266, in _constant_impl
t = convert_to_eager_tensor (value, ctx, dtype)
File “D:\ProgramData\Anaconda3\lib\site-packages\tensorflow_core\python\framework\constant_op.py”, line 94, in convert_to_eager_tensor
dtype = dtypes.as_dtype (dtype).as_datatype_enum
File “D:\ProgramData\Anaconda3\lib\site-packages\tensorflow_core\python\framework\dtypes.py”, line 720, in as_dtype
return _ANY_TO_TF [type_value]
TypeError: unhashable type: ‘list’
谢谢。。。。
with tf.GradientTape () as tape:
L = tf.reduce_sum (tf.square (tf.matmul (X, w) + b - y))
w_grad, b_grad = tape.gradient (L, [w, b])
请问这里 L 涉及到一个求和，是如何求得偏导的
1 reply
这里的求偏导是由 TensorFlow 根据求导的链式法则而自动进行的。求偏导的具体表达式可以参考一下注释 4 和 https://www.math.uwaterloo.ca/~hwolkowi/matrixcookbook.pdf
1 reply
谢谢回复，我在这里找到了答案
https://blog.csdn.net/li6016265/article/details/86746135
原来是矩阵的链式法则，以前一直不会算矩阵求导我自己却没有发现。
然后如果我没有弄错的话，注释 4 中 a 的偏导最后的 x_i 应该有个转置
学到了，谢谢各位大佬！
我结合你的代码重新看了一下，确实是有这样的现象。我的理解是，这个例子中的计算相对而言非常简单（只有一个乘法和一个加法），计算量也非常小，所以没能发挥出 TensorFlow 在 GPU 方面的优势。同时每次都自动求导也有额外的开销，肯定不如已经手工写好的求导来得快。在这个例子中，如果我们把数据量增大，例如
X_raw = np.array (range (1, 1000000), dtype=np.float32)
y_raw = np.array (range (1, 1000000), dtype=np.float32)
learning_rate = 5e-7
你会发现 TensorFlow 的速度比 NumPy 快（NumPy 32.12s，TensorFlow 22.73s，Colab GPU 模式）。
链式求导是指复合函数求导的链式法则，见 https://baike.baidu.com/item/链式法则
都是 warning，前面应该是 numpy 等级高了，可以查一下你 tf 等级对应的 numpy 等级，或者直接进去改。把_np_quint8 = np.dtype ([(“quint8”, np.uint8, 1)]) 改成_np_quint8 = np.dtype ([(“quint8”, np.uint8, (1,))])
写的也太好了吧，看哭了，太牛了！！！
相见恨晚！！！真的写的太好~谢谢大神！！
参考 https://tf.wiki/zh_hans/basic/basic.html#zh-hans-optimizer ，指定待求导的函数 f 和求导的变量 x （也就是，哪个函数对哪个变量求导， \frac{\partial f}{\partial x} ），即可自动计算梯度。
懂了，非常感谢
我的电脑是集成显卡，code如下
import tensorflow as tf
x = tf.Variable(initial_value=3.)
with tf.GradientTape() as tape:
y = tf.square(x)
y_grad = tape.gradient(y, x)
print(y, y_grad)
运行后结果没问题，但出现了一些提示，费解！请问是怎么回事呢？
tf.Tensor(9.0, shape=(), dtype=float32) tf.Tensor(6.0, shape=(), dtype=float32)
2021-03-09 13:56:02.374852: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2021-03-09 13:56:02.375186: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
\frac{\partial L(w,b)}{\partial w} = 2X^\top(Xw+b-y) = \begin{bmatrix}1 & 3 \\ 2 & 4\end{bmatrix}\begin{bmatrix}10 \\ 20\end{bmatrix} = \begin{bmatrix}70 \\ 100\end{bmatrix}
\frac{\partial L(w,b)}{\partial b} = 2\begin{bmatrix}1 & 1\end{bmatrix}(Xw+b-y) = \begin{bmatrix}1 & 1\end{bmatrix}\begin{bmatrix}10 \\ 20\end{bmatrix} = 30
