You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: src/dense.jl
+22-13Lines changed: 22 additions & 13 deletions
Original file line number
Diff line number
Diff line change
@@ -62,30 +62,39 @@ However, in the case of closures or callable structs which contain differentiate
62
62
63
63
# Notes
64
64
65
-
If `constant_function = true` but the enclosed data is not truly constant, then Enzyme.jl will not compute the correct derivative values.
66
-
An example of such a function is:
65
+
We now give several examples of functions.
66
+
For each one, we explain how `constant_function` should be set in order to compute the correct derivative with respect to the input `x`.
67
67
68
68
```julia
69
-
cache = [0.0]
70
-
function f(x)
71
-
cache[1] = x[1]^2
72
-
cache[1] + x[1]
69
+
function f1(x)
70
+
return x[1]
73
71
end
74
72
```
75
73
76
-
In this case, the enclosed cache is a function of the differentiated input, and thus its values are non-constant with respect to the input.
77
-
Thus, in order to compute the correct derivative of the output, the derivative must propagate through the `cache` value, and said `cache` must not be treated as constant.
78
-
79
-
Conversely, the following function can treat `parameter` as a constant, because `parameter` is never modified based on the input `x`:
74
+
The function `f1` is not a closure, it does not contain any data.
75
+
Thus `f1` can be differentiated with `AutoEnzyme(constant_function=true)`.
80
76
81
77
```julia
82
78
parameter = [0.0]
83
-
function f(x)
84
-
parameter[1] + x[1]
79
+
function f2(x)
80
+
return parameter[1] + x[1]
81
+
end
82
+
```
83
+
84
+
The function `f2` is a closure over `parameter`, but `parameter` is never modified based on the input `x`.
85
+
Thus, `f2` can be differentiated with `AutoEnzyme(constant_function=true)`.
86
+
87
+
```julia
88
+
cache = [0.0]
89
+
function f3(x)
90
+
cache[1] = x[1]
91
+
return cache[1] + x[1]
85
92
end
86
93
```
87
94
88
-
In this case, `constant_function = true` would allow the chosen differentiation system to perform extra memory and compute optimizations, under the assumption that `parameter` is kept constant.
95
+
The function `f3` is a closure over `cache`, and `cache` is modified based on the input `x`.
96
+
That means `cache` cannot be treated as constant, since derivative values must be propagated through it.
97
+
Thus `f3` must be differentiated with `AutoEnzyme(constant_function=false)`.
0 commit comments