Skip to content

Commit ae81822

Browse files
committed
Clarify closure examples
1 parent cebe983 commit ae81822

File tree

1 file changed

+22
-13
lines changed

1 file changed

+22
-13
lines changed

src/dense.jl

Lines changed: 22 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -62,30 +62,39 @@ However, in the case of closures or callable structs which contain differentiate
6262
6363
# Notes
6464
65-
If `constant_function = true` but the enclosed data is not truly constant, then Enzyme.jl will not compute the correct derivative values.
66-
An example of such a function is:
65+
We now give several examples of functions.
66+
For each one, we explain how `constant_function` should be set in order to compute the correct derivative with respect to the input `x`.
6767
6868
```julia
69-
cache = [0.0]
70-
function f(x)
71-
cache[1] = x[1]^2
72-
cache[1] + x[1]
69+
function f1(x)
70+
return x[1]
7371
end
7472
```
7573
76-
In this case, the enclosed cache is a function of the differentiated input, and thus its values are non-constant with respect to the input.
77-
Thus, in order to compute the correct derivative of the output, the derivative must propagate through the `cache` value, and said `cache` must not be treated as constant.
78-
79-
Conversely, the following function can treat `parameter` as a constant, because `parameter` is never modified based on the input `x`:
74+
The function `f1` is not a closure, it does not contain any data.
75+
Thus `f1` can be differentiated with `AutoEnzyme(constant_function=true)`.
8076
8177
```julia
8278
parameter = [0.0]
83-
function f(x)
84-
parameter[1] + x[1]
79+
function f2(x)
80+
return parameter[1] + x[1]
81+
end
82+
```
83+
84+
The function `f2` is a closure over `parameter`, but `parameter` is never modified based on the input `x`.
85+
Thus, `f2` can be differentiated with `AutoEnzyme(constant_function=true)`.
86+
87+
```julia
88+
cache = [0.0]
89+
function f3(x)
90+
cache[1] = x[1]
91+
return cache[1] + x[1]
8592
end
8693
```
8794
88-
In this case, `constant_function = true` would allow the chosen differentiation system to perform extra memory and compute optimizations, under the assumption that `parameter` is kept constant.
95+
The function `f3` is a closure over `cache`, and `cache` is modified based on the input `x`.
96+
That means `cache` cannot be treated as constant, since derivative values must be propagated through it.
97+
Thus `f3` must be differentiated with `AutoEnzyme(constant_function=false)`.
8998
"""
9099
struct AutoEnzyme{M, constant_function} <: AbstractADType
91100
mode::M

0 commit comments

Comments
 (0)