Overloads for LinearProblems with ForwardDiff Dual numbers

[jClugstor]

Checklist

• Appropriate tests were added
• Any code changes were done in a way that does not break public API
• All documentation related to code changes were updated
• The new code follows the
  contributor guidelines, in particular the SciML Style Guide and
  COLPRAC.
• Any new documentation only uses public API

Additional context

Addresses #607

This gives the same result as when Dual numbers go through the actual solver, and can handle cases when either A or b do not depend on anything, i.e. when only one of A or b consists of Dual numbers.

[jClugstor]

@oscardssmith I need to make sure that the results here are the same as when the Dual numbers actually go through the solver. I tried using invoke, but I can't seem to make it go through the more general solve. Any ideas?

[oscardssmith]

You can run __solve on the duals.

[jClugstor]

LinearSolve doesn't have __solve. But A \ b works with Dual numbers, so I can just compare with that since they should be very close.

[jClugstor]

Left to do:

• Make sure that Dual u0 is handled and passed to the iterative solvers
  - [ ] Make sure that SciMLOperators are handled
• Make sure that things relying on LinearCache can still work with DualLinearCache

[jClugstor]

This just needs a fix for remaking LinearProblems with u0 (#624), and then the tests should pass.

So far this doesn't handle when A is a SciMLOperator, but it doesn't affect it either, in that case it will just go through the solver like before. It might be ok if that's handled later?

I did have to add a DualLinearCache. I overloaded getproperty for A and b, so that if cache = DualCache and cache.A or cache.b is accessed, it get's the A and b from the underlying LinearCache used in the primal part of the solve. Also overloaded setproperty! so that if somebody sets A or b with dual numbers it also sets the partials in the cache, used for the dual part of the solve. That way everything should be consistent with initializing a cache and updating A and b like it shows in the docs.

That does mean that if somebody is relying on the type of the output of init, or the fields of LinearCache when solving with Dual numbers it will break. I could add overloads for DualLinearCache so that if someone tries accessing it with the name of a field of LinearCache it accesses the underlying primal LinearCache to mitigate that problem.