That's an example where the derivative does not exist.
> I guess another issue that could happen is that the original sequence converges, and the derivative sequence converges, but they converge at different rates.
This is a lot closer to what's happening in the video. For a potentially simpler example than an ODE solver, if you had a series evaluator that given a series evaluated it at a point, AD would need a similar fix to make sure the convergence test is including the convergence of the derivative.
That's an example where the derivative does not exist.
> I guess another issue that could happen is that the original sequence converges, and the derivative sequence converges, but they converge at different rates.
This is a lot closer to what's happening in the video. For a potentially simpler example than an ODE solver, if you had a series evaluator that given a series evaluated it at a point, AD would need a similar fix to make sure the convergence test is including the convergence of the derivative.