Derivatives

AP E3200x

Defining Computational Derivatives

A practical and useful example of numerical error in computational physics is subtractive cancellation (a type of "round-off" error.) When two numbers are subtracted from each other, the result is less precise as the original two numbers! This problem becomes severe when the difference between the two numbers become small.

We will illustrate this by computing a first-order and second-order approximation to a derivative and compare this approximation with the "exact" value.

A first-order approximation to the derivative of the function f[x] is
	(∂_x) f ~ ((f[x + h] - f[x])/h)
(It is called "first-order" because it's accuracy improves linearly with decreasing h.)

A second-order approximation to the derivative of the function f[x] is
	(∂_x) f ~ ((f[x + h] - f[x-h])/(2h))
(It is called "second-order" because it's accuracy improves linearly with decreasing (h^2). This is called a "central difference" because the difference of f[x] on both sides of x are evaluated.)

f[x_] := x^3;

First Order Derivative

error1[h_,x_:1] := Abs[3 x^2 - (f[x+h] - f[x])/h]

err1List = Table[{10.0^h,error1[10.0^h]},{h,-14,-1,0.5}];

err1List // Shallow

ListLogLogPlot[err1List,PlotStyle->PointSize[0.02]]

Often, I set Graphics options to make the plot easier to read.

myPlot[items__]:=ListLogLogPlot[items,Frame->True,GridLines->Automatic,PlotStyle->PointSize[0.02],BaseStyle->{FontWeight->"Bold",FontSize->12}]

p1 = myPlot[err1List,PlotLabel->"1st Order Difference Approximation"];

Second Order Derivative

error2[h_,x_:1] := Abs[3 x^2 - (f[x+h] - f[x-h])/(2h)]

err2List = Table[{10.0^h,error2[10.0^h]},{h,-14,-1,0.5}];

p2 = myPlot[err2List,PlotLabel->"2nd Order Difference Approximation"];

Show[p2,p1,PlotLabel->"Difference Error"]

Arbitrary-Precision

When you enter an approximate real number, Mathematica to decide whether to treat it as machine precision or arbitrary precision.Unless you specify otherwise,then if you give less than $MachinePrecisiondigits, Mathematica will treat the number as machine precision,and if you give more digits,it will treat the number as arbitrary precision.

10.0^-16

10.000000000000000000000000000000^-16

10.0`32^-16

10.0`200^-16

err3List = Table[{10.0^h,error2[10.0`32^h]},{h,-14,-1, 0.5`32}];

err3List // Shallow

p3 = myPlot[err3List,PlotLabel->"2nd Order Arbitrary-Precision"];

Show[p3,p2,p1,PlotLabel->"Finite  Difference  Error"]

Summary

We have displayed the formula for the finite difference approximations for first and second derivatives. This begins with a Taylor series expansion at x = x0 and used to relate the derivatives to nearby, known values of f[x].

We find that central difference approximations are significantly more accurate than equivalent forward (and backward) differences. The error associated with the approximation is called "truncation error" since we are truncating the Taylor series expansion. Reduced truncation error gives a more accurate solution to the differential equation and allows for longer integration times.