Floating-point Numbers Aren't Real - Revision history

http://commons.oreilly.com/wiki/index.php?title=Floating-point_Numbers_Aren't_Real&action=history&feed=atom Floating-point Numbers Aren't Real - Revision history 2013-06-19T19:13:09Z Revision history for this page on the wiki MediaWiki 1.11.0 http://commons.oreilly.com/wiki/index.php?title=Floating-point_Numbers_Aren%27t_Real&diff=25205&oldid=prev Harmonious at 04:48, 8 August 2009 2009-08-08T04:48:38Z

<p></p> <table style="background-color: white; color:black;"> <col class='diff-marker' /> <col class='diff-content' /> <col class='diff-marker' /> <col class='diff-content' /> <tr> <td colspan='2' style="background-color: white; color:black;">←Older revision</td> <td colspan='2' style="background-color: white; color:black;">Revision as of 04:48, 8 August 2009</td> </tr> <tr><td colspan="2" class="diff-lineno">Line 11:</td> <td colspan="2" class="diff-lineno">Line 11:</td></tr> <tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Smearing can occur in even more subtle ways. Suppose a library naively computes ''e<sup>x</sup>'' by the formula ''1 + x + x<sup>2</sup>/2 + x<sup>3</sup>/3! + ...''. This works fine for positive ''x'', but consider what happens when ''x'' is a large negative number. The even-powered terms result in large positive numbers, and subtracting the odd-powered magnitudes will not even affect the result. The problem here is that the roundoff in the large, positive terms is in a digit position of much greater significance than the true answer. The answer diverges toward positive infinity! The solution here is also simple: for negative ''x'', compute ''e<sup>x</sup> = 1/e<sup>|x|</sup>''.</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Smearing can occur in even more subtle ways. Suppose a library naively computes ''e<sup>x</sup>'' by the formula ''1 + x + x<sup>2</sup>/2 + x<sup>3</sup>/3! + ...''. This works fine for positive ''x'', but consider what happens when ''x'' is a large negative number. The even-powered terms result in large positive numbers, and subtracting the odd-powered magnitudes will not even affect the result. The problem here is that the roundoff in the large, positive terms is in a digit position of much greater significance than the true answer. The answer diverges toward positive infinity! The solution here is also simple: for negative ''x'', compute ''e<sup>x</sup> = 1/e<sup>|x|</sup>''.</div></td></tr> <tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr> <tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>It should go without saying that you shouldn't use floating-point numbers for financial <del style="color: red; font-weight: bold; text-decoration: none;">applic</del></div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>It should go without saying that you shouldn't use floating-point numbers for financial <ins style="color: red; font-weight: bold; text-decoration: none;">appliction's — that's what decimal classes in languages like Python and C# are for. Floating-point numbers are intended for efficient scientific computation. But efficiency is worthless without accuracy, so remember the source of rounding errors and code accordingly!</ins></div></td></tr> <tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div> </div></td></tr> <tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;">By Chuck Allison</ins></div></td></tr>  </table>

Harmonious http://commons.oreilly.com/wiki/index.php?title=Floating-point_Numbers_Aren%27t_Real&diff=25204&oldid=prev Harmonious at 04:45, 8 August 2009 2009-08-08T04:45:25Z

<p></p> <table style="background-color: white; color:black;"> <col class='diff-marker' /> <col class='diff-content' /> <col class='diff-marker' /> <col class='diff-content' /> <tr> <td colspan='2' style="background-color: white; color:black;">←Older revision</td> <td colspan='2' style="background-color: white; color:black;">Revision as of 04:45, 8 August 2009</td> </tr> <tr><td colspan="2" class="diff-lineno">Line 1:</td> <td colspan="2" class="diff-lineno">Line 1:</td></tr> <tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>Floating-point numbers are not "real numbers" in the mathematical sense, even though they are called ''real'' in some programming languages, such as Pascal and Fortran. Real numbers have infinite precision and are therefore continuous and non-lossy; floating-point numbers have limited precision, so they are finite, and they resemble "badly-behaved" integers, because they're not evenly spaced throughout <del style="color: red; font-weight: bold; text-decoration: none;">the </del>range.</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>Floating-point numbers are not "real numbers" in the mathematical sense, even though they are called ''real'' in some programming languages, such as Pascal and Fortran. Real numbers have infinite precision and are therefore continuous and non-lossy; floating-point numbers have limited precision, so they are finite, and they resemble "badly-behaved" integers, because they're not evenly spaced throughout <ins style="color: red; font-weight: bold; text-decoration: none;">their </ins>range.</div></td></tr> <tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr> <tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>To illustrate, assign 2147483647 (the largest signed 32-bit integer) to a 32-bit <code>float</code> variable (<code>x</code>, say), and print it. You'll see 2147483648. Now print <code>x - 64</code>. Still 2147483648. Now print <code>x-65</code> and you'll get 2147483520! Why? Because the spacing between adjacent floats in that range is 128, and floating-point operations round to the nearest floating-point number.</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>To illustrate, assign 2147483647 (the largest signed 32-bit integer) to a 32-bit <code>float</code> variable (<code>x</code>, say), and print it. You'll see 2147483648. Now print <code>x - 64</code>. Still 2147483648. Now print <code>x-65</code> and you'll get 2147483520! Why? Because the spacing between adjacent floats in that range is 128, and floating-point operations round to the nearest floating-point number.</div></td></tr> <tr><td colspan="2" class="diff-lineno">Line 11:</td> <td colspan="2" class="diff-lineno">Line 11:</td></tr> <tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Smearing can occur in even more subtle ways. Suppose a library naively computes ''e<sup>x</sup>'' by the formula ''1 + x + x<sup>2</sup>/2 + x<sup>3</sup>/3! + ...''. This works fine for positive ''x'', but consider what happens when ''x'' is a large negative number. The even-powered terms result in large positive numbers, and subtracting the odd-powered magnitudes will not even affect the result. The problem here is that the roundoff in the large, positive terms is in a digit position of much greater significance than the true answer. The answer diverges toward positive infinity! The solution here is also simple: for negative ''x'', compute ''e<sup>x</sup> = 1/e<sup>|x|</sup>''.</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Smearing can occur in even more subtle ways. Suppose a library naively computes ''e<sup>x</sup>'' by the formula ''1 + x + x<sup>2</sup>/2 + x<sup>3</sup>/3! + ...''. This works fine for positive ''x'', but consider what happens when ''x'' is a large negative number. The even-powered terms result in large positive numbers, and subtracting the odd-powered magnitudes will not even affect the result. The problem here is that the roundoff in the large, positive terms is in a digit position of much greater significance than the true answer. The answer diverges toward positive infinity! The solution here is also simple: for negative ''x'', compute ''e<sup>x</sup> = 1/e<sup>|x|</sup>''.</div></td></tr> <tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr> <tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>It should go without saying that you shouldn't use floating-point numbers for financial <del style="color: red; font-weight: bold; text-decoration: none;">application's &mdash; that's what decimal classes in languages like Python and C# are for. Floating-point numbers are intended for efficient scientific computation. But efficiency is worthless without accuracy, so remember the source of rounding errors and code accordingly!</del></div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>It should go without saying that you shouldn't use floating-point numbers for financial <ins style="color: red; font-weight: bold; text-decoration: none;">applic</ins></div></td></tr> <tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div> </div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div></div></td></tr> <tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div><del style="color: red; font-weight: bold; text-decoration: none;">By [[Chuck Allison]]</del></div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div></div></td></tr> <tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div> </div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div></div></td></tr> <tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div><del style="color: red; font-weight: bold; text-decoration: none;">This work is licensed under a [http://creativecommons.org/licenses/by/3.0/us/ Creative Commons Attribution 3] </del></div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div></div></td></tr> <tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div> </div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div></div></td></tr> <tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div> </div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div></div></td></tr> <tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div> </div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div></div></td></tr> <tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div><del style="color: red; font-weight: bold; text-decoration: none;">Back to [[97 Things Every Programmer Should Know]] home page</del></div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div></div></td></tr>  </table>

Harmonious http://commons.oreilly.com/wiki/index.php?title=Floating-point_Numbers_Aren%27t_Real&diff=23428&oldid=prev Kevlin at 22:06, 16 February 2009 2009-02-16T22:06:09Z

<p></p> <table style="background-color: white; color:black;"> <col class='diff-marker' /> <col class='diff-content' /> <col class='diff-marker' /> <col class='diff-content' /> <tr> <td colspan='2' style="background-color: white; color:black;">←Older revision</td> <td colspan='2' style="background-color: white; color:black;">Revision as of 22:06, 16 February 2009</td> </tr> <tr><td colspan="2" class="diff-lineno">Line 11:</td> <td colspan="2" class="diff-lineno">Line 11:</td></tr> <tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Smearing can occur in even more subtle ways. Suppose a library naively computes ''e<sup>x</sup>'' by the formula ''1 + x + x<sup>2</sup>/2 + x<sup>3</sup>/3! + ...''. This works fine for positive ''x'', but consider what happens when ''x'' is a large negative number. The even-powered terms result in large positive numbers, and subtracting the odd-powered magnitudes will not even affect the result. The problem here is that the roundoff in the large, positive terms is in a digit position of much greater significance than the true answer. The answer diverges toward positive infinity! The solution here is also simple: for negative ''x'', compute ''e<sup>x</sup> = 1/e<sup>|x|</sup>''.</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Smearing can occur in even more subtle ways. Suppose a library naively computes ''e<sup>x</sup>'' by the formula ''1 + x + x<sup>2</sup>/2 + x<sup>3</sup>/3! + ...''. This works fine for positive ''x'', but consider what happens when ''x'' is a large negative number. The even-powered terms result in large positive numbers, and subtracting the odd-powered magnitudes will not even affect the result. The problem here is that the roundoff in the large, positive terms is in a digit position of much greater significance than the true answer. The answer diverges toward positive infinity! The solution here is also simple: for negative ''x'', compute ''e<sup>x</sup> = 1/e<sup>|x|</sup>''.</div></td></tr> <tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr> <tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>It <del style="color: red; font-weight: bold; text-decoration: none;">goes </del>without saying that you shouldn't use floating-point numbers for financial application</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>It <ins style="color: red; font-weight: bold; text-decoration: none;">should go </ins>without saying that you shouldn't use floating-point numbers for financial application<ins style="color: red; font-weight: bold; text-decoration: none;">'</ins>s <ins style="color: red; font-weight: bold; text-decoration: none;">&mdash; </ins>that's what decimal classes in languages like Python and C# are for. Floating-point numbers are intended for efficient scientific computation. But efficiency is worthless without accuracy, so remember the source of rounding errors and code accordingly!</div></td></tr> <tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>s <del style="color: red; font-weight: bold; text-decoration: none;">-- </del>that's what decimal classes in languages like Python and C# are for. Floating-point numbers are intended for efficient scientific computation. But efficiency is worthless without accuracy, so remember the source of rounding errors and code accordingly!</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div></div></td></tr> <tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr> <tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>By [[Chuck Allison]]</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>By [[Chuck Allison]]</div></td></tr>  </table>

Kevlin http://commons.oreilly.com/wiki/index.php?title=Floating-point_Numbers_Aren%27t_Real&diff=23425&oldid=prev Harmonious at 21:09, 16 February 2009 2009-02-16T21:09:25Z

<p></p> <table style="background-color: white; color:black;"> <col class='diff-marker' /> <col class='diff-content' /> <col class='diff-marker' /> <col class='diff-content' /> <tr> <td colspan='2' style="background-color: white; color:black;">←Older revision</td> <td colspan='2' style="background-color: white; color:black;">Revision as of 21:09, 16 February 2009</td> </tr> <tr><td colspan="2" class="diff-lineno">Line 10:</td> <td colspan="2" class="diff-lineno">Line 10:</td></tr> <tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr> <tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Smearing can occur in even more subtle ways. Suppose a library naively computes ''e<sup>x</sup>'' by the formula ''1 + x + x<sup>2</sup>/2 + x<sup>3</sup>/3! + ...''. This works fine for positive ''x'', but consider what happens when ''x'' is a large negative number. The even-powered terms result in large positive numbers, and subtracting the odd-powered magnitudes will not even affect the result. The problem here is that the roundoff in the large, positive terms is in a digit position of much greater significance than the true answer. The answer diverges toward positive infinity! The solution here is also simple: for negative ''x'', compute ''e<sup>x</sup> = 1/e<sup>|x|</sup>''.</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Smearing can occur in even more subtle ways. Suppose a library naively computes ''e<sup>x</sup>'' by the formula ''1 + x + x<sup>2</sup>/2 + x<sup>3</sup>/3! + ...''. This works fine for positive ''x'', but consider what happens when ''x'' is a large negative number. The even-powered terms result in large positive numbers, and subtracting the odd-powered magnitudes will not even affect the result. The problem here is that the roundoff in the large, positive terms is in a digit position of much greater significance than the true answer. The answer diverges toward positive infinity! The solution here is also simple: for negative ''x'', compute ''e<sup>x</sup> = 1/e<sup>|x|</sup>''.</div></td></tr> <tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"></td></tr> <tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>It goes without saying that you shouldn't use floating-point numbers for financial application</div></td></tr> <tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>s -- that's what decimal classes in languages like Python and C# are for. Floating-point numbers are intended for efficient scientific computation. But efficiency is worthless without accuracy, so remember the source of rounding errors and code accordingly!</div></td></tr> <tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr> <tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>By [[Chuck Allison]]</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>By [[Chuck Allison]]</div></td></tr>  </table>

Harmonious http://commons.oreilly.com/wiki/index.php?title=Floating-point_Numbers_Aren%27t_Real&diff=23248&oldid=prev Kevlin at 11:45, 3 February 2009 2009-02-03T11:45:32Z

<p></p> <table style="background-color: white; color:black;"> <col class='diff-marker' /> <col class='diff-content' /> <col class='diff-marker' /> <col class='diff-content' /> <tr> <td colspan='2' style="background-color: white; color:black;">←Older revision</td> <td colspan='2' style="background-color: white; color:black;">Revision as of 11:45, 3 February 2009</td> </tr> <tr><td colspan="2" class="diff-lineno">Line 7:</td> <td colspan="2" class="diff-lineno">Line 7:</td></tr> <tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Knowing the spacing in the neighborhood of a floating-point number can help you avoid classic numerical blunders. For example, if you're performing an iterative calculation, such as searching for the root of an equation, there's no sense in asking for greater precision than the number system can give in the neighborhood of the answer. Make sure that the tolerance you request is no smaller than the spacing there; otherwise you'll loop forever.</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Knowing the spacing in the neighborhood of a floating-point number can help you avoid classic numerical blunders. For example, if you're performing an iterative calculation, such as searching for the root of an equation, there's no sense in asking for greater precision than the number system can give in the neighborhood of the answer. Make sure that the tolerance you request is no smaller than the spacing there; otherwise you'll loop forever.</div></td></tr> <tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr> <tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>Since floating-point numbers are approximations of real numbers, there is inevitably a little error present. This error, called ''roundoff'', can lead to surprising results. When you subtract nearly equal numbers, for example, the most significant digits cancel each other out, so what was the least significant digit (where the roundoff error resides) gets promoted to the most significant position in the floating-point result, essentially contaminating any further related computations (a phenomenon known as ''smearing''). You need to look closely at your algorithms to prevent such ''catastrophic cancellation''. To illustrate, consider solving the equation <del style="color: red; font-weight: bold; text-decoration: none;"><code></del>x<sup>2</sup> - 100000x + 1<del style="color: red; font-weight: bold; text-decoration: none;"></code> </del>with the quadratic formula. Since the operands in the expression <del style="color: red; font-weight: bold; text-decoration: none;"><code></del>-b + sqrt(b<sup>2</sup?> - 4)<del style="color: red; font-weight: bold; text-decoration: none;"></code> </del>are nearly equal in magnitude, you can instead compute the root <del style="color: red; font-weight: bold; text-decoration: none;"><code></del>r<sub>1</sub> = -b - sqrt(b<sup>2</sup> - 4)<del style="color: red; font-weight: bold; text-decoration: none;"></code></del>, and then obtain <del style="color: red; font-weight: bold; text-decoration: none;"><code></del>r<sub>2</sub> = 1/r<sub>1</sub><del style="color: red; font-weight: bold; text-decoration: none;"></code></del>, since for any quadratic equation, <del style="color: red; font-weight: bold; text-decoration: none;"><code></del>ax<sup>2</sup> + bx + c = 0<del style="color: red; font-weight: bold; text-decoration: none;"></code></del>, the roots satisfy <del style="color: red; font-weight: bold; text-decoration: none;"><code></del>r<sub>1</sub>r<sub>2</sub> = c/a<del style="color: red; font-weight: bold; text-decoration: none;"></code></del>.</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>Since floating-point numbers are approximations of real numbers, there is inevitably a little error present. This error, called ''roundoff'', can lead to surprising results. When you subtract nearly equal numbers, for example, the most significant digits cancel each other out, so what was the least significant digit (where the roundoff error resides) gets promoted to the most significant position in the floating-point result, essentially contaminating any further related computations (a phenomenon known as ''smearing''). You need to look closely at your algorithms to prevent such ''catastrophic cancellation''. To illustrate, consider solving the equation <ins style="color: red; font-weight: bold; text-decoration: none;">''</ins>x<sup>2</sup> - 100000x + 1 <ins style="color: red; font-weight: bold; text-decoration: none;">= 0'' </ins>with the quadratic formula. Since the operands in the expression <ins style="color: red; font-weight: bold; text-decoration: none;">''</ins>-b + sqrt(b<sup>2</sup?> - 4)<ins style="color: red; font-weight: bold; text-decoration: none;">'' </ins>are nearly equal in magnitude, you can instead compute the root <ins style="color: red; font-weight: bold; text-decoration: none;">''</ins>r<sub>1</sub> = -b - sqrt(b<sup>2</sup> - 4)<ins style="color: red; font-weight: bold; text-decoration: none;">''</ins>, and then obtain <ins style="color: red; font-weight: bold; text-decoration: none;">''</ins>r<sub>2</sub> = 1/r<sub>1</sub><ins style="color: red; font-weight: bold; text-decoration: none;">''</ins>, since for any quadratic equation, <ins style="color: red; font-weight: bold; text-decoration: none;">''</ins>ax<sup>2</sup> + bx + c = 0<ins style="color: red; font-weight: bold; text-decoration: none;">''</ins>, the roots satisfy <ins style="color: red; font-weight: bold; text-decoration: none;">''</ins>r<sub>1</sub>r<sub>2</sub> = c/a<ins style="color: red; font-weight: bold; text-decoration: none;">''</ins>.</div></td></tr> <tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr> <tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>Smearing can occur in even more subtle ways. Suppose a library naively computes <del style="color: red; font-weight: bold; text-decoration: none;"><code></del>e<sup>x</sup><del style="color: red; font-weight: bold; text-decoration: none;"></code> </del>by the formula <del style="color: red; font-weight: bold; text-decoration: none;"><code></del>1 + x + x<sup>2</sup>/2 + x<sup>3</sup>/3! + ...<del style="color: red; font-weight: bold; text-decoration: none;"></code> </del>This works fine for positive <del style="color: red; font-weight: bold; text-decoration: none;"><code></del>x<del style="color: red; font-weight: bold; text-decoration: none;"></code></del>, but consider what happens when <del style="color: red; font-weight: bold; text-decoration: none;"><code></del>x<del style="color: red; font-weight: bold; text-decoration: none;"></code> </del>is a large negative number. The even-powered terms result in large positive numbers, and subtracting the odd-powered magnitudes will not even affect the result. The problem here is that the roundoff in the large, positive terms is in a digit position of much greater significance than the true answer. The answer diverges toward positive infinity! The solution here is also simple: for negative <del style="color: red; font-weight: bold; text-decoration: none;"><code></del>x<del style="color: red; font-weight: bold; text-decoration: none;"></code></del>, compute <del style="color: red; font-weight: bold; text-decoration: none;"><code></del>e<sup>x</sup> = 1/e<sup>|x|</sup><del style="color: red; font-weight: bold; text-decoration: none;"><</del>/<del style="color: red; font-weight: bold; text-decoration: none;">code></del>.</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>Smearing can occur in even more subtle ways. Suppose a library naively computes <ins style="color: red; font-weight: bold; text-decoration: none;">''</ins>e<sup>x</sup><ins style="color: red; font-weight: bold; text-decoration: none;">'' </ins>by the formula <ins style="color: red; font-weight: bold; text-decoration: none;">''</ins>1 + x + x<sup>2</sup>/2 + x<sup>3</sup>/3! + ...<ins style="color: red; font-weight: bold; text-decoration: none;">''. </ins>This works fine for positive <ins style="color: red; font-weight: bold; text-decoration: none;">''</ins>x<ins style="color: red; font-weight: bold; text-decoration: none;">''</ins>, but consider what happens when <ins style="color: red; font-weight: bold; text-decoration: none;">''</ins>x<ins style="color: red; font-weight: bold; text-decoration: none;">'' </ins>is a large negative number. The even-powered terms result in large positive numbers, and subtracting the odd-powered magnitudes will not even affect the result. The problem here is that the roundoff in the large, positive terms is in a digit position of much greater significance than the true answer. The answer diverges toward positive infinity! The solution here is also simple: for negative <ins style="color: red; font-weight: bold; text-decoration: none;">''</ins>x<ins style="color: red; font-weight: bold; text-decoration: none;">''</ins>, compute <ins style="color: red; font-weight: bold; text-decoration: none;">''</ins>e<sup>x</sup> = 1/e<sup>|x|</sup><ins style="color: red; font-weight: bold; text-decoration: none;">''.</ins></div></td></tr> <tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div> </div></td></tr> <tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;">By [[Chuck Allison]]</ins></div></td></tr> <tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div> </div></td></tr> <tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;">This work is licensed under a [http:</ins>/<ins style="color: red; font-weight: bold; text-decoration: none;">/creativecommons</ins>.<ins style="color: red; font-weight: bold; text-decoration: none;">org/licenses/by/3.0/us/ Creative Commons Attribution 3] </ins></div></td></tr> <tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div> </div></td></tr> <tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div> </div></td></tr> <tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div> </div></td></tr> <tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;">Back to [[97 Things Every Programmer Should Know]] home page</ins></div></td></tr>  </table>

Kevlin http://commons.oreilly.com/wiki/index.php?title=Floating-point_Numbers_Aren%27t_Real&diff=22924&oldid=prev Harmonious at 20:05, 2 January 2009 2009-01-02T20:05:57Z

<p></p> <table style="background-color: white; color:black;"> <col class='diff-marker' /> <col class='diff-content' /> <col class='diff-marker' /> <col class='diff-content' /> <tr> <td colspan='2' style="background-color: white; color:black;">←Older revision</td> <td colspan='2' style="background-color: white; color:black;">Revision as of 20:05, 2 January 2009</td> </tr> <tr><td colspan="2" class="diff-lineno">Line 9:</td> <td colspan="2" class="diff-lineno">Line 9:</td></tr> <tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Since floating-point numbers are approximations of real numbers, there is inevitably a little error present. This error, called ''roundoff'', can lead to surprising results. When you subtract nearly equal numbers, for example, the most significant digits cancel each other out, so what was the least significant digit (where the roundoff error resides) gets promoted to the most significant position in the floating-point result, essentially contaminating any further related computations (a phenomenon known as ''smearing''). You need to look closely at your algorithms to prevent such ''catastrophic cancellation''. To illustrate, consider solving the equation <code>x<sup>2</sup> - 100000x + 1</code> with the quadratic formula. Since the operands in the expression <code>-b + sqrt(b<sup>2</sup?> - 4)</code> are nearly equal in magnitude, you can instead compute the root <code>r<sub>1</sub> = -b - sqrt(b<sup>2</sup> - 4)</code>, and then obtain <code>r<sub>2</sub> = 1/r<sub>1</sub></code>, since for any quadratic equation, <code>ax<sup>2</sup> + bx + c = 0</code>, the roots satisfy <code>r<sub>1</sub>r<sub>2</sub> = c/a</code>.</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Since floating-point numbers are approximations of real numbers, there is inevitably a little error present. This error, called ''roundoff'', can lead to surprising results. When you subtract nearly equal numbers, for example, the most significant digits cancel each other out, so what was the least significant digit (where the roundoff error resides) gets promoted to the most significant position in the floating-point result, essentially contaminating any further related computations (a phenomenon known as ''smearing''). You need to look closely at your algorithms to prevent such ''catastrophic cancellation''. To illustrate, consider solving the equation <code>x<sup>2</sup> - 100000x + 1</code> with the quadratic formula. Since the operands in the expression <code>-b + sqrt(b<sup>2</sup?> - 4)</code> are nearly equal in magnitude, you can instead compute the root <code>r<sub>1</sub> = -b - sqrt(b<sup>2</sup> - 4)</code>, and then obtain <code>r<sub>2</sub> = 1/r<sub>1</sub></code>, since for any quadratic equation, <code>ax<sup>2</sup> + bx + c = 0</code>, the roots satisfy <code>r<sub>1</sub>r<sub>2</sub> = c/a</code>.</div></td></tr> <tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr> <tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>Smearing can occur in even more subtle ways. Suppose a library naively computes <code>e<sup>x</sup></code> by the formula <code>1 + x + x<sup>2</sup>/2 + x<sup>3</sup>/3! + ...</code> This works fine for positive <code>x</code>, but consider what happens when <code>x</code> is a large negative number. The even-powered terms result in large positive numbers, and subtracting the odd-powered magnitudes will not even affect the result. The problem here is that the roundoff in the large, positive terms is in a digit position of much greater significance than the true answer. The answer diverges toward positive infinity! The solution here is also simple: for negative <code>x</code>, compute <code>e<sup>x</sup> = 1/e<sup>|x|</sup></div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>Smearing can occur in even more subtle ways. Suppose a library naively computes <code>e<sup>x</sup></code> by the formula <code>1 + x + x<sup>2</sup>/2 + x<sup>3</sup>/3! + ...</code> This works fine for positive <code>x</code>, but consider what happens when <code>x</code> is a large negative number. The even-powered terms result in large positive numbers, and subtracting the odd-powered magnitudes will not even affect the result. The problem here is that the roundoff in the large, positive terms is in a digit position of much greater significance than the true answer. The answer diverges toward positive infinity! The solution here is also simple: for negative <code>x</code>, compute <code>e<sup>x</sup> = 1/e<sup>|x|</sup><ins style="color: red; font-weight: bold; text-decoration: none;"></code>.</ins></div></td></tr>  </table>

Harmonious http://commons.oreilly.com/wiki/index.php?title=Floating-point_Numbers_Aren%27t_Real&diff=22923&oldid=prev Harmonious at 20:05, 2 January 2009 2009-01-02T20:05:28Z

<p></p> <table style="background-color: white; color:black;"> <col class='diff-marker' /> <col class='diff-content' /> <col class='diff-marker' /> <col class='diff-content' /> <tr> <td colspan='2' style="background-color: white; color:black;">←Older revision</td> <td colspan='2' style="background-color: white; color:black;">Revision as of 20:05, 2 January 2009</td> </tr> <tr><td colspan="2" class="diff-lineno">Line 9:</td> <td colspan="2" class="diff-lineno">Line 9:</td></tr> <tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Since floating-point numbers are approximations of real numbers, there is inevitably a little error present. This error, called ''roundoff'', can lead to surprising results. When you subtract nearly equal numbers, for example, the most significant digits cancel each other out, so what was the least significant digit (where the roundoff error resides) gets promoted to the most significant position in the floating-point result, essentially contaminating any further related computations (a phenomenon known as ''smearing''). You need to look closely at your algorithms to prevent such ''catastrophic cancellation''. To illustrate, consider solving the equation <code>x<sup>2</sup> - 100000x + 1</code> with the quadratic formula. Since the operands in the expression <code>-b + sqrt(b<sup>2</sup?> - 4)</code> are nearly equal in magnitude, you can instead compute the root <code>r<sub>1</sub> = -b - sqrt(b<sup>2</sup> - 4)</code>, and then obtain <code>r<sub>2</sub> = 1/r<sub>1</sub></code>, since for any quadratic equation, <code>ax<sup>2</sup> + bx + c = 0</code>, the roots satisfy <code>r<sub>1</sub>r<sub>2</sub> = c/a</code>.</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Since floating-point numbers are approximations of real numbers, there is inevitably a little error present. This error, called ''roundoff'', can lead to surprising results. When you subtract nearly equal numbers, for example, the most significant digits cancel each other out, so what was the least significant digit (where the roundoff error resides) gets promoted to the most significant position in the floating-point result, essentially contaminating any further related computations (a phenomenon known as ''smearing''). You need to look closely at your algorithms to prevent such ''catastrophic cancellation''. To illustrate, consider solving the equation <code>x<sup>2</sup> - 100000x + 1</code> with the quadratic formula. Since the operands in the expression <code>-b + sqrt(b<sup>2</sup?> - 4)</code> are nearly equal in magnitude, you can instead compute the root <code>r<sub>1</sub> = -b - sqrt(b<sup>2</sup> - 4)</code>, and then obtain <code>r<sub>2</sub> = 1/r<sub>1</sub></code>, since for any quadratic equation, <code>ax<sup>2</sup> + bx + c = 0</code>, the roots satisfy <code>r<sub>1</sub>r<sub>2</sub> = c/a</code>.</div></td></tr> <tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr> <tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>Smearing can occur in even more subtle ways. Suppose a library naively computes <code>e<sup>x</sup></code> by the formula <code>1 + x + x<sup>2</sup>/2 + x<sup>3</sup>/3! + ...</code> This works fine for positive <code>x</code>, but consider what happens when <code>x</code> is a large negative number. The even-powered terms result in large positive numbers, and subtracting the odd-powered magnitudes will not even affect the result. The problem here is that the roundoff in the large, positive terms is in a digit position of much greater significance than the true answer. The answer diverges toward positive infinity! The solution here is also simple: for negative <code>x</code>, compute <code>e<sup>x</sup> = 1/e<sup>|x|</sup><del style="color: red; font-weight: bold; text-decoration: none;"></co</del></div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>Smearing can occur in even more subtle ways. Suppose a library naively computes <code>e<sup>x</sup></code> by the formula <code>1 + x + x<sup>2</sup>/2 + x<sup>3</sup>/3! + ...</code> This works fine for positive <code>x</code>, but consider what happens when <code>x</code> is a large negative number. The even-powered terms result in large positive numbers, and subtracting the odd-powered magnitudes will not even affect the result. The problem here is that the roundoff in the large, positive terms is in a digit position of much greater significance than the true answer. The answer diverges toward positive infinity! The solution here is also simple: for negative <code>x</code>, compute <code>e<sup>x</sup> = 1/e<sup>|x|</sup></div></td></tr> <tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div><del style="color: red; font-weight: bold; text-decoration: none;">de>.</del></div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div></div></td></tr>  </table>

Harmonious http://commons.oreilly.com/wiki/index.php?title=Floating-point_Numbers_Aren%27t_Real&diff=22922&oldid=prev Harmonious at 20:04, 2 January 2009 2009-01-02T20:04:28Z

<p></p> <table style="background-color: white; color:black;"> <col class='diff-marker' /> <col class='diff-content' /> <col class='diff-marker' /> <col class='diff-content' /> <tr> <td colspan='2' style="background-color: white; color:black;">←Older revision</td> <td colspan='2' style="background-color: white; color:black;">Revision as of 20:04, 2 January 2009</td> </tr> <tr><td colspan="2" class="diff-lineno">Line 9:</td> <td colspan="2" class="diff-lineno">Line 9:</td></tr> <tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Since floating-point numbers are approximations of real numbers, there is inevitably a little error present. This error, called ''roundoff'', can lead to surprising results. When you subtract nearly equal numbers, for example, the most significant digits cancel each other out, so what was the least significant digit (where the roundoff error resides) gets promoted to the most significant position in the floating-point result, essentially contaminating any further related computations (a phenomenon known as ''smearing''). You need to look closely at your algorithms to prevent such ''catastrophic cancellation''. To illustrate, consider solving the equation <code>x<sup>2</sup> - 100000x + 1</code> with the quadratic formula. Since the operands in the expression <code>-b + sqrt(b<sup>2</sup?> - 4)</code> are nearly equal in magnitude, you can instead compute the root <code>r<sub>1</sub> = -b - sqrt(b<sup>2</sup> - 4)</code>, and then obtain <code>r<sub>2</sub> = 1/r<sub>1</sub></code>, since for any quadratic equation, <code>ax<sup>2</sup> + bx + c = 0</code>, the roots satisfy <code>r<sub>1</sub>r<sub>2</sub> = c/a</code>.</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Since floating-point numbers are approximations of real numbers, there is inevitably a little error present. This error, called ''roundoff'', can lead to surprising results. When you subtract nearly equal numbers, for example, the most significant digits cancel each other out, so what was the least significant digit (where the roundoff error resides) gets promoted to the most significant position in the floating-point result, essentially contaminating any further related computations (a phenomenon known as ''smearing''). You need to look closely at your algorithms to prevent such ''catastrophic cancellation''. To illustrate, consider solving the equation <code>x<sup>2</sup> - 100000x + 1</code> with the quadratic formula. Since the operands in the expression <code>-b + sqrt(b<sup>2</sup?> - 4)</code> are nearly equal in magnitude, you can instead compute the root <code>r<sub>1</sub> = -b - sqrt(b<sup>2</sup> - 4)</code>, and then obtain <code>r<sub>2</sub> = 1/r<sub>1</sub></code>, since for any quadratic equation, <code>ax<sup>2</sup> + bx + c = 0</code>, the roots satisfy <code>r<sub>1</sub>r<sub>2</sub> = c/a</code>.</div></td></tr> <tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr> <tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>Smearing can occur in even more subtle ways. Suppose a library naively computes <code>e<sup>x</sup></code> by the formula <code>1 + x + x<sup>2</sup>/2 + x<sup>3</sup>/3! + ...</code> This works fine for positive <code>x</code>, but consider what happens when <code>x</code> is a large negative number. The even-powered terms result in large positive numbers, and subtracting the odd-powered magnitudes will not even affect the result. The problem here is that the roundoff in the large, positive terms is in a digit position of much greater significance than the true answer. The answer diverges toward positive infinity! The solution here is also simple: for negative <code>x</code>, compute <code>e<sup>x</sup> = 1/e<sup>|x|</sup></<del style="color: red; font-weight: bold; text-decoration: none;">c</del></div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>Smearing can occur in even more subtle ways. Suppose a library naively computes <code>e<sup>x</sup></code> by the formula <code>1 + x + x<sup>2</sup>/2 + x<sup>3</sup>/3! + ...</code> This works fine for positive <code>x</code>, but consider what happens when <code>x</code> is a large negative number. The even-powered terms result in large positive numbers, and subtracting the odd-powered magnitudes will not even affect the result. The problem here is that the roundoff in the large, positive terms is in a digit position of much greater significance than the true answer. The answer diverges toward positive infinity! The solution here is also simple: for negative <code>x</code>, compute <code>e<sup>x</sup> = 1/e<sup>|x|</sup></<ins style="color: red; font-weight: bold; text-decoration: none;">co</ins></div></td></tr> <tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;">de>.</ins></div></td></tr>  </table>

Harmonious http://commons.oreilly.com/wiki/index.php?title=Floating-point_Numbers_Aren%27t_Real&diff=22921&oldid=prev Harmonious at 20:03, 2 January 2009 2009-01-02T20:03:24Z

<p></p> <table style="background-color: white; color:black;"> <col class='diff-marker' /> <col class='diff-content' /> <col class='diff-marker' /> <col class='diff-content' /> <tr> <td colspan='2' style="background-color: white; color:black;">←Older revision</td> <td colspan='2' style="background-color: white; color:black;">Revision as of 20:03, 2 January 2009</td> </tr> <tr><td colspan="2" class="diff-lineno">Line 9:</td> <td colspan="2" class="diff-lineno">Line 9:</td></tr> <tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Since floating-point numbers are approximations of real numbers, there is inevitably a little error present. This error, called ''roundoff'', can lead to surprising results. When you subtract nearly equal numbers, for example, the most significant digits cancel each other out, so what was the least significant digit (where the roundoff error resides) gets promoted to the most significant position in the floating-point result, essentially contaminating any further related computations (a phenomenon known as ''smearing''). You need to look closely at your algorithms to prevent such ''catastrophic cancellation''. To illustrate, consider solving the equation <code>x<sup>2</sup> - 100000x + 1</code> with the quadratic formula. Since the operands in the expression <code>-b + sqrt(b<sup>2</sup?> - 4)</code> are nearly equal in magnitude, you can instead compute the root <code>r<sub>1</sub> = -b - sqrt(b<sup>2</sup> - 4)</code>, and then obtain <code>r<sub>2</sub> = 1/r<sub>1</sub></code>, since for any quadratic equation, <code>ax<sup>2</sup> + bx + c = 0</code>, the roots satisfy <code>r<sub>1</sub>r<sub>2</sub> = c/a</code>.</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Since floating-point numbers are approximations of real numbers, there is inevitably a little error present. This error, called ''roundoff'', can lead to surprising results. When you subtract nearly equal numbers, for example, the most significant digits cancel each other out, so what was the least significant digit (where the roundoff error resides) gets promoted to the most significant position in the floating-point result, essentially contaminating any further related computations (a phenomenon known as ''smearing''). You need to look closely at your algorithms to prevent such ''catastrophic cancellation''. To illustrate, consider solving the equation <code>x<sup>2</sup> - 100000x + 1</code> with the quadratic formula. Since the operands in the expression <code>-b + sqrt(b<sup>2</sup?> - 4)</code> are nearly equal in magnitude, you can instead compute the root <code>r<sub>1</sub> = -b - sqrt(b<sup>2</sup> - 4)</code>, and then obtain <code>r<sub>2</sub> = 1/r<sub>1</sub></code>, since for any quadratic equation, <code>ax<sup>2</sup> + bx + c = 0</code>, the roots satisfy <code>r<sub>1</sub>r<sub>2</sub> = c/a</code>.</div></td></tr> <tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr> <tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>Smearing can occur in even more subtle ways. Suppose a library naively computes <code>e<sup>x</sup></code> by the formula <code>1 + x + x<sup>2</sup>/2 + x<sup>3</sup>/3! + ...</code>This works fine for positive <code>x</code>, but consider what happens when <code>x</code> is a large negative number. The even-powered terms result in large positive numbers, and subtracting the odd-powered magnitudes will not even affect the result. The problem here is that the roundoff in the large, positive terms is in a digit position of much greater significance than the true answer<del style="color: red; font-weight: bold; text-decoration: none;">! </del>The answer diverges toward positive infinity! The solution here is also simple: for negative <code>x</code>, compute <code>e<sup>x</sup> = 1/e<</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>Smearing can occur in even more subtle ways. Suppose a library naively computes <code>e<sup>x</sup></code> by the formula <code>1 + x + x<sup>2</sup>/2 + x<sup>3</sup>/3! + ...</code> This works fine for positive <code>x</code>, but consider what happens when <code>x</code> is a large negative number. The even-powered terms result in large positive numbers, and subtracting the odd-powered magnitudes will not even affect the result. The problem here is that the roundoff in the large, positive terms is in a digit position of much greater significance than the true answer<ins style="color: red; font-weight: bold; text-decoration: none;">. </ins>The answer diverges toward positive infinity! The solution here is also simple: for negative <code>x</code>, compute <code>e<sup>x</sup> = 1/e<sup>|x|</sup></<ins style="color: red; font-weight: bold; text-decoration: none;">c</ins></div></td></tr> <tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>sup>|x|</sup></<del style="color: red; font-weight: bold; text-decoration: none;">co</del></div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div></div></td></tr> <tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div><del style="color: red; font-weight: bold; text-decoration: none;">de>.</del></div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div></div></td></tr> <tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div> </div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div></div></td></tr> <tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div><del style="color: red; font-weight: bold; text-decoration: none;">By [[Chuck Allison]]</del></div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div></div></td></tr> <tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div> </div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div></div></td></tr> <tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div><del style="color: red; font-weight: bold; text-decoration: none;">This work is licensed under a [http://creativecommons.org/licenses/by/3.0/us/ Creative Commons Attribution 3] </del></div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div></div></td></tr> <tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div> </div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div></div></td></tr> <tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div> </div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div></div></td></tr> <tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div> </div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div></div></td></tr> <tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div><del style="color: red; font-weight: bold; text-decoration: none;">Back to [[97 Things Every Programmer Should Know]] home page</del></div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div></div></td></tr>  </table>

Harmonious http://commons.oreilly.com/wiki/index.php?title=Floating-point_Numbers_Aren%27t_Real&diff=22919&oldid=prev Harmonious at 20:02, 2 January 2009 2009-01-02T20:02:07Z

<p></p> <table style="background-color: white; color:black;"> <col class='diff-marker' /> <col class='diff-content' /> <col class='diff-marker' /> <col class='diff-content' /> <tr> <td colspan='2' style="background-color: white; color:black;">←Older revision</td> <td colspan='2' style="background-color: white; color:black;">Revision as of 20:02, 2 January 2009</td> </tr> <tr><td colspan="2" class="diff-lineno">Line 9:</td> <td colspan="2" class="diff-lineno">Line 9:</td></tr> <tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Since floating-point numbers are approximations of real numbers, there is inevitably a little error present. This error, called ''roundoff'', can lead to surprising results. When you subtract nearly equal numbers, for example, the most significant digits cancel each other out, so what was the least significant digit (where the roundoff error resides) gets promoted to the most significant position in the floating-point result, essentially contaminating any further related computations (a phenomenon known as ''smearing''). You need to look closely at your algorithms to prevent such ''catastrophic cancellation''. To illustrate, consider solving the equation <code>x<sup>2</sup> - 100000x + 1</code> with the quadratic formula. Since the operands in the expression <code>-b + sqrt(b<sup>2</sup?> - 4)</code> are nearly equal in magnitude, you can instead compute the root <code>r<sub>1</sub> = -b - sqrt(b<sup>2</sup> - 4)</code>, and then obtain <code>r<sub>2</sub> = 1/r<sub>1</sub></code>, since for any quadratic equation, <code>ax<sup>2</sup> + bx + c = 0</code>, the roots satisfy <code>r<sub>1</sub>r<sub>2</sub> = c/a</code>.</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>Since floating-point numbers are approximations of real numbers, there is inevitably a little error present. This error, called ''roundoff'', can lead to surprising results. When you subtract nearly equal numbers, for example, the most significant digits cancel each other out, so what was the least significant digit (where the roundoff error resides) gets promoted to the most significant position in the floating-point result, essentially contaminating any further related computations (a phenomenon known as ''smearing''). You need to look closely at your algorithms to prevent such ''catastrophic cancellation''. To illustrate, consider solving the equation <code>x<sup>2</sup> - 100000x + 1</code> with the quadratic formula. Since the operands in the expression <code>-b + sqrt(b<sup>2</sup?> - 4)</code> are nearly equal in magnitude, you can instead compute the root <code>r<sub>1</sub> = -b - sqrt(b<sup>2</sup> - 4)</code>, and then obtain <code>r<sub>2</sub> = 1/r<sub>1</sub></code>, since for any quadratic equation, <code>ax<sup>2</sup> + bx + c = 0</code>, the roots satisfy <code>r<sub>1</sub>r<sub>2</sub> = c/a</code>.</div></td></tr> <tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr> <tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>Smearing can occur in even more subtle ways. <del style="color: red; font-weight: bold; text-decoration: none;">Consider what happens if </del>a library naively computes <code>e<sup>x</sup></code> by the formula <code>1 + x + x<sup>2</sup>/2 + x<sup>3</sup>/3! + ...</code>This works fine for positive <code>x</code>, but consider what happens when <code>x</code> is a large negative number. The even-powered terms result in large positive numbers, and subtracting the odd-powered magnitudes will not even affect the result. The problem here is that the roundoff in the large, positive terms is in a digit position of much greater significance than the true answer! The answer diverges toward positive infinity! The solution here is also simple: for negative <code>x</code>, compute <code>e<sup>x</sup> = 1/e<</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>Smearing can occur in even more subtle ways. <ins style="color: red; font-weight: bold; text-decoration: none;">Suppose </ins>a library naively computes <code>e<sup>x</sup></code> by the formula <code>1 + x + x<sup>2</sup>/2 + x<sup>3</sup>/3! + ...</code>This works fine for positive <code>x</code>, but consider what happens when <code>x</code> is a large negative number. The even-powered terms result in large positive numbers, and subtracting the odd-powered magnitudes will not even affect the result. The problem here is that the roundoff in the large, positive terms is in a digit position of much greater significance than the true answer! The answer diverges toward positive infinity! The solution here is also simple: for negative <code>x</code>, compute <code>e<sup>x</sup> = 1/e<</div></td></tr> <tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>sup>|x|</sup></<del style="color: red; font-weight: bold; text-decoration: none;">code</del>>.</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>sup>|x|</sup></<ins style="color: red; font-weight: bold; text-decoration: none;">co</ins></div></td></tr> <tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;">de</ins>>.</div></td></tr> <tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr> <tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>By [[Chuck Allison]]</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>By [[Chuck Allison]]</div></td></tr>  </table>

Harmonious