Xojo Conferences
MBSOct2019CologneDE

significant digits? (Real Studio network user group Mailinglist archive)

Back to the thread list
Previous thread: Primary Keys
Next thread: tab keys...


Re: REALbasic-NUG Digest #6108   -   Mike Richardson
  significant digits?   -   David E. Garrett
   Re: significant digits?   -   john roberts

significant digits?
Date: 06.05.02 04:42 (Sun, 05 May 2002 21:42:43 -0600)
From: David E. Garrett
I am working on a project that needs me to round numbers to a specific
number of significant digits (usually 3). I envision something along the
lines of:
rounded = RoundToSD(unroundeddouble,howmanySD)

To illustrate the rules of SD rounding (all to 3 SD):
2.345678 becomes 2.35
12.764 becomes 12.8
0.03453 becomes 0.0345
6.03456 becomes 6.03
9.3021 becomes 9.30

This is more complex than simple rounding to the nearest integer. I can
mentally picture a method to do this, but it is so full of if this then
that statements, I am sure I am thinking the hard way. Has anyone created
such a method that handles this kind of rounding easily and elegantly? I
am sure there is a clever and elegant mathematical approach one can take,
but it escapes me.

Any ideas or already done methods I can borrow?

Thanks
Dave

-----------------------------------
David E. Garrett
Teacher & Technology Coordinator
Morinville Community High School
Phone: (780) 939-6891 email: <email address removed>
Fax: (780) 939-6896 http://www.mchs.gsacrd.ab.ca

---
Subscribe to the digest:
<mailto:<email address removed>>
Unsubscribe:
<mailto:<email address removed>>

Re: significant digits?
Date: 06.05.02 07:09 (Mon, 06 May 2002 02:09:55 -0400)
From: john roberts
on 5/5/02 11:42 PM, David E. Garrett at <email address removed> wrote:

> I am working on a project that needs me to round numbers to a specific
> number of significant digits (usually 3). I envision something along the
> lines of:
> rounded = RoundToSD(unroundeddouble,howmanySD)
>
> To illustrate the rules of SD rounding (all to 3 SD):
> 2.345678 becomes 2.35
> 12.764 becomes 12.8
> 0.03453 becomes 0.0345
> 6.03456 becomes 6.03
> 9.3021 becomes 9.30
>

You might take a look at the format function and the scientific notation
operator E. For example, let s be a string and x be a double. Using

s = format(x,"-#.00")

will put the string value of x into s with exactly 3 significant digits.
Similarly,

s = format(x,"-#.0000")

will put the string value of x into s with exactly 5 significant digits.
Both cases will be properly rounded BUT expressed in scientific notation. If
this is OK, then you are done; however, judging from what you said above,
you are not done. :)

To guarantee keeping all significant digits you will have to shift the
decimal yourself if the exponent is not zero. This will get you started; if
you need some more drop me a note.

Best regards,
John Roberts

mailto:<email address removed>

---
Subscribe to the digest:
<mailto:<email address removed>>
Unsubscribe:
<mailto:<email address removed>>