Xojo Developer Conference
25/27th April 2018 in Denver.
MBS Xojo Conference
6/7th September 2018 in Munich, Germany.

Re: problem with format (Real Studio network user group Mailinglist archive)

Back to the thread list
Previous thread: Sandboxing and Apple devbugs
Next thread: shown event


TypeLibs and API Interfaces for Windows   -   Garth Hjelte
  Re: problem with format   -   Carlo Rubini
   Re: problem with format   -   Stéphane Mons <
    problem with format   -   Carlo Rubini
     Re: problem with format   -   Stéphane Mons <
     Re: problem with format   -   Norman Palardy
     Re: problem with format   -  
     Re: problem with format   -   Stéphane Mons <
     Re: problem with format   -   Stéphane Mons <
     Re: problem with format   -   Tim Jones
      Re: problem with format   -   Dan Paymar
     Re: problem with format   -   Tim Jones
     Re: problem with format   -   Eric Williams
      Re: problem with format   -   Dan Paymar
     Re: problem with format   -   Stéphane Mons <

Re: problem with format
Date: 02.08.11 02:08 (Tue, 2 Aug 2011 07:08:41 +0600)
From: Carlo Rubini
I thank all those who answered.

At present I solved the problem not allowing users to enter more than 15 digits (before limitation was 16 digits).

I would only say that if format is meant to show something understandable to USERS (and not scientific notation etc.), than it should be able to deal also with integers.

BTW, doing the same operations (with 16 digits) with Apple Calculator would not give better results, both with 'Show thousand separator' selected and unselected.

Thanks again,

Carlo

_______________________________________________
Unsubscribe or switch delivery mode:
<http://www.realsoftware.com/support/listmanager/>

Search the archives:
<http://support.realsoftware.com/listarchives/lists.html>

Re: problem with format
Date: 02.08.11 03:42 (Tue, 2 Aug 2011 04:42:31 +0200)
From: Stéphane Mons <

Le 2 août 2011 à 03:08, Carlo Rubini a écrit :

> I thank all those who answered.
>
> At present I solved the problem not allowing users to enter more than 15 digits (before limitation was 16 digits).
>
> I would only say that if format is meant to show something understandable to USERS (and not scientific notation etc.), than it should be able to deal also with integers.
>
> BTW, doing the same operations (with 16 digits) with Apple Calculator would not give better results, both with 'Show thousand separator' selected and unselected.
>
> Thanks again,

Well anyway there is a problem with Format and Str with format specification. You should report it.

5 REM My Signature
10 PRINT "Stéphane"
20 GOTO 10



_______________________________________________
Unsubscribe or switch delivery mode:
<http://www.realsoftware.com/support/listmanager/>

Search the archives:
<http://support.realsoftware.com/listarchives/lists.html>

problem with format
Date: 01.08.11 15:39 (Mon, 1 Aug 2011 20:39:51 +0600)
From: Carlo Rubini
Hello,

dim v as int64 = 9999999999999999
MsgBox Format(v, "###,###")

I get 10,000,000,000,000,000

using: dim v as int32 or int16 I get still different outputs.

How to get 9,000,000,000,000,000 ?

Thanks,

RB 2010 5.1

Carlo

_______________________________________________
Unsubscribe or switch delivery mode:
<http://www.realsoftware.com/support/listmanager/>

Search the archives:
<http://support.realsoftware.com/listarchives/lists.html>

Re: problem with format
Date: 01.08.11 19:02 (Mon, 1 Aug 2011 20:02:56 +0200)
From: Stéphane Mons <

Le 1 août 2011 à 19:17, Norman Palardy a écrit :

>
> On Aug 1, 2011, at 10:55 AM, Stéphane Mons wrote:
>
>>> The original post was
>>>
>>> dim v as int64 = 9999999999999999
>>>
>>> A 16 decimal digit value stored as a 64-bit integer is more like 10 pounds in an 11 pound bag. No overflow. But converting it to a double tries to cram 10 pounds into a 9 pound bag, and that's a bug.
>>
>>
>> AND you cannot do math with it because it gets rounded somewhere in the process even when it is unnecessary.
>
>
> Literals come in a VERY few flavors
>
> Integers (32 bit)
> Floating point (doubles)
> Strings (in UTF-8 &u)
> Booleans
> Colors
> Hex/Binary/Octal (preceded by &h, &b or &o)
> (I might have neglected a case)
>
> But there's no such thing as a Int64 literal

So it really is a problem with Format and Str:

dim v1, v2 as Int64

v1 = 99999999
v2 = 100000000

v2 = (v1 * v2) + v1

Str( v2 ) -> 9999999999999999
Str( v2, "###,###" ) -> 10,000,000,000,000,000

Giving a format specification to Str causes unwanted rounding.


5 REM My Signature
10 PRINT "Stéphane"
20 GOTO 10



_______________________________________________
Unsubscribe or switch delivery mode:
<http://www.realsoftware.com/support/listmanager/>

Search the archives:
<http://support.realsoftware.com/listarchives/lists.html>

Re: problem with format
Date: 01.08.11 18:17 (Mon, 1 Aug 2011 11:17:33 -0600)
From: Norman Palardy

On Aug 1, 2011, at 10:55 AM, Stéphane Mons wrote:

>> The original post was
>>
>> dim v as int64 = 9999999999999999
>>
>> A 16 decimal digit value stored as a 64-bit integer is more like 10
>> pounds in an 11 pound bag. No overflow. But converting it to a
>> double tries to cram 10 pounds into a 9 pound bag, and that's a bug.
>
> AND you cannot do math with it because it gets rounded somewhere in
> the process even when it is unnecessary.

Literals come in a VERY few flavors

Integers (32 bit)
Floating point (doubles)
Strings (in UTF-8 &u)
Booleans
Colors
Hex/Binary/Octal (preceded by &h, &b or &o)
(I might have neglected a case)

But there's no such thing as a Int64 literal

Norman Palardy

Real World 2012, THE Real Studio Event of the year!
http://realsoftware.com/community/realworld.php

_______________________________________________
Unsubscribe or switch delivery mode:
<http://www.realsoftware.com/support/listmanager/>

Search the archives:
<http://support.realsoftware.com/listarchives/lists.html>

Re: problem with format
Date: 01.08.11 18:06 (Mon, 01 Aug 2011 13:06:14 -0400)
From:

On Mon, 01 Aug 2011 10:20:45 -0600, Dan Paymar <<email address removed>>
wrote:
> On 08/01/2011 10:08 AM, Eric Williams wrote:
>> On Aug 1, 2011, at 8:58 AM, Stéphane Mons wrote:
>>
>>> Int16 and Int32 cannot hold such a large number, so it is normal that
>>> you get weird results. Furthermore, you cannot express an Int64
>>> literally in RB.
>>>
>>> However, there is definitely a problem with that. I tried to write:
>>>
>>> dim i64, v as Int64
>>> i64 = 99999999
>>> v = 1
>>> i64 = ( i64 + v ) * i64 + i64
>>>
>>> but I got the same bad result. It looks like Int64 are converted to
>>> double for the calculation and it creates rounding problems.
>> That's exactly the problem.
>>
>>> Worth a feedback report
>> Better make it a feature request; the function is acting as expected.
> Why would you expect a value defined as an integer to be unnecessarily
> rounded? It's certainly not what I would expect. I consider it a bug.

Eeek, looks like I responded to the wrong email. I was trying to convey
that Format(double, string) is casting your Int64 to a double, with all the
baggage one might expect from that operation.

As for i64=99999999, here is your answer:

http://support.realsoftware.com/listarchives/realbasic-nug/2009-01/msg00387.html

99999999 is an integer literal.
99999999.0 is a double literal.

Eric M. Williams
Oxalyn

_______________________________________________
Unsubscribe or switch delivery mode:
<http://www.realsoftware.com/support/listmanager/>

Search the archives:
<http://support.realsoftware.com/listarchives/lists.html>

Re: problem with format
Date: 01.08.11 17:55 (Mon, 1 Aug 2011 18:55:27 +0200)
From: Stéphane Mons <

Le 1 août 2011 à 18:44, Dan Paymar a écrit :

>
>
> On 08/01/2011 10:27 AM, Tim Jones wrote:
>> On Aug 1, 2011, at 9:20 AM, Dan Paymar wrote:
>>
>>> On 08/01/2011 10:08 AM, Eric Williams wrote:
>>>> On Aug 1, 2011, at 8:58 AM, Stéphane Mons wrote:
>>>>
>>>>> Int16 and Int32 cannot hold such a large number, so it is normal that you get weird results. Furthermore, you cannot express an Int64 literally in RB.
>>>>>
>>>>> However, there is definitely a problem with that. I tried to write:
>>>>>
>>>>> dim i64, v as Int64
>>>>> i64 = 99999999
>>>>> v = 1
>>>>> i64 = ( i64 + v ) * i64 + i64
>>>>>
>>>>> but I got the same bad result. It looks like Int64 are converted to double for the calculation and it creates rounding problems.
>>>> That's exactly the problem.
>>>>
>>>>> Worth a feedback report
>>>> Better make it a feature request; the function is acting as expected.
>>> Why would you expect a value defined as an integer to be unnecessarily rounded? It's certainly not what I would expect. I consider it a bug.
>> 10 pounds in a 5 pound bag = spillage (overflow).
>
> The original post was
>
> dim v as int64 = 9999999999999999
>
> A 16 decimal digit value stored as a 64-bit integer is more like 10 pounds in an 11 pound bag. No overflow. But converting it to a double tries to cram 10 pounds into a 9 pound bag, and that's a bug.

AND you cannot do math with it because it gets rounded somewhere in the process even when it is unnecessary.

5 REM My Signature
10 PRINT "Stéphane"
20 GOTO 10



_______________________________________________
Unsubscribe or switch delivery mode:
<http://www.realsoftware.com/support/listmanager/>

Search the archives:
<http://support.realsoftware.com/listarchives/lists.html>

Re: problem with format
Date: 01.08.11 17:45 (Mon, 1 Aug 2011 18:45:43 +0200)
From: Stéphane Mons <

Le 1 août 2011 à 18:20, Dan Paymar a écrit :

>
>
> On 08/01/2011 10:08 AM, Eric Williams wrote:
>> On Aug 1, 2011, at 8:58 AM, Stéphane Mons wrote:
>>
>>> Int16 and Int32 cannot hold such a large number, so it is normal that you get weird results. Furthermore, you cannot express an Int64 literally in RB.
>>>
>>> However, there is definitely a problem with that. I tried to write:
>>>
>>> dim i64, v as Int64
>>> i64 = 99999999
>>> v = 1
>>> i64 = ( i64 + v ) * i64 + i64
>>>
>>> but I got the same bad result. It looks like Int64 are converted to double for the calculation and it creates rounding problems.
>> That's exactly the problem.
>>
>>> Worth a feedback report
>> Better make it a feature request; the function is acting as expected.
> Why would you expect a value defined as an integer to be unnecessarily rounded? It's certainly not what I would expect. I consider it a bug.

He was talking about Format (since the thread name is "Problem with format"), and Format works as expected.

5 REM My Signature
10 PRINT "Stéphane"
20 GOTO 10



_______________________________________________
Unsubscribe or switch delivery mode:
<http://www.realsoftware.com/support/listmanager/>

Search the archives:
<http://support.realsoftware.com/listarchives/lists.html>

Re: problem with format
Date: 01.08.11 17:27 (Mon, 1 Aug 2011 09:27:11 -0700)
From: Tim Jones
On Aug 1, 2011, at 9:20 AM, Dan Paymar wrote:

> On 08/01/2011 10:08 AM, Eric Williams wrote:
>> On Aug 1, 2011, at 8:58 AM, Stéphane Mons wrote:
>>
>>> Int16 and Int32 cannot hold such a large number, so it is normal that you get weird results. Furthermore, you cannot express an Int64 literally in RB.
>>>
>>> However, there is definitely a problem with that. I tried to write:
>>>
>>> dim i64, v as Int64
>>> i64 = 99999999
>>> v = 1
>>> i64 = ( i64 + v ) * i64 + i64
>>>
>>> but I got the same bad result. It looks like Int64 are converted to double for the calculation and it creates rounding problems.
>>
>> That's exactly the problem.
>>
>>> Worth a feedback report
>>
>> Better make it a feature request; the function is acting as expected.
>
> Why would you expect a value defined as an integer to be unnecessarily rounded? It's certainly not what I would expect. I consider it a bug.

10 pounds in a 5 pound bag = spillage (overflow).

Tim

_______________________________________________
Unsubscribe or switch delivery mode:
<http://www.realsoftware.com/support/listmanager/>

Search the archives:
<http://support.realsoftware.com/listarchives/lists.html>

Re: problem with format
Date: 01.08.11 17:44 (Mon, 01 Aug 2011 10:44:42 -0600)
From: Dan Paymar


On 08/01/2011 10:27 AM, Tim Jones wrote:
> On Aug 1, 2011, at 9:20 AM, Dan Paymar wrote:
>
>> On 08/01/2011 10:08 AM, Eric Williams wrote:
>>> On Aug 1, 2011, at 8:58 AM, Stéphane Mons wrote:
>>>
>>>> Int16 and Int32 cannot hold such a large number, so it is normal that you get weird results. Furthermore, you cannot express an Int64 literally in RB.
>>>>
>>>> However, there is definitely a problem with that. I tried to write:
>>>>
>>>> dim i64, v as Int64
>>>> i64 = 99999999
>>>> v = 1
>>>> i64 = ( i64 + v ) * i64 + i64
>>>>
>>>> but I got the same bad result. It looks like Int64 are converted to double for the calculation and it creates rounding problems.
>>> That's exactly the problem.
>>>
>>>> Worth a feedback report
>>> Better make it a feature request; the function is acting as expected.
>> Why would you expect a value defined as an integer to be unnecessarily rounded? It's certainly not what I would expect. I consider it a bug.
> 10 pounds in a 5 pound bag = spillage (overflow).

The original post was

dim v as int64 = 9999999999999999

A 16 decimal digit value stored as a 64-bit integer is more like 10 pounds in an 11 pound bag. No overflow. But converting it to a double tries to cram 10 pounds into a 9 pound bag, and that's a bug.

Dan


_______________________________________________
Unsubscribe or switch delivery mode:
<http://www.realsoftware.com/support/listmanager/>

Search the archives:
<http://support.realsoftware.com/listarchives/lists.html>

Re: problem with format
Date: 01.08.11 17:25 (Mon, 1 Aug 2011 09:25:04 -0700)
From: Tim Jones
On Aug 1, 2011, at 8:58 AM, Stéphane Mons wrote:

>
> Le 1 août 2011 à 16:39, Carlo Rubini a écrit :
>
>> Hello,
>>
>> dim v as int64 = 9999999999999999
>> MsgBox Format(v, "###,###")
>>
>> I get 10,000,000,000,000,000
>>
>>
>> using: dim v as int32 or int16 I get still different outputs.
>>
>>
>> How to get 9,000,000,000,000,000 ?
>>
>
> Int16 and Int32 cannot hold such a large number, so it is normal that you get weird results. Furthermore, you cannot express an Int64 literally in RB.
>
> However, there is definitely a problem with that. I tried to write:
>
> dim i64, v as Int64
> i64 = 99999999
> v = 1
> i64 = ( i64 + v ) * i64 + i64
>
> but I got the same bad result. It looks like Int64 are converted to double for the calculation and it creates rounding problems.
>
> Worth a feedback report

If you really require precision in very large numbers, one solution for the interim is to use Bob Delaney's free big number math plugins:

http://homepage.mac.com/delaneyrm/

Specifically -

http://homepage.mac.com/delaneyrm/fpPlugin.html

Tim


_______________________________________________
Unsubscribe or switch delivery mode:
<http://www.realsoftware.com/support/listmanager/>

Search the archives:
<http://support.realsoftware.com/listarchives/lists.html>

Re: problem with format
Date: 01.08.11 17:08 (Mon, 1 Aug 2011 09:08:09 -0700)
From: Eric Williams

On Aug 1, 2011, at 8:58 AM, Stéphane Mons wrote:

> Int16 and Int32 cannot hold such a large number, so it is normal that you get weird results. Furthermore, you cannot express an Int64 literally in RB.
>
> However, there is definitely a problem with that. I tried to write:
>
> dim i64, v as Int64
> i64 = 99999999
> v = 1
> i64 = ( i64 + v ) * i64 + i64
>
> but I got the same bad result. It looks like Int64 are converted to double for the calculation and it creates rounding problems.

That's exactly the problem.

> Worth a feedback report

Better make it a feature request; the function is acting as expected.

Eric M. Williams
Oxalyn
_______________________________________________
Unsubscribe or switch delivery mode:
<http://www.realsoftware.com/support/listmanager/>

Search the archives:
<http://support.realsoftware.com/listarchives/lists.html>

Re: problem with format
Date: 01.08.11 17:20 (Mon, 01 Aug 2011 10:20:45 -0600)
From: Dan Paymar


On 08/01/2011 10:08 AM, Eric Williams wrote:
> On Aug 1, 2011, at 8:58 AM, Stéphane Mons wrote:
>
>> Int16 and Int32 cannot hold such a large number, so it is normal that you get weird results. Furthermore, you cannot express an Int64 literally in RB.
>>
>> However, there is definitely a problem with that. I tried to write:
>>
>> dim i64, v as Int64
>> i64 = 99999999
>> v = 1
>> i64 = ( i64 + v ) * i64 + i64
>>
>> but I got the same bad result. It looks like Int64 are converted to double for the calculation and it creates rounding problems.
> That's exactly the problem.
>
>> Worth a feedback report
> Better make it a feature request; the function is acting as expected.
Why would you expect a value defined as an integer to be unnecessarily
rounded? It's certainly not what I would expect. I consider it a bug.

Dan Paymar

_______________________________________________
Unsubscribe or switch delivery mode:
<http://www.realsoftware.com/support/listmanager/>

Search the archives:
<http://support.realsoftware.com/listarchives/lists.html>

Re: problem with format
Date: 01.08.11 16:58 (Mon, 1 Aug 2011 17:58:28 +0200)
From: Stéphane Mons <
Int16 and Int32 cannot hold such a large number, so it is normal that you get weird results. Furthermore, you cannot express an Int64 literally in RB.

However, there is definitely a problem with that. I tried to write:

dim i64, v as Int64
i64 = 99999999
v = 1
i64 = ( i64 + v ) * i64 + i64

but I got the same bad result. It looks like Int64 are converted to double for the calculation and it creates rounding problems.

Worth a feedback report


Le 1 août 2011 à 16:39, Carlo Rubini a écrit :

> Hello,
>
> dim v as int64 = 9999999999999999
> MsgBox Format(v, "###,###")
>
> I get 10,000,000,000,000,000
>
>
> using: dim v as int32 or int16 I get still different outputs.
>
>
> How to get 9,000,000,000,000,000 ?
>
>
> Thanks,
>
> RB 2010 5.1
>
> Carlo
>
> _______________________________________________
> Unsubscribe or switch delivery mode:
> <http://www.realsoftware.com/support/listmanager/>
>
> Search the archives:
> <http://support.realsoftware.com/listarchives/lists.html>

5 REM My Signature
10 PRINT "Stéphane"
20 GOTO 10



_______________________________________________
Unsubscribe or switch delivery mode:
<http://www.realsoftware.com/support/listmanager/>

Search the archives:
<http://support.realsoftware.com/listarchives/lists.html>