Re: [OT] Rounding v Truncation, was: Re: Platform Support vs.
From: Dave Froble (davef_at_tsoft-inc.com)
Date: 07/26/05
- Next message: Dave Froble: "Re: Response issues on GS1280, VMS 7.3-2"
- Previous message: Peter Weaver: "Re: simple image processing on VMS"
- In reply to: Simon Clubley: "[OT] Rounding v Truncation, was: Re: Platform Support vs."
- Next in thread: JF Mezei: "Re: [OT] Rounding v Truncation, was: Re: Platform Support vs."
- Reply: JF Mezei: "Re: [OT] Rounding v Truncation, was: Re: Platform Support vs."
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Date: Tue, 26 Jul 2005 02:25:08 -0400
Simon Clubley wrote:
> In article <11e6169idjh2n3a@corp.supernews.com>, Dave Froble <davef@tsoft-inc.com> writes:
>
>>When I had a problem with Visual Basic she defended VB and wouldn't
>>consider it wrong. So I'll ask, actually maybe take a poll. If a real
>>value is placed into an integer variable, how many people would expect
>>the fraction to be truncated? I'd expect most would. I did. VAX/DEC
>>BASIC truncates. Well, VB rounds, and fractional parts of .5 and
>>greater round up. Sure screwed with the logic from the converted VAX
>>Basic applications. Sure screwed with my mind.
>>
>
>
> I would regard rounding to be the correct thing to do from a maths viewpoint.
I'll throw my rather aged degree in Mathematics onto the pile. My
memories of integer operation rules is that any fractional part is
always truncated. Hey, could be wrong.
> However, I created two examples, one in Ada and another in C which
> exhibit different behaviour. Ada rounds and C truncates:
<snip>
> The Ada behaviour is specified in the Ada Language Reference Manual (section
> 4.6, line 33), see: http://www.adapower.com/rm95/RM-4-6.html :
>
> If the target type is an integer type and the operand type is real,
> the result is rounded to the nearest integer (away from zero if
> exactly halfway between two integers).
Now this I have no problem with. If there is a specification such as
above, fine, anyone who makes use of the fine manual can determine what
to expect.
I could find no reference to such rules for Visual Basic. Not saying
that they don't exist, just that I cannot find such.
Regardless, my past experiences are that in a computer language, moving
a real number into an integer variable causes any fractional part to be
truncated. Try it in MACRO-32. Ok, so I'm a bit lazy, and don't
include the FIX function when that's the desired result. That comes
from over 30 years of programming and being used to a particular
behavior. Perhaps I should learn to be more specific.
-- David Froble Tel: 724-529-0450 Dave Froble Enterprises, Inc. Fax: 724-529-0596 DFE Ultralights, Inc. E-Mail: davef@tsoft-inc.com 170 Grimplin Road Vanderbilt, PA 15486
- Next message: Dave Froble: "Re: Response issues on GS1280, VMS 7.3-2"
- Previous message: Peter Weaver: "Re: simple image processing on VMS"
- In reply to: Simon Clubley: "[OT] Rounding v Truncation, was: Re: Platform Support vs."
- Next in thread: JF Mezei: "Re: [OT] Rounding v Truncation, was: Re: Platform Support vs."
- Reply: JF Mezei: "Re: [OT] Rounding v Truncation, was: Re: Platform Support vs."
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Relevant Pages
|