Bug 6531 - Precision issues with decimals
Summary: Precision issues with decimals
Status: RESOLVED WONTFIX
Alias: None
Product: SQL
Classification: Unclassified
Component: all (show other bugs)
Version: 11.27.11 (Jul2017-SP3)
Hardware: All All
: Normal normal
Assignee: SQL devs
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-02-01 12:40 CET by Hannes Muehleisen
Modified: 2018-02-04 10:34 CET (History)
1 user (show)



Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Hannes Muehleisen cwiconfidential 2018-02-01 12:40:47 CET
User-Agent:       Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.119 Safari/537.36
Build Identifier: 

select 0.7/987654321.0; returns 0.0

It should probably not? More correct result obtained with

select cast(0.7 as double)/987654321.0 ; (7.087499999911405e-10)


Reproducible: Always
Comment 1 Niels Nes cwiconfidential 2018-02-04 10:34:11 CET
this is a cast between 2 decimals (x,1). Automatically casting to doubles is not done currently.