Decimal precision is not correct

[skykep]

Current Behavior

Returns extra decimal places that do not exist in PLC

Expected Behavior

Return exact value as exists in PLC

Possible Solution (Optional)

Unknown...

Context

Values are not true to what exist in the PLC.

Steps to Reproduce (for bugs only)

1. Set a value of 0.12345679 in the PLC.
2. ethernet-ip returns 0.12345679104328156

Your Environment

• Package version (Use npm list - e.g. 1.0.6):
  [email protected]

• Node Version (Use node --version - e.g. 9.8.0):
  v12.18.2

• Operating System and version:
  Ubuntu 18.04

• Controller Type (eg 1756-L83E/B):
  1756-L73 or 1769-L18ER/B

• Controller Firmware (eg 30.11):
  30.11 or 27.11, respectively

[jhenson29]

This is largely an issue with the PLC using 32-bit IEEE 754 floating point and Node using 64-bit IEEE 754 floating point.

Here is how Node stores 0.123456789 in hex (BE):
3f bf 9a dd 37 39 63 5f

Here is how the PLC stores it:
3d fc d6 ea

When the data comes into the EIP driver, it it's as a byte array. Then it's converted to a float. The bytes match. The issue is the conversion to a float. Node stores it as a 64-bit number and fills the rest with zeros.

3f bf 9a dd 37 39 63 5f - 64-bit rep of 0.123456789
3f bf 9a dd 40 00 00 00 - 64-bit rep of 0.123456789 converted from a float

Maybe a custom float conversion could be written to handle it, but it hasn't been an issue for me in practice.

You can get back to the original data and do the conversion yourself after the fact if you want.
If you take 0.12345679104328156 and write it to a buffer as a float, you'll get the original bytes back.
3d fc d6 ea

[skykep]

Interesting. Thanks for the insight. I'll have to look and see if there's a way I can do something in my script. The error, while very small, is actually a bigger deal for what I'm trying to do since I need to capture the exact value.

[jhenson29]

If you want to do that, you might be better off encoding it in a DINT.
Send 0.1234567889 as 123456789 and then divide by 1000000000 for the data.
Float/Real isn't the best for 'exact' representation, especially across systems with different representations.

Float/Real is typically okay if the binary error is less than your required precision, which doesn't sound like your use case.

[skykep]

Yeah that's exactly what I thought about doing.
To make it easier, I'm thinking a few different arrays. Maybe DINTX10, DINTX100, DINTX1000 to signify the scaling.