Using the Calibrate Option of AutoVue When Performing a Measurement Returns a Value With Wrong Precision (Doc ID 2102226.1)

Last updated on FEBRUARY 05, 2016

Applies to:

Oracle AutoVue 3D Professional Advanced - Version 21.0.0 and later
Oracle AutoVue Electro-Mechanical Professional - Version 21.0.0 and later
Oracle AutoVue 2D Professional - Version 21.0.0 and later
Oracle AutoVue EDA Professional - Version 21.0.0 and later
Information in this document applies to any platform.

Symptoms

Using AutoVue and making a 5 inch measurement for example and trying to calibrate the measurement to 5 it comes back at a distance of 4.999999 instead of 5. It is not possible to set the value to 5.

This is a measurement taken on a file:

Cause

Sign In with your My Oracle Support account

Don't have a My Oracle Support account? Click to get started

My Oracle Support provides customers with access to over a
Million Knowledge Articles and hundreds of Community platforms