over the total length of the scale; error in spacing of successive divisions is not
to exceed 0.5 microns. Above and beyond this, we compare each scale with
one of the scales selected to serve as a master scale. In making this comparison,
the scale to be tested is placed in face-to-face contact with the master, a
spacing of nominally ten microns (or one linewidth) being set between
corresponding divisions. To facilitate the alignment of the two scales, the ends
of the master are cemented into position in the device pictured in Fig. 4. The
scale to be tested is then aligned with the aid of positioning screws, whereupon it
is clamped vertically at both ends to prevent further movement relative to the
master. The center-to-center spacing between corresponding divisions of the
two scales is easily measured to an accuracy of 40.2 microns by means of a
Watson Image Shearing Micrometer employed at a magnification of 200X. - If the
spacings were perfectly constant for all divisions, the scale to be tested would
perfectly match the master. Therefore, the variation of the spacing about the
mean is the measure of the discrepancy between the two scales. In this manner all
scales are referred to the master scale to an accuracy of 40.2 microns. The master
scale itself has been calibrated interferometrically to a certified accuracy of 40.5
microns by the National Bureau of Standards. By virtue of the comparative process
just described, all scales inherit the absolute accuracies of the master with virtually
no dilution, for rms errors of 0.5 and 0.2 microns combine vectorially to
[ (0.5)? * (0.2)? ]8 = 0.54 microns. The computer program is designed to apply
automatically the calibrated corrections for a given scale.
The second critical element of the comparator, the micrometer screw, is
calibrated to an accuracy of 410.3 microns over its measuring range of one
millimeter. This is accomplished with the aid of a microscope scale calibrated at
50 micron intervals over a length of one millimeter to an accuracy of +0.2 microns.
Inasmuch as the screw is calibrated when mounted in the comparator, the
calibration also accounts for any periodic errors resulting from eccentricity of the
measuring drum or from progressive errors in the graduation of the drum (the former
may amount to as much as £0.5 microns; the latter, by virtue of a dividing
accuracy of +15 seconds of arc, should not exceed +0. 12 microns). The calibrated
corrections for the screw are automatically applied by the computer program.
The primary requirement of the third critical element of the comparator, the
pivot, is that it provide a stationary point of rotation. This point is the center of a
0.75 inch ball bearing having a sphericity of £0.25 microns. The center of the ball
is located in the plane of the scale thereby making the measurements insensitive to
any up and down motion of the far end of the scale. Wobble of the bearing is
restricted by means of a spring loaded thrust bearing that provides a controlled
loading against the thrust bracket. The thrust is directed precisely along the axis
of rotation. When the comparator is in its normal operating position and the thrust
on the pivot is set at 3kg, a radial 'wobble' amounting to as much as two microns
can be detected over the sweep of the measuring arm; with a thrust of 6kg, there
is no discernible radial wobble (the lateral component of 'wobble' has no effect
of the measurements). To provide a comfortable margin of safety, the thrust on
production models is set at lOkg.