Poor Marc St. Hilaire. No one understands him—even though over 130 years have elapsed since his 1875 publication describing his new method of celestial sight reduction.
Perhaps this situation has arisen because it’s easy to learn the mechanics of the St. Hilaire method (also called the intercept method) without understanding the true fundamentals. After all, for many decades hundreds of navigators have easily, and successfully, navigated their vessels over the open oceans using the heavenly bodies. In writing my book, Celestial Navigation in the GPS Age (2007), I’ve had occasions to review many books on celestial navigation (CN). In all of these, the true fundamental idea is conveniently skipped over, or when explained, it’s explained incorrectly—I can only conclude that most celestial navigators haven’t had an opportunity to understand what they’re actually doing.
Even though the method can be used without understanding it, most boaters wish to know, as far as possible, every detail of their vessel and its equipment. So why not correctly understand the celestial navigation method that’s been universally used for the last century? It will make you a more confident celestial navigator.
It’s all about that assumed position, the infamous AP that many beginners find mysterious. And you can’t blame them. After all, in getting other lines of positions (LOPs), such as from the bearing of a known object or from a lighthouse’s height, we don’t assume anything. We measure one angle and plot the LOP from that. Why should CN be any different where we also measure one angle (the body’s altitude)?
The sextant’s measurement determines one thing — the great circle distance from the body’s GP (its geographical position on earth) to a point on the ship’s LOP. And when combined with the sun’s position, obtained from the Nautical Almanac using the sight’s time, we have all the needed information to plot the celestial LOP.
Nonetheless, many authors (including one who wrote a navigation classic) state that this is not enough information to plot the LOP — we need to introduce an assumed position to solve for the LOP’s location. Others claim that we must assume some position as a starting point because it requires an unreasonably large chart to plot the distance from the body’s GP to a point on the LOP. Another author states that no one has been able to figure out the curvature of the LOP, so it can only be plotted as a straight line. None of this is correct.
The celestial LOP
To place the mysterious assumed position into its proper perspective, we’ll take a vicarious stroll through direct ways of plotting the equal-altitude LOP. We’ll use the sun as our exemplary celestial body. The celestial navigator measures the sun’s angle above the horizon (called the sun’s altitude) with a sextant. But for explaining CN, we’ll use the angle between the sun and the point directly overhead the observer (their zenith). This angle is called the co-altitude — it’s simply 90° minus the altitude.
Figure 1 is drawn in the plane of the ship’s location, the sun’s GP, and the earth’s center. Because the sun is so far away, its rays from any particular point on its surface reach the earth essentially parallel to each other. This means the angle at the observer between the sun and their zenith, the co-altitude, is also the angle shown at the earth’s center. So the figure shows us that the great circle distance, R, is proportional to the co-altitude (i.e., if one is increased by a certain percentage, so is the other). The proportionality constant is approximately 60 nautical miles per degree of co-altitude. Thus R, in nautical miles, equals 60 times the co-altitude, in degrees.
So the sextant’s co-altitude gives us R, the ship’s great-circle distance from the sun’s GP. And since the Nautical Almanac gives us the location of the sun for the time of the sight, we know everything that there is to know. As shown in the 3D drawing of Figure 2, the ship must be on a small circle of radius R, centered on the sun’s GP. The ship could be anywhere on this LOP; every navigator on this circular LOP would see the same sun’s altitude, the same co-altitude, and would compute the same great-circle distance, R, as shown in the figure.
The purpose of sight reduction is to determine the latitude and longitude of some points on this celestial equal-altitude LOP. Latitude and longitude can specify the position of any point on earth, as every sailor knows. It follows that the same coordinates also specify the position of the point directly under a celestial body, its GP. However, for celestial bodies these coordinates are called declination, instead of latitude, and Greenwich hour angle, GHA, instead of longitude. These locations of the sun and an arbitrary point X on the LOP are shown in Figure 3, where we’ve labeled the distances from the North Pole rather than distances from the equator. That is, instead of latitude, we used the co-latitude which is just 90° minus the latitude. And we’ve done the same for the sun, using the co-declination, instead of the declination. The polar angle, called LHA (local hour angle), is the difference between the longitude of point X and the sun’s GHA.
The triangle formed by the three points, North Pole, point X, and the sun’s GP, is called the navigation triangle. As can been seen, it has three sides and three included angles. All we need to know about this triangle is that if we know any three of these six sides and angles, we can determine any of the remaining three (using a calculator or tables). But a celestial observation only gives us two pieces of information: the sun’s altitude and its declination. (We don’t know the LHA because that requires knowledge of the ship’s longitude.) Thus we’re stumped: we are short one piece of information. But that’s as it should be. We can’t expect one altitude observation to determine a unique point on the 2D surface of the earth — it takes two pieces of information relative to the sun’s GP to fix a point on a 2D surface.
However, if we use the two known sides of the triangle (the co-altitude and co-declination) and specify a value for a third parameter, a plotting variable, we can solve for the latitude and longitude of a point on the LOP that has that third parameter’s value. Thus by varying this plotting variable and keeping the co-declination and co-altitude constant, we can trace out the latitude and longitude coordinates of points on the LOP, plotting them on a chart.
Plotting the LOP — exactly
Any plotting variable will work. We could, for example, pick longitude. Then using three known quantities, LHA (from this specified longitude and the sun’s GHA), observed co-altitude and co-declination, we could calculate the latitude of that point on the LOP. Repeating this process for selected longitudes traces out the LOP by its latitude and longitude coordinates — no assumed position used.
We could also do the reverse: we could pick the latitude as the plotting variable. Then we would calculate the longitude of a point on the LOP from this latitude, co-declination and observed co-altitude. (The resulting longitude comes from calculating the LHA, and using the known GHA of the sun.) This is exactly what Capt. Thomas H. Sumner did while commanding the ship Cabot off the coast of Ireland on December 18, 1837. He was bound ENE up St. George’s Channel, closed-hauled with a SE wind making Ireland’s coast a lee shore. By calculating the longitude for three separate latitudes, he demonstrated that it’s possible to compute, and hence to plot, the location of any number of points on the LOP.
A third plotting variable (which is particularly convenient for computer use) is the bearing angle Bx between the sun’s meridian and the co-altitude arc, as shown in Figure 4. By incrementing Bx, we can conveniently sweep out any desired arc of the LOP. Now, the three known quantities are the co-altitude, the co-declination and the variable angle Bx.
It’s significant to point out that all of these methods produce the same exact LOP (i.e., within the accuracy of the Almanac and the altitude measurement). There are no assumptions — no assumed position, no dead reckoning. Using any of these methods, we can easily and exactly plot the latitude and longitude coordinates of the LOP on any map, of any scale and any projection.
This shows that there is no problem calculating the points on an equal-altitude LOP directly and accurately — anywhere and everywhere. But at sea we only want to plot a segment of the LOP in the region of our interest. In two of the above methods, we need to decide either a latitude or a longitude that we’re interested in. But this in an incomplete specification of our region of interest, leading to potential problems. For example, picking a latitude could lead to a longitude on the LOP that’s far from our region of interest. This is where Capt. St. Hilaire saves the ship.
The St. Hilaire method
In his method we specify both the latitude and longitude of a point in our region of interest. We’ll call this point the RP, the reference point. Using the coordinates of both this RP and the sun, we calculate the sun’s altitude and azimuth that would be observed at this RP at the time of the sight. Plotting a straight line at this azimuth on a Mercator chart from the RP toward the sun points directly and accurately toward the sun. Since this line points toward the sun, if we move a little away from this line at right angles to the azimuth line, we would still see the sun’s same altitude, as can be appreciated from Figures 2 or 3. Therefore, a short line draw perpendicular to the azimuth line is an equal-altitude sun LOP.
Figure 5 shows a plot of this azimuth line draw toward the sun from the RP. In this example, the calculated altitude (labeled Hc) at the RP is 34° 40’. Other contours of equal-altitude LOPs are also shown, crossing the azimuth line at right angles. Now let’s say that our actual observed altitude on our ship (labeled Ho) measures 34° 50’. We would then know that the ship’s LOP lies 10’ (or 10 nautical miles) toward the sun, at point A, as shown in the figure (it’s toward because the observed altitude was greater than Hc.) This 10 mile distance, Ho minus Hc, is called the intercept distance. Point A is exactly on the ship’s LOP within the error of approximating the intercept distance by a straight line (a rhumb line on a Mercator chart). In practice, this error is always negligible.
Using other RPs, we could calculate other points like A, tracing out the LOP exactly. But there’s no need to. As noted above, a straight line draw perpendicular to the azimuth line must show no change in altitude along it (at least for short distances). So it must be tangent to the true LOP, and thus it’s an approximation to the true LOP. This line is St. Hilaire’s LOP.
In practice, the only error in this method arises from the divergence of the straight-line LOP from the true-curved LOP. This depends on the ship’s distance along the LOP from the tangent point, A, compared to the curvature of the LOP. For altitudes less than about 75°, the curvature of the LOP is small enough that this discrepancy is acceptable.
The misleading assumed position
We’ve shown that we assume nothing in using the St. Hilaire method. We decide which region we wish to plot the sun’s LOP by specifying the latitude and longitude of a reference point. And the method gives us a straight-line approximation to the segment of the true LOP that is nearest that position — there’s no assumption, no approximation (other than the slight curvature of the true LOP).
Unfortunately, poor old Capt. St. Hilaire invited confusion by referring to this geographical reference point as an estimated position in his 1875 publication (at least in my English language version). This term later morphed into the assumed position, the AP, the term currently used for the reference point.
Even though St. Hilaire used the unfortunate term “estimated position,” it’s clear that he, and even Capt. Sumner in his 1843 publication, understood how to exactly plot the celestial LOP by calculating the latitude and longitude of several points on the curve.
Of course in normal navigation, we wish to know the segment of the LOP which is nearest where we think our ship is. So we pick our latest DR position, or (to simplify calculations) another position near the DR position for our RP; and unfortunately we call it the assumed position.
Many readers might think that I’m putting too fine a point on the distinction between the terms “reference point” and “assumed position.” But assumed position is a misnomer incorrectly implying that an assumption is necessary for some reason: because the distance between the sun’s GP and the ship is too great to plot, because there’s insufficient information to plot the LOP, or because we don’t know how to plot the exact LOP. We’ve just seen that none of this is correct.
John Karl is the author of Celestial Navigation in the GPS Age, published by Paradise Cay.
