Most GPS users are aware that the Department of Defense deliberately degrades the civilian Global Positioning System (GPS) signal. But how many users are aware that there may be an even greater error between the displayed latitude and longitude on the GPS and the resulting fix when it is plotted on a chart? At issue here are complex questions concerning the shape of the Earth and how it is modeled for chart-making purposes.
As GPS increasingly supplants traditional navigational tools, it becomes all the more important for sailors to have a firm grasp of its limitations. This was brought home to me forcefully last year on the south coast of Cuba when I discovered a half-mile difference between our actual position and the GPS position when plotted on a recently published large-scale (i.e., detailed) chart of Santiago de Cuba. What was going on? Peculiar though it may seem, the easiest way to come up with an explanation is to look at the history of chart-making.
By the 17th century it was possible to make sufficiently accurate astronomical observations and distance measurements on the Earth’s surface to discover that latitude measurements at different points on the globe were not equal. If the world were a perfect sphere, this would be an impossibility. Thus, the Earth could not be spherical. From these discoveries, a tremendous scientific debate erupted. The primary protagonists were the French and the British.
On orders from Louis XIV, who needed accurate maps to govern his country and tax his people, first the astronomer Jean Picard and then the astronomer Gian Domenico Cassini had taken on the task of mapping France. In keeping with time-honored methodology, surveyors combined astronomical observations and a measured base line with a process called triangulation.
It is interesting to see how this survey work proceeded, because the French methods reflected a norm that remained essentially unchanged until recent decades for both cartographic and inshore hydrographic surveys. The baseline measurements commenced in Paris at a point determined by exceedingly precise astronomical observations. From Paris, carefully calibrated wooden rods were used to accurately measure distances due north and south. The surveyors also measured changes in vertical elevation in order to discount the effects of these on the horizontal distances covered. In this way, a very precise log of horizontal distances was maintained. The process was slow and painstaking and took years to complete.
Once a baseline had been established north and south of Paris, angular measurements were taken from both ends to a third position. Knowing the length of the baseline and the two angles, simple trigonometry established the distances to the third point without having to make field measurements. The sides of the triangle thus established were now used as fresh baselines to extend the survey, once again without having to make actual distance measurements in the field. When the English Channel and the Mediterranean were reached, fresh astronomical observations established the length of the arcs from Paris to the north, and Paris to the south, in degrees of latitude. The measured baselines plus the process of triangulation gave the horizontal distances on the ground.
Cassini’s mistake
After the results were in and tabulated, Cassini announced to the world that a degree of latitude between Paris and Dunkirk is longer than one between Paris and the Mediterranean. The Earth must be pointed toward the poles and flattened at the equator.
England’s Sir Isaac Newton, who in the meantime had been perfecting his theory of gravity, was sure that the French had it reversed. Using various pendulum devices, he demonstrated that the pull of gravity is less at the equator (the pendulum swung fractionally slower) than it is toward the poles. Newton postulated that the reason for this must be that the equator is farther from the center of the earth than the poles. In other words, the earth must be flattened at the poles and elongated at the equator, in which case the French were guilty of sloppy survey work.
National pride was at stake. The French Academy of Sciences decided to settle the dispute once and for all. Two expeditions were dispatched, one as far north as it was practical to go, which was to the Arctic Circle in Lapland on the northern shores of the Gulf of Bothnia, and the other as close to the equator as possible in Peru (in an area that is now part of Ecuador). These expeditions were instructed to measure the precise horizontal distance of a substantial north/south arc of latitude at their respective locations.
The Lapland expedition spent 17 months (April 1736 to August 1737) hacking its way through thick forests, working its way up and down mountainsides, establishing triangulation points, making astronomical observations, and braving hunger, wolves, the hordes of mosquitoes, and the incredibly harsh winter. When the expedition returned home, even without the results of the Peruvian expedition, it was known that Newton was correct.
It was to be another six years (1743) before the remnants of the Peruvian expedition limped back to Paris to finally, and incontrovertibly, corroborate the results. Voltaire, the French writer and noted wit, had the last word. "You have found by prolonged toil," he remarked, "what Newton found without even leaving his home!"
From sphere to ellipsoid
How to model this non-spherical world? This was more than an academic question. To make maps, national surveyors now universally used an astronomically determined starting point and a measured baseline, working away from the beginning point by the process of triangulation. This methodology was adopted because the complexity, expense, and time involved in obtaining precise astronomical fixes made it impractical to get these fixes on a regular basis.
As the surveyors progressed farther afield, if the mapped latitudes and longitudes were to be kept in sync with the occasional astronomical observations (i.e., "real-life" latitudes and longitudes) there had to be a model showing the relationship between the distance on the ground and latitude and longitude and indicating how this relationship changed as the surveyors moved away from their astronomically determined starting point. This model had to be such that with available trigonometrical and computational methods the map makers could make the necessary adjustments to their data to accurately calculate changing latitudes and longitudes over substantial distancesin other words, the model had to be mathematically predictable.
The model that was adopted, and which is used to this day even with satellite-based map-making and navigation, is an ellipsoid (also called a spheroid). In essence, an ellipsoid is nothing more than a flattened sphere. It is characterized by two measurements: its radius at the equator, and the degree of flattening at the poles. Clearly, the key questions become: what is this radius, and what is the degree of flattening?
During the nineteenth century the continents were first accurately mapped based upon this concept of the world as an ellipsoid. For each of the great surveys, preliminary work extending over years used astronomical observations and measured baselines to establish the key dimensions of the ellipsoid that was to underlie the survey. In the U.K. a geodesist (the people who do this kind of research) named Sir George Airy developed an ellipsoid (known as Airy 1830) that became the basis for an incredibly detailed survey of the British Isles. His ellipsoid is still utilized to this day.
Using this ellipsoid, the surveyors commenced at a precisely determined astronomical point on Salisbury Plain, measured a baseline, and triangulated their way across the British Isles. The accuracy of the survey work and the ellipsoid was such that when western Ireland was reached decades later, and the original base line was checked by computation from the Irish base line 350 miles away, the two values differed by only five inches.
Another British geodesist, Alexander Clarke, who developed an ellipsoid for mapping France and Africa, came to the U.S. and was instrumental in developing the ellipsoid that has underlain the mapping of North America. This is known as the Clarke 1866 ellipsoid, and it was the basis of map- and chart-making on the North American continent until the advent of satellite-derived ellipsoids.
Using the Clarke 1866 ellipsoid and commencing at a single astronomically derived point and a measured baseline at Meade’s Ranch, in Osborne County, Kan., the American surveyors from the U.S. Coast and Geodetic Survey (now the National Geodetic Survey) fanned out, establishing triangulation points and mapping the entire continent as they went. This combination of an underlying ellipsoid, a specific astronomically determined starting point, and measured baseline is known as a Geodetic Datum, and, in this case, is now known as the North American Datum of 1927 (NAD 27 for short). Such is the accuracy of the NAD 27 surveys and the correlation of the Clarke 1866 ellipsoid with the real world that, at the margins of the survey (the northeast and northwestthose areas farthest from the starting point), the discrepancies between mapped latitudes and longitudes and astronomically derived latitudes and longitudes are no more than 40 or 50 meters.
From ellipsoid to geoid
By the end of the 19th century there were numerous ellipsoids in use, all of them differing slightly from one another. Surely they couldn’t all be correct, or could they? The answer lies in a more sophisticated understanding of our planet.
The individual ellipsoids closely model the shape of the world in the areas in which the surveys were conducted, producing a correlation between mapped positions and astronomically derived positions even at the margins of the survey. But although these ellipsoids are based on very accurate measurements over large areas of land, these are still only small areas of the world. When extrapolated to the globe as a whole, these ellipsoids produce increasingly serious discrepancies between ellipsoid-derived latitudes and longitudes and astronomically derived positions. Geodesists realized that, not only is the world not a sphere, but it is also not an ellipsoid. It in fact does not have a geometrically uniform shape at all, but has numerous humps and hollows.
Another concept was needed to deal with this shape. It is the geoid. The geoid is defined as the real shape of the surface of the world if we discount all elevations above sea level. In other words, if we were to bulldoze the mountains and valleys down to sea level, we would end up with the geoid. This is, in effect, the two-dimensional world as surveyed by map makers, since the vertical element in the earth’s topography is discounted when measuring baselines and other distances; they are all painstakingly reduced to the horizontal, using sea level as the base elevation. Whereas an ellipsoid is a mathematically defined regular surface, the geoid is a very irregular (mathematically unpredictable) shape. Irrespective of the ellipsoid used to model the world, at times the surface of the geoid will be above that of the ellipsoid, and at times it will be below itthis is known as geoid undulation.
If we take two positions on an ellipsoid and define them in terms of latitude and longitude, the distance between them can be mathematically determined. But no such relationship holds with the geoid. If the geoid undulates above the ellipsoid, the horizontal distance between the two points will be greater than the corresponding distance on the ellipsoid; if the geoid undulates below the ellipsoid, the horizontal distance will be less.
Astronomically derived positions are real-life points on the surface of the Earth that have been determined in relation to observable celestial phenomena. As such they are referenced to the mathematically unpredictable geoid, as opposed to map-makers’ positions, which are mostly derived from some mathematical model (an ellipsoid) of the world. Because of the mathematically unpredictable nature of the geoid, there is no mathematical relationship between astronomically determined positions and positions determined by reference to an ellipsoid: the only way to correlate the two is through individual measurements or by modeling the geoid and ellipsoid and measuring the offsets.
What this means is that there can be no ellipsoid that produces a precise correlation between ellipsoid-derived latitudes and longitudes and those derived astronomically. Which is why we currently have more than 20 different ellipsoids in use around the world, each of which forms the basis for a different map datum, and none of which are compatible. In their own areas, these ellipsoids and datums create a "best fit" between latitudes and longitudes derived from the ellipsoid, and those derived astronomically (those referenced to the geoid). But if expanded to worldwide coverage, latitudes and longitudes based on these ellipsoids exhibit increasingly large discrepancies from those derived astronomically.
The age of satellites
Today all this has changed. Satellites and advanced technology (such as electro-optical distance measuring devices) have finally unified the globe from a surveyor’s perspective. In the past four decades an incredible mass of geodetic data has become available from all parts of the world. On the basis of this a succession of "unified" world datums, called World Geodetic Systems (WGS), have been developed. These have culminated in WGS 84, which now has worldwide acceptance.
WGS 84 is another ellipsoid. This one, however, has been developed as a best fit with the geoid as a whole, as opposed to having a best fit with just one specific region of the geoid. The ironic thing about this is that, given the irregularities in the geoid, the divergence between WGS 84 and the geoid is actually greater in many areas than the divergence between older ellipsoids and the geoid. For example, in North America the difference between the Clarke 1866 ellipsoid and the geoid is generally less than 10 meters, whereas with WGS 84 it is at least 15 meters and often 30 to 35 meters. However, on a worldwide scale, WGS 84 makes a better fit than Clarke 1866. But what this means is that the difference between map-derived and astronomically derived latitudes and longitudes is greater on a WGS 84-based map than it is on a NAD 27 map.
However, satellite navigation systems, first Transit (satnav) and now GPS and GLONASS (the Russian equivalent of GPS), have finally broken the umbilical cord that tied our map-making to the stars. We now can use satellite-based survey techniques that directly relate surveyed positions to the WGS 84 ellipsoid. But whereas astronomically determined latitudes and longitudes are "absolute" in the sense that every point on the globe has a fixed, unchanging astronomical latitude and longitude, ellipsoid-derived latitudes and longitudes are only absolute in relation to a particular ellipsoid, which makes them relative in relation to the geoid. A change in ellipsoidal assumptions will alter the latitude and longitude of real-life points on the globe.
At first sight, this would seem to make it impossible to have precise position fixes. But with a little more thought it will be seen that this relativity of ellipsoid-derived latitudes and longitudes is irrelevant as long as the equipment used to derive a latitude and longitude bases its calculations on the same ellipsoid as the map or chart on which the position is plotted. If the maps and charts are made to a particular set of assumptions, and the position-fixing equipment operates on the same set of assumptions, the results will be precise fixesin some cases, incredibly precise fixes, down to centimeter accuracy at a continental scale.
The rub comes if someone is navigating with electronic equipment that is not operating on the same set of assumptions as those used to make a given map or chart. If the GPS is using WGS 84 and the chart is based on Clarke 1866 (NAD 27) the resulting position error may be as high as 100 meters in the conterminous U.S.; if the chart is based on the U.K.’s Ordnance Survey datum it is around 100 meters; for charts using the 1950 European Datum (also used in the U.K.) it may be up to 150 meters; and in the case of the Tokyo datum charts, used in much of the Far East, the difference may be as high as 900 meters.
Nautical peculiarities
Finally, there are all those nautical surveys made without reference to any ellipsoid at all. Coastal surveys were traditionally done by setting up triangulation points on shore and continuing the land-based process of triangulation out to sea. But once the surveyors moved beyond the range of the shore-based triangulation system there was no way to tie the surveys into any ellipsoid or shore-based datum. The necessarily precise astronomical and baseline measurements simply could not be made from the moving platform of a ship.
For transoceanic surveys this inability to tie into a given ellipsoid or datum was immaterial, since precise position-fixing was not necessary. Mariners navigating the oceans could not, in any case, fix their position with any degree of precision using a sextant and other traditional means of celestial navigation.
Problems have always arisen, however, in relation to charts of remote islands, rocks and other navigational features. Unable to establish a relationship to any ellipsoid or chart datum, the surveyors had to establish a local astronomically determined position and then conduct a survey working away from this point using traditional methods of triangulation. Apart from the fact that the astronomically derived starting points are often seriously in error (sometimes by milesaccording to the British Admiralty, the worst discrepancy, which is in the South Pacific, is seven miles), there are frequent surveying errors on these older charts (imprecisely measured angles between features or poorly calculated distances, etc.). This resulted in charts that cannot be tied into any ellipsoid or datum and that are unreliable in relation to all forms of celestial navigation. Nevertheless, prior to the satellite age, once a landfall was made these surveys, too, were often adequate for mariners because navigation was by traditional methods using bearings from identified points of land, changes in depth, and so on, all of which are unrelated to latitudes and longitudes.
But with the advent of satellite-based navigation systems, mariners have been able to precisely locate themselves anywhere in the world in terms of latitude and longitude. What few of them realize is that this position is with reference to a particular ellipsoid (in the case of GPS, WGS 84) that may differ markedly from the underlying ellipsoid or datum of the chart that they are using. It serves little purpose, other than confusing the navigator and providing a false sense of confidence, to know exactly where you are (latitude and longitude) in relation to WGS 84 if the chart you are using is based on an ellipsoid and datum that result in the lines of latitude and longitude running through substantially different real-world locations. The half-mile difference we experienced in Cuba between our actual position and our plotted position was an example of the kind of differences that can arise when satellite-based navigation equipment is operating on a different datum than that used for the chart. Neither the GPS nor chart were "wrong"; in fact, the chart is a very good one. The two were simply referencing latitude and longitude to a different set of assumptions.
This is when blindly following GPS becomes potentially quite dangerous. Unless a GPS receiver is operating on the same datum as that underpinning the chart that is being used for navigation, a GPS fix may result in considerable navigational errors. If the GPS cannot be reset to match the chart datum, or the lines of latitude and longitude on the chart cannot be shifted to match the GPS datum, the GPS must be treated as an unreliable navigational tool.
My thanks to David Doyle, senior geodesist with the National Geodetic Survey, for a great deal of help and advice in preparing this article, and also to Messrs. Simpson and Shimmin at the British Admiralty.
Contributing editor Nigel Calder is the author of several books, including Cuba, A Cruising Guide, published by Imray, Laurie, Norie & Wilson in Britain.
