The table and equation are based on a spherical earth, especially when you look at the derivation of the equation.

The horizon distance (H) on a globe with radius (R) and observer height (X) can be accurately calculated with the right triangle rule. However this isn't easy to work out in your head and it requires both the observer height and radius to use the same units.

H = [(R+X)^2 - R^2]^1/2

If we factor out the exponent

= [R^2 + 2*R*X + X^2 - R^2] ^1/2

The R^2 and -R^2 cancel each other out

= [2*R*X + X^2]^1/2

Pull the X out

= [(2*R + X) * X]^1/2

Which can be rewritten as

= (2*R + X)^1/2 * X^1/2

The format of this looks pretty similar to the equation posted by Z.W. Wolf. In the expression (2*R + X), we know that R will be much much bigger than X, therefore lets assume X is 0 for this section. We then get...

H = (2*R + 0)^1/2 * X^1/2

= (2*R)^1/2 * X^1/2

Remember, this equation requires both X and R to have the same units, but the equation used by the table has an input of ft and output of Nmi. Lets say both R and X are in feet which will give the horizon distance in feet, so lets add a conversion to change the horizon distance to Nmi.

H = ((2*R)^1/2 * X^1/2) ft

= ((2*R)^1/2 * X^1/2) ft * (1Nmi / 6076ft)

= ((2*R)^1/2) / 6076 * X^1/2 Nmi

Since R is a constant, the first half of the equation will not change based on observer height. Now we have an equation that allows us to input the observer height X in ft, and we'll get out the Horizon distance in Nmi and we can do the calculation pretty easily in our head. If we use the 1.17 from the equation posted by Z.W. Wolf, we can calculate the value of R (in ft) used.

1.17 = ((2*R)^1/2) / 6076

R = [(1.17 * 6076)^ 2] / 2

R = 25268372 ft

Convert this to miles.

R = 4785.7 miles

The coast guard table is based on a sphere with radius 4786 miles. The standard refraction radius 7/6*R is 4618 miles. So the coast guard either assumes light bends even more than the standard refraction model (approx 1.2 instead of 1.16) or that the earth is bigger than conventionally thought. Either way, it does nothing to prove that the earth is flat.