public static double getSqSegDist(double px, double py, double p1x, double p1y, double p2x, double p2y) {

    double x = p1x,
      y = p1y,
      dx = p2x - x,
      dy = p2y - y;

    if (dx != 0d || dy != 0d) {

      double t = ((px - x) * dx + (py - y) * dy) / (dx * dx + dy * dy);

      if (t > 1) {
        x = p2x;
        y = p2y;

      } else if (t > 0) {
        x += dx * t;
        y += dy * t;
      }
    }

    dx = px - x;
    dy = py - y;

    return dx * dx + dy * dy;
  }
Returns the square of the number of units that (px, p1) is away from the line segment from (p1x, py1) to (p2x,
p2y).
planetiler-core/src/main/java/com/onthegomap/planetiler/geo/DouglasPeuckerSimplifier.java