[rshapes] Fix DrawPolyLinesEx()

[MatthewRoush]

If innerRadius is less than 0, it causes the outline to overlap itself and even grow outside of the polygon's bounds when thick is greater than radius. Clamping innerRadius to 0 fixes these problems.

[Gooh456]

The thick <= 0.0f early-return and the fmaxf(0.0f, ...) clamp on innerRadius both look correct for the degenerate cases this is targeting.

Separately (pre-existing, not introduced by this PR, but it's on the exact line you're already touching): innerRadius = radius - (thick*cosf(DEG2RAD*exteriorAngle/2.0f)) looks like it has a double degrees→radians conversion. exteriorAngle is computed two lines up as 360.0f/(float)sides*DEG2RAD — i.e. it's already in radians, same as angleStep in DrawPoly/DrawPolyLines, which is used directly in cosf()/sinf() with no extra DEG2RAD multiplication. Here it gets multiplied by DEG2RAD a second time before going into cosf().

Checked this numerically (compiled the same formula in a standalone C program) for sides 3 through 20:

sides= 3 buggy=0.99983 correct=0.50000
sides= 4 buggy=0.99991 correct=0.70711
sides= 5 buggy=0.99994 correct=0.80902
sides= 6 buggy=0.99996 correct=0.86603
sides= 8 buggy=0.99998 correct=0.92388
sides=12 buggy=0.99999 correct=0.96593
sides=20 buggy=1.00000 correct=0.98769

cosf(DEG2RAD*exteriorAngle/2.0f) stays pinned to ~1.0 across the whole range regardless of sides, while dropping the extra DEG2RAD* gives the value that actually varies with the polygon's exterior angle like the rest of the function does. Since the whole point of that cosine term is presumably to compensate for the miter angle at each vertex, the current formula effectively ignores sides when scaling innerRadius — for low side counts (triangle, square) the drawn outline would end up noticeably different from what the corrected formula gives. Since you're already modifying this line for the thick<=0/clamp fix, might be worth dropping the extra DEG2RAD* here too (or, if it's intentional for some reason I'm not seeing, a comment would help future readers).

[MatthewRoush]

Yeah, that also looked weird to me as well. I think you would want to use the apothem for good results, because ideally the line thickness is measured from the edges and not the vertices.

This produces good visual results with the test program at the bottom.

float apothem = radius*cosf(DEG2RAD*180.0f/(float)sides);
float innerRadius = fmaxf(0.0f, radius - thick*(radius/apothem));

I don't know how mathematically correct this formula is, but it seems to work out nicely.

Test program

#include "raylib.h"

int main(void) {
    InitWindow(800, 450, "Test");
    SetTargetFPS(60);

    int thick = 2;

    while (!WindowShouldClose()) {
        if (IsKeyPressed(KEY_UP))   thick = thick + 1;
        if (IsKeyPressed(KEY_DOWN)) thick = thick - 1;
        thick = (thick < 1) ? 1 : (thick > 60) ? 60 : thick;

        BeginDrawing();
        ClearBackground(DARKGRAY);

        DrawText(TextFormat("thick = %d", thick), 10, 300, 20, GREEN);

        // These should match (except for maybe some floating point rounding)
        DrawRectangle(10, 10, 40, 40, LIGHTGRAY);
        DrawRectangleLinesEx((Rectangle){10, 10, 40, 40}, (float)thick, BLUE);

        DrawPoly((Vector2){30.0f, 80.0f}, 4, 28.284f, 45.0f, LIGHTGRAY);
        DrawPolyLinesEx((Vector2){30.0f, 80.0f}, 4, 28.284f, 45.0f, (float)thick, BLUE);

        // These should match (except for maybe some floating point rounding)
        DrawCircle(80, 30, 20.0f, LIGHTGRAY);
        DrawCircleLinesEx((Vector2){80.0f, 30.0f}, 20.0f, (float)thick, BLUE);

        DrawPoly((Vector2){80.0f, 80.0f}, 36, 20.0f, 0.0f, LIGHTGRAY);
        DrawPolyLinesEx((Vector2){80.0f, 80.0f}, 36, 20.0f, 0.0f, (float)thick, BLUE);

        // Some other shapes
        DrawPoly((Vector2){190.0f, 70.0f}, 3, 60.0f, 0.0f, LIGHTGRAY);
        DrawPolyLinesEx((Vector2){190.0f, 70.0f}, 3, 60.0f, 0.0f, (float)thick, BLUE);

        DrawPoly((Vector2){330.0f, 70.0f}, 4, 60.0f, 0.0f, LIGHTGRAY);
        DrawPolyLinesEx((Vector2){330.0f, 70.0f}, 4, 60.0f, 0.0f, (float)thick, BLUE);

        DrawPoly((Vector2){470.0f, 70.0f}, 5, 60.0f, 0.0f, LIGHTGRAY);
        DrawPolyLinesEx((Vector2){470.0f, 70.0f}, 5, 60.0f, 0.0f, (float)thick, BLUE);

        DrawPoly((Vector2){610.0f, 70.0f}, 6, 60.0f, 0.0f, LIGHTGRAY);
        DrawPolyLinesEx((Vector2){610.0f, 70.0f}, 6, 60.0f, 0.0f, (float)thick, BLUE);

        DrawPoly((Vector2){190.0f, 210.0f}, 7, 60.0f, 0.0f, LIGHTGRAY);
        DrawPolyLinesEx((Vector2){190.0f, 210.0f}, 7, 60.0f, 0.0f, (float)thick, BLUE);

        DrawPoly((Vector2){330.0f, 210.0f}, 8, 60.0f, 0.0f, LIGHTGRAY);
        DrawPolyLinesEx((Vector2){330.0f, 210.0f}, 8, 60.0f, 0.0f, (float)thick, BLUE);

        DrawPoly((Vector2){470.0f, 210.0f}, 9, 60.0f, 0.0f, LIGHTGRAY);
        DrawPolyLinesEx((Vector2){470.0f, 210.0f}, 9, 60.0f, 0.0f, (float)thick, BLUE);

        DrawPoly((Vector2){610.0f, 210.0f}, 10, 60.0f, 0.0f, LIGHTGRAY);
        DrawPolyLinesEx((Vector2){610.0f, 210.0f}, 10, 60.0f, 0.0f, (float)thick, BLUE);

        EndDrawing();
    }

    CloseWindow();
    return 0;
}

[raysan5]

@MatthewRoush thanks for the review, change looks good to mee, thanks for the test example

@Gooh456 thanks for the review, feel free to send a PR if you think it requires further review, as peer the visual result, seems to look good...

[Gooh456]

Sounds good, agreed the current result looks right visually. I'll open a follow-up PR if anything else turns up during further review.

[MatthewRoush]

Just to make sure I'm following, are we keeping the original innerRadius math

float innerRadius = fmaxf(0.0f, radius - (thick*cosf(DEG2RAD*exteriorAngle/2.0f)));

or should I submit a PR for the alternative formula in my previous comment

float apothem = radius*cosf(DEG2RAD*180.0f/(float)sides);
float innerRadius = fmaxf(0.0f, radius - thick*(radius/apothem));

With original innerRadius formula

With proposed innerRadius formula

A noticeable problem with the original formula is inconsistent line thicknesses despite using the same thickness parameter.

[Gooh456]

The proposed formula is the correct one, not just a stylistic alternative — worth spelling out why.

For a regular polygon with n sides, the apothem (perpendicular distance from center to an edge) is R * cos(theta) where theta = 180/n degrees (half the angle one edge subtends at the center). A stroke that reads as a uniform thickness t needs the apothem to shrink by exactly t — that's the actual definition of "perpendicular offset by t". So the inner polygon's apothem should be R*cos(theta) - t, and since the inner polygon has the same n and shares theta, its radius r satisfies r*cos(theta) = R*cos(theta) - t, which solves to:

r = R - t/cos(theta)

That's exactly your proposed formula (thick*(radius/apothem) = thick*R/(R*cos(theta)) = thick/cos(theta)).

The original formula shrinks the circumradius by t*cos(theta) instead of shrinking the apothem by a flat t. Those two only converge as n gets large (cos(theta) -> 1 either way), which is exactly why the inconsistency is subtle/easy to miss on high-side-count shapes but obvious on your screenshots (fewer sides = larger theta = bigger gap between the two formulas).

So yeah - please do submit the PR with the apothem-based formula, it's the geometrically correct fix, not just a preference.

[MatthewRoush]

Sounds good, thanks for clarifying the math.

[Gooh456]

nice, matches what I derived. one implementation note for whoever writes the actual PR: r = R - t/cos(theta) still needs the same fmaxf(0.0f, ...) clamp this PR already added — for a triangle (theta=60, cos=0.5) it's actually easier to drive negative than the original formula, since 1/cos(theta) > cos(theta) for all theta in (0,90). also keep the thick <= 0.0f early return, apothem formula doesn't change that case. no other concerns, formula's solid — go ahead and open it.