Spencer/bugfix 171 docker pg credentials by spencertipping · Pull Request #172 · morphik-org/morphik-core

spencertipping · 2025-05-26T14:26:25Z

Fixes #171 by parsing credentials from POSTGRES_URI, making the check compatible with containerized deployments. This PR also adds some debugging statements and stderr logging to make it easier to track down connection issues.

I also pulled the files out of the dockerfile to make them easier to edit.

(My branch is misnamed: it should be bugfix-171, not bugfix-174.)

jazzberry-ai · 2025-05-26T14:26:28Z

Bug Report

Name	Severity	Example test case	Description
Incorrect Password Parsing	High	`postgresql://user:pass!@#word@host:5432/database`	The script incorrectly parses passwords containing special characters, leading to authentication failures.
Incorrect Port Parsing	Medium	`postgresql://user:password@host/database`	The script sets the PORT variable to the HOST when the port is missing in the URI.
Incorrect Database Parsing	Medium	`postgresql://user:password@host:5432`	The script sets the DB variable to the HOST:PORT when the database is missing in the URI.

_{Comments? Email us.}

CLAassistant · 2025-05-26T14:26:31Z

All committers have signed the CLA.

…GRES_URI

jazzberry-ai · 2025-05-26T14:27:34Z

Bug Report

Name	Severity	Example test case	Description
Incorrect PostgreSQL port parsing	High	`POSTGRES_URI=postgresql://user:password@host/database`	When the `POSTGRES_URI` does not contain a port, the `docker-entrypoint.sh` script incorrectly assigns the hostname to the `PORT` variable. This causes connection errors when trying to connect to the database.

_{Comments? Email us.}

jazzberry-ai · 2025-05-26T14:34:31Z

Bug Report

Name	Severity	Example test case	Description
Incorrect parsing of POSTGRES_URI	Critical	POSTGRES_URI="postgresql://user:password@host:5432/db"	The `awk` script in `docker/docker-entrypoint.sh` does not correctly parse the username, password, and host from the `POSTGRES_URI` environment variable. This can lead to authentication failures and prevent the application from connecting to the database.

_{Comments? Email us.}

…lleged failure cases

jazzberry-ai · 2025-05-26T14:44:59Z

Bug Report

Name	Severity	Example test case	Description
Incorrect PostgreSQL URI Parsing	Critical	`postgresql://user:password@host:5432/database`	The `parse-postgres-uri.sh` script fails to correctly parse connection parameters from the `POSTGRES_URI`.
Query Parameter Handling Issue	Critical	`postgresql://user:password@host:5432/database?sslmode=require`	The `parse-postgres-uri.sh` script fails to correctly parse connection parameters when URI contains query parameters.
Special Character Handling	Critical	`postgresql://user%40test:password@host:5432/database`	The `parse-postgres-uri.sh` script fails to correctly parse usernames containing special characters.

_{Comments? Email us.}

jazzberry-ai · 2025-05-26T14:54:48Z

Bug Report

Name	Severity	Example test case	Description
PostgreSQL URI parsing failure with special characters	Critical	`postgresql://user%40example:pass%23word@host:port/db`	The script `parse-postgres-uri.sh` fails to correctly parse the username and password when they contain URL-encoded special characters (e.g., `%40` for `@`, `%23` for `#`). This prevents users from using passwords with common special characters.

_{Comments? Email us.}

spencertipping · 2025-05-26T15:03:12Z

@jazzberry-ai that is a lie.

…ed or corrupt URIs

jazzberry-ai · 2025-05-26T15:29:28Z

Bug Report

Name: URI escape injection
Severity: Critical
Example test case: postgresql://user:pass@host:port/db%0Arm -rf /
Description: The script decodes URI escapes without validating the decoded characters. This could allow an attacker to inject arbitrary commands into the environment variables by using the newline character (%0A).

_{Comments? Email us.}

…alhost:5432/morphik", which happens during docker build

jazzberry-ai · 2025-05-26T15:41:35Z

Bug Report

Name	Severity	Example test case	Description
Command Injection in URI Parsing	Critical	`POSTGRES_URI="postgresql://user:pass\`rm -rf /tmp/*`@host:port/db"`	The `parse-postgres-uri.sh` script uses `awk` and `eval` to parse the PostgreSQL URI. A malicious URI can inject arbitrary commands, leading to potential data loss or system compromise. The fix is to sanitize the URI components before using them in `eval`. For example, escape shell-sensitive characters in the `user`, `pass`, `host`, `port`, and `db` variables.

_{Comments? Email us.}

… has no strtonum function

jazzberry-ai · 2025-05-26T15:46:41Z

Bug Report

Name	Severity	Example test case	Description
Incorrectly parses postgres URI with encoded characters	Critical	`source /root/morphik-core/docker/parse-postgres-uri.sh "postgresql://user%40example:pass%25word@host%2Fname:5432/db%2Bname"`	The script incorrectly parses postgres URIs that contain encoded characters, such as `%40`, `%25`, and `%2F`. This can lead to the application connecting to the wrong database, or failing to connect to the database at all. This can have serious security implications, as it can allow unauthorized access to sensitive data.

_{Comments? Email us.}

jazzberry-ai · 2025-05-26T16:01:39Z

Bug Report

Name	Severity	Example test case	Description
Potentially unavailable perl dependency	Medium	Run the docker image on a system without Perl installed.	The `parse-postgres-uri.sh` script relies on `perl` for URL decoding, which might not be available in all environments.
Incorrect default values in case of parse failure	Low	Set POSTGRES_URI to an invalid value and check that the application exits with an error.	The `parse-postgres-uri.sh` script defines default values for the postgres variables, but the `docker-entrypoint.sh` script continues execution with the default values when parsing fails. The application should exit with an error in this case.
Potential npm package version incompatibility	Low	Check the changelog for @morphik/ui between versions 0.2.22 and 0.2.24 for any breaking changes or known bugs.	The npm package version update might introduce new bugs or break compatibility with existing code.

_{Comments? Email us.}

jazzberry-ai · 2025-05-26T16:52:41Z

Bug Report

Name	Severity	Example test case	Description
Incorrect parsing of query parameters	High	export POSTGRES_URI='postgresql://user:pass@host:port/db?password=secret' && /root/morphik-core/docker/parse-postgres-uri.py "$POSTGRES_URI"	The script doesn't correctly parse the password if it's passed as a query parameter in the URI. This can lead to the password not being set and could expose the password in logs or environment variables.

_{Comments? Email us.}

jazzberry-ai · 2025-05-26T16:57:37Z

Bug Report

Name	Severity	Example test case	Description
Missing default values for HOST and PORT	Medium	Run the Docker container without setting POSTGRES_URI.	If `POSTGRES_URI` is not defined, the `HOST` and `PORT` variables will be empty, causing issues when starting the application.
Incorrect regex for parsing postgres URI	High	Use an invalid URI like `postgresql://:password@host:5432/db`	The regex pattern in `parse-postgres-uri.py` doesn't correctly validate the URI format, allowing invalid URIs to be parsed, which can lead to connection errors.

_{Comments? Email us.}

spencertipping · 2025-05-26T18:35:03Z

Not ready to merge yet; I'm fixing up some integrations with the docker builder action.

jazzberry-ai · 2025-05-26T18:39:17Z

Bug Report

| Name | Severity | Example test case | Description |\n|---|---|---|---|\n| Shell Injection in Postgres URI | Critical | postgresql://\touch /tmp/shell_injection`@localhost/db` | Backticks are not properly escaped in the username/password, leading to arbitrary shell command execution. |\n

_{Comments? Email us.}

spencertipping · 2025-05-26T18:40:01Z

OK, now everything should work (see https://github.com/ribera-ai/morphik-core/tree/feat/issue-158-publish-docker-image)

jazzberry-ai · 2025-05-26T18:56:40Z

Bug Report

Name	Severity	Example test case	Description
Potential Race Condition during Docker Build	Medium	Run the docker build workflow.	The application container may start before the database and Redis are fully initialized, leading to intermittent test failures. The /ping endpoint may become available before the app is fully ready. This is a race condition.

_{Comments? Email us.}

jazzberry-ai · 2025-05-26T22:21:55Z

Bug Report

Name	Severity	Example test case	Description
Shell Injection Vulnerability in POSTGRES_URI Parsing	Critical	`postgresql://user:'pass; rm -rf /tmp/*'@host:5432/db`	The `parse-postgres-uri.py` script does not properly escape shell metacharacters in the parsed URI components, allowing for arbitrary shell commands to be executed via the `eval` command in `docker-entrypoint.sh`.

_{Comments? Email us.}

… earlier

jazzberry-ai · 2025-05-26T22:35:22Z

Bug Report

Name	Severity	Example test case	Description
PostgreSQL URI Parsing Failure Leads to Incorrect Uvicorn Binding	Medium	Set POSTGRES_URI to an invalid value (e.g., 'malformed_uri').	When the PostgreSQL URI parsing fails, the PG_HOST and PG_PORT variables are not set. The docker-entrypoint.sh script then uses these unset variables for the uvicorn command, causing uvicorn to bind to localhost:8000 instead of the intended 0.0.0.0:8000, making the application inaccessible from outside the container.

_{Comments? Email us.}

spencertipping and others added 3 commits May 26, 2025 09:27

Pulling scripts into separate files, parsing PG credentials from POST…

c947271

…GRES_URI

Preserving stderr to help debug connection issues

fa9c481

update npm package version

0d42518

spencertipping force-pushed the spencer/bugfix-174-docker-pg-credentials branch from fc26848 to 0d42518 Compare May 26, 2025 14:27

Trying awk parse to fix edge cases

2555066

Split postgres parser into its own script so I can test jazzberry's a…

63c4595

…lleged failure cases

More edge case fixes

71b663f

Postgres need not use a password, especially when dealing with obscur…

bbb7272

…ed or corrupt URIs

Allowing postgres URI parse failures to accommodate the value "***loc…

38fd569

…alhost:5432/morphik", which happens during docker build

Modified script to work with github awk version, which is non-GNU and…

efba09c

… has no strtonum function

You're killing me github

c3d9933

OK, doing everything in python since we know the container has that

52a3654

Log PG values only on failure

a60bbe7

spencertipping added 3 commits May 26, 2025 13:38

Spinning up DB container during test

a9ce671

Using async PG driver

25fa23e

Trying longer timeout

6bc2eec

spencertipping added 3 commits May 26, 2025 13:39

Added redis container

dffb99a

Fixed redis container name

145e66c

Fixed postgres credentials to not collide with primary bind addr/port

24aaef1

Using test toml file instead of repo-default

2f716ad

Create test file if it doesn't exist (avoids mounting an empty dir)

1077fe9

Properly locating the test file, which must exist since we created it…

67b35cc

… earlier

Adityav369 closed this Jun 15, 2025

Uh oh!

Conversation

spencertipping commented May 26, 2025

Uh oh!

jazzberry-ai Bot commented May 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Bug Report

Uh oh!

CLAassistant commented May 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jazzberry-ai Bot commented May 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Bug Report

Uh oh!

jazzberry-ai Bot commented May 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Bug Report

Uh oh!

jazzberry-ai Bot commented May 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Bug Report

Uh oh!

jazzberry-ai Bot commented May 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Bug Report

Uh oh!

spencertipping commented May 26, 2025

Uh oh!

jazzberry-ai Bot commented May 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Bug Report

Uh oh!

jazzberry-ai Bot commented May 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Bug Report

Uh oh!

jazzberry-ai Bot commented May 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Bug Report

Uh oh!

jazzberry-ai Bot commented May 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Bug Report

Uh oh!

jazzberry-ai Bot commented May 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Bug Report

Uh oh!

jazzberry-ai Bot commented May 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Bug Report

Uh oh!

spencertipping commented May 26, 2025

Uh oh!

jazzberry-ai Bot commented May 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Bug Report

Uh oh!

spencertipping commented May 26, 2025

Uh oh!

jazzberry-ai Bot commented May 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Bug Report

Uh oh!

jazzberry-ai Bot commented May 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Bug Report

Uh oh!

jazzberry-ai Bot commented May 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Bug Report

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

jazzberry-ai Bot commented May 26, 2025 •

edited

Loading

CLAassistant commented May 26, 2025 •

edited

Loading

jazzberry-ai Bot commented May 26, 2025 •

edited

Loading

jazzberry-ai Bot commented May 26, 2025 •

edited

Loading

jazzberry-ai Bot commented May 26, 2025 •

edited

Loading

jazzberry-ai Bot commented May 26, 2025 •

edited

Loading

jazzberry-ai Bot commented May 26, 2025 •

edited

Loading

jazzberry-ai Bot commented May 26, 2025 •

edited

Loading

jazzberry-ai Bot commented May 26, 2025 •

edited

Loading

jazzberry-ai Bot commented May 26, 2025 •

edited

Loading

jazzberry-ai Bot commented May 26, 2025 •

edited

Loading

jazzberry-ai Bot commented May 26, 2025 •

edited

Loading

jazzberry-ai Bot commented May 26, 2025 •

edited

Loading

jazzberry-ai Bot commented May 26, 2025 •

edited

Loading

jazzberry-ai Bot commented May 26, 2025 •

edited

Loading

jazzberry-ai Bot commented May 26, 2025 •

edited

Loading