Skip to content

Are the DML & DDL statements failing due concurrency conflicts retriable? #147

Description

@findinpath

While testing on https://github.com/trinodb/trino I have seen on CI several times this kind of exception:

Caused by: com.amazon.redshift.util.RedshiftException: ERROR: could not complete because of conflict with concurrent transaction
	at com.amazon.redshift.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2648)
...
	at io.trino.plugin.jdbc.BaseJdbcClient.execute(BaseJdbcClient.java:1629)
	at io.trino.plugin.jdbc.BaseJdbcClient.finishInsertTable(BaseJdbcClient.java:1195)
	... 27 more
	Suppressed: java.lang.RuntimeException: Query: INSERT INTO "***"."test_schema"."test_row_updateesa4qfftu2" ("nationkey", "name", "regionkey", "comment") SELECT "nationkey", "name", "regionkey", "comment" FROM "***"."test_schema"."tmp_trino_97a22eb1_bdfe6d29" temp_table WHERE EXISTS (SELECT 1 FROM "***"."test_schema"."tmp_trino_97a22eb1_5a157fbd" page_sink_table WHERE page_sink_table.trino_page_sink_id = temp_table.trino_page_sink_id)
		at io.trino.plugin.jdbc.BaseJdbcClient.execute(BaseJdbcClient.java:1632)
		... 28 more

While searching the web, I've stumbled as well on dbt-labs/dbt-core#2967 which basically reports a problem very similar to the one that I'm experiencing.

tldr; The Redshift exception "could not complete because of conflict with concurrent transaction" seems to happen (on the server-side) even in situations where multiple concurrent operations only do DML operations by selecting from the same source table.

My question here is applies rather generically - can such an exception as the one listed in the above stacktrace be considered "retriable" ?
If yes, I plan to apply retries for generic operations executed on Redshift through the trino-redshift connector.
I'm considering to add this kind of handling to be focused on DML operations.

Code taken from a chatgpt output (obviously to be taken with a grain of salt):

private boolean isRetriable(RedshiftException e) {
    String msg = e.getMessage();
    if (msg == null) return false;

    // Redshift serialization-conflict messages
    return msg.contains("conflict with concurrent transaction")
        || msg.contains("Serializable isolation violation")
        || msg.contains("Serialization error")
        || msg.contains("1023"); // Redshift error code for these conflicts
}

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions