Performance Log

[yaozhihang]

One feature reqeust: It is usually helpful to know the cumulative execution times of the major long-running tasks during the import/export process. It can help developer to identify performance bottlenecks. The performance log is potentially useful not only for PostgreSQL but for other database adapters. Without this high level performance log, we often have to spend a lot of time digging into database logs and/or asking our performance experts for help.

The following is a list of items:
• Total time to processGlobalObjects
• Number of calls to get sequence numbers
• Cumulative time to get sequence numbers
• Number of calls to batchExecute
• Cumulative time to batchExecute
• Cumulative time to convert geometry
• ...

For export
• Initial feature hierarchy query time
• Cumulative time to execute sql queries for each of the exporters: feature/geometry/implicit geometry/property/...
• Cumulative time to process result sets for the above queries
• Cumulative time to convert geometry
• ...

The performance log can use the existing logging framework that can be enabled/disabled by setting the log level.

[clausnagel]

I agree that having cumulative performance statistics would be useful. However, I don't think this should be part of the general logging infrastructure. The intended audience is developers investigating performance bottlenecks of database adapters rather than end users, and performance data has a different nature than operational logs.

I would prefer implementing this as an optional profiling mode enabled through an internal JVM system property or environment variable rather than exposing it on the public CLI surface. When enabled, the tool could collect the relevant statistics throughout the import/export process using a lightweight instrumentation utility that is a no-op when profiling is disabled, so it adds no overhead in normal operation, and produce a dedicated performance summary at the end of the run. Logging could be one possible output sink for this summary.

This keeps performance profiling independent of the normal logging infrastructure and avoids exposing it as a regular user-facing logging feature, while still making it easy to enable and extend when needed.

[BWibo]

I fully agree. Logging and performance metrics are two substantially different things and should not be mixed.
For the implementation of metrics I'd look into OpenTelemetry, a widly adopted open standard, that would allow the easy integration with Grafana/Prometheus/Loki/ stacks.