refactor: migrate SQL parser from arana-db to pingcap/tidb#1120
refactor: migrate SQL parser from arana-db to pingcap/tidb#1120yunfan24 wants to merge 15 commits into
Conversation
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #1120 +/- ##
==========================================
- Coverage 58.36% 54.79% -3.58%
==========================================
Files 282 308 +26
Lines 19405 20670 +1265
==========================================
Hits 11326 11326
- Misses 7102 8366 +1264
- Partials 977 978 +1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
Resolve the CI problems and add some unit-test cases. |
There was a problem hiding this comment.
Pull request overview
This PR migrates Seata-Go’s MySQL SQL parsing layer (used by AT-mode executors / undo-log builders) from github.com/arana-db/parser to the upstream TiDB parser (github.com/pingcap/tidb/pkg/parser), and updates call sites/tests accordingly. It also includes a broad Go toolchain / dependency update and various small refactors.
Changes:
- Replace Arana parser imports with TiDB parser imports across SQL parsing, executors, and undo-log builders; adapt AST API differences (e.g.,
PatternLikeExpr→PatternLikeOrIlikeExpr). - Add parameter-marker ordering assignment (
assignParamMarkerOrders) to support TiDB parser’s param marker order handling. - Update Go module/tooling and CI configs (Go version bump, golangci-lint config schema, dependency tidy) plus minor cleanups (error strings,
ioutilremoval, etc.).
Reviewed changes
Copilot reviewed 92 out of 93 changed files in this pull request and generated 11 comments.
Show a summary per file
| File | Description |
|---|---|
| pkg/util/reflectx/unmarkshaler.go | Minor reflection validation refactor in setter signature check. |
| pkg/util/reflectx/unexpoert_field.go | Pointer-kind handling updated in GetElemDataValue. |
| pkg/util/reflectx/reference_service.go | Pointer-kind handling updated for service reference derivation. |
| pkg/util/net/address_validator.go | Regex literals simplified using raw string literals. |
| pkg/util/convert/convert.go | Reflection-based assignment pointer-kind handling updated. |
| pkg/tm/transaction_executor.go | Error message punctuation normalized (tests updated accordingly). |
| pkg/tm/transaction_executor_test.go | Update expected error messages to match new strings. |
| pkg/tm/global_transaction.go | Error message punctuation normalized. |
| pkg/tm/global_transaction_test.go | Update expected error strings to match new strings. |
| pkg/saga/statemachine/store/db/statelog.go | Simplify nil/len check for serialized error bytes. |
| pkg/saga/statemachine/store/db/db.go | Add rows.Err() checks to SelectOne/SelectList. |
| pkg/saga/statemachine/statelang/parser/sub_state_machine_parser.go | Parser method call cleanups (GetString, task attr parsing). |
| pkg/saga/statemachine/statelang/parser/statemachine_parser.go | Simplify make(map...) initialization. |
| pkg/saga/statemachine/statelang/parser/statemachine_json_parser.go | Simplify boolean return helper. |
| pkg/saga/statemachine/statelang/parser/statemachine_config_parser.go | Simplify empty content checks for JSON/YAML config parsers. |
| pkg/saga/statemachine/process_ctrl/process_context.go | Simplify map lookups / deletes. |
| pkg/saga/statemachine/process_ctrl/handlers/service_task_state_handler.go | Simplify empty slice check for sub-instances. |
| pkg/saga/statemachine/process_ctrl/event_bus.go | Log formatting adjusted to avoid treating dynamic strings as format strings. |
| pkg/saga/statemachine/engine/strategy/status_decision.go | Simplify slice length check. |
| pkg/saga/statemachine/engine/pcext/state_router_impl.go | Use promoted AbstractTaskState on embedded service task impl. |
| pkg/saga/statemachine/engine/pcext/process_router.go | Simplify map nil/len check. |
| pkg/saga/statemachine/engine/pcext/process_handler.go | Remove redundant nil checks; simplify expression resolver variable. |
| pkg/saga/statemachine/engine/pcext/loop_task_utils.go | Use promoted AbstractTaskState on embedded service task impl. |
| pkg/saga/statemachine/engine/pcext/instruction.go | Replace errors.New(fmt.Sprintf(...)) with fmt.Errorf. |
| pkg/saga/statemachine/engine/invoker/local_invoker.go | Pointer-type detection updated when converting params. |
| pkg/saga/statemachine/engine/invoker/grpc_invoker.go | Replace errors.New(fmt.Sprintf(...)) with fmt.Errorf. |
| pkg/saga/statemachine/engine/invoker/grpc_invoker_test.go | Simplify slice empty checks in tests. |
| pkg/saga/statemachine/engine/invoker/func_invoker_test.go | Simplify slice empty checks in tests. |
| pkg/saga/statemachine/engine/exception/forward_invalid_exception_test.go | Update exception message field access in tests. |
| pkg/saga/statemachine/engine/exception/exception_test.go | Update exception message field access in tests. |
| pkg/saga/statemachine/engine/core/process_ctrl_statemachine_engine.go | Simplify nil checks; log call simplification; map iteration simplification. |
| pkg/rm/tcc/tcc_service.go | Use proxy methods for action metadata; map/type switch simplification; pointer-kind handling updated. |
| pkg/rm/tcc/tcc_resource.go | Use promoted proxy methods; call commit/rollback through resource wrapper. |
| pkg/rm/tcc/fence/store/db/dao/tcc_fence_db.go | Add rows.Err() check after scanning query results. |
| pkg/rm/rm_remoting.go | Normalize error message casing. |
| pkg/remoting/getty/session_manager.go | Pointer-kind handling updated for reflection-based XID access. |
| pkg/remoting/getty/readwriter.go | Refactor heartbeat message type handling to switch. |
| pkg/integration/grpc/grpc_transaction_interceptor.go | Simplify metadata slice length checks. |
| pkg/discovery/etcd3_test.go | Simplify logging call. |
| pkg/datasource/sql/xa/oracle_xa_connection.go | Normalize “oracle” error string casing. |
| pkg/datasource/sql/xa/mysql_xa_connection.go | Remove redundant break; simplify infinite loop syntax. |
| pkg/datasource/sql/xa_resource_manager.go | Refactor ticker loop; simplify error logging formatting. |
| pkg/datasource/sql/util/lockkey.go | Use fmt.Fprintf instead of fmt.Sprintf for builder writes. |
| pkg/datasource/sql/util/escape.go | Use strings.Contains instead of strings.Index. |
| pkg/datasource/sql/util/convert.go | Reflection-based assignment pointer-kind handling updated. |
| pkg/datasource/sql/undo/executor/utils.go | Use fmt.Fprintf for builder writes when emitting PK column names. |
| pkg/datasource/sql/undo/builder/mysql_update_undo_log_builder.go | Migrate parser imports to TiDB; simplify before-image checks. |
| pkg/datasource/sql/undo/builder/mysql_update_undo_log_builder_test.go | Update parser test driver import to TiDB. |
| pkg/datasource/sql/undo/builder/mysql_multi_update_undo_log_builder.go | Migrate parser imports to TiDB. |
| pkg/datasource/sql/undo/builder/mysql_multi_update_undo_log_builder_test.go | Update parser imports/test driver to TiDB. |
| pkg/datasource/sql/undo/builder/mysql_multi_undo_log_builder.go | Remove redundant break statements in switches. |
| pkg/datasource/sql/undo/builder/mysql_multi_undo_log_builder_test.go | Migrate AST/model imports to TiDB. |
| pkg/datasource/sql/undo/builder/mysql_multi_delete_undo_log_builder.go | Migrate parser imports to TiDB. |
| pkg/datasource/sql/undo/builder/mysql_insertonduplicate_update_undo_log_builder.go | Migrate AST imports to TiDB. |
| pkg/datasource/sql/undo/builder/mysql_insert_undo_log_builder.go | Migrate AST imports; normalize error messages; simplify nil/len checks. |
| pkg/datasource/sql/undo/builder/mysql_insert_undo_log_builder_test.go | Migrate parser imports/test driver to TiDB. |
| pkg/datasource/sql/undo/builder/mysql_delete_undo_log_builder.go | Migrate parser imports to TiDB. |
| pkg/datasource/sql/undo/builder/basic_undo_log_builder.go | Migrate parser/test driver; refactor AST traversal; use fmt.Fprintf for builders. |
| pkg/datasource/sql/undo/base/undo.go | Switch parser mysql import to TiDB; add rows.Err() check; simplify map lookup. |
| pkg/datasource/sql/types/image.go | Pointer-kind handling updated for actual value extraction. |
| pkg/datasource/sql/types/executor.go | Migrate AST/format imports to TiDB. |
| pkg/datasource/sql/types/executor_test.go | Migrate AST imports to TiDB. |
| pkg/datasource/sql/parser/parser_factory.go | Switch to TiDB parser; add param marker order assignment visitor. |
| pkg/datasource/sql/parser/parse_factory_test.go | Migrate parser imports/test driver to TiDB. |
| pkg/datasource/sql/exec/select_for_update_executor.go | Migrate parser imports; normalize error strings; use fmt.Fprintf. |
| pkg/datasource/sql/exec/at/update_join_executor.go | Migrate parser imports; normalize error strings. |
| pkg/datasource/sql/exec/at/update_executor.go | Migrate parser imports; normalize error strings. |
| pkg/datasource/sql/exec/at/select_for_update_executor.go | Migrate parser imports; rename conflict error var; normalize errors; use fmt.Fprintf. |
| pkg/datasource/sql/exec/at/select_for_update_executor_test.go | Adjust assertion expression. |
| pkg/datasource/sql/exec/at/multi_update_excutor.go | Migrate parser imports to TiDB. |
| pkg/datasource/sql/exec/at/multi_delete_executor.go | Migrate parser imports; normalize error strings; minor string conversion change. |
| pkg/datasource/sql/exec/at/insert_on_update_executor.go | Migrate AST imports to TiDB. |
| pkg/datasource/sql/exec/at/insert_executor.go | Migrate AST imports; normalize error strings; simplify nil/len checks. |
| pkg/datasource/sql/exec/at/insert_executor_test.go | Migrate parser imports/test driver to TiDB. |
| pkg/datasource/sql/exec/at/delete_executor.go | Migrate parser imports to TiDB. |
| pkg/datasource/sql/exec/at/base_executor.go | Migrate parser/model/test driver; adapt LIKE AST node type; refactor traversal logic; use fmt.Fprintf. |
| pkg/datasource/sql/driver.go | Fix error message spelling (“unsupported”). |
| pkg/datasource/sql/datasource/utils.go | Pointer-kind handling updated for DeepEqual. |
| pkg/datasource/sql/datasource/mysql/trigger.go | Minor comparison style change. |
| pkg/datasource/sql/conn_xa.go | Use c.targetConn directly when creating XA resource. |
| pkg/datasource/sql/conn_at.go | Simplify ExecResult handling in query wrapper logic. |
| pkg/compressor/gzip_compress.go | Replace deprecated ioutil.ReadAll with io.ReadAll. |
| pkg/compressor/deflate_compress_test.go | Simplify variable declaration. |
| pkg/compressor/bzip2_compress.go | Replace deprecated ioutil.ReadAll with io.ReadAll. |
| pkg/client/config.go | Replace deprecated ioutil.ReadFile with os.ReadFile. |
| go.sum | Update checksums to reflect dependency migration and tidy. |
| go.mod | Replace parser dependency; bump Go version; update various direct/indirect deps. |
| .golangci.yml | Migrate golangci-lint config to v2 schema; adjust enabled linters/formatters. |
| .github/workflows/unit-test.yml | Update Go version and GitHub Actions versions. |
| .github/workflows/license.yml | Update Go version used by license check job. |
| .github/workflows/integrate-test.yml | Update Go version used by integration tests. |
| .github/workflows/golangci-lint.yml | Update Go version and golangci-lint action/version. |
| .github/workflows/build.yml | Update Go version used by build job. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
https://github.com/pingcap/tidb/releases/tag/v8.5.6 为了项目长久发展,去当前连链接中去把源码down一下~,不要用go.mod的形式 需要在license和notice里面标注一下项目出处,参考seata-java |
https://github.com/pingcap/tidb/releases/tag/v8.5.6 For the long-term development of the project, go to the current link to down the source code ~, do not use the form go.mod Need to mark the project provenance in the license and notice, refer to seata-java |
…aser # Conflicts: # go.mod # pkg/datasource/sql/exec/at/base_executor.go # pkg/datasource/sql/exec/at/insert_executor_test.go # pkg/datasource/sql/undo/executor/utils.go
|
plz fix ci |
|
Among the 29 merged PRs in arana-db/parser, a large portion are Arana-specific features, such as sharding rule inspection, topology management, multi-tenant syntax, and DRDS partitioning. seata-go has zero references to these features, and they are unrelated to distributed transactions. |
|
@ssshr-66 @thunguo @Ethan-Xingyue ready for review and merge |
|
解决一下冲突 |
Resolve a conflict |
@thunguo done |
@ thunguo done |
What this PR does:
github.com/arana-db/parsertogithub.qkg1.top/pingcap/tidb/pkg/parserPatternLikeExpr→PatternLikeOrIlikeExprassignParamMarkerOrdersto handle parameter marker ordering required by the new parsergo mod tidyto resolve transitive dependenciesWhich issue(s) this PR fixes:
#1104
Special notes for your reviewer:
go.modversion bump (1.20 → 1.25)Does this PR introduce a user-facing change?: