[2628-a2] feat(checkpoint): add createTime to memory checkpoint key#2637
[2628-a2] feat(checkpoint): add createTime to memory checkpoint key#2637Takuka0311 wants to merge 3 commits into
Conversation
The in-memory checkpoint key was (dev, inode, configName), which cannot distinguish different physical files that reuse the same inode (e.g. a same-name rebuild). Such checkpoints overwrote each other, so on recovery one file's state was lost and it was re-collected from the beginning. Add an int64_t createTime dimension to CheckPointKey and CheckPoint so these entries no longer collide. Comparison order is dev -> inode -> createTime -> configName. createTime defaults to 0, which preserves the legacy keying behavior; disk dump/load serialization and the actual createTime plumbing are handled in follow-up steps (A3/A4). Closes alibaba#2631 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
这个pr的改动是不是没有调用点,纯粹为后续的阶段搭建框架? |
|
你的观察是准确的:本 PR 在生产代码路径上确实没有会填入非零 生产调用点(全部走默认
为什么此 PR 不含真正的填值调用点
安全性 / 兼容性
综上,这是有意为之的增量:本 PR 只引入 key 维度与结构并以 UT 证明机制正确,wiring 按范围锁留给 A3 及后续阶段。若你更希望把 A2 与后续 wiring 合并为单一 PR、或调整阶段边界,请示下,我另行开范围处理(当前不在本 issue 允许改动集内)。 from=agent action=none |
There was a problem hiding this comment.
Pull request overview
Note
Copilot couldn't run its full agentic review because no GitHub Actions runner was available. Make sure your repository has a runner available to run Copilot's review, or add a copilot-setup-steps.yml file specifying one with the runs-on attribute. See the docs for more details.
This PR extends the in-memory checkpoint key to include createTime, preventing collisions when different physical files reuse the same (dev, inode) and ensuring checkpoints don’t overwrite each other in same-inode rebuild scenarios.
Changes:
- Add
mCreateTimetoCheckPointandCheckPointKey, and updateCheckPointKey::operator<to includecreateTimein ordering. - Update in-memory
Add/Get/Deleteto use the extended key with a defaultcreateTime = 0for backward compatibility. - Add unit tests covering same-inode/different-createTime storage and backward-compatible behavior.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
| core/file_server/checkpoint/CheckPointManager.h | Adds mCreateTime, extends key structure and public APIs with defaulted createTime. |
| core/file_server/checkpoint/CheckPointManager.cpp | Uses createTime when building the map key for Add/Get/Delete in memory. |
| core/unittest/checkpoint/CheckpointManagerUnittest.cpp | Adds UTs validating non-overwrite behavior and default-0 compatibility. |
Address PR review: introduce kUnsetCreateTime so the "createTime not assigned" value is explicit rather than a bare 0, document that the memory key is exact-match (not a wildcard), and use insert_or_assign in AddCheckPoint to replace the same key in a single lookup. Tests now hold owning unique_ptr and release() into AddCheckPoint. Refs alibaba#2631 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
背景 / Motivation
Epic #2629 · Discussion #2628 的 W1(checkpoint key 增加物理文件区分维度)内存层步骤。
内存 checkpoint 的 key 为
(dev, inode, configName),无法区分复用同一 inode 的不同物理文件(同名重建时同 inode)→ 互相覆盖 → 恢复时丢失一方状态、从头重采。本 PR 在内存层给 key 增加createTime维度,根治同 inode 覆盖。改动 / Changes
CheckPoint增加int64_t mCreateTime(默认 0),构造函数新增尾部默认形参createTime = 0。CheckPointKey增加mCreateTime,operator<比较顺序改为dev → inode → createTime → configName。AddCheckPoint从CheckPoint::mCreateTime取值构建 key;GetCheckPoint/DeleteCheckPoint新增createTime形参(默认 0)。createTime == 0时行为与旧逻辑完全一致,现有调用点(LogFileReader、EventDispatcher、CheckpointManagerV2)无需改动即可编译。范围锁 / Scope
仅改动(符合 Issue 范围锁):
core/file_server/checkpoint/CheckPointManager.hcore/file_server/checkpoint/CheckPointManager.cpp(仅内存 Add/Get/Delete 与 key/结构)core/unittest/checkpoint/CheckpointManagerUnittest.cpp(新增 UT)未涉及:磁盘 dump/load 的
create_time序列化(A3)、createTime 实际填充(A4)、LogFileReader.*/EventDispatcher.*/CheckpointManagerV2.*/FileSystemUtil.*。Test plan
新增 UT(
CheckpointManagerUnittest),本地实际跑通(Docker build image 增量编译 + 运行):TestAddCheckPointSameInodeDifferentCreateTime(dev,inode)、不同 createTime 两条 → map size==2,互不覆盖TestSameNameRebuildNoOverwriteTestGetDeleteCheckPointWithCreateTimeTestCreateTimeZeroBackwardCompatible运行结果:
(含原有
TestSearchFilePathByDevInodeInDirectory,4 个为本 PR 新增)构建/测试命令:以
loongcollector-build-linux:2.1.17镜像编译checkpoint_manager_unittest(devtoolset-9 + ASAN,-Werror通过),并直接运行该 UT 二进制。clang-format 18 已格式化。Closes #2631