Skip to content

Add selective index updates (n/m) to the heap table AM#23

Draft
gburd wants to merge 39 commits intomasterfrom
tepid
Draft

Add selective index updates (n/m) to the heap table AM#23
gburd wants to merge 39 commits intomasterfrom
tepid

Conversation

@gburd
Copy link
Copy Markdown
Owner

@gburd gburd commented Apr 6, 2026

Updates trigger n of m index updates rather than all, none, or only summarizing. Table AMs have the ability to influence this behavior by changing modified_idx_attrs.

@github-actions github-actions Bot force-pushed the master branch 20 times, most recently from 70bedd9 to 6ab967f Compare April 7, 2026 11:20
@gburd @github-actions
Rebase on upstream hourly, add AI/LLM PR review
c3a1172 
  - Hourly upstream sync from postgres/postgres (24x daily)
  - AI-powered PR reviews using AWS Bedrock Claude Sonnet 4.5
  - Multi-platform CI via existing Cirrus CI configuration
  - Cost tracking and comprehensive documentation

  Features:
  - Automatic issue creation on sync conflicts
  - PostgreSQL-specific code review prompts (C, SQL, docs, build)
  - Cost limits: $15/PR, $200/month
  - Inline PR comments with security/performance labels
  - Skip draft PRs to save costs

  Documentation:
  - .github/SETUP_SUMMARY.md - Quick setup overview
  - .github/QUICKSTART.md - 15-minute setup guide
  - .github/PRE_COMMIT_CHECKLIST.md - Verification checklist
  - .github/docs/ - Detailed guides for sync, AI review, Bedrock

  See .github/README.md for complete overview

Complete Phase 3: Windows builds + fix sync for CI/CD commits

Phase 3: Windows Dependency Build System
- Implement full build workflow (OpenSSL, zlib, libxml2)
- Smart caching by version hash (80% cost reduction)
- Dependency bundling with manifest generation
- Weekly auto-refresh + manual triggers
- PowerShell download helper script
- Comprehensive usage documentation

Sync Workflow Fix:
- Allow .github/ commits (CI/CD config) on master
- Detect and reject code commits outside .github/
- Merge upstream while preserving .github/ changes
- Create issues only for actual pristine violations

Documentation:
- Complete Windows build usage guide
- Update all status docs to 100% complete
- Phase 3 completion summary

All three CI/CD phases complete (100%):
✅ Hourly upstream sync with .github/ preservation
✅ AI-powered PR reviews via Bedrock Claude 4.5
✅ Windows dependency builds with smart caching

Cost: $40-60/month total
See .github/PHASE3_COMPLETE.md for details

Fix sync to allow 'dev setup' commits on master

The sync workflow was failing because the 'dev setup v19' commit
modifies files outside .github/. Updated workflows to recognize
commits with messages starting with 'dev setup' as allowed on master.

Changes:
- Detect 'dev setup' commits by message pattern (case-insensitive)
- Allow merge if commits are .github/ OR dev setup OR both
- Update merge messages to reflect preserved changes
- Document pristine master policy with examples

This allows personal development environment commits (IDE configs,
debugging tools, shell aliases, Nix configs, etc.) on master without
violating the pristine mirror policy.

Future dev environment updates should start with 'dev setup' in the
commit message to be automatically recognized and preserved.

See .github/docs/pristine-master-policy.md for complete policy
See .github/DEV_SETUP_FIX.md for fix summary

Optimize CI/CD costs by skipping builds for pristine commits

Add cost optimization to Windows dependency builds to avoid expensive
builds when only pristine commits are pushed (dev setup commits or
.github/ configuration changes).

Changes:
- Add check-changes job to detect pristine-only pushes
- Skip Windows builds when all commits are dev setup or .github/ only
- Add comprehensive cost optimization documentation
- Update README with cost savings (~40% reduction)

Expected savings: ~$3-5/month on Windows builds, ~$40-47/month total
through combined optimizations.

Manual dispatch and scheduled builds always run regardless.
@github-actions github-actions Bot force-pushed the master branch 4 times, most recently from dc10453 to e74507c Compare April 7, 2026 15:29
gburd added 5 commits April 7, 2026 12:09
@gburd
dev setup v27
0664676
@gburd
Add tests to cover a variety of heap HOT update behaviors
adf2a93 
This commit introduces test infrastructure for verifying Heap-Only Tuple
(HOT) update functionality in PostgreSQL. It provides a baseline for
demonstrating and validating HOT update behavior.

Regression tests:
- Basic HOT vs non-HOT update decisions
- All-or-none property for multiple indexes
- Partial indexes and predicate handling
- BRIN (summarizing) indexes allowing HOT updates
- TOAST column handling with HOT
- Unique constraints behavior
- Multi-column indexes
- Partitioned table HOT updates

Isolation tests:
- HOT chain formation and maintenance
- Concurrent HOT update scenarios
- Index scan behavior with HOT chains
@gburd
Identify modified indexed attributes in the executor on UPDATE
e3ebfe7 
Refactor executor update logic to determine which indexed columns have
actually changed during an UPDATE operation rather than leaving this up
to HeapDetermineColumnsInfo() in heap_update().

Applied patch v38-0002 with offsets (-16 lines in heapam.h, various
other files with 1-10 line offsets).

# Conflicts:
#	src/backend/access/heap/heapam.c
#	src/backend/access/heap/heapam_handler.c
#	src/include/access/heapam.h
#	src/include/access/tableam.h
@gburd
Use ii_IndexUnchanged for hint rather than expression checking
6a95241 
Replace expression-based index_unchanged_by_update() with per-UPDATE
hint mechanism using ExecSetIndexUnchanged() based on modified_idx_attrs.

Applied patch v20260316c-0009 with offsets.
@gburd
Replace TU_UpdateIndexes enum with per-index bitmapset tracking
a244142 
Replaces the three-value TU_UpdateIndexes enum with a cleaner
bitmapset-based approach using MODIFIED_IDX_ATTRS_ALL_IDX sentinel bit.

Major API changes:
- Removed TU_UpdateIndexes enum (TU_None, TU_All, TU_Summarizing)
- Changed simple_heap_update() signature to use Bitmapset **modified_idx_attrs
- Removed summarized_only parameter from HeapUpdateHotAllowable()
- Updated ExecSetIndexUnchanged() signature
- Replaced EIIT_ONLY_SUMMARIZING with EIIT_ALL_INDEXES

Applied patch v20260316c-0010 with manual conflict resolution in:
- heapam.c (13 hunks manually applied)
- nodeModifyTable.c (2 hunks manually applied)
- execReplication.c (1 hunk manually applied)
@github-actions github-actions Bot force-pushed the master branch 30 times, most recently from 8f7ca27 to 9c2f5e9 Compare April 13, 2026 19:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@gburd