Work log since September 2023 (joshlf)

[joshlf]

Major features shipped

• Everything in 0.8
  • Data-ful enums
  • Dynamically-sized types (via KnownLayout)
  • Fallible conversions (via TryFromBytes and try_ macros)
  • UnsafeCells + types with interior mutability (including atomics, Cell)
  • Richer error reporting; ability to prove at compile time that certain errors are impossible
  • Support for external implementations of ByteSlice, SplitByteSlice
• SplitAt, supporting splitting arbitrary slice DSTs
• ByteEq, ByteHash (optimized implementations of stdlib traits)
• Improvements in internal robustness (refactors, Kani) to increase confidence in correctness/soundness
• Support for new SIMD types
• Slice DST transmutes, including metadata fix-up (supporting sized/sized, sized/unsized, and unsized/unsized transmutes)
• Shrinking value transmutes
• Lower MSRV to 1.56 (enabling dependencies from important crates)
• Support for Option<T> where T is subject to NPO (e.g. see FromZeros impls for Option types)
• Better, more helpful error messages
• Various improvements in derives to support broader range of types
• core::ops implementations on byteorder types
• include_bytes! macro

Highlights

• KnownLayout::validate_cast #349
• Initial commit of Ptr type #406
• Teach transmute_{ref,mut}! to handle slice DSTs #2428
• [macro_util] Add trailing_field_offset! macro #540
• Implement TryFromBytes for UnsafeCell #912
• [pointer] Support generic TransmuteFrom framework #2408
• Generalize SizeEq to SizeFrom, support shrinking #2580
• Add round_down_to_next_multiple_of_alignment #393
  • Example of use of Kani
• [pointer] Improve soundness of invariant modeling #2397
• Impl traits for TransparentWrapper types safely #1001
• Implement TransparentWrapper for atomic types  #1011
• TransparentWrapper supports UnsafeCell variance #1024
• [cargo.sh] Initial commit #363
• Support unsized try_transmute_{ref,mut}! #2944
• Support sized-to-unsized transmute_{ref,mut}! #2943
• [layout] Support sized->unsized, sized->sized casts #2950
• [derive] Allow TryFromBytes on non-Immutable unions #2876
• Use ReadOnly in TryFromBytes::is_bit_valid #2873
• [project] Support projecting enum tags #2940
• Introduce ReadOnly<T> which is unconditionally Immutable #2866
• [transmute] Encode cast in TryTransmuteFromPtr params #2921
• [transmute] Remove SizeEq super-trait bound #2920
• Make SizeEq safe #2907
• [derive] Implement #[zerocopy(on_error = skip/fail)] #2982
• [pointer] Add generic projection/cast framework #2860

Feature Work

• Add KnownLayout trait #347
• KnownLayout::validate_cast #349
• [byteorder] Implement core::ops traits #434
• Add transmute_ref! macro #183
• Initial commit of Ptr type #406
• Add include_bytes! macro #35
• [derive] Support KnownLayout for sized types #514
• Lower MSRV to 1.60 #561
• [simd] Add support for new x86[_64] types #571
• [macro_util] Add trailing_field_offset! macro #540
• [doc] Use doc(cfg(...)) in docs.rs #579
• Implement KnownLayout, FromZeroes for raw pointers #584
• FromZeroes for Option<T> where T is subject to NPO #595
• Implement FromZeroes for Option<[extern "C"] fn()> #600
• Add TryFromBytes trait #641
• Add T: TryFromBytes where T: FromBytes #644
• Add TryFromBytes::try_from_ref, impl for bool #647
• Implement TryFromBytes for some external types  #648
• Initial commit of NoCell #656
• [derive] Support deriving NoCell #667
• Traits other than NoCell permit UnsafeCells #682
• [derive] Support TryFromBytes on field-less enums #803
• Add TryFromBytes::try_read_from #808
• [derive] TryFromBytes on repr(C) enums #806
• Lower our MSRV to 1.59.0 #791
• [byteorder] Implement TryFromBytes #824
• [byteorder] Make some methods const #851
• Implement TryFromBytes for more types #817
• Lower MSRV to 1.56 #855
• [derive] Support IntoBytes on some generic structs #862
• Relax type bounds in Ptr::from_maybe_uninit_ref #900
• Add TryFromBytes::try_from_mut #892
• Implement <Unalign as TryFromBytes>::is_bit_valid #903
• Implement TryFromBytes for Option<F> for fn types #904
• [ptr] Relax some method bounds #902
• [derive] Automatically derive super-traits #926
• [derive] Support deriving deprecated AsBytes #938
• Implement TryFromBytes for UnsafeCell #912
• Make TryFromBytes a super-trait of FromZeros #952
• Implement Copy and Clone for Ref #989
• Split ByteSlice and SplitByteSlice #991
• Implement traits for atomic types #1028
• Add non-slice Ref constructors w/ slice elem count #996
• [pointer] Support as_bytes on unsized types #1180
• [pointer] Relax UnsafeCell requirements #1211
• Remove Immutable where it's no longer needed #1225
• [ref] Change some Ref constructor names #1239
• Rename FromBytes methods using with_elems suffix #1245
• Add Ref more with_elems Ref constructors #1258
• Rename FromBytes::ref_from[_with_elems] #1278
• Update deprecated methods for FromBytes, Ref #1282
• [error] Implement std::error::Error on errors #1298
• [Ref] Make methods into associated functions #1618
• FromZeros boxed slice method supports slice DSTs #1478
• Implement TryFromBytes for unsized UnsafeCell #1619
• Make extend_vec_zeroed, insert_vec_zeroed fallible #1658
• [error] Implement core::error::Error for errors #1663
• Use #[diagnostic::on_unimplemented] on traits #1682
• [Unalign::update] Optimize when T is unaligned #1703
• Move Vec zero extension functions to FromZeros #1690
• Improve IntoBytes derive error message #1713
• [derive] Improve IntoBytes error messages #1712
• Encode infallible alignment errors in types #1718
• [Ref] Remove unaligned constructors #1723
• Add TryFromBytes::try_read_from_{prefix,suffix} #1738
• Add TryFromBytes::try_ref_from_bytes_with_elems #1740
• Add TryFromBytes::try_{ref,mut}_from_{bytes,prefix,suffix}_with_elems #1747
• Derive trivial is_bit_valid when possible #1303
• [derive] Support IntoBytes on repr(Rust) structs #1796
• Improve ZST DST error message #1873
• Support 16-bit target platforms #2401
• [pointer] Support generic TransmuteFrom framework #2408
• Implement FromZeros for AtomicPtr #2426
• [pointer] SizeEq supports raw pointer transmutes #2427
• Implement traits for Cell #2421
• Teach transmute_{ref,mut}! to handle slice DSTs #2428
• [macros] Support shrinking value transmutes #2581
• [project] Distinguish structs/union/enums #2857
• Invert build.rs version detection flags and update CI #2804
• Support x86 AVX-12 SIMD types on 1.89.0 stable #2574

Enabling agentic development

• [agents] Update review guidelines for TODOs #2967
• Add more instructions for coding agents #2834
• [agents] Add instructions for reviewing #2855

CI

• [CI] Configure dependabot commit message prefix #323
• [CI] Remove CARGO_NET_RETRY override #324
• [CI] Only run Clippy on nightly #332
• [CI] Use generate_cache job in more places #339
• [byteorder] Optimize tests; large impact on Miri #350
• [CI] Use cargo-semver-checks to check backwards-compatibility #351
• Add round_down_to_next_multiple_of_alignment #393
• [miri] Skip running UI tests #400
• [ci] Skip MIPS tests on the stable toolchain #459
• [ci] Fix all-jobs-succeeded job #460
• [roll] Install rust-src component when rolling #469
• [ci][kani] Pass correct RUSTFLAGS #493
• [ci] Don't test on MIPS #499
• [ci] Work around Miri bug on aarch64 targets #519
• [ci][check_versions] Refactor bash script #581
• Lower MSRV to 1.57 #848
• [ci] Check with tests #852
• [ci][generate_cache] Run steps in parallel #940
• [ci] Run cargo fix when rolling toolchains #943
• read_from_{prefix,suffix} return remaining bytes #1270
• [ci] Make matrix names more concise #1306
• [ci] Run Miri tests on multiple threads #1307
• [cargo-zerocopy] Pass --cfg __ZEROCOPY_INTERNAL_USE_ONL_TOOLCHAIN=... #2983
• [ci] Update Cargo.lock from releasing script #2958
• [cargo-zerocopy] Pass --cfg __ZEROCOPY_INTERNAL_USE_ONLY_DEV_MODE #2960
• [ci] Roll toolchains weekly #2773
• Update Cargo.lock #2957
• ci: Fix yq installation failure by pinning version and disabling sumdb check #2962
• Commit Cargo.lock and tools/Cargo.lock #2932
• [ci] Replace Rust caching with generic caching #2931
• [ci] Roll cargo-llvm-cov to 0.8.0 #2929
• [ci] Work around bugs in nextest and llvm-cov #2924
• [tools] Remove now-obsolete rm Cargo.lock #2916
• [ci] Check for orphaned UI test .stderr files #2909
• [ci] Fix Rust toolchain roller #3068
• [githooks] Don't run ci/check_todo.sh #3028
• [ci] Pin dependencies to hashes #2869
• [ci] Roll pinned nightly toolchain #2753
• [ci] In check_todo.sh, permit disabling check #2839
• [ci][dependabot] Run weekly, group updates #2832
• [ci] Add hide-from-release-notes label to Dependabot PRs #2828
• [ci] Various performance improvements #2829
• Fix typos in .github/workflows/ci.yml #2802
• [ci] Integrate zizmor static analysis #2810
• [ci] Integrate action-validator into CI and pre-push hooks #2811
• [ci] Add MSRV minimality check #2812
• [ci] Fix bug where PR labels are not added #2782

Fixes

• Restrict MaybeUninit trait impls to fix soundness #308
• [docs] Fix broken doc link #387
• Fix bug in validate_cast_and_convert_metadata #376
• Improve test_new_aligned_sized to catch more bugs #511
• [derive][tests] Fix KnownLayout test #555
• Fix soundness hole in Ref::into_ref and into_mut #721
• [derive] FromZeros requires repr on unit-only enums #802
• [derive] Disable prelude when testing; fix bugs #886
• TryFromBytes for MaybeUninit<T> requires T: NoCell #889
• [ptr] Fix soundness hole in "at least" invariants #909
• Fix alignment bug in test_impls #944
• Add IntoByteSlice trait, use as bound for into_ref #966
• Add KnownLayout, remove Sized from some APIs #967
• Use KnownLayout in more APIs #970
• Deprecate some Ref methods in favor of KnownLayout #971
• Make IntoByteSlice an unsafe trait #1261
• Strengthen soundness proof for size_of_val_raw #1574
• Fix unsoundness in impl_for_transparent_wrapper! #1585
• Write some safety proofs #1615
• Tighten soundness of addr_of! uses #1617
• Add safety comment for MaybeUninit UnsafeCell #1620
• Derive packed #1781
• [derive] Don't support IntoBytes on repr(Rust) types #1783
• [derive] Correctly handle #[repr(align)] enums #1784
• Fix FromZeros method signatures #1805
• [derive] IntoBytes on unions requires --cfg #1804
• Warn more loudly about IntoBytes on unions #1812
• [pointer] Fix soundness hole in read_unaligned #1893
• [derive] Fix bug with KnownLayout on repr(packed) #2307
• [pointer] Rename Any -> Unknown/Inaccessible #1909
• [pointer][invariant] Remove AliasingMapping, Inaccessible #2301
• [pointer] Fix Ptr[Inner] variance #2351
• [pointer] Improve soundness of invariant modeling #2397
• [transmute] Update TryTransmuteFromPtr proof #2455
• Update SizeEq safety invariant #2564
• [pointer] Fix bug in try_cast_into_no_leftover #2937
• [derive] Avoid ambiguities referring to associated items #2917
• [cast] Fix safety comment bug for CastUnsized #2908
• [derive] Fix unused variable bug in tests #2891
• [zerocopy-derive] Fix panic with raw identifiers #2871
• [derive] Fix IntoBytes on repr(C) DST #2836

Reducing Technical Debt

• Simplify AsAddress impls #327
• Move location of Sealed impls #334
• Simplify unsafe_impl! macro #335
• Move wrappers to their own module #337
• Simplify unaligned Ref constructors #329
• [macros] safety_comment! permits multiple comments #342
• [derive] Simplify code, remove obsolete features #375
• Refactor unsafe blocks; update TODO comments #432
• [macro_util] Rename from derive_util, include more #455
• [byteorder] Simplify tests #435
• [util] Ptr doesn't track byte length separately #645
• Use KnownLayout to construct and operate on Refs #678
• [byteorder] Remove byteorder feature and crate dep #583
• [derive] Rename is_c_like -> is_fieldless #804
• [util] Use new padding_needed_for implementation #833
• [ptr] Add TransparentWrapper #958
• Deprecate FromBytes::slice_from #978
• [wrappers] Replace some unsafe code with Ptr #1007
• Remove unnecessary Ptr "assume" calls #1002
• Impl traits for TransparentWrapper types safely #1001
• Implement TransparentWrapper for atomic types  #1011
• TransparentWrapper supports UnsafeCell variance #1024
• Implement TransparentWrapper for UnsafeCell #1025
• Move deprecated items to dedicated module #1036
• Remove zeroing Ref constructors #1135
• KnownLayout tracks pointer metadata type #995
• [FromBytes] Replace Ref with Ptr in some methods #1163
• [FromBytes] Rename some methods for consistency #1210
• Move Ref to its own module #1216
• [ref] Move Ref definition into its own module #1218
• Add AliasingSafe framework #1224
• Factor out some duplicated code in FromBytes #1234
• Move tests to more appropriate locations #1240
• [impls] Move trait impls from crate root #1242
• Add more safety proof to KnownLayout derive #1302
• Reorganize macro modules under util module #1623
• Module reorg finish #1624
• [derive] Overhaul repr parsing #1752
• Refactor try_read_from_xxx #1810
• [pointer] Add separate PtrInner #1891
• [pointer] Simplify AliasingSafe, rename to Read #1908
• [pointer] Replace project with cast_unsized_unchecked #2405
• [pointer] Remove ReadReason from Read trait #2409
• [ptr] Simplify PtrInner's invariants #2450
• [derive] Replace impl_block with builder pattern #2456
• Move split_at module to its own file #2470
• Replace safety_comment! with const blocks #2483
• Simplify some Ptr operations #2577
• Add generic Ptr::try_with #2946
• Simplify try_transmute! (and friends) internals #2938
• [util] use more items to make code more concise #2942
• Replace unsafe_with_size_eq! with transmute_with #2923
• [transmute] Remove some SizeEq impls #2922
• Clarify semantics of size-preserving transmutes #2911
• [derive] Refactor into more modules #2901
• [derive] Factor out core_reexport path generation #2899
• [tests] Rely on cargo-zerocopy toolchain resolution #2989
• [derive] Introduce Ctx type #2890
• [derive] Move find_zero_variant to enum module #2889
• [derive] Split repr(C) logic from derive_known_layout_inner #2888
• Fix stale comments #2896
• Simplify projection methods #2895
• Tighten impl of CastUnsized #2887
• [derive] Rename module ext -> util #2892
• [derive] Unify handling of #[allow] attributes #2882
• Simplify MutationCompatible impls #2877

Testing Improvements

• [ci] Run byteorder tests under Kani #394
• [miri] Optimize tests #395
• [tests] Test more failure conditions #396
• [ci] Don't run ignored tests under Miri #399
• [tests] Fix UI tests with missing imports #706
• Support testing TryFromBytes for !NoCell types #924
• Add tests for some uncovered lines #1277
• [derive] Test exact output of derives #1664
• [derive] Test TryFromBytes output on dataful enums #1665
• Test elem count preservation for ZST slice tails #1689
• [derive] More complex TryFromBytes enum output tests #1683
• [derive] Make output tests insensitive to syntax #1711
• [kani] Use function contracts instead of proofs #2449
• [byte_slice] Add Kani proofs #2451
• [derive] Test with UnsafeCells #2463
• [derive] In tests, assert hygiene of output #2918
• [derive] Automatically bless output test output #2905
• [tests] Annotate UI test files with error requirements #3000
• [tests] Merge UI test directories #2997
• [ci] Support XODO comments in commit messages #2998
• [tests] Delete disabled test file #2996
• [tests] Migrate from trybuild to ui-test #2994
• [test] Move impl_or_verify! tests to doctests #2988
• [derive] Improve hygiene testing #2898
• [derive] Refactor is_bit_valid tests #2879
• Add tests targeted at un-covered lines of code #2833

Documentation Improvements

• Improve trait safety documentation #283
• [docs] Improve crate-level docs #386
• Improve ManuallyDoc safety comments #522
• [byteorder] Minor doc improvements #546
• Safety comments quote new bit validity guarantees #462
• [docs] Fix missing doc link in TryFromBytes #655
• [docs] Fix typo in include_value! doc comment #662
• [policies] Document yanking policy #677
• Release 0.8.0-alpha #681
• [readme] Link to GitHub Releases #692
• [issues] Add "Customer request" template #708
• [docs] Add yank log #732
• [docs] Link to customer request issue template #738
• [doc] TryFromBytes, IntoBytes aren't symmetrical #860
• [ptr] Clean up prose for field invariants #901
• Document slice DSTs and size validity #1273
• [error] Document validation order #1456
• [doc] Update wording regarding transmute macros #1662
• [docs] Clarify derive dependency recommendation #1672
• [byte_slice] Clean up docs and methods #1700
• [Unalign] Update docs #1701
• [byteorder] Clean up docs #1707
• [Ref] Clean up docs #1706
• [docs] Make infallible align errors discoverable #1737
• Fix broken doc links #1823
• [pointer] Clarify semantics of aliasing invariants #1889
• [pointer] Document validity safety invariant #2403
• Improve SplitAt docs #2489
• Minor doc comment improvements #2936
• [macros] Clarify size/align behavior of try macros #2975
• [macros] Fix doc bug in transmute_mut! #2951
• [byteorder] Note that network-endian is a synonym for big-endian #2785

Development Experience

• Deny clippy::missing_inline_in_public_items #341
• [cargo.sh] Initial commit #363
• [testutil] Initial commit #412
• [ci] Roll pinned nightly toolchain automatically #408
• Roll stable and nightly toolchains separately #446
• [POLICIES.md] Initial commit #485
• [derive] Include LICENSE file when publishing #489
• [third-party][rust] Add README.fuchsia file #492
• Deny strict provenance lints during testing #572
• [ci] Add code coverage reporting for PRs and main #1274
• [ci] Only run Miri tests in merge queue #1313
• [ci] Test some targets only in the merge queue #1314
• Ban TODO comments, require FIXME comments #2511
• [projection] Clarify comment explaining fallibility #2939
• Vendor dependencies #2883
• [derive] Store output test goldens in separate files #2904
• [tests] Coalesce UI test files #3001
• [ci][check_fmt] Add --fix flag #2991
• [test] Detect and warn about using cargo for development #2990
• [cargo-zerocopy] Pass RUSTDOCFLAGS #2987
• [cargo-zerocopy] Auto-install env var toggle #2840

Misc

• Don't inline functions that are never called #346
• [derive] Exclude large test files when publishing #743
• Re-parent orphaned comment #2935
• Update Cargo.lock #2949
• [derive] Document use of Field trait #2865
• [README] Update Thanks section #2766

Releases

• Release 0.7.9 #486
• Release 0.7.18 #558
• Release 0.7.23 #587
• Release 0.7.28 #663
• Release 0.7.29 #676
• Release 0.8.0-alpha.1 #698
• Release 0.8.0-alpha.5 #869
• Release 0.8.0-alpha.6 #979
• Release 0.8.0-alpha.7 #1027
• Release 0.8.0-alpha.8 #1138