from_reader() reads the multihash code and length as single bytes, assuming they are always 1-byte varints. For codec codes > 0x7F (e.g., dag-cbor at 0x71, blake2b-256 at 0xB220), the varint encoding uses multiple bytes, causing incorrect parsing.
Problem
In cid/cid.py, the CIDv1 path of from_reader():
elif version == 1:
# Read codec (varint)
codec_bytes = bytearray()
codec_bytes.append(first_byte[0])
bytes_read = 1
while True:
byte = reader.read(1)
codec_bytes.append(byte[0])
bytes_read += 1
if (byte[0] & 0x80) == 0:
break
# Now read multihash
peek = reader.read(2) # ← Assumes 2 bytes for mh code + length
mh_length = int(peek[1]) # ← Assumes length is second byte
remaining = mh_length
multihash_bytes = reader.read(remaining)The peek = reader.read(2) assumes the multihash code is 1 byte and the length is 1 byte. But:
- Multihash codes > 0x7F use multi-byte varints (e.g.,
blake2b-256 = 0xB220 = 3 bytes)
- Digest lengths > 127 use multi-byte varints
Similarly, the CIDv0 path has the same problem:
if version == 0:
peek = reader.read(2)
mh_length = int(peek[1]) # ← Assumes single-byte code and lengthProposed Solution
Use proper varint reading for all fields:
def from_reader(reader):
import varint
first_byte = reader.read(1)
if not first_byte:
raise ValueError("Not enough data to read CID")
version = int(first_byte[0])
if version == 0:
# CIDv0: first byte is actually the multihash code
# Read the multihash properly
mh_code = version # The first byte IS the mh code for CIDv0
# ... use varint.decode_stream() for length ...
elif version == 1:
# Read codec varint
codec_code = varint.decode_stream(reader)
# Read multihash code varint
mh_code = varint.decode_stream(reader)
# Read multihash length varint
mh_length = varint.decode_stream(reader)
# Read digest
digest = reader.read(mh_length)
# ...Related
- File:
cid/cid.py, from_reader() function
- Dependency:
varint library (already imported)
from_reader()reads the multihash code and length as single bytes, assuming they are always 1-byte varints. For codec codes > 0x7F (e.g.,dag-cborat 0x71,blake2b-256at 0xB220), the varint encoding uses multiple bytes, causing incorrect parsing.Problem
In
cid/cid.py, the CIDv1 path offrom_reader():The
peek = reader.read(2)assumes the multihash code is 1 byte and the length is 1 byte. But:blake2b-256= 0xB220 = 3 bytes)Similarly, the CIDv0 path has the same problem:
Proposed Solution
Use proper varint reading for all fields:
Related
cid/cid.py,from_reader()functionvarintlibrary (already imported)