I suggest that Sparv should use pyfory instead of pickle for serialization and deserialization of work files.
Benefits
- Faster serialization and deserialization
- Smaller data size
- Stable dataformat, dataformat depends on fory version, not on Python version (which
pickle does).
Migration
From pyfory's migration guide:
# Before (pickle)
import pickle
data = pickle.dumps(obj)
result = pickle.loads(data)
# After (Fory - drop-in replacement with better performance)
import pyfory
f = pyfory.Fory(xlang=False, ref=True, strict=False)
data = f.dumps(obj) # Faster and more compact
result = f.loads(data) # Faster deserialization
# Benefits:
# - 2-10x faster serialization
# - 2-5x faster deserialization
# - Up to 3x smaller data size
# - Same API, better performance
I suggest that Sparv should use
pyforyinstead ofpicklefor serialization and deserialization of work files.Benefits
pickledoes).Migration
From pyfory's migration guide: