Skip to content

Use pyfory instead of pickle #218

Description

@kod-kristoff

I suggest that Sparv should use pyfory instead of pickle for serialization and deserialization of work files.

Benefits

  • Faster serialization and deserialization
  • Smaller data size
  • Stable dataformat, dataformat depends on fory version, not on Python version (which pickle does).

Migration

From pyfory's migration guide:

# Before (pickle)
import pickle
data = pickle.dumps(obj)
result = pickle.loads(data)

# After (Fory - drop-in replacement with better performance)
import pyfory
f = pyfory.Fory(xlang=False, ref=True, strict=False)
data = f.dumps(obj)      # Faster and more compact
result = f.loads(data)   # Faster deserialization

# Benefits:
# - 2-10x faster serialization
# - 2-5x faster deserialization
# - Up to 3x smaller data size
# - Same API, better performance

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions