Skip to content

feat: add to_sedonadb() method #2511

Description

@MrPowers

It would be nice to have an interface that converts a SedonaSpark DataFrame to a SedonaDB DataFrame easily. Here is a current solution that works:

import sedona.db
sd = sedona.db.connect()

df = sd.create_data_frame(dataframe_to_arrow(spark_df))

This could be nice:

spark_df.to_sedonadb()

But maybe we'd have to do this:

spark_df.to_sedonadb(sd)

This would allow for cool spatial workflows, like this:

  • Read an Iceberg table with SedonaSpark and perform big data operations with a filtering operation at the end to make the data small enough to fit on a single machine
  • Convert the SedonaSpark DataFrame to SedonaDB
  • Use a library that's compatible with SedonaDB, like lonboard, to create a graph

Let me know what you think!

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions