-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathSQLiteArrowKit.txt
More file actions
123 lines (104 loc) · 5.92 KB
/
Copy pathSQLiteArrowKit.txt
File metadata and controls
123 lines (104 loc) · 5.92 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
/*JULIA.
SQLiteArrowKit.main("/content/songs.csv", "songs") */
/*****************************************************************/
DATA SIZE~437 MB
****************************************
LECTURE TIME, FROM CSV TO DATAFRAMES: 17.6778
PROCESSING TIME, USING DATAFRAMES:
2×2 DataFrame
Row │ mode frequency
│ Int64 Int64
─────┼──────────────────
1 │ 0 978222
2 │ 1 1132094
4×4 DataFrame
Row │ spotify_id name mode persistenceInPopularity
│ String31 String Int64 Int64
─────┼────────────────────────────────────────────────────────────────────────────
1 │ 6dOtVTDdiauQNBQEDOtlAB BIRDS OF A FEATHER 1 14043
2 │ 2plbrEY59IikOBgBGLjaoe Die With A Smile 0 11238
3 │ 2qSkIjg1o9h3YT9RAgYN75 Espresso 1 6597
4 │ 3rUGC1vUpkDG9CZFHMur1t greedy 0 7658
time~2.253
****************************************
LECTURE TIME (EXPLICIT EXPORT), FROM CURSOR TO ARROW TABLE:
81.854
┌ Info: arrow table created:
│ conversion_time = 81.3088
└ lecture_time = 0.5402
PROCESSING TIME, USING DUCKDB QUERIES: 0.8765
2×2 DataFrame
Row │ mode frequency
│ Int64 Int64
─────┼──────────────────
1 │ 0 978222
2 │ 1 1132094
4×3 DataFrame
Row │ name mode persistenceInPopularity95
│ String Int64 Int64
─────┼──────────────────────────────────────────────────────
1 │ Die With A Smile 0 11238
2 │ BIRDS OF A FEATHER 1 14043
3 │ Espresso 1 6597
4 │ greedy 0 7658
/*Python. */
/*****************************************************************/
DATA SIZE~437 MB
::::::::::::::::::::::::::::::::::::::::::::::::::
LECTURE TIME, FROM CSV TO POLARS (EAGER MODE):
time~7.4685
PROCESSING TIME, USING POLARS (EAGER MODE):
shape: (2, 2)
┌──────┬───────────┐
│ mode ┆ frequency │
│ --- ┆ --- │
│ i64 ┆ u32 │
╞══════╪═══════════╡
│ 0 ┆ 978222 │
│ 1 ┆ 1132094 │
└──────┴───────────┘
shape: (4, 4)
┌────────────────────────┬────────────────────┬──────┬────────────────────────────┐
│ spotify_id ┆ name ┆ mode ┆ persistenceInPopularity>95 │
│ --- ┆ --- ┆ --- ┆ --- │
│ str ┆ str ┆ i64 ┆ u32 │
╞════════════════════════╪════════════════════╪══════╪════════════════════════════╡
│ 3rUGC1vUpkDG9CZFHMur1t ┆ greedy ┆ 0 ┆ 7658 │
│ 2qSkIjg1o9h3YT9RAgYN75 ┆ Espresso ┆ 1 ┆ 6597 │
│ 2plbrEY59IikOBgBGLjaoe ┆ Die With A Smile ┆ 0 ┆ 11238 │
│ 6dOtVTDdiauQNBQEDOtlAB ┆ BIRDS OF A FEATHER ┆ 1 ┆ 14043 │
└────────────────────────┴────────────────────┴──────┴────────────────────────────┘
time~0.3408
::::::::::::::::::::::::::::::::::::::::::::::::::
LECTURE TIME, FROM CURSOR (NATIVE ARROW) TO ARROW TABLE: 9.6044
QUERYING DIRECTLY FROM SQLITE:
name mode
0 Ordinary 1
1 Manchild 1
2 back to friends 1
3 Die With A Smile 0
4 BIRDS OF A FEATHER 1
5 Don’t Say You Love Me 1
6 La Plena - W Sound 05 1
7 undressed 1
8 APT. 0
9 Love Me Not 1
::::::::::::::::::::::::::::::::::::::::::::::::::
PROCESSING TIME, USING DUCKDB QUERIES:
┌───────┬───────────┐
│ mode │ frequency │
│ int64 │ int64 │
├───────┼───────────┤
│ 0 │ 978222 │
│ 1 │ 1132094 │
└───────┴───────────┘
┌────────────────────┬───────┬───────────────────────────┐
│ name │ mode │ persistenceInPopularity95 │
│ varchar │ int64 │ int64 │
├────────────────────┼───────┼───────────────────────────┤
│ Die With A Smile │ 0 │ 11238 │
│ BIRDS OF A FEATHER │ 1 │ 14043 │
│ Espresso │ 1 │ 6597 │
│ greedy │ 0 │ 7658 │
└────────────────────┴───────┴───────────────────────────┘
time~1.8701