Skip to content

Commit 62e0dde

Browse files
committed
Describe how to use the custom saving functions in the README.
1 parent 1e4664c commit 62e0dde

1 file changed

Lines changed: 47 additions & 5 deletions

File tree

README.md

Lines changed: 47 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -96,15 +96,15 @@ typedef knncolle_annoy::AnnoyBuilder<
9696

9797
To save and reload Annoy indices from disk, we need to register a loading function into **knncolle**'s `load_prebuilt()` registry.
9898
This is a little complicated as we must decide on which combinations of types and distances we want to deal with.
99-
Here, we only consider the obvious distance metrics and the defaults for the internal Annoy types,
99+
In the example below, we only consider the obvious distance metrics and the defaults for the internal Annoy types,
100100
though more combinations could also be supported at the cost of larger binaries and longer compile times.
101101

102102
```cpp
103103
auto& reg = knncolle::load_prebuilt_registry<int, double, double>();
104104
reg[knncolle_annoy::save_name] = [](const std::string& prefix) -> Prebuilt<int, double, double>* {
105-
auto config = knncolle_annoy::scan_prebuilt_save_config(prefix);
105+
auto config = knncolle_annoy::load_annoy_prebuilt_types(prefix);
106106

107-
// Maybe add some checks that the types are as expected.
107+
// Checks that the AnnoyIndex_ and AnnoyData_ types are the same as the defaults.
108108
if (config.index != knncolle_annoy::get_numeric_type<int>()) {
109109
throw std::runtime_error("unexpected type for the Annoy index");
110110
}
@@ -122,14 +122,56 @@ reg[knncolle_annoy::save_name] = [](const std::string& prefix) -> Prebuilt<int,
122122
```
123123

124124
Then we can save and reload the `Prebuilt` Annoy indices.
125-
Note the caveats on `knncolle::Prebuilt::save()` - specifically, the files are not guaranteed to be portable between machines or even different versions of **knncolle_annoy**.
125+
Note the caveats on `knncolle::Prebuilt::save()` -
126+
specifically, the files are not guaranteed to be portable between machines or even different versions of **knncolle_annoy**.
126127

127128
```cpp
128-
std::string path_prefix = "anno/location/here_";
129+
std::string path_prefix = "annoy/location/here_";
129130
an_index.save(path_prefix);
130131
auto reloaded = knncolle::load_prebuilt(path_prefix);
131132
```
132133

134+
For custom Annoy-related types and distances, users can define their own saving and loading methods.
135+
For example, if we had a custom distance:
136+
137+
```cpp
138+
// Mock up a custom distance.
139+
class MyCustomDistance : public Annoy::Euclidean {};
140+
141+
// Define a function to save information about this custom distance during .save() calls.
142+
knncolle_annoy::custom_save_for_annoy_data<MyCustomDistance>() = [](const std::string& prefix) -> void {
143+
const auto custdistpath = prefix + "_custom";
144+
// write something about MyCustomDistance at 'custdistpath'.
145+
};
146+
147+
// Make a new loading function that uses the saved custom information.
148+
reg[knncolle_annoy::save_name] = [](const std::string& prefix) -> Prebuilt<int, double, double>* {
149+
auto config = knncolle_annoy::load_annoy_prebuilt_types(prefix);
150+
if (config.index != knncolle_annoy::get_numeric_type<int>()) {
151+
throw std::runtime_error("unexpected type for the Annoy index");
152+
}
153+
if (config.data != knncolle_annoy::get_numeric_type<float>()) {
154+
throw std::runtime_error("unexpected type for the Annoy data");
155+
}
156+
157+
if (config.distance == "euclidean") {
158+
return knncolle_annoy::load_annoy_prebuilt<int, double, double, Annoy::Euclidean>();
159+
} else if (config.distance == "manhattan") {
160+
return knncolle_annoy::load_annoy_prebuilt<int, double, double, Annoy::Manhattan>();
161+
} else {
162+
const auto custdispath = prefix + "_custom";
163+
// read something from 'custdistpath' to determine the custom distance to use.
164+
return knncolle_annoy::load_annoy_prebuilt<int, double, double, MyCustomDistance>();
165+
}
166+
};
167+
168+
std::string custom_prefix = "annoy/location/custom_";
169+
knncolle_annoy::AnnoyBuilder<int, double, double, MyCustomDistance> custom_builder;
170+
auto custom_index = custom_builder.build_unique(mat);
171+
custom_index.save(custom_prefix);
172+
auto custom_reloaded = knncolle::load_prebuilt(custom_prefix);
173+
```
174+
133175
## Building projects
134176

135177
### CMake with `FetchContent`

0 commit comments

Comments
 (0)