-
Notifications
You must be signed in to change notification settings - Fork 4
Feat: implement hnsw index #43
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: refactor
Are you sure you want to change the base?
Conversation
e160cf5 to
3a275c3
Compare
HNSW implementation code from my PR on sdslabs/VortexDB sdslabs/VortexDB#43 Signed-off-by: Arshdeep54 <balarsh535@gmail.com>
1d98363 to
f883ad4
Compare
Signed-off-by: Arshdeep54 <balarsh535@gmail.com>
f883ad4 to
9f8ca6d
Compare
| // Per-node, per-level neighbor lists (bounded by M/M0) | ||
| pub nodes: HashMap<PointId, Node>, | ||
| // Number of points inserted | ||
| pub nb_point: usize, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nb_point is never utilized or updated
| // Cached dimension of stored vectors | ||
| pub data_dimension: Dimension, | ||
| // Guard against concurrent mutation during queries | ||
| pub searching: bool, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this flag is never utilized?
| let mut current = ep; | ||
| loop { | ||
| let cur_vec = self.get_vec(current); | ||
| let mut best_score = distance(query.to_vec(), cur_vec.to_vec(), self.similarity); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
need to change distance function to take references instead of copies, otherwise it will be slower. I changed distance function accordingly in KD Tree PR, please check that.
| /// - greedy descend from current entry to l+1 to get a pivot | ||
| /// - for each level down to 0: ef-construction, diversity pruning, bidirectional connect with caps | ||
| /// - if l is above current max level, update the entry point | ||
| fn insert(&mut self, vector: IndexedVector) -> Result<(), DbError> { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
add a check that a point with same id does not exist already.
HNSW implementation referenced from HNSW (Malkov & Yashunin)
Includes the following public functions :