Skip to content

use double hashing instead of linear probing - #7418

Merged
ben-schwen merged 19 commits into
truehashfrom
cuckoo_hash
Dec 4, 2025
Merged

ben-schwen merged 19 commits into
truehashfrom
cuckoo_hash

Conversation

@ben-schwen

@ben-schwen ben-schwen commented Nov 6, 2025 •

Copy link
Copy Markdown
Member

So I implemented double hashing and adjusted the table size to the power of 2 (both suggestions from Ivan in #6694)

knuth is Ivans linear probing with knuth hash
double is the first try of double hashing where some mixing of XOR happens to spare multiplications
noxor is an easier variation without mixing
khash is using the khash.h from klib

What really made (double and noxor) competitive with master and khash was probably parallelizing the probing in chmatch after building the hashtable (khash also uses this). What is left is to benchmark chmatchdup which is probably slower and either needs adjustment of the hash table to also save a list of elements or at least cache the 3x lookups.

image image image
Details
library(atime)
library(data.table)

pkg.path <- '.'
limit <- 1
# taken from .ci/atime/tests.R
pkg.edit.fun <- function(old.Package, new.Package, sha, new.pkg.path) {
  pkg_find_replace <- function(glob, FIND, REPLACE) {
    atime::glob_find_replace(file.path(new.pkg.path, glob), FIND, REPLACE)
  }
  Package_regex <- gsub(".", "_?", old.Package, fixed = TRUE)
  Package_ <- gsub(".", "_", old.Package, fixed = TRUE)
  new.Package_ <- paste0(Package_, "_", sha)
  pkg_find_replace(
    "DESCRIPTION",
    paste0("Package:\\s+", old.Package),
    paste("Package:", new.Package))
  pkg_find_replace(
    file.path("src", "Makevars.*in"),
    Package_regex,
    new.Package_)
  pkg_find_replace(
    file.path("R", "onLoad.R"),
    Package_regex,
    new.Package_)
  pkg_find_replace(
    file.path("R", "onLoad.R"),
    sprintf('packageVersion\\("%s"\\)', old.Package),
    sprintf('packageVersion\\("%s"\\)', new.Package))
  pkg_find_replace(
    file.path("src", "init.c"),
    paste0("R_init_", Package_regex),
    paste0("R_init_", gsub("[.]", "_", new.Package_)))
  pkg_find_replace(
    "NAMESPACE",
    sprintf('useDynLib\\("?%s"?', Package_regex),
    paste0('useDynLib(', new.Package_))
}

versions <- c(
  master = '3c90b0f80e1d5b54ea97f3b56f28df07b93e820a',
  knuth = 'd7a9a1707ec94ec4f2bd86a5dfb5609207029ba4',
  double = '1d88ad4c9dd179fb815266fb3acb687a89b8b7c7',
  noxor = '48b19422dd331a9d7eaddff507563bcf9643ebdd',
  khash = '3178f10af7860223b7bea0fba519104d6a9d55ca'
)

set.seed(3)
sample_strings = function(N=10, len=4) {
   do.call(paste0, replicate(len, sample(LETTERS, N, TRUE), FALSE))
}

N <- 10^seq(2, 7.5, .25)

# Benchmark 1: Large table 
tab_full = sample_strings(1e6, 10)
tab_small = sample(tab_full, 9e5)
chmatch_work1 <- lapply(setNames(nm = N), \(N)
  sample(tab_full, N, TRUE)
)

chmatch1 <- atime_versions(
  pkg.path, N,
  expr = data.table::chmatch(chmatch_work1[[as.character(N)]], tab_small),
  seconds.limit = limit, verbose = TRUE, sha.vec = versions,
  pkg.edit.fun = pkg.edit.fun
)
plot(chmatch1)

# Benchmark 2: Small table (expected case: a few distinct strings - 26 letters)
chmatch_work2 <- lapply(setNames(nm = N), \(N)
  sample(letters, N, TRUE)
)
chmatch2 <- atime_versions(
  pkg.path, N,
  expr = data.table::chmatch(chmatch_work2[[as.character(N)]], letters),
  seconds.limit = limit, verbose = TRUE, sha.vec = versions,
  pkg.edit.fun = pkg.edit.fun
)
plot(chmatch2)

# Benchmark 3: Medium table (10K unique strings)
medium_table <- sample_strings(10000, len=4)
chmatch_work3 <- lapply(setNames(nm = N), \(N)
  sample(medium_table, N, TRUE)
)
chmatch3 <- atime_versions(
  pkg.path, N,
  expr = data.table::chmatch(chmatch_work3[[as.character(N)]], medium_table),
  seconds.limit = limit, verbose = TRUE, sha.vec = versions,
  pkg.edit.fun = pkg.edit.fun
)
plot(chmatch3)

# Benchmark 4: High miss rate (most items not found)
miss_table <- sample_strings(100, len=4)
chmatch_work4 <- lapply(setNames(nm = N), \(N)
  sample_strings(N, len=4)
)
chmatch4 <- atime_versions(
  pkg.path, N,
  expr = data.table::chmatch(chmatch_work4[[as.character(N)]], miss_table),
  seconds.limit = limit, verbose = TRUE, sha.vec = versions,
  pkg.edit.fun = pkg.edit.fun
)
plot(chmatch4)

# Benchmark 5: All hits (every item found)
hit_table <- sample_strings(1000, len=4)
chmatch_work5 <- lapply(setNames(nm = N), \(N)
  sample(hit_table, N, TRUE)
)
chmatch5 <- atime_versions(
  pkg.path, N,
  expr = data.table::chmatch(chmatch_work5[[as.character(N)]], hit_table),
  seconds.limit = limit, verbose = TRUE, sha.vec = versions,
  pkg.edit.fun = pkg.edit.fun
)
plot(chmatch5)

# Benchmark 6: chin (logical return) with small table
chin_work <- lapply(setNames(nm = N), \(N)
  sample(letters, N, TRUE)
)
chin_bench <- atime_versions(
  pkg.path, N,
  expr = data.table::`%chin%`(chin_work[[as.character(N)]], letters),
  seconds.limit = limit, verbose = TRUE, sha.vec = versions,
  pkg.edit.fun = pkg.edit.fun
)
plot(chin_bench)

# Benchmark 7: Long strings (16 chars)
long_table <- sample_strings(1000, len=16)
chmatch_work7 <- lapply(setNames(nm = N), \(N)
  sample(long_table, N, TRUE)
)
chmatch7 <- atime_versions(
  pkg.path, N,
  expr = data.table::chmatch(chmatch_work7[[as.character(N)]], long_table),
  seconds.limit = limit, verbose = TRUE, sha.vec = versions,
  pkg.edit.fun = pkg.edit.fun
)
plot(chmatch7)

# Benchmark 8: Realistic join keys (90% hit rate)
join_table <- sample_strings(10000, len=8)
chmatch_work8 <- lapply(setNames(nm = N), \(N) {
  # 90% from join_table (will be found), 10% new (won't be found)
  c(
    sample(join_table, as.integer(N * 0.9), TRUE),
    sample_strings(as.integer(N * 0.1), len=8)
  )
})
chmatch8 <- atime_versions(
  pkg.path, N,
  expr = data.table::chmatch(chmatch_work8[[as.character(N)]], join_table),
  seconds.limit = limit, verbose = TRUE, sha.vec = versions,
  pkg.edit.fun = pkg.edit.fun
)
plot(chmatch8)


N <- 10^seq(2, 7.5, .25)
# expected case: a few distinct strings
forderv1_work <- lapply(setNames(nm = N), \(N)
  sample(letters, N, TRUE)
)
forderv1 <- atime_versions(
  pkg.path, N,
  expr = data.table:::forderv(forderv1_work[[as.character(N)]]),
  sha.vec = versions, seconds.limit = limit, verbose = TRUE,
  pkg.edit.fun = pkg.edit.fun
)
plot(forderv1)
rm(forderv1_work); gc(full = TRUE)

# worst case: all strings different
# (a challenge for the allocator too due to many small immovable objects)
N <- 10^seq(2, 7.5, .25)
forderv2_work <- lapply(setNames(nm = N), \(N)
  format(runif(N), digits = 16)
)
forderv2 <- atime_versions(
  pkg.path, N,
  expr = data.table:::forderv(forderv2_work[[as.character(N)]]),
  sha.vec = versions, seconds.limit = limit, verbose = TRUE,
  pkg.edit.fun = pkg.edit.fun
)
plot(forderv2)
rm(forderv2_work); gc(full = TRUE)

# expected case: all columns named the same
N <- 10^seq(1, 5.5, .25) # number of data.tables in the list
k <- 10 # number of columns per data.table
rbindlist1_work <- lapply(setNames(nm = N), \(N)
  rep(list(setNames(as.list(1:k), letters[1:k])), N)
)
rbindlist1 <- atime_versions(
  pkg.path, N,
  expr = data.table::rbindlist(rbindlist1_work[[as.character(N)]]),
  sha.vec = versions, seconds.limit = limit, verbose = TRUE,
  pkg.edit.fun = pkg.edit.fun
)
plot(rbindlist1)
rm(rbindlist1_work); gc(full = TRUE)

# worst case: all columns different
N <- 10^seq(1, 4.5, .25) # number of data.tables in the list
k <- 10 # number of columns per data.table
rbindlist2_work <- lapply(setNames(nm = N), \(N)
  replicate(N, setNames(as.list(1:k), format(runif(k), digits = 16)), FALSE)
)
rbindlist2 <- atime_versions(
  pkg.path, N,
  expr = data.table::rbindlist(rbindlist2_work[[as.character(N)]], fill = TRUE),
  sha.vec = versions, seconds.limit = limit, verbose = TRUE,
  pkg.edit.fun = pkg.edit.fun
)
plot(rbindlist2)
rm(rbindlist2_work); gc(full = TRUE)

@codecov

codecov Bot commented Nov 6, 2025 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 85.45455% with 8 lines in your changes missing coverage. Please review.
✅ Project coverage is 99.08%. Comparing base (2003929) to head (aba17d5).
⚠️ Report is 1 commits behind head on truehash.

Files with missing lines Patch % Lines
src/hash.c 84.90% 8 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##           truehash    #7418      +/-   ##
============================================
- Coverage     99.12%   99.08%   -0.05%     
============================================
  Files            86       86              
  Lines         16665    16686      +21     
============================================
+ Hits          16519    16533      +14     
- Misses          146      153       +7     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@tdhock

tdhock commented Nov 7, 2025

Copy link
Copy Markdown
Member

interesting atime result data! Thanks for sharing.
I tried running the code and got this error

Error in value[[3L]](cond) : 
  Error in revparse_single(object, branch): Error in 'git2r_revparse_single': Requested object could not be found

 when trying to checkout 3178f10af7860223b7bea0fba519104d6a9d55ca

is khash = '3178f10af7860223b7bea0fba519104d6a9d55ca' from a deleted branch?

@tdhock

tdhock commented Nov 7, 2025

Copy link
Copy Markdown
Member

on my windows machine I get similar results for the first test.
image
I also fit asymptotic reference lines, which suggest that everything is linear (differences are constant factors).
image

Details
library(data.table)
pkg.path <- '~/R/data.table'
limit <- 1
# taken from .ci/atime/tests.R
pkg.edit.fun <- function(old.Package, new.Package, sha, new.pkg.path) {
  pkg_find_replace <- function(glob, FIND, REPLACE) {
    atime::glob_find_replace(file.path(new.pkg.path, glob), FIND, REPLACE)
  }
  Package_regex <- gsub(".", "_?", old.Package, fixed = TRUE)
  Package_ <- gsub(".", "_", old.Package, fixed = TRUE)
  new.Package_ <- paste0(Package_, "_", sha)
  pkg_find_replace(
    "DESCRIPTION",
    paste0("Package:\\s+", old.Package),
    paste("Package:", new.Package))
  pkg_find_replace(
    file.path("src", "Makevars.*in"),
    Package_regex,
    new.Package_)
  pkg_find_replace(
    file.path("R", "onLoad.R"),
    Package_regex,
    new.Package_)
  pkg_find_replace(
    file.path("R", "onLoad.R"),
    sprintf('packageVersion\\("%s"\\)', old.Package),
    sprintf('packageVersion\\("%s"\\)', new.Package))
  pkg_find_replace(
    file.path("src", "init.c"),
    paste0("R_init_", Package_regex),
    paste0("R_init_", gsub("[.]", "_", new.Package_)))
  pkg_find_replace(
    "NAMESPACE",
    sprintf('useDynLib\\("?%s"?', Package_regex),
    paste0('useDynLib(', new.Package_))
}

versions <- c(
  master = '3c90b0f80e1d5b54ea97f3b56f28df07b93e820a',
  knuth = 'd7a9a1707ec94ec4f2bd86a5dfb5609207029ba4',
  double = '1d88ad4c9dd179fb815266fb3acb687a89b8b7c7',
  noxor = '48b19422dd331a9d7eaddff507563bcf9643ebdd'
  #khash = '3178f10af7860223b7bea0fba519104d6a9d55ca'
)

set.seed(3)
sample_strings = function(N=10, len=4) {
   do.call(paste0, replicate(len, sample(LETTERS, N, TRUE), FALSE))
}

N <- 10^seq(2, 7.5, .25)

# Benchmark 1: Large table 
tab_full = sample_strings(1e6, 10)
tab_small = sample(tab_full, 9e5)
chmatch_work1 <- lapply(setNames(nm = N), \(N)
  sample(tab_full, N, TRUE)
)

chmatch1 <- atime::atime_versions(
  pkg.path, N,
  expr = data.table::chmatch(chmatch_work1[[as.character(N)]], tab_small),
  seconds.limit = limit, verbose = TRUE, sha.vec = versions,
  pkg.edit.fun = pkg.edit.fun
)
plot(chmatch1)
chmatch1.refs <- atime::references_best(chmatch1)
plot(chmatch1.refs)

@tdhock

tdhock commented Nov 7, 2025

Copy link
Copy Markdown
Member

atime CI failure can probably be fixed by merging master into this branch.

@ben-schwen

Copy link
Copy Markdown
Member Author

is khash = '3178f10af7860223b7bea0fba519104d6a9d55ca' from a deleted branch?

The khash commit is only locally and not committed since I didn't want to push other licensed files to the repo.

@aitap aitap left a comment •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you very much!

Since you made it possible for struct hashtab to dynamically grow, hopefully this obviates the need for struct dhashtab?

Comment thread src/hash.c Resolved
@github-actions

github-actions Bot commented Dec 3, 2025 •

Copy link
Copy Markdown

No obvious timing issues in HEAD=cuckoo_hash
Comparison Plot

Generated via commit aba17d5

Download link for the artifact containing the test results: ↓ atime-results.zip

Task Duration
R setup and installing dependencies 2 minutes and 57 seconds
Installing different package versions 21 seconds
Running and plotting the test cases 2 minutes and 37 seconds

@ben-schwen
ben-schwen merged commit 09fe9ae into truehash Dec 4, 2025
11 of 13 checks passed
@ben-schwen
ben-schwen deleted the cuckoo_hash branch December 4, 2025 08:08
ben-schwen added a commit that referenced this pull request Dec 17, 2025
* Implement the hash table

* memrecycle(): replace TRUELENGTH marks with a hash

* rbindlist(): replace 1/2 TRUELENGTH with hashing

Also avoid crashing when creating a 0-size hash.

* rbindlist(): replace 2/2 TRUELENGTH with hashing

This may likely require a dynamically growing hash of TRUELENGTHs
instead of the current pre-allocation approach with a very conservative
over-estimate.

* chmatchMain(): replace TRUELENGTH marks with hash

* copySharedColumns(): hash instead of TRUELENGTH

* combineFactorLevels(): hash instead of TRUELENGTH

* anySpecialStatic(): hash instead of TRUELENGTH

* forder(): hash instead of TRUELENGTH

The hash needs O(n) memory (actually 2*n/load_factor entries) which
isn't great.

* Remove savetl()

* Add codecov suppressions

* Dynamically grow the hash table with bound unknown

In forder() and rbindlist(), there is no good upper boundary on the
number of elements in the hash known ahead of time. Grow the hash table
dynamically. Since the R/W locks are far too slow and OpenMP atomics are
too limited, rely on strategically placed flushes, which isn't really a
solution.

* Minor hash improvements

Use only 28 bits of the pointer (lower 32 but discard the lowest 4).
Inline the linear search by advancing the pointer instead of repeatedly
computing and dividing the hash value. Average improvement of 10%.

* dhash: no need to keep previous table

The hash can only be enlarged from (1) a single-thread context, or (2)
under a critical section, so there is no need to worry about other
threads getting a use-after-free due to a reallocation. This should
halve the memory use by the hash table.

* use double hashing instead of linear probing (#7418)

* add lookup or insert

* use lookup or insert

* use lookup_or_insert

* really use lookup or insert

* use cuckoo hashing

* add rehash

* use power of 2 and mask instead of modulo

* mix instead of multiplication

* use different mixes

* change multipliers

* use double hashing

* remove xor folding

* Fix allocation non-overflow precondition

* Set the default load factor

* Inline hash_rehash()

* update comments

* Leave overflow checking to R_alloc

* internal_error() is not covered

---------

Co-authored-by: Ivan K <krylov.r00t@gmail.com>

* replace dhashtab with hashtab in rbindlist

* Use hashtab in forder()

Since range_str() runs a parallel OpenMP loop that may update the hash
table in a critical section, use a special form of hash_set that returns
the newly reallocated hash table instead of overwriting it in place.

* Drop dhashtab

* rbindlist: initial hash size = upperBoundUniqueNames

* Don't bother cleaning the hash before an error

* Avoid setting same key&value twice

* chmatch: hash x instead of table (#7454)

* add hash x branch in chmatch

* adapt kick-in threshold

* make chin branch more explicit

* Use linear probing instead of double hashing (#7455)

* use linear probing instead of double hashing

* remove mask from struct

* fix comment

* rbindlist(): better initial allocation size

Also adjust the minimal hash table size to avoid shift overflow on
size=0 and free=0 on size=1.

* dogroups: replace remaining uses of SETLENGTH

* hashtab: switch to C allocator to avoid longjumps

PROTECT() the corresponding EXTPTRSXP while used.

Introduce a separate hash_set_shared() operation that avoids long jumps.
Deallocate the previous hash table when growing a non-shared hash table.

* range_str: use hash_set_shared in OpenMP region

Explicitly check the return value and update the shared pointer when
necessary. If a reallocation attempt fails, signal an error when
possible.

* range_str: allocate temporaries on the R heap

Avoid memory leaks from potential long jumps in hash_set().

* memrecycle: allocate temporary on the R heap

This avoids a memory leak in case growVector or hash_set fails.

* chmatchdup: allocate temporaries on the R heap

Prevent memory leak in case hash_set() causes a long jump.

* rbindlist: drop 'uniq'

Turns out it wasn't used.

* rbindlist: use hash_set_shared

Where C-heap allocations also exist, catch and handle potential
allocation failures from the hash table.

* range_str: propagate rehashed marks to caller

* fix stale comment

* NEWS item

Co-authored-by: Benjamin Schwendinger <benjaminschwe@gmail.com>
Co-authored-by: HughParsonage <hugh.parsonage@gmail.com>
Co-authored-by: Jan Gorecki <J.Gorecki@wit.edu.pl>

* glci: update status expectations for R-devel

* copyAsGrowable: use the correct number of arguments

---------

Co-authored-by: Benjamin Schwendinger <benjaminschwe@gmail.com>
Co-authored-by: Benjamin Schwendinger <52290390+ben-schwen@users.noreply.github.com>
Co-authored-by: HughParsonage <hugh.parsonage@gmail.com>
Co-authored-by: Jan Gorecki <J.Gorecki@wit.edu.pl>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants