redisbench-admin

Redis benchmark run helper can help you with the following tasks:

• Setup abd teardown of benchmarking infrastructure specified on redis-performance/testing-infrastructure
• Setup and teardown of an Redis and Redis Modules DBs for benchmarking
• Management of benchmark data and specifications across different setups
• Running benchmarks and recording results
• Exporting performance results in several formats (CSV, RedisTimeSeries, JSON)
• Finding on-cpu, off-cpu, io, and threading performance problems by attaching profiling tools/probers ( perf (a.k.a. perf_events), bpf tooling, vtune )
• [SOON] Finding performance problems by attaching telemetry probes

Current supported benchmark tools:

• redis-benchmark
• memtier_benchmark
• redis-benchmark-go
• YCSB
• tsbs
• redisgraph-benchmark-go
• ftsb_redisearch
• ann-benchmarks

Remote SSH connections retry transient connection failures (e.g. a refused connection or a brief host outage) up to SSH_CONNECT_MAX_RETRIES times (default 5), waiting 1, 2, 4, ... seconds between attempts. Each attempt is bounded by SSH_CONNECT_TIMEOUT seconds (default 30), so an unreachable host fails after about 3.5 minutes with the defaults; refused connections fail immediately and only pay the backoff (31 seconds). Only opening the connection is retried, before any remote command runs, so setup steps and benchmark measurements are never replayed. Authentication errors are not retried. Fetching result files retries transient SSH/SFTP errors the same way, up to SSH_FETCH_MAX_RETRIES times (default 3).

Installation

Installation is done using pip, the package installer for Python, in the following manner:

python3 -m pip install redisbench-admin

Profiler daemon

You can use the profiler daemon by itself in the following manner. On the target machine do as follow:

pip3 install --upgrade pip
pip3 install redisbench-admin --ignore-installed PyYAML

# install perf
apt install linux-tools-common linux-tools-generic linux-tools-`uname -r` -y

# ensure perf is working
perf --version

# install awscli
snap install aws-cli --classic


# configure aws
aws configure

# start the perf-daemon
perf-daemon start
WARNING:root:Unable to detected github_actor. caught the following error: No section: 'user'
Writting log to /tmp/perf-daemon.log
Starting perf-daemon. PID file /tmp/perfdaemon.pid. Daemon workdir: /root/RedisGraph

# check daemon is working appropriatelly
curl localhost:5000/ping

# start a profile
curl -X POST localhost:5000/profiler/perf/start/<pid to profile>

# stop a profile
curl -X POST -d '{"aws_access_key_id":$AWS_ACCESS_KEY_ID,"aws_secret_access_key":$AWS_SECRET_ACCESS_KEY}' localhost:5000/profiler/perf/stop/<pid to profile>

Development

1. Install pypoetry to manage your dependencies and trigger tooling.

pip install poetry

2. Installing dependencies from lock file

poetry install

Running formaters

poetry run black .

Running linters

poetry run flake8

Running tests

A test suite is provided, and can be run with:

$ tox

To run a specific test:

$ tox -- tests/test_defaults_purpose_built_env.py

To run a specific test and persist the docker container used for timeseries:

tox  --docker-dont-stop=rts_datasink  -- -vv --log-cli-level=INFO  tests/test_defaults_purpose_built_env.py 

To run a specific test with verbose logging:

# tox -- -vv --log-cli-level=INFO tests/test_run.py

License

redisbench-admin is distributed under the BSD3 license - see LICENSE