DARE -- Direct Access REplication
=================================

Brief: A wait-free direct access protocol that uses RDMA features such as QP disconnect and QP timeouts, to ensure highest performance and reliability.

Build DARE
----------
1. Prerequisites
    - libev (version >= 4.15)
    - libibverbs (version >= 1.0.0)
2. ./configure --help
Script for building DARE
usage  : $0 [options]
options: [--prefix=DIR]                # Installation directory
         [--with-ev=DIR]               # libev installation directory: /usr/local/
         [--with-ibv=DIR]              # libibverbs installation directory
3. make

Running benchmarks (from [1])
-----------------------------
1. Latency benchmark (i.e., a single client sends requests (according to a trace) to a DARE group)
./benchmarks/latency_bench.sh --help
Script for starting DARE's latency benchmark
usage  : $0 [options]
options: [--dare=DIR]               # path to DARE bin and lib
         [--size=INT]               # server count [default 3]
         [--op=put|get]             # operation type [default put]

2. Throughput benchmark (i.e., multiple clients send requests (according to a trace) to a DARE group)
./benchmarks/throughput_bench.sh --help
Script for starting DARE's throughput benchmark
usage  : $0 [options]
options: --dare=DIR           # path to DARE bin and lib
         [--scount=INT]       # server count [default 3]
         [--ccount=INT]       # client count [default 1]   
         [--op=put|get]       # operation type [default put]
         [--bsize=(8-1024)]   # blob size [default 64]   
         [--proc=(0-100)]     # percentage of op operation [default 100]

3. Group reconfiguration benchmark (i.e., DARE’s write throughput during group reconfiguration)
./benchmarks/reconf_bench.sh --help
Script for starting DARE's group reconfiguration benchmark
usage  : $0 [options]
options: --dare=DIR           # path to DARE bin and lib
         [--op=put|get]       # operation type [default put]
         [--bsize=(8-1024)]   # blob size [default 64]

Note: The benchmarks generate data into $CWD/data, where $CWD is the folder from where the scripts are launched. 

Folder overview
---------------
./bin -- binaries
./lib -- libraries
./src -- source files
./include -- header files
./trace -- trace generator
./test -- main functions for server/client
./utils 
    - rbtree -- red-black Trees implementation (Linux)
    - libev-4.15.tar.gz 
 
[1]  M. Poke, T. Hoefler: DARE: High-Performance State Machine Replication on RDMA Networks. 
In Proceedings of the 24th ACM International Symposium on High-Performance Parallel and 
Distributed Computing (HPDC’15)

./kvs_trace:
--loop: generate only two requests, one is put and the other is get. Their size is the same as what "-s" designates.
--trace: when "-s" is 1024, it will generate request of size 8, 16, 32, 64, 128 ... until 1024
default --loop, not --trace

for example: ./kvs_trace --loop --put -s 128 -o throughput
./kvs_trace -s 128 -o throughput

Please remember that if the leader bathces 6 write requests (which are sent from 6 clients) and every request is 128 bytes in size
then it could possibly exceed RDMA maximum transport size.

Remember to #define or #undef in both dare_server.c and dare_ibv_rc.c

--------------------------------------------------------------------------------------------------------------------------
1. Event Order
Do the following things in two files: dare_server.c, dare_ibv_rc.c
只保留TEST_POST_SEND_INTERVAL和RDTSC的define，将其他的都undef掉

2. ACK of D
Do the following things in two files: dare_server.c, dare_ibv_rc.c
只保留TEST_CONSENSUS_LATENCY_NEW和RDTSC的define，将其他的都undef掉