boost/redis
2
0
Fork 0
mirror of https://github.com/boostorg/redis.git synced 2026-01-19 04:42:09 +00:00
Code Issues Projects Releases Wiki Activity

Fixes documentation.

Browse Source
This commit is contained in:
Marcelo Zimbres
2022-07-16 13:31:47 +02:00
parent 59b5d35672
commit ffc4230368
6 changed files with 70 additions and 240 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
Makefile.am
View File

@@ -70,7 +70,6 @@ nobase_include_HEADERS =\
$(top_srcdir)/aedis/resp3/detail/parser.hpp\
$(top_srcdir)/aedis/resp3/type.hpp\
$(top_srcdir)/aedis/resp3/read.hpp\
$(top_srcdir)/aedis/resp3/exec.hpp\
$(top_srcdir)/aedis/resp3/write.hpp\
$(top_srcdir)/aedis/resp3/request.hpp\
$(top_srcdir)/aedis/resp3/detail/impl/parser.ipp\
@@ -88,13 +87,15 @@ EXTRA_DIST += $(top_srcdir)/doc/DoxygenLayout.xml
EXTRA_DIST += $(top_srcdir)/doc/aedis.css
EXTRA_DIST += $(top_srcdir)/doc/htmlfooter.html
EXTRA_DIST += $(top_srcdir)/doc/htmlheader.html
EXTRA_DIST += $(top_srcdir)/benchmarks/benchmarks.md
EXTRA_DIST += $(top_srcdir)/benchmarks/benchmarks.tex
EXTRA_DIST += $(top_srcdir)/benchmarks/c/libuv/echo_server_direct.c
EXTRA_DIST += $(top_srcdir)/benchmarks/c/libuv/README.md
EXTRA_DIST += $(top_srcdir)/benchmarks/go/echo_server_direct.go
EXTRA_DIST += $(top_srcdir)/benchmarks/nodejs/echo_server_direct.js
EXTRA_DIST += $(top_srcdir)/benchmarks/nodejs/echo_server_over_redis.js
EXTRA_DIST += $(top_srcdir)/benchmarks/nodejs/package.json
EXTRA_DIST += $(top_srcdir)/benchmarks/nodejs/package-lock.json
EXTRA_DIST += $(top_srcdir)/benchmarks/nodejs/echo_server_direct/echo_server_direct.js
EXTRA_DIST += $(top_srcdir)/benchmarks/nodejs/echo_server_direct/package.json
EXTRA_DIST += $(top_srcdir)/benchmarks/nodejs/echo_server_over_redis/echo_server_over_redis.js
EXTRA_DIST += $(top_srcdir)/benchmarks/nodejs/echo_server_over_redis/package.json
EXTRA_DIST += $(top_srcdir)/benchmarks/rust/echo_server_direct/Cargo.toml
EXTRA_DIST += $(top_srcdir)/benchmarks/rust/echo_server_direct/src/main.rs
EXTRA_DIST += $(top_srcdir)/benchmarks/rust/echo_server_over_redis/Cargo.toml
@@ -102,7 +103,6 @@ EXTRA_DIST += $(top_srcdir)/benchmarks/rust/echo_server_over_redis/src/main.rs
.PHONY: doc
doc:
rm -rf ../aedis-gh-pages/*
doxygen doc/Doxyfile
.PHONY: bench

66
aedis/aedis.hpp
View File

@@ -13,7 +13,6 @@
#include <aedis/connection.hpp>
#include <aedis/resp3/read.hpp>
#include <aedis/resp3/write.hpp>
#include <aedis/resp3/exec.hpp>
#include <aedis/resp3/request.hpp>
#include <aedis/adapter/adapt.hpp>
@@ -66,7 +65,7 @@
\endcode
For a detailed comparison of Redis clients and the design
rationale behind Aedis jump to \ref why-aedis.
rationale behind Aedis jump to \ref why-aedis. For benchmarks see [](https://github.com/mzimbres/aedis/blob/master/benchmarks/benchmarks.md)
\section requests Requests
@@ -80,7 +79,7 @@
// Command with variable length of arguments.
req.push("SET", "key", "some value", value, "EX", "2");
// Pushes a set.
// Pushes a list.
std::list<std::string> list
{"channel1", "channel2", "channel3"};
req.push_range("SUBSCRIBE", list);
@@ -176,34 +175,32 @@
subset of the RESP3 specification. Now let us see some examples
@code
auto dbuffer = dynamic_buffer(buffer);
// To ignore the response.
co_await resp3::async_read(socket, dbuffer, adapt());
co_await db->async_exec(req, adapt());
// Read in a std::string e.g. get.
std::string str;
co_await resp3::async_read(socket, dbuffer, adapt(str));
co_await db->async_exec(req, adapt(resp));
// Read in a long long e.g. rpush.
long long number;
co_await resp3::async_read(socket, dbuffer, adapt(number));
long long resp;
co_await db->async_exec(req, adapt(resp));
// Read in a std::set e.g. smembers.
std::set<T, U> set;
co_await resp3::async_read(socket, dbuffer, adapt(set));
std::set<T, U> resp;
co_await db->async_exec(req, adapt(resp));
// Read in a std::map e.g. hgetall.
std::map<T, U> set;
co_await resp3::async_read(socket, dbuffer, adapt(map));
std::map<T, U> resp;
co_await db->async_exec(req, adapt(resp));
// Read in a std::unordered_map e.g. hgetall.
std::unordered_map<T, U> umap;
co_await resp3::async_read(socket, dbuffer, adapt(umap));
std::unordered_map<T, U> resp;
co_await db->async_exec(req, adapt(resp));
// Read in a std::vector e.g. lrange.
std::vector<T> vec;
co_await resp3::async_read(socket, dbuffer, adapt(vec));
std::vector<T> resp;
co_await db->async_exec(req, adapt(resp));
@endcode
In other words, it is straightforward, just pass the result of \c
@@ -219,8 +216,8 @@
wrap your type around \c boost::optional like this
@code
boost::optional<std::unordered_map<T, U>> umap;
co_await resp3::async_read(socket, dynamic_buffer(buffer), adapt(umap));
boost::optional<std::unordered_map<T, U>> resp;
co_await db->async_exec(req, adapt(resp));
@endcode
Everything else stays the same, before accessing data, users will
@@ -261,17 +258,22 @@
can be read in the following way
@code
std::tuple<
boost::optional<std::string>, // Response to get
boost::optional<std::vector<std::string>>, // Response to lrange
boost::optional<std::map<std::string, std::string>> // Response to hgetall
> trans;
using trans_type =
std::tuple<
boost::optional<std::string>, // get
boost::optional<std::vector<std::string>>, // lrange
boost::optional<std::map<std::string, std::string>> // hgetall
>;
co_await resp3::async_read(socket, dynamic_buffer(buffer)); // Ignore multi
co_await resp3::async_read(socket, dynamic_buffer(buffer)); // Ignore get
co_await resp3::async_read(socket, dynamic_buffer(buffer)); // Ignore lrange
co_await resp3::async_read(socket, dynamic_buffer(buffer)); // Ignore hgetall
co_await resp3::async_read(socket, dynamic_buffer(buffer), adapt(trans));
std::tuple<
aedis::ignore, // multi
aedis::ignore, // get
aedis::ignore, // lrange
aedis::ignore, // hgetall
trans_type, // exec
> resp;
co_await db->async_exec(req, adapt(resp));
@endcode
Note that above we are not ignoring the response to the commands
@@ -317,7 +319,7 @@
\subsection gen-case The general case
As already mentioned, there are cases where responses to Redis
There are cases where responses to Redis
commands won't fit in the model presented above, some examples are
@li Commands (like \c set) whose response don't have a fixed
@@ -356,11 +358,11 @@
@code
// Receives any RESP3 simple data type.
node<std::string> resp;
co_await resp3::async_read(socket, dynamic_buffer(buffer), adapt(resp));
co_await db->async_exec(req, adapt(resp));
// Receives any RESP3 simple or aggregate data type.
std::vector<node<std::string>> resp;
co_await resp3::async_read(socket, dynamic_buffer(buffer), adapt(resp));
co_await db->async_exec(req, adapt(resp));
@endcode
For example, suppose we want to retrieve a hash data structure

1
aedis/detail/connection_ops.hpp
View File

@@ -22,7 +22,6 @@
#include <aedis/resp3/type.hpp>
#include <aedis/resp3/detail/parser.hpp>
#include <aedis/resp3/read.hpp>
#include <aedis/resp3/exec.hpp>
#include <aedis/resp3/write.hpp>
#include <aedis/resp3/request.hpp>

176
aedis/resp3/exec.hpp
View File

@@ -1,176 +0,0 @@
/* Copyright (c) 2018-2022 Marcelo Zimbres Silva (mzimbres@gmail.com)
*
* Distributed under the Boost Software License, Version 1.0. (See
* accompanying file LICENSE.txt)
*/
#ifndef AEDIS_RESP3_EXEC_HPP
#define AEDIS_RESP3_EXEC_HPP
#include <boost/assert.hpp>
#include <boost/asio/ip/tcp.hpp>
#include <boost/asio/read.hpp>
#include <boost/asio/write.hpp>
#include <boost/asio/coroutine.hpp>
#include <boost/asio/compose.hpp>
#include <boost/asio/steady_timer.hpp>
#include <boost/asio/experimental/parallel_group.hpp>
#include <aedis/error.hpp>
#include <aedis/resp3/read.hpp>
#include <aedis/resp3/request.hpp>
namespace aedis {
namespace resp3 {
namespace detail {
#include <boost/asio/yield.hpp>
template <
class AsyncStream,
class Adapter,
class DynamicBuffer
>
struct exec_op {
AsyncStream* socket;
request const* req;
Adapter adapter;
DynamicBuffer dbuf;
boost::asio::coroutine coro;
template <class Self>
void operator()( Self& self
, boost::system::error_code ec = {}
, std::size_t n = 0)
{
reenter (coro)
{
yield
boost::asio::async_write(
*socket,
boost::asio::buffer(req->payload()),
std::move(self));
if (ec) {
self.complete(ec, 0);
return;
}
yield resp3::async_read(*socket, dbuf, adapter, std::move(self));
self.complete(ec, n);
}
}
};
#include <boost/asio/unyield.hpp>
} // detail
template <
class AsyncStream,
class Adapter,
class DynamicBuffer,
class CompletionToken = boost::asio::default_completion_token_t<typename AsyncStream::executor_type>
>
auto async_exec(
AsyncStream& socket,
request const& req,
Adapter adapter,
DynamicBuffer dbuf,
CompletionToken token = CompletionToken{})
{
return boost::asio::async_compose
< CompletionToken
, void(boost::system::error_code, std::size_t)
>(detail::exec_op<AsyncStream, Adapter, DynamicBuffer>
{&socket, &req, adapter, dbuf}, token, socket);
}
namespace detail {
#include <boost/asio/yield.hpp>
template <
class AsyncStream,
class Adapter,
class DynamicBuffer
>
struct exec_with_timeout_op {
AsyncStream* socket;
boost::asio::steady_timer* timer;
request const* req;
Adapter adapter;
DynamicBuffer dbuf;
boost::asio::coroutine coro;
template <class Self>
void operator()( Self& self
, std::array<std::size_t, 2> order = {}
, boost::system::error_code ec1 = {}
, std::size_t n = 0
, boost::system::error_code ec2 = {})
{
reenter (coro)
{
yield
boost::asio::experimental::make_parallel_group(
[this](auto token) { return resp3::async_exec(*socket, *req, adapter, dbuf, token);},
[this](auto token) { return timer->async_wait(token);}
).async_wait(
boost::asio::experimental::wait_for_one(),
std::move(self));
switch (order[0]) {
case 0:
{
if (ec1) {
self.complete(ec1, 0);
return;
}
} break;
case 1:
{
if (!ec2) {
self.complete(aedis::error::idle_timeout, 0);
return;
}
} break;
default: BOOST_ASSERT(false);
}
self.complete({}, n);
}
}
};
#include <boost/asio/unyield.hpp>
} // detail
template <
class AsyncStream,
class Adapter,
class DynamicBuffer,
class CompletionToken = boost::asio::default_completion_token_t<typename AsyncStream::executor_type>
>
auto async_exec(
AsyncStream& socket,
boost::asio::steady_timer& timer,
request const& req,
Adapter adapter,
DynamicBuffer dbuf,
CompletionToken token = CompletionToken{})
{
return boost::asio::async_compose
< CompletionToken
, void(boost::system::error_code, std::size_t)
>(detail::exec_with_timeout_op<AsyncStream, Adapter, DynamicBuffer>
{&socket, &timer, &req, adapter, dbuf}, token, socket, timer);
}
} // resp3
} // aedis
#endif // AEDIS_RESP3_EXEC_HPP

53
benchmarks/benchmarks.md
View File

@@ -1,20 +1,21 @@
# TCP echo server performance.
This document describe benchmakrs I made in implementations of TCP
echo server I implemented in different languages. The main
motivations for choosing a TCP echo server as a benchmark program are
This document describe benchmarks the performance of TCP echo servers
I implemented in different languages using different Redis clients.
The main motivations for choosing a TCP echo server as a benchmark
program are
* Simple to implement and does not require expertise level in most languages.
* I/O bound: Echo servers have very low CPU consumption in general
and therefore an excelent measure of the ability of a program to
server concurrent requests.
and therefore are excelent as a measure of the ability of a
program to server concurrent requests.
* It simulates very well a typical backend in regard to concurrency.
I also imposed some constraints on the implementations
* It should not require me to write too much code.
* It should be simple enough and not require writing too much code.
* Favor the use standard idioms and avoid optimizations that require expert level.
* Makes no use of complex things like connection and thread pool.
* Avoid the use of complex things like connection and thread pool.
## No Redis
@@ -24,9 +25,10 @@ be seen below
![](https://mzimbres.github.io/aedis/tcp-echo-direct.png)
The tests were performed with 1000 TCP connection on the localhost
where latency is 0.07ms on average. On higher latency networks the
difference among libraries is expected to decrease.
The tests were performed with a 1000 concurrent TCP connection on the
localhost where latency is 0.07ms on average on my machine. On higher
latency networks the difference among libraries is expected to
decrease.
### Remarks:
@@ -34,7 +36,7 @@ difference among libraries is expected to decrease.
* I did expect nodejs to come a little behind given it is is
javascript code. Otherwise I did expect it to have similar
performance to libuv since it is the framework behind it.
* The go performance was no surprise: decent and not some much far behind nodejs.
* Go performance did not surprise me: decent and not some much far behind nodejs.
The code used in the benchmarks can be found at
@@ -46,38 +48,41 @@ The code used in the benchmarks can be found at
## Echo over Redis
This is similar to the echo server described above but the message is
echoed by Redis. The echo server works as a proxy between the
client and the Redis server. The result can be seen below
This is similar to the echo server described above but messages are
echoed by Redis and not by the echo-server itself, which acts
as a proxy between the client and the Redis server. The result
can be seen below
![](https://mzimbres.github.io/aedis/tcp-echo-over-redis.png)
The tests were also performed with 1000 TCP connections on a network
latency is 35ms on average.
The tests were performed on a network where latency is 35ms on
average, otherwise it is equal to the benchmarks above regarding the
number of TCP connection. The result can be seen below
### Remarks
As the reader can see, the Libuv and the Rust test are not depicted
above, reasons are
in the graph, the reasons are
* [redis-rs](https://github.com/redis-rs/redis-rs): This client
comes so far behind that it can't even be represented together
with the other benchmarks without making them insignificant. I
don't know for sure why it is so slow, I suppose however it has
with the other benchmarks without making them look insignificant. I
don't know for sure why it is so slow, I suppose it has
something to do with its lack of proper
[pipelining](https://redis.io/docs/manual/pipelining/) support.
In fact, the more TCP connections I lauch the worst its
performance gets.
* Libuv: I left it out because it would require too much work to
make it have a good performance. More specifically, I would have
to use hiredis and implement support for pipelines manually.
write it and make it have a good performance. More specifically,
I would have to use hiredis and implement support for pipelines
manually.
The code used in the benchmarks can be found at
* [Aedis](https://github.com/mzimbres/aedis): [Code](https://github.com/mzimbres/aedis/blob/3fb018ccc6138d310ac8b73540391cdd8f2fdad6/examples/echo_server.cpp)
* [node-redis](https://github.com/redis/node-redis): [Code](https://github.com/mzimbres/aedis/tree/3fb018ccc6138d310ac8b73540391cdd8f2fdad6/benchmarks/nodejs/echo_server_over_redis)
* [go-redis](https://github.com/go-redis/redis): [Go](https://github.com/mzimbres/aedis/blob/3fb018ccc6138d310ac8b73540391cdd8f2fdad6/benchmarks/go/echo_server_over_redis.go)
* [Aedis](https://github.com/mzimbres/aedis): [code](https://github.com/mzimbres/aedis/blob/3fb018ccc6138d310ac8b73540391cdd8f2fdad6/examples/echo_server.cpp)
* [node-redis](https://github.com/redis/node-redis): [code](https://github.com/mzimbres/aedis/tree/3fb018ccc6138d310ac8b73540391cdd8f2fdad6/benchmarks/nodejs/echo_server_over_redis)
* [go-redis](https://github.com/go-redis/redis): [code](https://github.com/mzimbres/aedis/blob/3fb018ccc6138d310ac8b73540391cdd8f2fdad6/benchmarks/go/echo_server_over_redis.go)
## Contributing

2
configure.ac
View File

@@ -1,5 +1,5 @@
AC_PREREQ([2.69])
AC_INIT([Aedis], [0.2.0], [mzimbres@gmail.com])
AC_INIT([Aedis], [0.2.1], [mzimbres@gmail.com])
AC_CONFIG_MACRO_DIR([m4])
#AC_CONFIG_SRCDIR([src/aedis.cpp])
AC_CONFIG_HEADERS([config.h])