2
0
mirror of https://github.com/boostorg/redis.git synced 2026-01-19 16:52:08 +00:00

Compare commits

...

5 Commits

Author SHA1 Message Date
Marcelo Zimbres
7ac785100d Fixes narrowing conversion.
NOTE: I had to disable the TLS tests because I shotdown the server I was
running on my domain occase.de. Once this ticket is merged I will open a
new one to fix that and reenable the tests.
2024-03-20 11:28:15 +01:00
Marcelo Zimbres
69bb69001e Some fixes in the article about the costs of async abstractions [skip ci] 2024-03-15 11:31:30 +01:00
Marcelo Zimbres
1378b32c2e Adds endian to the list of dependencies 2024-02-17 21:06:04 +01:00
Marcelo Zimbres
30fcbc9782 Refactors add_hello and adds unit tests. 2024-02-17 13:15:49 +01:00
Marcelo Zimbres
a6b8017fb2 Fixes issue 181. 2024-02-06 21:14:36 +01:00
13 changed files with 207 additions and 35 deletions

View File

@@ -77,6 +77,7 @@ if (BOOST_REDIS_MAIN_PROJECT)
functional
test
json
endian
)
foreach(dep IN LISTS deps)

View File

@@ -693,6 +693,23 @@ https://lists.boost.org/Archives/boost/2023/01/253944.php.
apps need only one connection for their entire application, which
makes the overhead of one ssl-context per connection negligible.
* ([Issue 181](https://github.com/boostorg/redis/issues/181)).
See a detailed description of this bug in
[this](https://github.com/boostorg/redis/issues/181#issuecomment-1913346983)
comment.
* ([Issue 182](https://github.com/boostorg/redis/issues/182)).
Sets `"default"` as the default value of `config::username`. This
makes it simpler to use the `requirepass` configuration in Redis.
* ([Issue 189](https://github.com/boostorg/redis/issues/189)).
Fixes narrowing convertion by using `std::size_t` instead of
`std::uint64_t` for the sizes of bulks and aggregates. The code
relies now on `std::from_chars` returning an error if a value
greater than 32 is received on platforms on which the size
of`std::size_t` is 32.
### Boost 1.84 (First release in Boost)
* Deprecates the `async_receive` overload that takes a response. Users

View File

@@ -17,7 +17,7 @@ here influenced my views on the proposed
(a.k.a. Senders and Receivers), which is likely to become the basis of
networking in upcoming C++ standards.
Although the analysis presented here uses the Redis communication
Although the analysis presented in this article uses the Redis communication
protocol for illustration I expect it to be useful in general since
[RESP3](https://github.com/antirez/RESP3/blob/master/spec.md) shares
many similarities with other widely used protocols such as HTTP.
@@ -98,7 +98,7 @@ This is illustrated in the diagram below
|<---- offset threshold ---->|
| |
"+PONG\r\n:100\r\n+OK\r\n_\r\n+PONG\r\n"
| # Initial message
| # Initial offset
"+PONG\r\n:100\r\n+OK\r\n_\r\n+PONG\r\n"
|<------>| # After 1st message
@@ -110,7 +110,7 @@ This is illustrated in the diagram below
|<--------------------->| # After 3rd message
"+PONG\r\n:100\r\n+OK\r\n_\r\n+PONG\r\n"
|<-------------------------->| # 4th message crosses the threashold
|<-------------------------->| # Threshold crossed after the 4th message
"+PONG\r\n"
| # After rotation
@@ -255,7 +255,7 @@ avoided, this is what worked for Boost.Redis
`try_send_via_dispatch` for a more aggressive optimization).
3. Coalescing of individual requests into a single payload to reduce
the number of necessary writes on the socket,this is only
the number of necessary writes on the socket, this is only
possible because Redis supports pipelining (good protocols
help!).
@@ -282,8 +282,8 @@ avoided, this is what worked for Boost.Redis
`resp3::async_read` is IO-less.
Sometimes it is not possible to avoid asynchronous operations that
complete synchronously, in the following sections we will therefore
see how favoring throughput over fairness works in Boost.Asio.
complete synchronously, in the following sections we will see how to
favor throughput over fairness in Boost.Asio.
### Calling the continuation inline
@@ -299,7 +299,7 @@ async_read_until(socket, buffer, "\r\n", continuation);
// Immediate completions are executed in exec2 (otherwise equal to the
// version above). The completion is called inline if exec2 is the
same // executor that is running the operation.
// same executor that is running the operation.
async_read_until(socket, buffer, "\r\n", bind_immediate_executor(exec2, completion));
```
@@ -388,7 +388,7 @@ Although faster, this strategy has some downsides
- Requires additional layers of complexity such as
`bind_immediate_executor` in addition to `bind_executor`.
- Not compliat with more strict
- Non-compliat with more strict
[guidelines](https://en.wikipedia.org/wiki/The_Power_of_10:_Rules_for_Developing_Safety-Critical_Code)
that prohibits reentrat code.
@@ -432,7 +432,7 @@ instructing the asynchronous operation to call the completion inline
on immediate completion. It turns out however that coroutine support
for _tail-calls_ provide a way to completely sidestep this problem.
This feature is described by
[Backer](https://lewissbaker.github.io/2020/05/11/understanding_symmetric_transfer)
[Lewis Baker](https://lewissbaker.github.io/2020/05/11/understanding_symmetric_transfer)
as follows
> A tail-call is one where the current stack-frame is popped before
@@ -581,7 +581,7 @@ reentracy, allowing
[sixteen](https://github.com/NVIDIA/stdexec/blob/83cdb92d316e8b3bca1357e2cf49fc39e9bed403/include/exec/trampoline_scheduler.hpp#L52)
levels of inline calls by default. While in Boost.Asio it is possible to use
reentracy as an optimization for a corner cases, here it is made its
_modus operandi_, my opinion about this has already been stated in a
_modus operandi_, the downsides of this approach have already been stated in a
previous section so I won't repeat it here.
Also the fact that a special scheduler is needed by specific

View File

@@ -38,7 +38,7 @@ struct config {
* [HELLO](https://redis.io/commands/hello/) command. If left
* empty `HELLO` will be sent without authentication parameters.
*/
std::string username;
std::string username = "default";
/** @brief Password passed to the
* [HELLO](https://redis.io/commands/hello/) command. If left

View File

@@ -506,6 +506,9 @@ public:
usage get_usage() const noexcept
{ return usage_; }
auto run_is_canceled() const noexcept
{ return cancel_run_called_; }
private:
using receive_channel_type = asio::experimental::channel<executor_type, void(system::error_code, std::size_t)>;
using runner_type = runner<executor_type>;
@@ -539,6 +542,7 @@ private:
});
reqs_.erase(point, std::end(reqs_));
std::for_each(std::begin(reqs_), std::end(reqs_), [](auto const& ptr) {
return ptr->mark_waiting();
});
@@ -575,6 +579,12 @@ private:
} break;
case operation::run:
{
// Protects the code below from being called more than
// once, see https://github.com/boostorg/redis/issues/181
if (std::exchange(cancel_run_called_, true)) {
return;
}
close();
writer_timer_.cancel();
receive_channel_.cancel();
@@ -601,8 +611,9 @@ private:
// partition of unwritten requests instead of them all.
std::for_each(std::begin(reqs_), std::end(reqs_), [](auto const& ptr) {
BOOST_ASSERT_MSG(ptr != nullptr, "Expects non-null pointer.");
if (ptr->is_staged())
if (ptr->is_staged()) {
ptr->mark_written();
}
});
}
@@ -921,6 +932,7 @@ private:
read_buffer_.clear();
parser_.reset();
on_push_ = false;
cancel_run_called_ = false;
}
asio::ssl::context ctx_;
@@ -942,6 +954,7 @@ private:
reqs_type reqs_;
resp3::parser parser_{};
bool on_push_ = false;
bool cancel_run_called_ = false;
usage usage_;
};

View File

@@ -30,6 +30,8 @@
namespace boost::redis::detail
{
void push_hello(config const& cfg, request& req);
template <class Runner, class Connection, class Logger>
struct hello_op {
Runner* runner_ = nullptr;
@@ -42,9 +44,6 @@ struct hello_op {
{
BOOST_ASIO_CORO_REENTER (coro_)
{
runner_->hello_req_.clear();
if (runner_->hello_resp_.has_value())
runner_->hello_resp_.value().clear();
runner_->add_hello();
BOOST_ASIO_CORO_YIELD
@@ -232,17 +231,10 @@ private:
void add_hello()
{
if (!cfg_.username.empty() && !cfg_.password.empty() && !cfg_.clientname.empty())
hello_req_.push("HELLO", "3", "AUTH", cfg_.username, cfg_.password, "SETNAME", cfg_.clientname);
else if (cfg_.username.empty() && cfg_.password.empty() && cfg_.clientname.empty())
hello_req_.push("HELLO", "3");
else if (cfg_.clientname.empty())
hello_req_.push("HELLO", "3", "AUTH", cfg_.username, cfg_.password);
else
hello_req_.push("HELLO", "3", "SETNAME", cfg_.clientname);
if (cfg_.database_index && cfg_.database_index.value() != 0)
hello_req_.push("SELECT", cfg_.database_index.value());
hello_req_.clear();
if (hello_resp_.has_value())
hello_resp_.value().clear();
push_hello(cfg_, hello_req_);
}
bool has_error_in_response() const noexcept

View File

@@ -0,0 +1,27 @@
/* Copyright (c) 2018-2024 Marcelo Zimbres Silva (mzimbres@gmail.com)
*
* Distributed under the Boost Software License, Version 1.0. (See
* accompanying file LICENSE.txt)
*/
#include <boost/redis/detail/runner.hpp>
namespace boost::redis::detail
{
void push_hello(config const& cfg, request& req)
{
if (!cfg.username.empty() && !cfg.password.empty() && !cfg.clientname.empty())
req.push("HELLO", "3", "AUTH", cfg.username, cfg.password, "SETNAME", cfg.clientname);
else if (cfg.password.empty() && cfg.clientname.empty())
req.push("HELLO", "3");
else if (cfg.clientname.empty())
req.push("HELLO", "3", "AUTH", cfg.username, cfg.password);
else
req.push("HELLO", "3", "SETNAME", cfg.clientname);
if (cfg.database_index && cfg.database_index.value() != 0)
req.push("SELECT", cfg.database_index.value());
}
} // boost::redis::detail

View File

@@ -1,4 +1,4 @@
/* Copyright (c) 2018-2023 Marcelo Zimbres Silva (mzimbres@gmail.com)
/* Copyright (c) 2018-2024 Marcelo Zimbres Silva (mzimbres@gmail.com)
*
* Distributed under the Boost Software License, Version 1.0. (See
* accompanying file LICENSE.txt)
@@ -13,7 +13,7 @@
namespace boost::redis::resp3 {
void to_int(int_type& i, std::string_view sv, system::error_code& ec)
void to_int(std::size_t& i, std::string_view sv, system::error_code& ec)
{
auto const res = std::from_chars(sv.data(), sv.data() + std::size(sv), i);
if (res.ec != std::errc())
@@ -29,7 +29,7 @@ void parser::reset()
{
depth_ = 0;
sizes_ = {{1}};
bulk_length_ = (std::numeric_limits<unsigned long>::max)();
bulk_length_ = (std::numeric_limits<std::size_t>::max)();
bulk_ = type::invalid;
consumed_ = 0;
sizes_[0] = 2; // The sentinel must be more than 1.
@@ -189,7 +189,7 @@ parser::consume_impl(
case type::attribute:
case type::map:
{
int_type l = -1;
std::size_t l = -1;
to_int(l, elem, ec);
if (ec)
return {};

View File

@@ -1,4 +1,4 @@
/* Copyright (c) 2018-2023 Marcelo Zimbres Silva (mzimbres@gmail.com)
/* Copyright (c) 2018-2024 Marcelo Zimbres Silva (mzimbres@gmail.com)
*
* Distributed under the Boost Software License, Version 1.0. (See
* accompanying file LICENSE.txt)
@@ -16,8 +16,6 @@
namespace boost::redis::resp3 {
using int_type = std::uint64_t;
class parser {
public:
using node_type = basic_node<std::string_view>;
@@ -38,7 +36,7 @@ private:
std::array<std::size_t, max_embedded_depth + 1> sizes_;
// Contains the length expected in the next bulk read.
int_type bulk_length_;
std::size_t bulk_length_;
// The type of the next bulk. Contains type::invalid if no bulk is
// expected.

View File

@@ -10,6 +10,7 @@
#include <boost/redis/impl/ignore.ipp>
#include <boost/redis/impl/connection.ipp>
#include <boost/redis/impl/response.ipp>
#include <boost/redis/impl/runner.ipp>
#include <boost/redis/resp3/impl/type.ipp>
#include <boost/redis/resp3/impl/parser.ipp>
#include <boost/redis/resp3/impl/serialization.ipp>

View File

@@ -30,7 +30,8 @@ macro(make_test TEST_NAME STANDARD)
endmacro()
make_test(test_conn_quit 17)
make_test(test_conn_tls 17)
# TODO: Configure a Redis server with TLS in the CI and reenable this test.
#make_test(test_conn_tls 17)
make_test(test_low_level 17)
make_test(test_conn_exec_retry 17)
make_test(test_conn_exec_error 17)
@@ -47,6 +48,7 @@ make_test(test_conn_exec_cancel2 20)
make_test(test_conn_echo_stress 20)
make_test(test_conn_run_cancel 20)
make_test(test_issue_50 20)
make_test(test_issue_181 17)
# Coverage
set(

64
test/test_issue_181.cpp Normal file
View File

@@ -0,0 +1,64 @@
/* Copyright (c) 2018-2024 Marcelo Zimbres Silva (mzimbres@gmail.com)
*
* Distributed under the Boost Software License, Version 1.0. (See
* accompanying file LICENSE.txt)
*/
#include <boost/redis/connection.hpp>
#include <boost/redis/logger.hpp>
#include <boost/asio/awaitable.hpp>
#include <boost/asio/use_awaitable.hpp>
#define BOOST_TEST_MODULE conn-quit
#include <boost/test/included/unit_test.hpp>
#include <chrono>
#include <iostream>
#include <memory>
#include <optional>
#include <string>
#include "common.hpp"
namespace net = boost::asio;
using boost::redis::request;
using boost::redis::request;
using boost::redis::response;
using boost::redis::ignore;
using boost::redis::logger;
using boost::redis::config;
using boost::redis::operation;
using boost::redis::connection;
using boost::system::error_code;
using namespace std::chrono_literals;
BOOST_AUTO_TEST_CASE(issue_181)
{
using connection_base = boost::redis::detail::connection_base<net::any_io_executor>;
auto const level = boost::redis::logger::level::debug;
net::io_context ioc;
auto ctx = net::ssl::context{net::ssl::context::tlsv12_client};
connection_base conn{ioc.get_executor(), std::move(ctx), 1000000};
net::steady_timer timer{ioc};
timer.expires_after(std::chrono::seconds{1});
auto run_cont = [&](auto ec){
std::cout << "async_run1: " << ec.message() << std::endl;
};
auto cfg = make_test_config();
cfg.health_check_interval = std::chrono::seconds{0};
cfg.reconnect_wait_interval = std::chrono::seconds{0};
conn.async_run(cfg, boost::redis::logger{level}, run_cont);
BOOST_TEST(!conn.run_is_canceled());
// Uses a timer to wait some time until run has been called.
auto timer_cont = [&](auto ec){
std::cout << "timer_cont: " << ec.message() << std::endl;
BOOST_TEST(!conn.run_is_canceled());
conn.cancel(operation::run);
BOOST_TEST(conn.run_is_canceled());
};
timer.async_wait(timer_cont);
ioc.run();
}

View File

@@ -4,6 +4,7 @@
* accompanying file LICENSE.txt)
*/
#include <boost/redis/detail/runner.hpp>
#include <boost/redis/resp3/serialization.hpp>
#include <boost/redis/adapter/adapt.hpp>
#define BOOST_TEST_MODULE conn-quit
@@ -11,6 +12,9 @@
#include <string>
#include <iostream>
using boost::redis::request;
using boost::redis::config;
using boost::redis::detail::push_hello;
using boost::redis::adapter::adapt2;
using boost::redis::adapter::result;
using boost::redis::resp3::detail::deserialize;
@@ -31,3 +35,56 @@ BOOST_AUTO_TEST_CASE(low_level_sync_sans_io)
exit(EXIT_FAILURE);
}
}
BOOST_AUTO_TEST_CASE(config_to_hello)
{
config cfg;
cfg.clientname = "";
request req;
push_hello(cfg, req);
std::string_view const expected = "*2\r\n$5\r\nHELLO\r\n$1\r\n3\r\n";
BOOST_CHECK_EQUAL(req.payload(), expected);
}
BOOST_AUTO_TEST_CASE(config_to_hello_with_select)
{
config cfg;
cfg.clientname = "";
cfg.database_index = 10;
request req;
push_hello(cfg, req);
std::string_view const expected =
"*2\r\n$5\r\nHELLO\r\n$1\r\n3\r\n"
"*2\r\n$6\r\nSELECT\r\n$2\r\n10\r\n";
BOOST_CHECK_EQUAL(req.payload(), expected);
}
BOOST_AUTO_TEST_CASE(config_to_hello_cmd_clientname)
{
config cfg;
request req;
push_hello(cfg, req);
std::string_view const expected = "*4\r\n$5\r\nHELLO\r\n$1\r\n3\r\n$7\r\nSETNAME\r\n$11\r\nBoost.Redis\r\n";
BOOST_CHECK_EQUAL(req.payload(), expected);
}
BOOST_AUTO_TEST_CASE(config_to_hello_cmd_auth)
{
config cfg;
cfg.clientname = "";
cfg.username = "foo";
cfg.password = "bar";
request req;
push_hello(cfg, req);
std::string_view const expected = "*5\r\n$5\r\nHELLO\r\n$1\r\n3\r\n$4\r\nAUTH\r\n$3\r\nfoo\r\n$3\r\nbar\r\n";
BOOST_CHECK_EQUAL(req.payload(), expected);
}