boost/redis
2
0
Fork 0
mirror of https://github.com/boostorg/redis.git synced 2026-01-19 04:42:09 +00:00
Code Issues Projects Releases Wiki Activity

Adds support for optional fields

Browse Source
This commit is contained in:
Marcelo Zimbres
2025-02-08 21:59:39 +01:00
parent 31ceed9f8f
commit 412f5535ea
8 changed files with 287 additions and 112 deletions
Download Patch File Download Diff File Expand all files Collapse all files

112
README.md
View File

@@ -4,41 +4,39 @@ Boost.Redis is a high-level [Redis](https://redis.io/) client library built on t
[Boost.Asio](https://www.boost.org/doc/libs/release/doc/html/boost_asio.html)
that implements the Redis protocol
[RESP3](https://github.com/redis/redis-specifications/blob/master/protocol/RESP3.md).
The requirements for using Boost.Redis are:
The requirements for using Boost.Redis are
* Boost. The library is included in Boost distributions starting with 1.84.
* Boost 1.84 or higher.
* C++17 or higher.
* Redis 6 or higher (must support RESP3).
* Gcc (11, 12), Clang (11, 13, 14) and Visual Studio (16 2019, 17 2022).
* GCC (11, 12), Clang (11, 13, 14) and Visual Studio (16 2019, 17 2022).
* Have basic-level knowledge about [Redis](https://redis.io/docs/)
and [Boost.Asio](https://www.boost.org/doc/libs/1_82_0/doc/html/boost_asio/overview.html).
The latest release can be downloaded on
https://github.com/boostorg/redis/releases. The library headers can be
found in the `include` subdirectory and a compilation of the source
To use the library it is necessary to include
```cpp
#include <boost/redis/src.hpp>
```
is required. The simplest way to do it is to included this header in
no more than one source file in your applications. To build the
examples and tests cmake is supported, for example
in no more than one source file in your applications. To build the
examples and tests with cmake run
```cpp
# Linux
$ BOOST_ROOT=/opt/boost_1_84_0 cmake --preset g++-11
$ BOOST_ROOT=/opt/boost_1_84_0 cmake -S <source-dir> -B <binary-dir>
# Windows
$ cmake -G "Visual Studio 17 2022" -A x64 -B bin64 -DCMAKE_TOOLCHAIN_FILE=C:/vcpkg/scripts/buildsystems/vcpkg.cmake
```
For more details see https://github.com/boostorg/cmake.
<a name="connection"></a>
## Connection
Let us start with a simple application that uses a short-lived
connection to send a [ping](https://redis.io/commands/ping/) command
to Redis
The code below uses a short-lived connection to
[ping](https://redis.io/commands/ping/) the Redis server
```cpp
auto co_main(config const& cfg) -> net::awaitable<void>
@@ -50,7 +48,7 @@ auto co_main(config const& cfg) -> net::awaitable<void>
request req;
req.push("PING", "Hello world");
// Response where the PONG response will be stored.
// Response object.
response<std::string> resp;
// Executes the request.
@@ -145,21 +143,18 @@ req.push_range("SUBSCRIBE", std::cbegin(list), std::cend(list));
req.push_range("HSET", "key", map);
```
Sending a request to Redis is performed with `boost::redis::connection::async_exec` as already stated.
### Config flags
The `boost::redis::request::config` object inside the request dictates how the
`boost::redis::connection` should handle the request in some important situations. The
reader is advised to read it carefully.
Sending a request to Redis is performed with `boost::redis::connection::async_exec` as already stated. The
`boost::redis::request::config` object inside the request dictates how
the `boost::redis::connection` the request is handled in some
situations. The reader is advised to read it carefully.
<a name="responses"></a>
## Responses
Boost.Redis uses the following strategy to support Redis responses
Boost.Redis uses the following strategy to deal with Redis responses
* `boost::redis::request` is used for requests whose number of commands are not dynamic.
* **Dynamic**: Otherwise use `boost::redis::generic_response`.
* `boost::redis::request` used for requests whose number of commands are not dynamic.
* `boost::redis::generic_response` used when the size is dynamic.
For example, the request below has three commands
@@ -170,8 +165,8 @@ req.push("INCR", "key");
req.push("QUIT");
```
and its response also has three commands and can be read in the
following response object
and therefore its response will also contain three elements which can
be read in the following reponse object
```cpp
response<std::string, int, std::string>
@@ -186,7 +181,7 @@ To ignore responses to individual commands in the request use the tag
```cpp
// Ignore the second and last responses.
response<std::string, boost::redis::ignore_t, std::string, boost::redis::ignore_t>
response<std::string, ignore_t, std::string, ignore_t>
```
The following table provides the resp3-types returned by some Redis
@@ -230,7 +225,7 @@ req.push("QUIT");
```
can be read in the tuple below
can be read in the response object below
```cpp
response<
@@ -243,7 +238,8 @@ response<
> resp;
```
Where both are passed to `async_exec` as showed elsewhere
Then, to execute the request and read the response use `async_exec` as
shown below
```cpp
co_await conn->async_exec(req, resp);
@@ -279,15 +275,13 @@ req.push("SUBSCRIBE", "channel");
req.push("QUIT");
```
must be read in this tuple `response<std::string, std::string>`,
that has static size two.
must be read in the response object `response<std::string, std::string>`.
### Null
It is not uncommon for apps to access keys that do not exist or
that have already expired in the Redis server, to deal with these
cases Boost.Redis provides support for `std::optional`. To use it,
wrap your type around `std::optional` like this
It is not uncommon for apps to access keys that do not exist or that
have already expired in the Redis server, to deal with these usecases
wrap the type with an `std::optional` as shown below
```cpp
response<
@@ -295,11 +289,9 @@ response<
std::optional<B>,
...
> resp;
co_await conn->async_exec(req, resp);
```
Everything else stays pretty much the same.
Everything else stays the same.
### Transactions
@@ -321,22 +313,18 @@ use the following response type
```cpp
using boost::redis::ignore;
using exec_resp_type =
response<
ignore_t, // multi
ignore_t, // QUEUED
ignore_t, // QUEUED
ignore_t, // QUEUED
response<
std::optional<std::string>, // get
std::optional<std::vector<std::string>>, // lrange
std::optional<std::map<std::string, std::string>> // hgetall
>;
response<
boost::redis::ignore_t, // multi
boost::redis::ignore_t, // get
boost::redis::ignore_t, // lrange
boost::redis::ignore_t, // hgetall
exec_resp_type, // exec
> // exec
> resp;
co_await conn->async_exec(req, resp);
```
For a complete example see cpp20_containers.cpp.
@@ -350,7 +338,7 @@ commands won't fit in the model presented above, some examples are
* Commands (like `set`) whose responses don't have a fixed
RESP3 type. Expecting an `int` and receiving a blob-string
will result in error.
results in an error.
* RESP3 aggregates that contain nested aggregates can't be read in STL containers.
* Transactions with a dynamic number of commands can't be read in a `response`.
@@ -411,7 +399,7 @@ the following customization points
void boost_redis_to_bulk(std::string& to, mystruct const& obj);
// Deserialize
void boost_redis_from_bulk(mystruct& obj, char const* p, std::size_t size, boost::system::error_code& ec)
void boost_redis_from_bulk(mystruct& u, node_view const& node, boost::system::error_code&)
```
These functions are accessed over ADL and therefore they must be
@@ -676,6 +664,28 @@ https://lists.boost.org/Archives/boost/2023/01/253944.php.
## Changelog
### Boost 1.88
* (Issue [233](https://github.com/boostorg/redis/issues/233))
To deal with keys that might not exits in the Redis server, the
library supports `std::optional`, for example
`response<std::optional<std::vector<std::string>>>`. In some cases
however, such as the [MGET](https://redis.io/docs/latest/commands/mget/) command,
each element in the vector might be non exiting, now it is possible
to specify a response as `response<std::optional<std::vector<std::optional<std::string>>>>`.
* (Issue [225](https://github.com/boostorg/redis/issues/225))
Use `deferred` as the connection default completion token.
* (Issue [128](https://github.com/boostorg/redis/issues/128))
Adds a new `async_exec` overload that allows passing response
adapters. This makes it possible to receive Redis responses directly
in custom data structures thereby avoiding uncessary data copying.
Thanks to Ruben Perez (@anarthal) for implementing this feature.
* There are also other multiple small improvements in this release,
users can refer to the git history for more details.
### Boost 1.87
* (Issue [205](https://github.com/boostorg/redis/issues/205))

46
example/cpp20_containers.cpp
View File

@@ -1,4 +1,4 @@
/* Copyright (c) 2018-2022 Marcelo Zimbres Silva (mzimbres@gmail.com)
/* Copyright (c) 2018-2025 Marcelo Zimbres Silva (mzimbres@gmail.com)
*
* Distributed under the Boost Software License, Version 1.0. (See
* accompanying file LICENSE.txt)
@@ -24,13 +24,25 @@ using boost::asio::awaitable;
using boost::asio::detached;
using boost::asio::consign;
template<class T>
std::ostream& operator<<(std::ostream& os, std::optional<T> const& opt)
{
if (opt.has_value())
std::cout << opt.value();
else
std::cout << "null";
return os;
}
void print(std::map<std::string, std::string> const& cont)
{
for (auto const& e: cont)
std::cout << e.first << ": " << e.second << "\n";
}
void print(std::vector<int> const& cont)
template <class T>
void print(std::vector<T> const& cont)
{
for (auto const& e: cont) std::cout << e << " ";
std::cout << "\n";
@@ -48,6 +60,7 @@ auto store(std::shared_ptr<connection> conn) -> awaitable<void>
request req;
req.push_range("RPUSH", "rpush-key", vec);
req.push_range("HSET", "hset-key", map);
req.push("SET", "key", "value");
co_await conn->async_exec(req, ignore);
}
@@ -67,6 +80,21 @@ auto hgetall(std::shared_ptr<connection> conn) -> awaitable<void>
print(std::get<0>(resp).value());
}
auto mget(std::shared_ptr<connection> conn) -> awaitable<void>
{
// A request contains multiple commands.
request req;
req.push("MGET", "key", "non-existing-key");
// Responses as tuple elements.
response<std::vector<std::optional<std::string>>> resp;
// Executes the request and reads the response.
co_await conn->async_exec(req, resp);
print(std::get<0>(resp).value());
}
// Retrieves in a transaction.
auto transaction(std::shared_ptr<connection> conn) -> awaitable<void>
{
@@ -74,19 +102,26 @@ auto transaction(std::shared_ptr<connection> conn) -> awaitable<void>
req.push("MULTI");
req.push("LRANGE", "rpush-key", 0, -1); // Retrieves
req.push("HGETALL", "hset-key"); // Retrieves
req.push("MGET", "key", "non-existing-key");
req.push("EXEC");
response<
ignore_t, // multi
ignore_t, // lrange
ignore_t, // hgetall
response<std::optional<std::vector<int>>, std::optional<std::map<std::string, std::string>>> // exec
ignore_t, // mget
response<
std::optional<std::vector<int>>,
std::optional<std::map<std::string, std::string>>,
std::optional<std::vector<std::optional<std::string>>>
> // exec
> resp;
co_await conn->async_exec(req, resp);
print(std::get<0>(std::get<3>(resp).value()).value().value());
print(std::get<1>(std::get<3>(resp).value()).value().value());
print(std::get<0>(std::get<4>(resp).value()).value().value());
print(std::get<1>(std::get<4>(resp).value()).value().value());
print(std::get<2>(std::get<4>(resp).value()).value().value());
}
// Called from the main function (see main.cpp)
@@ -98,6 +133,7 @@ awaitable<void> co_main(config cfg)
co_await store(conn);
co_await transaction(conn);
co_await hgetall(conn);
co_await mget(conn);
conn->cancel();
}

16
example/cpp20_json.cpp
View File

@@ -21,12 +21,14 @@
#include <boost/redis/resp3/serialization.hpp>
namespace asio = boost::asio;
namespace resp3 = boost::redis::resp3;
using namespace boost::describe;
using boost::redis::request;
using boost::redis::response;
using boost::redis::ignore_t;
using boost::redis::config;
using boost::redis::connection;
using boost::redis::resp3::node_view;
// Struct that will be stored in Redis using json serialization.
struct user {
@@ -40,10 +42,18 @@ BOOST_DESCRIBE_STRUCT(user, (), (name, age, country))
// Boost.Redis customization points (example/json.hpp)
void boost_redis_to_bulk(std::string& to, user const& u)
{ boost::redis::resp3::boost_redis_to_bulk(to, boost::json::serialize(boost::json::value_from(u))); }
{
resp3::boost_redis_to_bulk(to, boost::json::serialize(boost::json::value_from(u)));
}
void boost_redis_from_bulk(user& u, std::string_view sv, boost::system::error_code&)
{ u = boost::json::value_to<user>(boost::json::parse(sv)); }
void
boost_redis_from_bulk(
user& u,
node_view const& node,
boost::system::error_code&)
{
u = boost::json::value_to<user>(boost::json::parse(node.value));
}
auto co_main(config cfg) -> asio::awaitable<void>
{

12
example/cpp20_protobuf.cpp
View File

@@ -19,12 +19,14 @@
#if defined(BOOST_ASIO_HAS_CO_AWAIT)
namespace asio = boost::asio;
namespace resp3 = boost::redis::resp3;
using boost::redis::request;
using boost::redis::response;
using boost::redis::operation;
using boost::redis::ignore_t;
using boost::redis::config;
using boost::redis::connection;
using boost::redis::resp3::node_view;
// The protobuf type described in example/person.proto
using tutorial::person;
@@ -42,12 +44,16 @@ void boost_redis_to_bulk(std::string& to, person const& u)
if (!u.SerializeToString(&tmp))
throw boost::system::system_error(boost::redis::error::invalid_data_type);
boost::redis::resp3::boost_redis_to_bulk(to, tmp);
resp3::boost_redis_to_bulk(to, tmp);
}
void boost_redis_from_bulk(person& u, std::string_view sv, boost::system::error_code& ec)
void
boost_redis_from_bulk(
person& u,
node_view const& node,
boost::system::error_code& ec)
{
std::string const tmp {sv};
std::string const tmp {node.value};
if (!u.ParseFromString(tmp))
ec = boost::redis::error::invalid_data_type;
}

169
include/boost/redis/adapter/detail/adapters.hpp
View File

@@ -37,49 +37,120 @@
namespace boost::redis::adapter::detail
{
// Serialization.
template <class> struct is_integral : std::false_type {};
template <class T>
auto boost_redis_from_bulk(T& i, std::string_view sv, system::error_code& ec) -> typename std::enable_if<std::is_integral<T>::value, void>::type
{
auto const res = std::from_chars(sv.data(), sv.data() + std::size(sv), i);
if (res.ec != std::errc())
ec = redis::error::not_a_number;
}
template <> struct is_integral<long long int > : std::true_type {};
template <> struct is_integral<unsigned long long int> : std::true_type {};
template <> struct is_integral<int > : std::true_type {};
inline
void boost_redis_from_bulk(bool& t, std::string_view sv, system::error_code&)
{
t = *sv.data() == 't';
}
template<class T, bool = is_integral<T>::value>
struct converter;
inline
void boost_redis_from_bulk(double& d, std::string_view sv, system::error_code& ec)
{
template<class T>
struct converter<T, true> {
template <class String>
static void
apply(
T& i,
resp3::basic_node<String> const& node,
system::error_code& ec)
{
auto const res =
std::from_chars(node.value.data(), node.value.data() + node.value.size(), i);
if (res.ec != std::errc())
ec = redis::error::not_a_number;
}
};
template<>
struct converter<bool, false> {
template <class String>
static void
apply(
bool& t,
resp3::basic_node<String> const& node,
system::error_code& ec)
{
t = *node.value.data() == 't';
}
};
template<>
struct converter<double, false> {
template <class String>
static void
apply(
double& d,
resp3::basic_node<String> const& node,
system::error_code& ec)
{
#ifdef _LIBCPP_VERSION
// The string in sv is not null terminated and we also don't know
// if there is enough space at the end for a null char. The easiest
// thing to do is to create a temporary.
std::string const tmp{sv.data(), sv.data() + std::size(sv)};
char* end{};
d = std::strtod(tmp.data(), &end);
if (d == HUGE_VAL || d == 0)
ec = redis::error::not_a_double;
// The string in node.value is not null terminated and we also
// don't know if there is enough space at the end for a null
// char. The easiest thing to do is to create a temporary.
std::string const tmp{node.value.data(), node.value.data() + node.value.size()};
char* end{};
d = std::strtod(tmp.data(), &end);
if (d == HUGE_VAL || d == 0)
ec = redis::error::not_a_double;
#else
auto const res = std::from_chars(sv.data(), sv.data() + std::size(sv), d);
if (res.ec != std::errc())
ec = redis::error::not_a_double;
auto const res = std::from_chars(node.value.data(), node.value.data() + node.value.size(), d);
if (res.ec != std::errc())
ec = redis::error::not_a_double;
#endif // _LIBCPP_VERSION
}
}
};
template <class CharT, class Traits, class Allocator>
struct converter<std::basic_string<CharT, Traits, Allocator>, false> {
template <class String>
static void
apply(
std::basic_string<CharT, Traits, Allocator>& s,
resp3::basic_node<String> const& node,
system::error_code&)
{
s.append(node.value.data(), node.value.size());
}
};
template <class T>
struct from_bulk_impl {
template <class String>
static void
apply(
T& t,
resp3::basic_node<String> const& node,
system::error_code& ec)
{
converter<T>::apply(t, node, ec);
}
};
template <class T>
struct from_bulk_impl<std::optional<T>> {
template <class String>
static void
apply(
std::optional<T>& op,
resp3::basic_node<String> const& node,
system::error_code& ec)
{
if (node.data_type != resp3::type::null) {
op.emplace(T{});
converter<T>::apply(op.value(), node, ec);
}
}
};
template <class T, class String>
void
boost_redis_from_bulk(
std::basic_string<CharT, Traits, Allocator>& s,
std::string_view sv,
system::error_code&)
T& t,
resp3::basic_node<String> const& node,
system::error_code& ec)
{
s.append(sv.data(), sv.size());
from_bulk_impl<T>::apply(t, node, ec);
}
//================================================
@@ -138,14 +209,14 @@ public:
void on_value_available(Result&) {}
template <class String>
void operator()(Result& result, resp3::basic_node<String> const& n, system::error_code& ec)
void operator()(Result& result, resp3::basic_node<String> const& node, system::error_code& ec)
{
if (is_aggregate(n.data_type)) {
if (is_aggregate(node.data_type)) {
ec = redis::error::expects_resp3_simple_type;
return;
}
boost_redis_from_bulk(result, n.value, ec);
boost_redis_from_bulk(result, node, ec);
}
};
@@ -175,7 +246,7 @@ public:
}
typename Result::key_type obj;
boost_redis_from_bulk(obj, nd.value, ec);
boost_redis_from_bulk(obj, nd, ec);
hint_ = result.insert(hint_, std::move(obj));
}
};
@@ -208,11 +279,11 @@ public:
if (on_key_) {
typename Result::key_type obj;
boost_redis_from_bulk(obj, nd.value, ec);
boost_redis_from_bulk(obj, nd, ec);
current_ = result.insert(current_, {std::move(obj), {}});
} else {
typename Result::mapped_type obj;
boost_redis_from_bulk(obj, nd.value, ec);
boost_redis_from_bulk(obj, nd, ec);
current_->second = std::move(obj);
}
@@ -233,7 +304,7 @@ public:
result.reserve(result.size() + m * nd.aggregate_size);
} else {
result.push_back({});
boost_redis_from_bulk(result.back(), nd.value, ec);
boost_redis_from_bulk(result.back(), nd, ec);
}
}
};
@@ -266,7 +337,7 @@ public:
}
BOOST_ASSERT(nd.aggregate_size == 1);
boost_redis_from_bulk(result.at(i_), nd.value, ec);
boost_redis_from_bulk(result.at(i_), nd, ec);
}
++i_;
@@ -289,7 +360,7 @@ struct list_impl {
}
result.push_back({});
boost_redis_from_bulk(result.back(), nd.value, ec);
boost_redis_from_bulk(result.back(), nd, ec);
}
}
};
@@ -340,13 +411,14 @@ struct impl_map<std::deque<T, Allocator>> { using type = list_impl<std::deque<T,
template <class>
class wrapper;
template <class Result>
class wrapper<result<Result>> {
template <class T>
class wrapper<result<T>> {
public:
using response_type = result<Result>;
using response_type = result<T>;
private:
response_type* result_;
typename impl_map<Result>::type impl_;
typename impl_map<T>::type impl_;
bool called_once_ = false;
template <class String>
bool set_if_resp3_error(resp3::basic_node<String> const& nd) noexcept
@@ -366,7 +438,7 @@ public:
explicit wrapper(response_type* t = nullptr) : result_(t)
{
if (result_) {
result_->value() = Result{};
result_->value() = T{};
impl_.on_value_available(result_->value());
}
}
@@ -379,7 +451,7 @@ public:
if (result_->has_error())
return;
if (set_if_resp3_error(nd))
if (!std::exchange(called_once_, true) && set_if_resp3_error(nd))
return;
BOOST_ASSERT(result_);
@@ -395,6 +467,7 @@ public:
private:
response_type* result_;
typename impl_map<T>::type impl_{};
bool called_once_ = false;
template <class String>
bool set_if_resp3_error(resp3::basic_node<String> const& nd) noexcept
@@ -426,7 +499,7 @@ public:
if (set_if_resp3_error(nd))
return;
if (nd.data_type == resp3::type::null)
if (!std::exchange(called_once_, true) && nd.data_type == resp3::type::null)
return;
if (!result_->value().has_value()) {

2
include/boost/redis/adapter/detail/result_traits.hpp
View File

@@ -30,7 +30,7 @@ namespace boost::redis::adapter::detail
*/
template <class Result>
struct result_traits {
using adapter_type = adapter::detail::wrapper<typename std::decay<Result>::type>;
using adapter_type = wrapper<typename std::decay<Result>::type>;
static auto adapt(Result& r) noexcept { return adapter_type{&r}; }
};

7
include/boost/redis/resp3/node.hpp
View File

@@ -54,11 +54,16 @@ auto operator==(basic_node<String> const& a, basic_node<String> const& b)
&& a.value == b.value;
};
/** @brief A node in the response tree.
/** @brief A node in the response tree that owns its data
* @ingroup high-level-api
*/
using node = basic_node<std::string>;
/** @brief A node view in the response tree
* @ingroup high-level-api
*/
using node_view = basic_node<std::string_view>;
} // boost::redis::resp3
#endif // BOOST_REDIS_RESP3_NODE_HPP

35
test/test_low_level_sync_sans_io.cpp
View File

@@ -200,3 +200,38 @@ BOOST_AUTO_TEST_CASE(issue_210_no_nested)
}
}
BOOST_AUTO_TEST_CASE(issue_233_array_with_null)
{
try {
result<std::vector<std::optional<std::string>>> resp;
char const* wire = "*3\r\n+one\r\n_\r\n+two\r\n";
deserialize(wire, adapt2(resp));
BOOST_CHECK_EQUAL(resp.value().at(0).value(), "one");
BOOST_TEST(!resp.value().at(1).has_value());
BOOST_CHECK_EQUAL(resp.value().at(2).value(), "two");
} catch (std::exception const& e) {
std::cerr << e.what() << std::endl;
exit(EXIT_FAILURE);
}
}
BOOST_AUTO_TEST_CASE(issue_233_optional_array_with_null)
{
try {
result<std::optional<std::vector<std::optional<std::string>>>> resp;
char const* wire = "*3\r\n+one\r\n_\r\n+two\r\n";
deserialize(wire, adapt2(resp));
BOOST_CHECK_EQUAL(resp.value().value().at(0).value(), "one");
BOOST_TEST(!resp.value().value().at(1).has_value());
BOOST_CHECK_EQUAL(resp.value().value().at(2).value(), "two");
} catch (std::exception const& e) {
std::cerr << e.what() << std::endl;
exit(EXIT_FAILURE);
}
}