2
0
mirror of https://github.com/boostorg/parser.git synced 2026-03-05 03:02:12 +00:00

Merge branch 'master' into gh-pages

This commit is contained in:
Zach Laine
2024-01-22 21:51:53 -06:00
9 changed files with 1373 additions and 18 deletions

View File

@@ -86,6 +86,9 @@
[def _split_ [globalref boost::parser::split `boost::parser::split`]]
[def _split_v_ [classref boost::parser::split_view `boost::parser::split_view`]]
[def _split_vs_ [classref boost::parser::split_view `boost::parser::split_view`s]]
[def _replace_ [globalref boost::parser::replace `boost::parser::replace`]]
[def _replace_v_ [classref boost::parser::replace_view `boost::parser::replace_view`]]
[def _replace_vs_ [classref boost::parser::replace_view `boost::parser::replace_view`s]]
[def _std_str_ `std::string`]

View File

@@ -2926,6 +2926,7 @@ _search_all_ creates _search_all_vs_. _search_all_v_ is a `std::views`-style
view. It produces a range of subranges. Each subrange it produces is the
next match of the given parser in the parsed range.
namespace bp = boost::parser;
auto r = "XYZaaXYZbaabaXYZXYZ" | bp::search_all(bp::lit("XYZ"));
int count = 0;
// Prints XYZ XYZ XYZ XYZ.
@@ -2941,9 +2942,9 @@ _search_all_: its parser produces no attributes; it accepts C-style strings as
if they were ranges; and it knows how to get from the internally-used iterator
type back to the given iterator type, in typical cases.
_search_all_ can be called with, and and _search_all_v_ can be constructed
with, a skip parser or not, and you can always pass _trace_ at the end of any
of their overloads.
_search_all_ can be called with, and _search_all_v_ can be constructed with, a
skip parser or not, and you can always pass _trace_ at the end of any of their
overloads.
[heading _split_]
@@ -2956,6 +2957,7 @@ produced by _search_all_v_. _split_v_ has very similar semantics to
produce empty ranges between the beginning/end of the parsed range and an
adjacent match, or between adjacent matches.
namespace bp = boost::parser;
auto r = "XYZaaXYZbaabaXYZXYZ" | bp::split(bp::lit("XYZ"));
int count = 0;
// Prints '' 'aa' 'baaba' '' ''.
@@ -2967,13 +2969,96 @@ adjacent match, or between adjacent matches.
assert(count == 5);
All the details called out in the subsection on _search_ above apply to
_search_all_: its parser produces no attributes; it accepts C-style strings as
if they were ranges; and it knows how to get from the internally-used iterator
_split_: its parser produces no attributes; it accepts C-style strings as if
they were ranges; and it knows how to get from the internally-used iterator
type back to the given iterator type, in typical cases.
_search_all_ can be called with, and and _search_all_v_ can be constructed
with, a skip parser or not, and you can always pass _trace_ at the end of any
of their overloads.
_split_ can be called with, and _split_v_ can be constructed with, a skip
parser or not, and you can always pass _trace_ at the end of any of their
overloads.
[heading _replace_]
_replace_ creates _replace_vs_. _replace_v_ is a `std::views`-style view. It
produces a range of subranges from the parsed range `r` and the given
replacement range `replacement`. Wherever in the parsed range a match to the
given parser `parser` is found, `replacement` is the subrange produced. Each
subrange of `r` that does not match `parser` is produced as a subrange as
well. The subranges are produced in the order in which they occur in `r`.
Unlike _split_v_, _replace_v_ does not produce empty subranges, unless
`replacement` is empty.
namespace bp = boost::parser;
auto rng = "XYZaaXYZbaabaXYZXYZ" | bp::replace(bp::lit("XYZ"), "foo");
int count = 0;
// Prints foo aa foo baaba foo foo.
for (auto subrange : rng) {
std::cout << std::string_view(subrange.begin(), subrange.end() - subrange.begin()) << " ";
++count;
}
std::cout << "\n";
assert(count == 6);
If the iterator types `Ir` and `Ireplacement` for the `r` and `replacement`
ranges passed are identical (as in the example above), the iterator type for
the subranges produced is `Ir`. If they are different, an
implementation-defined type is used for the iterator. This type is the moral
equivalent of a `std::variant<Ir, Ireplacement>`. This works as long as `Ir`
and `Ireplacement` are compatible. To be compatible, they must have common
reference, value, and rvalue reference types, as determined by
`std::common_type_t`. One advantage to this scheme is that the range of
subranges represented by _replace_v_ is easily joined back into a single
range.
namespace bp = boost::parser;
char const str[] = "XYZaaXYZbaabaXYZXYZ";
auto rng = str | bp::replace(bp::lit("XYZ"), "foo") | std::views::join;
std::string replace_result;
for (auto ch : rng) {
replace_result.push_back(ch);
}
assert(replace_result == "fooaafoobaabafoofoo");
Note that we could not have written `std::string replace_result(r.begin(),
r.end())`. This is ill-formed because the `std::string` range constructor
takes two iterators, but `decltype(rng.end())` is a sentinel type different
from `decltype(rng.begin())`.
Though the ranges `r` and `replacement` can both be C-style strings,
_replace_v_ must know the end of `replacement` before it does any work. This
is because the subranges produced are all common ranges, and so if
`replacement` is not, a common range must be formed from it. If you expect to
pass very long C-style strings to _replace_ and not pay to see the end until
the range is used, don't.
You may wonder what happens when you pass a UTF-N range for `r`, and a UTF-M
range for `replacement`. What happens in this case is silent transcoding of
`replacement` from UTF-M to UTF-N by the _replace_ range adaptor. This
doesn't require memory allocation; _replace_ just slaps `|
boost::parser::as_utfN` onto `replacement`. However, since _Parser_ treats
`char` sequences as unknown encoding, _replace_ will not transcode from `char`
sequences. So calls like this won't work:
char const str[] = "some text";
char const replacement_str[] = "some text";
using namespace bp = boost::parser;
auto r = empty_str | bp::replace(parser, replacement_str | bp::as_utf8);
Notice that this does not work, even though `char` and UTF-8 are the same
size. If `r` and `replacement` are both sequences of `char`, everything will
work of course. It's just mixing `char` and UTF-encoded sequences that does
not work.
All the details called out in the subsection on _search_ above apply to
_replace_: its parser produces no attributes; it accepts C-style strings for
the `r` and `replacement` parameters as if they were ranges; and it knows how
to get from the internally-used iterator type back to the given iterator type,
in typical cases.
_replace_ can be called with, and _replace_v_ can be constructed with, a skip
parser or not, and you can always pass _trace_ at the end of any of their
overloads.
[endsect]

View File

@@ -308,7 +308,7 @@ namespace boost::parser::detail { namespace stl_interfaces {
{
#if BOOST_PARSER_DETAIL_STL_INTERFACES_USE_CONCEPTS
if constexpr (std::is_invocable_v<F const &, Args...>) {
return f((Args &&) args...);
return f_((Args &&) args...);
} else {
return closure(
stl_interfaces::bind_back(f_, (Args &&) args...));

View File

@@ -993,6 +993,11 @@ namespace boost { namespace parser {
using iter_reference_t = std::iter_reference_t<T>;
template<typename T>
using range_value_t = std::ranges::range_value_t<T>;
template<typename T>
using range_reference_t = std::ranges::range_reference_t<T>;
template<typename T>
using range_rvalue_reference_t =
std::ranges::range_rvalue_reference_t<T>;
template<typename T>
constexpr bool is_parsable_code_unit_v = code_unit<T>;
@@ -1010,7 +1015,14 @@ namespace boost { namespace parser {
template<typename T>
using iter_reference_t = decltype(*std::declval<T &>());
template<typename T>
using iter_rvalue_reference_t =
decltype(std::move(*std::declval<T &>()));
template<typename T>
using range_value_t = iter_value_t<iterator_t<T>>;
template<typename T>
using range_reference_t = iter_reference_t<iterator_t<T>>;
template<typename T>
using range_rvalue_reference_t = iter_rvalue_reference_t<iterator_t<T>>;
template<typename T>
using has_insert = decltype(std::declval<T &>().insert(

View File

@@ -0,0 +1,749 @@
#ifndef BOOST_PARSER_REPLACE_HPP
#define BOOST_PARSER_REPLACE_HPP
#include <boost/parser/search.hpp>
#if !defined(_MSC_VER) || BOOST_PARSER_USE_CONCEPTS
namespace boost::parser {
// TODO: transform_replace.
namespace detail {
template<typename T, bool = std::is_pointer_v<remove_cv_ref_t<T>>>
constexpr auto range_value_type =
wrapper<remove_cv_ref_t<range_value_t<T>>>{};
template<typename T>
constexpr auto range_value_type<T, true> = wrapper<
remove_cv_ref_t<std::remove_pointer_t<remove_cv_ref_t<T>>>>{};
template<typename T>
constexpr text::format range_utf_format()
{
using value_t = typename decltype(range_value_type<T>)::type;
if constexpr (std::is_same_v<value_t, char>) {
return no_format;
#if defined(__cpp_char8_t)
} else if constexpr (std::is_same_v<value_t, char8_t>) {
return format::utf8;
#endif
} else if constexpr (
std::is_same_v<value_t, char16_t>
#ifdef _MSC_VER
|| std::is_same_v<T, wchar_t>
#endif
) {
return format::utf16;
} else if constexpr (
std::is_same_v<value_t, char32_t>
#ifndef _MSC_VER
|| std::is_same_v<T, wchar_t>
#endif
) {
return format::utf32;
} else {
static_assert(
sizeof(T) && false,
"Looks like you're trying to pass a range to replace or "
"transform_replace that has a non-character type for its "
"value type. This is not supported.");
}
}
template<typename T>
constexpr text::format
range_utf_format_v = detail::range_utf_format<T>();
template<typename V1, typename V2>
using concat_reference_t =
std::common_type_t<range_reference_t<V1>, range_reference_t<V2>>;
template<typename V1, typename V2>
using concat_value_t =
std::common_type_t<range_value_t<V1>, range_value_t<V2>>;
template<typename V1, typename V2>
using concat_rvalue_reference_t = std::common_type_t<
range_rvalue_reference_t<V1>,
range_rvalue_reference_t<V2>>;
#if BOOST_PARSER_USE_CONCEPTS
// clang-format off
template<typename ReplacementV, typename V>
concept concatable = requires {
typename detail::concat_reference_t<ReplacementV, V>;
typename detail::concat_value_t<ReplacementV, V>;
typename detail::concat_rvalue_reference_t<ReplacementV, V>;
};
// clang-format on
#else
template<typename ReplacementV, typename V>
// clang-format off
using concatable_expr = decltype(
std::declval<concat_reference_t<ReplacementV, V>>(),
std::declval<concat_value_t<ReplacementV, V>>(),
std::declval<concat_rvalue_reference_t<ReplacementV, V>>());
// clang-format on
template<typename ReplacementV, typename V>
constexpr bool concatable =
is_detected_v<concatable_expr, ReplacementV, V>;
#endif
template<
typename V1,
typename V2
#if !BOOST_PARSER_USE_CONCEPTS
,
typename Enable = std::enable_if_t<concatable<V1, V2>>
#endif
>
#if BOOST_PARSER_USE_CONCEPTS
requires concatable<V1, V2>
#endif
struct either_iterator_impl
: detail::stl_interfaces::iterator_interface<
either_iterator_impl<V1, V2>,
std::forward_iterator_tag,
concat_value_t<V1, V2>,
concat_reference_t<V1, V2>>
{
constexpr either_iterator_impl() = default;
constexpr either_iterator_impl(iterator_t<V1> it) : it_(it) {}
template<typename V = V2>
constexpr either_iterator_impl(iterator_t<V> it) : it_(it)
{}
constexpr concat_reference_t<V1, V2> operator*() const
{
if (it_.index() == 0) {
return *std::get<0>(it_);
} else {
return *std::get<1>(it_);
}
}
constexpr either_iterator_impl & operator++()
{
if (it_.index() == 0)
++std::get<0>(it_);
else
++std::get<1>(it_);
return *this;
}
friend constexpr bool
operator==(either_iterator_impl lhs, either_iterator_impl rhs)
{
if (lhs.it_.index() != rhs.it_.index())
return false;
if (lhs.it_.index() == 0)
return std::get<0>(lhs.it_) == std::get<0>(rhs.it_);
else
return std::get<1>(lhs.it_) == std::get<1>(rhs.it_);
}
using base_type = detail::stl_interfaces::iterator_interface<
either_iterator_impl<V1, V2>,
std::forward_iterator_tag,
concat_value_t<V1, V2>,
concat_reference_t<V1, V2>>;
using base_type::operator++;
private:
std::variant<iterator_t<V1>, iterator_t<V2>> it_;
};
template<typename V1, typename V2>
using either_iterator = std::conditional_t<
std::is_same_v<iterator_t<V1>, iterator_t<V2>>,
iterator_t<V1>,
either_iterator_impl<V1, V2>>;
#if BOOST_PARSER_USE_CONCEPTS
// clang-format off
template<typename ReplacementV, typename V>
concept replacement_for = requires (ReplacementV replacement, V base) {
{ either_iterator<V, ReplacementV>(replacement.begin()) };
{ either_iterator<V, ReplacementV>(replacement.end()) };
{ either_iterator<V, ReplacementV>(base.begin()) };
};
// clang-format on
#else
template<typename ReplacementV, typename V>
using replacement_for_expr = decltype(
either_iterator<V, ReplacementV>(
std::declval<ReplacementV&>().begin()),
either_iterator<V, ReplacementV>(
std::declval<ReplacementV&>().end()),
either_iterator<V, ReplacementV>(std::declval<V&>().begin()));
template<typename ReplacementV, typename V>
constexpr bool replacement_for =
is_detected_v<replacement_for_expr, ReplacementV, V>;
#endif
}
/** Produces a range of subranges of a given range `base`. Each subrange
is either a subrange of `base` that does not match the given parser
`parser`, or is the given replacement for a match, `replacement`. */
template<
#if BOOST_PARSER_USE_CONCEPTS
std::ranges::viewable_range V,
std::ranges::viewable_range ReplacementV,
#else
typename V,
typename ReplacementV,
#endif
typename Parser,
typename GlobalState,
typename ErrorHandler,
typename SkipParser
#if !BOOST_PARSER_USE_CONCEPTS
,
typename Enable = std::enable_if_t<
detail::replacement_for<ReplacementV, V> &&
(detail::range_utf_format_v<V> ==
detail::range_utf_format_v<ReplacementV>)>
#endif
>
#if BOOST_PARSER_USE_CONCEPTS
requires detail::replacement_for<ReplacementV, V> &&
(detail::range_utf_format_v<V> ==
detail::range_utf_format_v<ReplacementV>)
#endif
struct replace_view
: detail::stl_interfaces::view_interface<replace_view<
V,
ReplacementV,
Parser,
GlobalState,
ErrorHandler,
SkipParser>>
{
constexpr replace_view() = default;
constexpr replace_view(
V base,
parser_interface<Parser, GlobalState, ErrorHandler> const & parser,
parser_interface<SkipParser> const & skip,
ReplacementV replacement,
trace trace_mode = trace::off) :
base_(std::move(base)),
replacement_(std::move(replacement)),
parser_(parser),
skip_(skip),
trace_mode_(trace_mode)
{}
constexpr replace_view(
V base,
parser_interface<Parser, GlobalState, ErrorHandler> const & parser,
ReplacementV replacement,
trace trace_mode = trace::off) :
base_(std::move(base)),
replacement_(std::move(replacement)),
parser_(parser),
skip_(),
trace_mode_(trace_mode)
{}
constexpr V base() const &
#if BOOST_PARSER_USE_CONCEPTS
requires std::copy_constructible<V>
#endif
{
return base_;
}
constexpr V base() && { return std::move(base_); }
constexpr V replacement() const &
#if BOOST_PARSER_USE_CONCEPTS
requires std::copy_constructible<V>
#endif
{
return replacement_;
}
constexpr V replacement() && { return std::move(replacement_); }
constexpr auto begin() { return iterator<false>{this}; }
constexpr auto end() { return sentinel<false>{}; }
constexpr auto begin() const
#if BOOST_PARSER_USE_CONCEPTS
requires std::ranges::range<const V>
#endif
{
return iterator<true>{this};
}
constexpr auto end() const
#if BOOST_PARSER_USE_CONCEPTS
requires std::ranges::range<const V>
#endif
{
return sentinel<true>{};
}
template<bool Const>
struct sentinel
{};
template<bool Const>
struct iterator : detail::stl_interfaces::proxy_iterator_interface<
iterator<Const>,
std::forward_iterator_tag,
BOOST_PARSER_SUBRANGE<detail::either_iterator<
detail::maybe_const<Const, V>,
detail::maybe_const<Const, ReplacementV>>>>
{
using I = detail::iterator_t<detail::maybe_const<Const, V>>;
using S = detail::sentinel_t<detail::maybe_const<Const, V>>;
using ref_t_iter = detail::either_iterator<
detail::maybe_const<Const, V>,
detail::maybe_const<Const, ReplacementV>>;
using reference_type = BOOST_PARSER_SUBRANGE<ref_t_iter>;
constexpr iterator() = default;
constexpr iterator(
detail::maybe_const<Const, replace_view> * parent) :
parent_(parent),
r_(parent_->base_.begin(), parent_->base_.end()),
curr_(r_.begin(), r_.begin()),
next_it_(r_.begin()),
in_match_(true)
{
++*this;
}
constexpr iterator & operator++()
{
if (in_match_) {
r_ = BOOST_PARSER_SUBRANGE<I, S>(next_it_, r_.end());
auto const new_match = parser::search(
r_,
parent_->parser_,
parent_->skip_,
parent_->trace_mode_);
if (new_match.begin() == curr_.end()) {
curr_ = new_match;
} else {
curr_ =
BOOST_PARSER_SUBRANGE(next_it_, new_match.begin());
in_match_ = false;
}
next_it_ = new_match.end();
} else {
if (!curr_.empty()) {
curr_ = BOOST_PARSER_SUBRANGE(curr_.end(), next_it_);
in_match_ = true;
}
if (curr_.empty())
r_ = BOOST_PARSER_SUBRANGE<I, S>(next_it_, r_.end());
}
return *this;
}
constexpr reference_type operator*() const
{
if (in_match_) {
return reference_type(
ref_t_iter(parent_->replacement_.begin()),
ref_t_iter(parent_->replacement_.end()));
} else {
return reference_type(
ref_t_iter(curr_.begin()), ref_t_iter(curr_.end()));
}
}
friend constexpr bool operator==(iterator lhs, iterator rhs)
{
return lhs.r_.begin() == rhs.r_.begin();
}
friend constexpr bool operator==(iterator it, sentinel<Const>)
{
return it.r_.begin() == it.r_.end();
}
using base_type = detail::stl_interfaces::proxy_iterator_interface<
iterator,
std::forward_iterator_tag,
reference_type>;
using base_type::operator++;
private:
detail::maybe_const<Const, replace_view> * parent_;
BOOST_PARSER_SUBRANGE<I, S> r_;
BOOST_PARSER_SUBRANGE<I> curr_;
I next_it_;
bool in_match_;
};
template<bool Const>
friend struct iterator;
private:
V base_;
ReplacementV replacement_;
parser_interface<Parser, GlobalState, ErrorHandler> parser_;
parser_interface<SkipParser> skip_;
trace trace_mode_;
};
// deduction guides
#if BOOST_PARSER_USE_CONCEPTS
template<
typename V,
typename ReplacementV,
typename Parser,
typename GlobalState,
typename ErrorHandler,
typename SkipParser>
replace_view(
V &&,
parser_interface<Parser, GlobalState, ErrorHandler>,
parser_interface<SkipParser>,
ReplacementV &&,
trace)
-> replace_view<
std::views::all_t<V>,
std::views::all_t<ReplacementV>,
Parser,
GlobalState,
ErrorHandler,
SkipParser>;
template<
typename V,
typename ReplacementV,
typename Parser,
typename GlobalState,
typename ErrorHandler,
typename SkipParser>
replace_view(
V &&,
parser_interface<Parser, GlobalState, ErrorHandler>,
parser_interface<SkipParser>,
ReplacementV &&)
-> replace_view<
std::views::all_t<V>,
std::views::all_t<ReplacementV>,
Parser,
GlobalState,
ErrorHandler,
SkipParser>;
template<
typename V,
typename ReplacementV,
typename Parser,
typename GlobalState,
typename ErrorHandler>
replace_view(
V &&,
parser_interface<Parser, GlobalState, ErrorHandler>,
ReplacementV &&,
trace)
-> replace_view<
std::views::all_t<V>,
std::views::all_t<ReplacementV>,
Parser,
GlobalState,
ErrorHandler,
parser_interface<eps_parser<detail::phony>>>;
template<
typename V,
typename ReplacementV,
typename Parser,
typename GlobalState,
typename ErrorHandler>
replace_view(
V &&,
parser_interface<Parser, GlobalState, ErrorHandler>,
ReplacementV &&)
-> replace_view<
std::views::all_t<V>,
std::views::all_t<ReplacementV>,
Parser,
GlobalState,
ErrorHandler,
parser_interface<eps_parser<detail::phony>>>;
#else
template<
typename V,
typename ReplacementV,
typename Parser,
typename GlobalState,
typename ErrorHandler>
replace_view(
V &&,
parser_interface<Parser, GlobalState, ErrorHandler>,
ReplacementV &&,
trace)
-> replace_view<
V,
ReplacementV,
Parser,
GlobalState,
ErrorHandler,
parser_interface<eps_parser<detail::phony>>>;
template<
typename V,
typename ReplacementV,
typename Parser,
typename GlobalState,
typename ErrorHandler>
replace_view(
V &&,
parser_interface<Parser, GlobalState, ErrorHandler>,
ReplacementV &&)
-> replace_view<
V,
ReplacementV,
Parser,
GlobalState,
ErrorHandler,
parser_interface<eps_parser<detail::phony>>>;
#endif
namespace detail {
template<
typename V,
typename ReplacementV,
typename Parser,
typename GlobalState,
typename ErrorHandler,
typename SkipParser>
using replace_view_expr = decltype(replace_view<
V,
ReplacementV,
Parser,
GlobalState,
ErrorHandler,
SkipParser>(
std::declval<V>(),
std::declval<
parser_interface<Parser, GlobalState, ErrorHandler> const &>(),
std::declval<parser_interface<SkipParser> const &>(),
std::declval<ReplacementV>(),
trace::on));
template<
typename V,
typename ReplacementV,
typename Parser,
typename GlobalState,
typename ErrorHandler,
typename SkipParser>
constexpr bool can_replace_view = is_detected_v<
replace_view_expr,
V,
ReplacementV,
Parser,
GlobalState,
ErrorHandler,
SkipParser>;
struct replace_impl
{
#if BOOST_PARSER_USE_CONCEPTS
template<
parsable_range_like R,
range_like ReplacementR,
typename Parser,
typename GlobalState,
typename ErrorHandler,
typename SkipParser>
requires
// clang-format off
(std::is_pointer_v<std::remove_cvref_t<R>> ||
std::ranges::viewable_range<R>) &&
(std::is_pointer_v<std::remove_cvref_t<ReplacementR>> ||
std::ranges::viewable_range<ReplacementR>) &&
// clang-format on
can_replace_view<
decltype(to_range<R>::call(std::declval<R>())),
decltype(to_range<
ReplacementR,
true,
detail::range_utf_format_v<R>>::
call(std::declval<ReplacementR>())),
Parser,
GlobalState,
ErrorHandler,
SkipParser>
// clang-format off
[[nodiscard]] constexpr auto operator()(
R && r,
parser_interface<Parser, GlobalState, ErrorHandler> const &
parser,
parser_interface<SkipParser> const & skip,
ReplacementR && replacement,
trace trace_mode = trace::off) const
// clang-format on
{
return replace_view(
to_range<R>::call((R &&) r),
parser,
skip,
to_range<
ReplacementR,
true,
detail::range_utf_format_v<R>>::call((ReplacementR &&)
replacement),
trace_mode);
}
template<
parsable_range_like R,
range_like ReplacementR,
typename Parser,
typename GlobalState,
typename ErrorHandler>
requires
// clang-format off
(std::is_pointer_v<std::remove_cvref_t<R>> ||
std::ranges::viewable_range<R>) &&
(std::is_pointer_v<std::remove_cvref_t<ReplacementR>> ||
std::ranges::viewable_range<ReplacementR>) &&
// clang-format on
can_replace_view<
decltype(to_range<R>::call(std::declval<R>())),
decltype(to_range<
ReplacementR,
true,
detail::range_utf_format_v<R>>::
call(std::declval<ReplacementR>())),
Parser,
GlobalState,
ErrorHandler,
parser_interface<eps_parser<detail::phony>>>
// clang-format off
[[nodiscard]] constexpr auto operator()(
R && r,
parser_interface<Parser, GlobalState, ErrorHandler> const &
parser,
ReplacementR && replacement,
trace trace_mode = trace::off) const
// clang-format on
{
return (*this)(
(R &&) r,
parser,
parser_interface<eps_parser<detail::phony>>{},
(ReplacementR &&) replacement,
trace_mode);
}
#else
template<
typename R,
typename Parser,
typename GlobalState,
typename ErrorHandler,
typename SkipParser,
typename ReplacementR = trace,
typename Trace = trace,
typename Enable = std::enable_if_t<is_parsable_range_like_v<R>>>
[[nodiscard]] constexpr auto operator()(
R && r,
parser_interface<Parser, GlobalState, ErrorHandler> const &
parser,
SkipParser && skip,
ReplacementR && replacement = ReplacementR{},
Trace trace_mode = Trace{}) const
{
if constexpr (
is_parser_iface<remove_cv_ref_t<SkipParser>> &&
is_range_like<remove_cv_ref_t<ReplacementR>> &&
std::is_same_v<Trace, trace>) {
// (r, parser, skip, replacement, trace) case
return impl(
(R &&) r,
parser,
skip,
(ReplacementR &&) replacement,
trace_mode);
} else if constexpr (
is_range_like<remove_cv_ref_t<SkipParser>> &&
std::is_same_v<remove_cv_ref_t<ReplacementR>, trace> &&
std::is_same_v<Trace, trace>) {
// (r, parser, replacement, trace) case
return impl(
(R &&) r,
parser,
parser_interface<eps_parser<detail::phony>>{},
(SkipParser &&) skip,
replacement);
} else {
static_assert(
sizeof(R) == 1 && false,
"Only the signatures replace(R, parser, skip, "
"replcement trace = trace::off) and replace(R, parser, "
"replacement, trace = trace::off) are supported.");
}
}
private:
template<
typename R,
typename ReplacementR,
typename Parser,
typename GlobalState,
typename ErrorHandler,
typename SkipParser>
[[nodiscard]] constexpr auto impl(
R && r,
parser_interface<Parser, GlobalState, ErrorHandler> const &
parser,
parser_interface<SkipParser> const & skip,
ReplacementR && replacement,
trace trace_mode = trace::off) const
{
return replace_view(
to_range<R>::call((R &&) r),
parser,
skip,
to_range<
ReplacementR,
true,
detail::range_utf_format_v<R>>::call((ReplacementR &&)
replacement),
trace_mode);
}
#endif
};
}
/** A range adaptor object ([range.adaptor.object]). Given subexpressions
`E` and `P`, `Q`, `R`, and 'S', each of the expressions `replace(E,
P)`, `replace(E, P, Q)`. `replace(E, P, Q, R)`, and `replace(E, P, Q,
R, S)` are expression-equivalent to `replace_view(E, P)`,
`replace_view(E, P, Q)`, `replace_view(E, P, Q, R)`, `replace_view(E,
P, Q, R, S)`, respectively. */
inline constexpr detail::stl_interfaces::adaptor<detail::replace_impl>
replace = detail::replace_impl{};
}
#if BOOST_PARSER_USE_CONCEPTS
template<
typename V,
typename ReplacementV,
typename Parser,
typename GlobalState,
typename ErrorHandler,
typename SkipParser>
constexpr bool std::ranges::enable_borrowed_range<boost::parser::replace_view<
V,
ReplacementV,
Parser,
GlobalState,
ErrorHandler,
SkipParser>> =
enable_borrowed_range<V> && enable_borrowed_range<ReplacementV>;
#endif
#endif
#endif

View File

@@ -4,6 +4,8 @@
#include <boost/parser/parser.hpp>
#include <boost/parser/transcode_view.hpp>
#include <cstring>
namespace boost::parser {
@@ -11,8 +13,22 @@ namespace boost::parser {
template<bool Const, typename T>
using maybe_const = std::conditional_t<Const, const T, T>;
inline constexpr text::format no_format = static_cast<text::format>(0);
template<text::format Format = text::format::utf8>
constexpr auto as_utf =
text::detail::as_utf_impl<text::utf8_view, text::format::utf8>{};
template<>
constexpr auto as_utf<text::format::utf16> =
text::detail::as_utf_impl<text::utf16_view, text::format::utf16>{};
template<>
constexpr auto as_utf<text::format::utf32> =
text::detail::as_utf_impl<text::utf32_view, text::format::utf32>{};
template<
typename R_,
bool ToCommonRange = false,
text::format OtherRangeFormat = no_format,
bool = std::is_pointer_v<remove_cv_ref_t<R_>> ||
text::detail::is_bounded_array_v<remove_cv_ref_t<R_>>>
struct to_range
@@ -23,20 +39,43 @@ namespace boost::parser {
static_assert(std::is_same_v<R, R_>);
using T = remove_cv_ref_t<R>;
if constexpr (std::is_pointer_v<T>) {
return BOOST_PARSER_SUBRANGE(r, null_sentinel_t{});
} else {
if constexpr (OtherRangeFormat == no_format) {
if constexpr (ToCommonRange)
return BOOST_PARSER_SUBRANGE(r, r + std::strlen(r));
else
return BOOST_PARSER_SUBRANGE(r, null_sentinel_t{});
} else {
if constexpr (ToCommonRange) {
return BOOST_PARSER_SUBRANGE(
r, r + std::strlen(r)) |
as_utf<OtherRangeFormat>;
} else {
return BOOST_PARSER_SUBRANGE(r, null_sentinel_t{}) |
as_utf<OtherRangeFormat>;
}
}
} else if constexpr (
std::is_pointer_v<T> ||
text::detail::is_bounded_array_v<T>) {
auto const first = std::begin(r);
auto last = std::end(r);
constexpr auto n = std::extent_v<T>;
if (n && !r[n - 1])
--last;
return BOOST_PARSER_SUBRANGE(first, last);
if constexpr (OtherRangeFormat == no_format) {
return BOOST_PARSER_SUBRANGE(first, last);
} else {
return BOOST_PARSER_SUBRANGE(first, last) |
as_utf<OtherRangeFormat>;
}
} else {
return (R &&) r | as_utf<OtherRangeFormat>;
}
}
};
template<typename R_>
struct to_range<R_, false>
template<typename R_, bool ToCommonRange>
struct to_range<R_, ToCommonRange, no_format, false>
{
template<typename R>
static constexpr R && call(R && r)
@@ -270,13 +309,19 @@ namespace boost::parser {
parser_interface<Parser, GlobalState, ErrorHandler> const & parser,
parser_interface<SkipParser> const & skip,
trace trace_mode = trace::off) :
base_(base), parser_(parser), skip_(skip), trace_mode_(trace_mode)
base_(std::move(base)),
parser_(parser),
skip_(skip),
trace_mode_(trace_mode)
{}
constexpr search_all_view(
V base,
parser_interface<Parser, GlobalState, ErrorHandler> const & parser,
trace trace_mode = trace::off) :
base_(base), parser_(parser), skip_(), trace_mode_(trace_mode)
base_(std::move(base)),
parser_(parser),
skip_(),
trace_mode_(trace_mode)
{}
constexpr V base() const &

View File

@@ -29,13 +29,19 @@ namespace boost::parser {
parser_interface<Parser, GlobalState, ErrorHandler> const & parser,
parser_interface<SkipParser> const & skip,
trace trace_mode = trace::off) :
base_(base), parser_(parser), skip_(skip), trace_mode_(trace_mode)
base_(std::move(base)),
parser_(parser),
skip_(skip),
trace_mode_(trace_mode)
{}
constexpr split_view(
V base,
parser_interface<Parser, GlobalState, ErrorHandler> const & parser,
trace trace_mode = trace::off) :
base_(base), parser_(parser), skip_(), trace_mode_(trace_mode)
base_(std::move(base)),
parser_(parser),
skip_(),
trace_mode_(trace_mode)
{}
constexpr V base() const &

View File

@@ -48,6 +48,7 @@ endmacro()
add_test_executable(search)
add_test_executable(split)
add_test_executable(replace)
add_test_executable(hl)
add_test_executable(aggr_tuple_assignment)
add_test_executable(parser_lazy_params)

454
test/replace.cpp Normal file
View File

@@ -0,0 +1,454 @@
/**
* Copyright (C) 2024 T. Zachary Laine
*
* Distributed under the Boost Software License, Version 1.0. (See
* accompanying file LICENSE_1_0.txt or copy at
* http://www.boost.org/LICENSE_1_0.txt)
*/
#include <boost/parser/replace.hpp>
#include <gtest/gtest.h>
#include "ill_formed.hpp"
#include <list>
// TODO: Document that replace{,_view} are not supported in MSVC in C++17 mode.
#if !defined(_MSC_VER) || BOOST_PARSER_USE_CONCEPTS
namespace bp = boost::parser;
#if BOOST_PARSER_USE_CONCEPTS
namespace deduction {
using namespace std::literals;
std::string str;
auto const parser = bp::char_;
auto const skip = bp::ws;
auto deduced_1 = bp::replace_view(str, parser, skip, "foo", bp::trace::on);
auto deduced_2 = bp::replace_view(str, parser, skip, "foo");
auto deduced_3 = bp::replace_view(str, parser, "foo", bp::trace::on);
auto deduced_4 = bp::replace_view(str, parser, "foo");
}
#endif
static_assert(
bp::detail::range_utf_format<char const *&>() == bp::detail::no_format);
TEST(replace, either_iterator)
{
{
std::list<int> l({1, 2, 3});
std::vector<int> v({4, 5, 6});
bp::detail::either_iterator<std::list<int>, std::vector<int>>
either_l_begin(l.begin());
bp::detail::either_iterator<std::list<int>, std::vector<int>>
either_l_end(l.end());
bp::detail::either_iterator<std::list<int>, std::vector<int>>
either_v_begin(v.begin());
bp::detail::either_iterator<std::list<int>, std::vector<int>>
either_v_end(v.end());
int const l_array[] = {1, 2, 3};
auto l_array_curr = l_array;
for (auto it = either_l_begin; it != either_l_end;
++it, ++l_array_curr) {
EXPECT_EQ(*it, *l_array_curr);
}
int const v_array[] = {4, 5, 6};
auto v_array_curr = v_array;
for (auto it = either_v_begin; it != either_v_end;
++it, ++v_array_curr) {
EXPECT_EQ(*it, *v_array_curr);
}
}
{
auto r1 = bp::detail::to_range<decltype("")>::call("");
auto r2 = bp::detail::to_range<decltype("foo")>::call("foo");
bp::detail::either_iterator<decltype(r1), decltype(r2)> either_r1_begin(
r1.begin());
bp::detail::either_iterator<decltype(r1), decltype(r2)> either_r1_end(
r1.end());
bp::detail::either_iterator<decltype(r1), decltype(r2)> either_r2_begin(
r2.begin());
bp::detail::either_iterator<decltype(r1), decltype(r2)> either_r2_end(
r2.end());
EXPECT_EQ(either_r1_begin, either_r1_end);
std::string copy;
for (auto it = either_r2_begin; it != either_r2_end; ++it) {
copy.push_back(*it);
}
EXPECT_EQ(copy, "foo");
}
}
TEST(replace, replace)
{
{
auto r = bp::replace("", bp::lit("XYZ"), bp::ws, "foo");
int count = 0;
for (auto subrange : r) {
(void)subrange;
++count;
}
EXPECT_EQ(count, 0);
}
{
char const str[] = "aaXYZb";
auto r = bp::replace(str, bp::lit("XYZ"), bp::ws, "foo");
int count = 0;
std::string_view const strs[] = {"aa", "foo", "b"};
for (auto subrange : r) {
std::string str(subrange.begin(), subrange.end());
EXPECT_EQ(str, strs[count]);
++count;
}
EXPECT_EQ(count, 3);
}
{
char const str[] = "aaXYZbaabaXYZ";
auto r =
str | bp::replace(bp::lit("XYZ"), bp::ws, "foo", bp::trace::off);
int count = 0;
std::string_view const strs[] = {"aa", "foo", "baaba", "foo"};
for (auto subrange : r) {
std::string str(subrange.begin(), subrange.end());
EXPECT_EQ(str, strs[count]);
++count;
}
EXPECT_EQ(count, 4);
}
#if !defined(__GNUC__) || 12 <= __GNUC__
// Older GCCs don't like the use of temporaries like the
// std::string("foo") below.
{
char const str[] = "aaXYZbaabaXYZ";
auto r = str | bp::replace(
bp::lit("XYZ"), std::string("foo"), bp::trace::off);
int count = 0;
std::string_view const strs[] = {"aa", "foo", "baaba", "foo"};
for (auto subrange : r) {
std::string str(subrange.begin(), subrange.end());
EXPECT_EQ(str, strs[count]);
++count;
}
EXPECT_EQ(count, 4);
}
#endif
{
char const str[] = "aaXYZbaabaXYZ";
auto r = str | bp::replace(bp::lit("XYZ"), "foo");
int count = 0;
std::string_view const strs[] = {"aa", "foo", "baaba", "foo"};
for (auto subrange : r) {
std::string str(subrange.begin(), subrange.end());
EXPECT_EQ(str, strs[count]);
++count;
}
EXPECT_EQ(count, 4);
}
{
char const str[] = "aaXYZbaabaXYZXYZ";
auto r = str | bp::replace(bp::lit("XYZ"), "foo");
int count = 0;
std::string_view const strs[] = {"aa", "foo", "baaba", "foo", "foo"};
for (auto subrange : r) {
std::string str(subrange.begin(), subrange.end());
EXPECT_EQ(str, strs[count]);
++count;
}
EXPECT_EQ(count, 5);
}
{
char const str[] = "XYZaaXYZbaabaXYZXYZ";
auto r = str | bp::replace(bp::lit("XYZ"), "foo");
int count = 0;
std::string_view const strs[] = {
"foo", "aa", "foo", "baaba", "foo", "foo"};
for (auto subrange : r) {
std::string str(subrange.begin(), subrange.end());
EXPECT_EQ(str, strs[count]);
++count;
}
EXPECT_EQ(count, 6);
}
{
char const str[] = "XYZXYZaaXYZbaabaXYZXYZ";
auto r = str | bp::replace(bp::lit("XYZ"), "foo");
int count = 0;
std::string_view const strs[] = {
"foo", "foo", "aa", "foo", "baaba", "foo", "foo"};
for (auto subrange : r) {
std::string str(subrange.begin(), subrange.end());
EXPECT_EQ(str, strs[count]);
++count;
}
EXPECT_EQ(count, 7);
}
{
char const * str = "XYZXYZaaXYZbaabaXYZXYZ";
char const * replacement = "foo";
auto r = str | bp::replace(bp::lit("XYZ"), replacement);
int count = 0;
std::string_view const strs[] = {
"foo", "foo", "aa", "foo", "baaba", "foo", "foo"};
for (auto subrange : r) {
std::string str(subrange.begin(), subrange.end());
EXPECT_EQ(str, strs[count]);
++count;
}
EXPECT_EQ(count, 7);
}
{
char const * str = "XYZXYZaaXYZbaabaXYZXYZ";
char const * replacement = "foo";
auto const r = str | bp::replace(bp::lit("XYZ"), replacement);
int count = 0;
std::string_view const strs[] = {
"foo", "foo", "aa", "foo", "baaba", "foo", "foo"};
for (auto subrange : r) {
std::string str(subrange.begin(), subrange.end());
EXPECT_EQ(str, strs[count]);
++count;
}
EXPECT_EQ(count, 7);
}
}
// MSVC produces hard errors here, so ill_formed does not work.
#if defined(__cpp_char8_t) && !defined(_MSC_VER)
char const empty_str[] = "";
template<typename T>
using char_str_utf8_replacement =
decltype(std::declval<T>() | bp::replace(bp::lit("XYZ"), std::declval<T>() | bp::as_utf8));
static_assert(ill_formed<char_str_utf8_replacement, decltype(empty_str)>{});
template<typename T>
using char_str_utf16_replacement =
decltype(std::declval<T>() | bp::replace(bp::lit("XYZ"), std::declval<T>() | bp::as_utf16));
static_assert(ill_formed<char_str_utf16_replacement, decltype(empty_str)>{});
template<typename T>
using utf8_str_char_replacement =
decltype(std::declval<T>() | bp::as_utf8 | bp::replace(bp::lit("XYZ"), std::declval<T>()));
static_assert(ill_formed<utf8_str_char_replacement, decltype(empty_str)>{});
#endif
TEST(replace, replace_unicode)
{
{
char const str_[] = "";
auto str = str_ | bp::as_utf8;
auto r = bp::replace(str, bp::lit("XYZ"), bp::ws, "foo" | bp::as_utf8);
int count = 0;
for (auto subrange : r) {
(void)subrange;
++count;
}
EXPECT_EQ(count, 0);
}
{
char const * str_ = "aaXYZb";
auto str = str_ | bp::as_utf16;
auto r = bp::replace(str, bp::lit("XYZ"), bp::ws, "foo" | bp::as_utf16);
int count = 0;
std::string_view const strs[] = {"aa", "foo", "b"};
for (auto subrange : r) {
std::string str(subrange.begin(), subrange.end());
EXPECT_EQ(str, strs[count]);
++count;
}
EXPECT_EQ(count, 3);
}
{
char const str_[] = "aaXYZbaabaXYZ";
auto str = str_ | bp::as_utf32;
auto r =
str |
bp::replace(
bp::lit("XYZ"), bp::ws, "foo" | bp::as_utf32, bp::trace::off);
int count = 0;
std::string_view const strs[] = {"aa", "foo", "baaba", "foo"};
for (auto subrange : r) {
std::string str(subrange.begin(), subrange.end());
EXPECT_EQ(str, strs[count]);
++count;
}
EXPECT_EQ(count, 4);
}
{
char const str_[] = "aaXYZbaabaXYZ";
auto str = str_ | bp::as_utf8;
auto r = str | bp::replace(bp::lit("XYZ"), "foo", bp::trace::off);
int count = 0;
std::string_view const strs[] = {"aa", "foo", "baaba", "foo"};
for (auto subrange : r) {
std::string str(subrange.begin(), subrange.end());
EXPECT_EQ(str, strs[count]);
++count;
}
EXPECT_EQ(count, 4);
}
{
char const str_[] = "aaXYZbaabaXYZ";
auto str = str_ | bp::as_utf16;
auto r = str | bp::replace(bp::lit("XYZ"), "foo");
int count = 0;
std::string_view const strs[] = {"aa", "foo", "baaba", "foo"};
for (auto subrange : r) {
std::string str(subrange.begin(), subrange.end());
EXPECT_EQ(str, strs[count]);
++count;
}
EXPECT_EQ(count, 4);
}
{
char const str_[] = "aaXYZbaabaXYZXYZ";
auto str = str_ | bp::as_utf32;
auto r = str | bp::replace(bp::lit("XYZ"), "foo");
int count = 0;
std::string_view const strs[] = {"aa", "foo", "baaba", "foo", "foo"};
for (auto subrange : r) {
std::string str(subrange.begin(), subrange.end());
EXPECT_EQ(str, strs[count]);
++count;
}
EXPECT_EQ(count, 5);
}
{
char const str_[] = "XYZaaXYZbaabaXYZXYZ";
auto str = str_ | bp::as_utf8;
auto r = str | bp::replace(bp::lit("XYZ"), "foo");
int count = 0;
std::string_view const strs[] = {
"foo", "aa", "foo", "baaba", "foo", "foo"};
for (auto subrange : r) {
std::string str(subrange.begin(), subrange.end());
EXPECT_EQ(str, strs[count]);
++count;
}
EXPECT_EQ(count, 6);
}
{
char const str_[] = "XYZXYZaaXYZbaabaXYZXYZ";
auto str = str_ | bp::as_utf16;
auto r = str | bp::replace(bp::lit("XYZ"), "foo");
int count = 0;
std::string_view const strs[] = {
"foo", "foo", "aa", "foo", "baaba", "foo", "foo"};
for (auto subrange : r) {
std::string str(subrange.begin(), subrange.end());
EXPECT_EQ(str, strs[count]);
++count;
}
EXPECT_EQ(count, 7);
}
{
char const str_[] = "XYZXYZaaXYZbaabaXYZXYZ";
auto str = str_ | bp::as_utf16;
auto r = str | bp::replace(bp::lit("XYZ"), "foo" | bp::as_utf8);
int count = 0;
std::string_view const strs[] = {
"foo", "foo", "aa", "foo", "baaba", "foo", "foo"};
for (auto subrange : r) {
std::string str(subrange.begin(), subrange.end());
EXPECT_EQ(str, strs[count]);
++count;
}
EXPECT_EQ(count, 7);
}
{
char const str_[] = "XYZXYZaaXYZbaabaXYZXYZ";
auto str = str_ | bp::as_utf16;
auto r = str | bp::replace(bp::lit("XYZ"), "foo" | bp::as_utf32);
int count = 0;
std::string_view const strs[] = {
"foo", "foo", "aa", "foo", "baaba", "foo", "foo"};
for (auto subrange : r) {
std::string str(subrange.begin(), subrange.end());
EXPECT_EQ(str, strs[count]);
++count;
}
EXPECT_EQ(count, 7);
}
}
// TODO: Document this.
#if BOOST_PARSER_USE_CONCEPTS && !defined(__GNUC__) || 12 <= __GNUC__
// Older GCCs don't like the use of temporaries like the std::string("foo")
// below. This causes | join to break.
TEST(replace, join_compat)
{
{
char const str[] = "XYZXYZaaXYZbaabaXYZXYZ";
auto rng = str | bp::as_utf32 |
bp::replace(bp::lit("XYZ"), "foo" | bp::as_utf8) |
std::views::join;
std::string replace_result;
for (auto ch : rng) {
static_assert(std::is_same_v<decltype(ch), char32_t>);
replace_result.push_back(ch);
}
EXPECT_EQ(replace_result, "foofooaafoobaabafoofoo");
}
{
char const str[] = "XYZXYZaaXYZbaabaXYZXYZ";
auto rng = str | bp::replace(bp::lit("XYZ"), "foo") | std::views::join;
std::string replace_result;
for (auto ch : rng) {
replace_result.push_back(ch);
}
EXPECT_EQ(replace_result, "foofooaafoobaabafoofoo");
}
{
std::string str = "XYZXYZaaXYZbaabaXYZXYZ";
auto rng = str | bp::replace(bp::lit("XYZ"), "foo") | std::views::join;
std::string replace_result;
for (auto ch : rng) {
replace_result.push_back(ch);
}
EXPECT_EQ(replace_result, "foofooaafoobaabafoofoo");
}
{
std::string const str = "XYZXYZaaXYZbaabaXYZXYZ";
auto rng = str | bp::replace(bp::lit("XYZ"), "foo") | std::views::join;
std::string replace_result;
for (auto ch : rng) {
replace_result.push_back(ch);
}
EXPECT_EQ(replace_result, "foofooaafoobaabafoofoo");
}
{
auto rng = std::string("XYZXYZaaXYZbaabaXYZXYZ") |
bp::replace(bp::lit("XYZ"), "foo") | std::views::join;
std::string replace_result;
for (auto ch : rng) {
replace_result.push_back(ch);
}
EXPECT_EQ(replace_result, "foofooaafoobaabafoofoo");
}
}
#endif
TEST(replace, doc_examples)
{
{
auto rng = "XYZaaXYZbaabaXYZXYZ" | bp::replace(bp::lit("XYZ"), "foo");
int count = 0;
// Prints foo aa foo baaba foo foo.
for (auto subrange : rng) {
std::cout << std::string_view(subrange.begin(), subrange.end() - subrange.begin()) << " ";
++count;
}
std::cout << "\n";
assert(count == 6);
}
}
#endif