2
0
mirror of https://github.com/boostorg/parser.git synced 2026-01-21 05:02:14 +00:00

15 Commits

Author SHA1 Message Date
Zach Laine
92c4993b87 Documentation pass on the changes made so far to support token parsing.
See #202.
2024-11-29 20:00:28 -06:00
Zach Laine
f298bfe59b Remove the support for token_view::iterator -> underlying iterator translation
in *parse_impl(); add support to the error handlers and their support
functions directly instead.  There are simply too many APIs there that need
the translation to leave it to other code.

Add lex_error exception type, and add support for all the APIs that used to
take a parse_error param to now take either a parse_error or a lex_error.

Throw lex_error from failed parsing of lexed tokens in detail::make_token().

See #202.
2024-11-29 16:29:55 -06:00
Zach Laine
092a76173b Change token<> to include the position of the start of the token in the
underlying sequence, and change the way that the error handler is invoked, so
that it detects token iterators, and passes iterators into the underlying
range to the error handler, instead of the token iterators.

See #202.
2024-11-29 16:29:55 -06:00
Zach Laine
e419ef2a60 Completely rethink the relationship among token_parser, token_spec_t, and
parser_interface.  token_spec is now a variable template that generates a
parser_interface wrapping a token_parser, which parameterized on the
token_spec_t.  This way, a single token_spec use can be used to specify how to
lex, and how to parse.

See #202.
2024-11-29 16:29:55 -06:00
Zach Laine
f00f4dfa75 Initial, partial sketch of token_parser.
See #202.
2024-11-29 16:29:55 -06:00
Zach Laine
358adf247a Address the remaining non-documentation TODOs in the lexer header and tests.
See #202.
2024-11-29 16:29:55 -06:00
Zach Laine
ee8ab13779 Extend the lexer tests, addressing some of the testing TODOs; fix errors.
See #202.
2024-11-29 16:29:55 -06:00
Zach Laine
391bb2b5b0 Factor out a lot of the lexer.cpp tests into a separate file.
See #202.
2024-11-29 16:29:55 -06:00
Zach Laine
6ed1152390 Switch from double to long double as token's floating point representation.
Therer's room for it, since it's in a union with a string_view anyway.

See #202.
2024-11-29 16:29:55 -06:00
Zach Laine
bf336fb096 Add much longer lexing test; fix errors.
See #202.
2024-11-29 16:29:55 -06:00
Zach Laine
63483cb284 Specify the true/false and number token_specs in the adobe lexer test code to
use a bool and a double as its value, repsectively, instead of a string_view.

See #202.
2024-11-29 16:29:55 -06:00
Zach Laine
49213c428e Change all the existing lex runs to compare their output against an expected
sequence of tokens.

See #202.
2024-11-29 16:29:55 -06:00
Zach Laine
79f34ef252 Add support for an externally-supplied token cache.
See #202.
2024-11-29 16:29:55 -06:00
Zach Laine
cd9c7492dd Add a range adaptor for token_view; token_view -> tokens_view. Break
detail::make_input_subrange() out into its own header for reuse.

See #202.
2024-11-29 16:29:55 -06:00
Zach Laine
e5c101378b Initial sketch of a lexer for Boost.Parser, based on using CTRE as an external
dependency.  Lots of TODOs, somewhat thin testing, but the basics work.

See #202.
2024-11-29 16:29:55 -06:00