Zach Laine
092a76173b
Change token<> to include the position of the start of the token in the
...
underlying sequence, and change the way that the error handler is invoked, so
that it detects token iterators, and passes iterators into the underlying
range to the error handler, instead of the token iterators.
See #202 .
2024-11-29 16:29:55 -06:00
Zach Laine
e419ef2a60
Completely rethink the relationship among token_parser, token_spec_t, and
...
parser_interface. token_spec is now a variable template that generates a
parser_interface wrapping a token_parser, which parameterized on the
token_spec_t. This way, a single token_spec use can be used to specify how to
lex, and how to parse.
See #202 .
2024-11-29 16:29:55 -06:00
Zach Laine
f00f4dfa75
Initial, partial sketch of token_parser.
...
See #202 .
2024-11-29 16:29:55 -06:00
Zach Laine
358adf247a
Address the remaining non-documentation TODOs in the lexer header and tests.
...
See #202 .
2024-11-29 16:29:55 -06:00
Zach Laine
ee8ab13779
Extend the lexer tests, addressing some of the testing TODOs; fix errors.
...
See #202 .
2024-11-29 16:29:55 -06:00
Zach Laine
391bb2b5b0
Factor out a lot of the lexer.cpp tests into a separate file.
...
See #202 .
2024-11-29 16:29:55 -06:00
Zach Laine
6ed1152390
Switch from double to long double as token's floating point representation.
...
Therer's room for it, since it's in a union with a string_view anyway.
See #202 .
2024-11-29 16:29:55 -06:00
Zach Laine
bf336fb096
Add much longer lexing test; fix errors.
...
See #202 .
2024-11-29 16:29:55 -06:00
Zach Laine
63483cb284
Specify the true/false and number token_specs in the adobe lexer test code to
...
use a bool and a double as its value, repsectively, instead of a string_view.
See #202 .
2024-11-29 16:29:55 -06:00
Zach Laine
49213c428e
Change all the existing lex runs to compare their output against an expected
...
sequence of tokens.
See #202 .
2024-11-29 16:29:55 -06:00
Zach Laine
79f34ef252
Add support for an externally-supplied token cache.
...
See #202 .
2024-11-29 16:29:55 -06:00
Zach Laine
cd9c7492dd
Add a range adaptor for token_view; token_view -> tokens_view. Break
...
detail::make_input_subrange() out into its own header for reuse.
See #202 .
2024-11-29 16:29:55 -06:00
Zach Laine
e5c101378b
Initial sketch of a lexer for Boost.Parser, based on using CTRE as an external
...
dependency. Lots of TODOs, somewhat thin testing, but the basics work.
See #202 .
2024-11-29 16:29:55 -06:00
