Integers with digit separators are already recognized as literal
tokens. This commit adds proper interpretation of them as integers for
use in expressions e.g. with #if
- Fix RE2C code for numbers (binary and digit separators)
- Revert to RE2C version 1.0.2, for now
- Revamp token ids to minimize changes
- Restore existing and more accurate name OPTSIGN in slex
- Add binary literal support to lexertl
- Fix xlex support for size_t literals
- Add test tokens for octal, binary, and hex literals
* fix segment fault of empty stringify under C++20
* Add tests for optional comma in variadic macro call
---------
Co-authored-by: Jeff Trull <edaskel@att.net>
* Make the library modular usable.
* Switch to library requirements instead of source. As source puts extra source in install targets.
* Add wave tool build to all target.
* Add missing NO_LIB usage requirements.
* Add requires-b2 check to top-level build file.
* Bump B2 require to 5.2
* Change all <source> references to <library>.
* Move inter-lib dependencies to a project variable and into the build targets.
* Update build deps.
The previous code checked for signed overflow on +, -, and * by doing
the operation and checking the result, which can invoke undefined
behavior. This replaces that code with initial tests on the operands.
In addition, a test for the sole integer division overflow case (that
is not division by zero) was missing: INT_MIN / -1
It has now been added, along with a test case.
* Update marker before and after fill() operation in cpplexer
* A basic test case for trigraph "pound" (octothorpe) on fill boundary (detects the marker issue)
This change resolves#202
---------
Co-authored-by: Chris Chisolm <chris_chisolm@intuit.com>
Co-authored-by: Jeff Trull <edaskel@att.net>
Updates the non-strict lexer to support e and E, both with value 27
(ASCII ESC). Also adds tests for lexing these literals, and checks
that they are usable in conditional preprocessor expressions.
* Fix line directives after endif when else clause is present
The initial part of a conditional block (the if/ifdef/ifndef) takes
the "slow" path through the code in cpp_iterator.hpp using the C++
expression grammar, while else and endif directives take the "fast"
path. The slow path sets must_emit_line_directive, and it will be
cleared after either else or endif is processed. Unfortunately when
an else precedes the endif, must_emit_line_directive is cleared and is
never set, as only the slow path sets it. The result is proper line
directives are issued for the else clause, but not for the subsequent
endif.
This changes adds setting of must_emit_line_directive in the "fast"
path for endif so line directives are considered after an intervening else.
* Check hooks after expected errors, not just when there are none
Some unit tests had intentional errors, and code to verify that the
appropriate hooks were called, but we never checked.
* Update unit tests to match hooks, now that we are checking
* Fix bug revealed by checking the hooks
Or in other words, bug #161 (thanks abakhirkin for the fix)
* Fix link in documentation
* Extend __has_include grammar for complex conditional expressions
The logic handling __has_include erroneously assumed it would always
be the last expression on a line. This code fixes that by extending
the grammar to only consume the __has_include() itself, and count
parentheses as necessary to find the correct input range.
Also add some unit tests to cover the error in the future.
* While I'm at it, a couple of small cleanups
1) Remove unneeded header
2) The wrong BOOST_SPIRIT_DEBUG_TRACE_RULE was commented out
In C++11 and later modes Wave by default does not require newline prior to EOF. This works
fine but there is one place where the test for this feature was used but Wave is not necessarily
looking at the end of the file: pp_is_last_on_line. One observable (and surprising)
result is that unknown directives are not getting flagged - there may be others. The unit tests
did not discover this problem because they generally do not set c++11 mode.
The necessary qualification is added, along with unit test to cover this issue.
At some point in the past the handling for #if and #ifdef diverged. The code that handles emitting a line directive when a conditional section is skipped worked for #if but not ifdef/ifndef.
This problem was not observable when the eat_whitespace hooks were used instead of the default_preprocessing hooks, because the former
signals skipped newlines through the may_skip_whitespace hook, hiding the problem. Furthermore, the majority of Wave tests use the eat_whitespace hooks, so it wasn't visible there.
This change restores ifdef/ifndef to the same section as #if, so any changes to conditional handling will happen uniformly. Also, a test case is added to cover the default hooks and this particular case.
These changes, a subset of what clang-format generates, ensure code
and comments are aligned properly for the level of the code. There
were a number of places where comments (many) and code (some) were
four spaces to the left of where they should be. This will make the
code easier to read.
One advantage of separating out these changes is they can be verified
to be whitespace-only with a simple diff.
Support was already in slex, though it wasn't covered by the token
test. This updates the tests while adding virtually identical code
to lexertl and xlex.
An (optional) extra position field is added to token_data and set for
any identifier token created from a macro expansion. This information
is used to correctly calculate the filename and line number.
Reduce build time by:
- using iosfwd instead of iostream, where possible
- where iostream is necessary but only used for debugging,
making its inclusion conditional
- removing old C-style headers
I used IWYU to guide this work
* Run hooks for certain predefined object-like macros
These are the "dynamic" macros with their own special execution path.
Most predefined macros can be computed at startup time; those were
already covered by the normal expansion code.
* __VA_OPT__ is supported as a function-like macro
It is referred to as such in the proposal. __VA_ARGS__ is not supported, under the theory that it represents a parameter or set of parameters, and no hooks are called for parameter substitutions.
* Pass tests by matching lexertl blank handling
* Remedial improvement to position tracking to avoid exception
* Fix token statistics sample's count map index calculation
* Add xlex to test suite
Also, prune includes somewhat.
* Introduce support for C++20 preprocessor features
- add __VA_OPT__ feature to variadic macros
- allow supplying 0 variadic arguments in more cases
- add related unit tests
