- Fix RE2C code for numbers (binary and digit separators)
- Revert to RE2C version 1.0.2, for now
- Revamp token ids to minimize changes
- Restore existing and more accurate name OPTSIGN in slex
- Add binary literal support to lexertl
- Fix xlex support for size_t literals
- Add test tokens for octal, binary, and hex literals
* Make the library modular usable.
* Switch to library requirements instead of source. As source puts extra source in install targets.
* Add wave tool build to all target.
* Add missing NO_LIB usage requirements.
* Add requires-b2 check to top-level build file.
* Bump B2 require to 5.2
* Change all <source> references to <library>.
* Move inter-lib dependencies to a project variable and into the build targets.
* Update build deps.
Updates the non-strict lexer to support e and E, both with value 27
(ASCII ESC). Also adds tests for lexing these literals, and checks
that they are usable in conditional preprocessor expressions.
Under Visual Studio 2019 (and possibly 2017), building with /permissive- causes three samples to fail to compile. Adding the /Zc:twoPhase- option makes them compile successfully again. See bug #160 for more details.
It's unclear what the source of this issue is, but using constexpr variables instead of a macro is better practice anyway, and fixes the errors.
Clang warns that a for loop in a sample increments an iterator in both
the header and the body. This seems to be done on purpose to skip
whitespace. I used std::advance in the header instead, for clarity.
Also cleaned up an unused type in the same file while I was there.
Support was already in slex, though it wasn't covered by the token
test. This updates the tests while adding virtually identical code
to lexertl and xlex.
An (optional) extra position field is added to token_data and set for
any identifier token created from a macro expansion. This information
is used to correctly calculate the filename and line number.
filesystem paths in MSVC are based on wchar_t so my 8b regex (and
std::string) were a problem for matching against native paths. Trying
again with the generic std::string interface to see if it works.
* Introduce a new sample that checks macro naming conventions
This has a direct use in Boost's own QA checks and demonstrates
the use of macro definition and include guard callbacks, as well
as Boost.Filesystem.
Reduce build time by:
- using iosfwd instead of iostream, where possible
- where iostream is necessary but only used for debugging,
making its inclusion conditional
- removing old C-style headers
I used IWYU to guide this work
* Pass tests by matching lexertl blank handling
* Remedial improvement to position tracking to avoid exception
* Fix token statistics sample's count map index calculation
* Add xlex to test suite
Also, prune includes somewhat.
The main programs of these samples use std::string to instantiate
their context object, which propagates down to the type of the lexer.
The manual instantiations use BOOST_WAVE_STRINGTYPE a.k.a flex_string
and so do not satisfy the missing symbols, giving a link error.
The macro `BOOST_NO_AUTO_PTR` is defined on systems where both `auto_ptr` and
`unique_ptr` are available (and `auto_ptr` usage produces a deprecation warning).
By using `BOOST_NO_CXX11_SMART_PTR` we will favor `unique_ptr` over `auto_ptr`.
```
slex/lexer.hpp:2423:13: error: 'template<class> class std::auto_ptr' is deprecated [-Werror=deprecated-declarations]
```
Jeff Trull