Previously the mapping was eager on construction and calls to
`next()`. This was fine when generators were always exhausted fully,
but with the new generator filtering, we might not want to call
the map function eagerly; rather we want to call it only once,
after the generator is moved to the target element.
Not being able to filter generators to specific element has been regularly
causing problems. It was possible to use a dynamic section to run tests
for specific element in a generator, at least if the element had a nice
string representation, but the test case would still run once per element
in the generator.
With this change, it is possible to have the generator return only one
specific element, and do so based on the index, rather than the string
representation of the element. This enables simple debugging of tests
that fail for specific generator element.
* Removed `IGeneratorTracker::hasGenerator`, as it was never used.
* Removed `IGeneratorTracker::setGenerator`, as actually using it
after the tracker was used would lead to inconsistent behaviour.
Instead, the tracker is given the generator it guards during
construction.
* Add validation for --benchmark-samples to prevent crash with zero samples
* Add automated test for benchmark samples validation
* Update baselines for benchmark-samples validation
* Add missing test for benchmark samples validation
---------
Co-authored-by: Chan Aung <chan@Thinkpad.localdomain>
Suppress `Global initializer calls a non-constexpr function 'Catch::makeTestInvoker' (i.22).`,
which is issued when using macro `TEST_CASE` for windows/MSVC builds.
---------
Co-authored-by: Martin Hořeňovský <martin.horenovsky@gmail.com>
The original implementation of `UNSCOPED_X` message macros used a
clever hack to make the original implementation simpler: construct
an instance of `ScopedMessage` to manage its lifetime, but store
it in a vector, so its lifetime is not actually scope-based, and
we can manage it through the vector instance.
This hack made it so that the lifetime of the vector that manages
the fake `ScopedMessage`s must be outlived by the vector with the
actual messages. Originally this wasn't a problem, because they both
lived inside the run context instance. However, since then these
vectors became globals and thread-local. When this happened, it
still wasn't a problem; the two globals were declared in the right
order, so they were destroyed in the right order as well.
Then, in f80956a43a, these globals
were turned into magic static globals to improve their behaviour
in MSVC's Debug build mode. This caused their lifetimes to be
runtime-dependent; if a specific test thread added its first scoped
message before it added first unscoped message, the lifetimes
would be correct. If it instead added first unscoped message
before adding first scoped message, then there **might** be
invalid reads during thread destruction.
The fix is simple: do things properly and manage the lifetime of
messages in `UNSCOPED_X` explicitly. Then we don't have to deal
with the destruction of fake `ScopedMessage`s while the thread is
being destroyed, and the lifetime of the two vectors is no longer
tied together.
I also threw them both into a new type, to encapsulate some of the
unscoped message logic.
This avoids calling the global's constructor on threads that will
never interact with them. Calling the constructor can have surprising
overhead, as e.g. MSVC's Debug mode `std::vector` will allocate in
the default constructor.
Closes#3050
Specifically, this commit makes the `-c`/`--section` parameter
strictly ordered and hierarchical, unlike how it behaved before,
which was a huge mess -- see #3038 for details.
Closes#3038
The matcher combinators do not take ownership of the matchers
being combined, which can catch the users off-guard, when code like
this
```cpp
using Catch::Matchers::EndsWith;
using Catch::Matchers::ContainsSubstring;
auto combinedMatcher = EndsWith("as a service")
&& ContainsSubstring("web scale");
REQUIRE_THAT( getSomeString(), combinedMatcher );
```
leads to use-after-free, as the `combinedMatcher` refers to matcher
temporaries that no longer exists. With this commit, users of Clang,
MSVC or other compiler that understands the `lifetimebound` attribute,
should get a warning.
This commit causes small (~5%) slowdown when processing strings
that consist mostly of escaped characters, but improves throughput
by about 100% for strings that consist mostly of characters that
do not need escaping.
We still keep the error check in the function, but hide it in an
outlined function inside a .cpp file, to promote inlining of the
retrieval part.
In the future, we should explore two things
1) Skipping over the context retrieval here, allowing direct access.
I currently do not see a way to do this while keeping the
"greppability" of mutable vs immutable accesses that is there now,
but it would help a lot when inlining is not enabled.
2) Removing the error check, to make the function trivially inlinable,
and without branches.
**runtime difference**
| --------- | Debug | Release |
|:----------|------:|--------:|
| Slow path | 0.98 | 1.07 |
| Fast path | 1.04 | 1.08 |
We lost bit of performance on the assertion slow path in debug mode,
but together with the previous commit, it comes out at net zero.
For other combinations, we see 5-10% perf improvement across the
two commits.
This allows us to remove the lazy init checks, improving the inlining
potential when retrieving current context, thus slightly improving
the performance of assertions.
**runtime difference**
| --------- | Debug | Release |
|:----------|------:|--------:|
| Slow path | 1.01 | 0.98 |
| Fast path | 1.02 | 1.02 |
There is small slowdown in case of Release build + assertions taking
the slow path, but
1) going through the slow path is rare
2) Given the code change, I believe this to be artifact of the
optimizer in the old GCC version I am using locally.
The change is very simple. If a handler previously existed, Catch2 will invoke it after printing out its output. I've also updated the comment to better reflect that it's returning EXCEPTION_CONTINUE_SEARCH even in scenarios where the exception is one that the library cares about.
Commit 582200a made `ReusableStringStream`'s index reservation thread safe, however it's still accessing the `m_streams` vector outside the lock. This makes it so that the `add` call returns the pointer in addition to the index so that the lock doesn't need to get acquired again until destruction.
The issue with accessing `m_streams` outside the lock is that `add` can call `push_back` on the vector, which might re-allocate. If this re-allocation occurs concurrently with anther thread trying to index into this array, you get UB (typically a null pointer read).
This cherry-picks the source changes from PR made by @pkleymonov-qnx,
without including the build path changes that are irrelevant unless
we can add a QNX target to our CI.
Close#2953
This tells Catch2 to create an empty file at specified path before
the tests start, and delete it after the tests finish. This allows
callers to catch cases where the test binary silently exits before
finishing (e.g. via call to `exit(0)` inside the code under test),
by looking whether the file still exists.
Closes#3020
