Because of the changed semantics of appending operations in v4, path
composition in absolute() would produce incorrect results because at some
point it would append root directory and therefore discard root name
that was potentially added before. The updated implementation fixes that,
and also fixes the case when the input path is already absolute and
starts with a root directory, and the base path has a root name.
Previously, the returned path would contain the root name from the
base path, while the correct thing to do is to return the input path
as is.
The new implementation is prepared for the removal of the implicit
trailing dots in v4 path. It also no longer uses recursion
internally and therefore is better protected against stack overflows.
As a side effect of this rewrite, create_directories no longer reports
error if the input path consists entirely of dot and dot-dot elements.
This is in line with C++20 std::filesystem behavior.
linux/magic.h was introduced in Linux kernel 2.6.19, building Boost.Filesystem
with older kernel headers would fail because of this. Only include the header
when it is found and fallback to our local constant definitions when it's not.
Windows APIs such as GetFileAttributesW perform lexical path normalization
internally, which means e.g. "C:\a\.." resolves to an existing path
even if "C:\a" doesn't. This breaks depection of the longest sequence
of existing path elements in weakly_canonical and results in an error
in canonical that is called on that sequence.
As a workaround, perform forward iteration on Windows, so that we
stop on the first path element that doesn't exist.
Also, while at it, corrected error code reported from weakly_canonical
when status fails with an error.
Closes https://github.com/boostorg/filesystem/issues/201.
Instead of using atomic<> to access global function pointers, use raw
pointers and atomic_ref to access them safely in multi-threaded builds.
This allows to ensure constant initialization of the function pointers,
even in C++03. This also solves the problem of undefined dynamic
initialization order that we previously tried to solve with the
init_priority attribute. The attribute turns out to not work if the
pointers were raw pointers (in single-threaded builds). It is also
not supported by Intel Compiler and possibly other, which required
us to avoid using the function pointer for fill_random.
The resulting code should be simpler and more portable. In order to
check for C++20 std::atomic_ref availability, the older check for <atomic>
header was replaced with a check for std::atomic_ref. If not available,
we're using Boost.Atomic, as before.
Fall back to reading /dev/(u)random if getrandom fails with ENOSYS.
Also, extracted the portability macros for atomics to a separate header
to be able to use them in unique_path.cpp. Rearranged function pointers
initialization to decouple the initializer object from the particular
system calls.
For getrandom, the ENOSYS failure is only cached if the compiler supports
specifying global object initialization priority, which is needed to ensure
that the function pointer is initialized before the syscall initializer
in a different TU. If the compiler does not support this feature, just
always attempt getrandom first.
This can be useful if the syscall is present at compile time but fails with
ENOSYS at run time (for example, in Docker containers that restrict the syscall,
even if available on the host).
Additionally, marked statx syscall wrappers with attributes to disable MSAN
for them. It was reported that MSAN on clang 10 is showing errors
accessing uninitialized data in stx_mask, which must be initialized by the
syscall.
Related to https://github.com/boostorg/filesystem/issues/172
Related to https://github.com/boostorg/filesystem/issues/185
If Windows is running in Developer mode, it is possible to specify
SYMBOLIC_LINK_FLAG_ALLOW_UNPRIVILEGED_CREATE flag to CreateSymbolicLinkW
so that the call doesn't require elevated privileges.
While at it, explicitly separated implementation of create_symlink and
create_directory_symlink for POSIX and Windows.
The print name can be empty for some reparse points (e.g. mount points
created by Box cloud storage driver and directory junctions created by
junction.exe). It is supposed to be mostly used for presenting a "simple"
path to the user and not to actually locate the file.
The substitute name is the actionable replacement path, but it is in
NT path format and can potentially point to unmounted volumes and
UNC resources. The implementation attempts to convert the NT path
to Win32 path by matching commonly known patterns against the NT path.
If no pattern matches, we create a Win32 path by converting the NT path
prefix to "\\?\".
Related to https://github.com/boostorg/filesystem/issues/187.
When current_path on POSIX falls back to the dynamically allocated
buffer for the resulting path, start with double the size of the
small stack buffer that was used initially.
When canonical() resolves symlink, it is possible that a symlink resolves
to an absolute path with a different root. We need to update the root
path so that when we restart symlink resolution the check for the
root path still works.
Also, slightly refactored the canonical() implementation to reduce code
size and possibly optimize the generated code.
There was a missing calling convention specification in the CopyFileEx
callback. This was not a problem in 64-bit builds since there is no
stdcall convention in 64-bit x86.
When CopyFileEx copies multi-stream files, the callback is executed for each
stream separately. Each stream is represented with a separate file handle,
so we have to flush buffers when the stream is fully copied, rather than the
whole file.
Use FlushFileBuffers to force any buffered data written to the permanent
storage. The previously used COPY_FILE_NO_BUFFERING flag only guarantees
that no data is left in the OS filesystem cache, but does not ensure
that any device buffers are flushed.
In single-threaded builds we can assume no thread synchronization
is necessary and avoid the dependency on Boost.Atomic. The dependency
caused single-threaded build failures because Boost.Atomic requires
multithreading to be enabled.
The CMake build currently does not support single-threaded builds, so
the dependency is left present there.
Closes https://github.com/boostorg/filesystem/issues/188.
The buffer size is now selected based on the file size and filesystem block
size and is limited with min and max. This allows to reduce memory consumption
and possibly increase performance when copying smaller files.
Some filesystems have regular files with generated content. Such files have
arbitrary size, including zero, but have actual content. Linux system calls
sendfile or copy_file_range will not copy contents of such files, so we must
use a read/write loop to handle them.
Check the type of the source filesystem before using sendfile or
copy_file_range and fallback to the read/write loop if it matches one of
the blacklisted filesystems: procfs, sysfs, tracefs or debugfs.
Also, added a test to verify that copy_file works on procfs.
Since sendfile and copy_file_range can fail for some filesystems
(e.g. eCryptFS), we have to fallback to the read/write loop in copy_file
implementation. Additionally, since we implement the fallback now,
fallback to sendfile if copy_file_range fails with EXDEV and use
copy_file_range on older kernels that don't implement it for
cross-filesystem copying. This may be beneficial if copy_file_range
is used within a filesystem, and is performed on a remote server NFS or CIFS).
Also, it was discovered that copy_file_range can also fail with EOPNOTSUPP
when it is performed on an NFSv4 filesystem and the remote server does
not support COPY operation. This happens on some patched kernels in RHEL/CentOS.
Lastly, to make sure the copy_file_data pointer is accessed atomically,
it is now declared as an atomic value. If std::atomic is unavailable,
Boost.Atomic is used.
Fixes https://github.com/boostorg/filesystem/issues/184.
These options allow to synchronize the copied data and attributes with
the permanent storage. Note that by default on POSIX systems copy_file
used to synchronize data in previous releases, and this commit changes
this. The caller now has to explicitly request syncing, as it has
significant performance implications.
Closes https://github.com/boostorg/filesystem/issues/186.
At least HP-UX is known to leave the file descriptor open if close() returns
EINTR. On other systems (Linux, BSD, Solaris, AIX) the file descriptor
is closed in the same situation, and closing it again may potentially close
the wrong descriptor if it is reused by another thread. We introduce
close_fd internal helper to abstract away these platform differences.
By defining these new config macros the user can configure the library
to avoid using some system APIs even if they are detected as available
by the library build scripts. This can be useful in case if the API
is known to consistently fail at runtime on the target system.
Related to https://github.com/boostorg/filesystem/issues/172.
create_directories used to ignore errors returned by status()
calls issued internally. The operation would likely fail anyway,
but the error codes returned by create_directories would be incorrect.
Also, it is better to terminate the operation as early as possible
when an error is encountered.
Reported in https://github.com/boostorg/filesystem/issues/182.
