boost/website-v2-docs
2
0
Fork 0
mirror of https://github.com/boostorg/website-v2-docs.git synced 2026-01-27 19:32:16 +00:00
Code Issues Projects Releases Wiki Activity
Files
develop
website-v2-docs/user-guide/modules/ROOT/pages/task-text-processing.adoc
Peter Turcan 126d849c83 Text Processing scenario updated (#497)
2025-09-04 11:34:01 -07:00

476 lines
16 KiB
Plaintext
Raw Permalink Blame History

////
Copyright (c) 2024 The C++ Alliance, Inc. (https://cppalliance.org)
Distributed under the Boost Software License, Version 1.0. (See accompanying
file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
Official repository: https://github.com/boostorg/website-v2-docs
////
= Text Processing
:navtitle: Text Processing
Developing a word processor, or other text based app, involves handling text, GUI (Graphical User Interface), file operations, and possibly networking for cloud features. Boost does not provide a library for creating a GUI. You may want to consider using a library like https://www.qt.io/product/development-tools[Qt] or https://wxwidgets.org/[wxWidgets] for the GUI part of your word processor.
[square]
* <<Libraries>>
* <<Sample of Regular Expression Parsing>>
* <<Add Robust Date and Time Parsing>>
* <<Culturally Aware Date Formatting>>
* <<Local Time>>
* <<See Also>>
== Libraries
Here are some Boost libraries that might assist you in processing text:
[circle]
* boost:regex[]: For some simpler parsing tasks, regular expressions can be sufficient and easier to use than full-blown parsing libraries. You could use these features to match specific patterns in your input text, like specific commands or phrases, word boundaries, etc.
* boost:locale[] : This library provides a way of handling and manipulating text in a culturally-aware manner. It provides localization and internationalization facilities, allowing your word processor to be used by people with different languages and locales.
* boost:spirit[] : This library is a parser framework that can parse complex data structures. If you're creating a word processor, it could be useful to interpret different markup and file formats.
* boost:date_time[] : If you need to timestamp changes or edits, or if you're implementing any kind of version history feature, this library can help.
* boost:filesystem[] : This library provides a way of manipulating files and directories. This would be critical in a word processor for opening, saving, and managing documents.
* boost:asio[] : If your word processor has network-related features, such as real-time collaboration or cloud-based storage, boost:asio[] provides a consistent asynchronous model for network programming.
* boost:serialization[] : This library provides a way of serializing and deserializing data, which could be useful for saving and loading documents in a specific format.
* boost:xpressive[] : Could be useful for implementing features like search and replace, spell-checking, and more.
* boost:algorithm[] : This library includes a variety of algorithms for string and sequence processing, which can be useful for handling text.
* boost:multi-index[] : This library provides a way of maintaining a set of items sorted according to multiple keys, which could be useful for implementing features like an index or a sorted list of items.
* boost:thread[] : If your application is multithreaded (for example, if you want to save a document while the user continues to work), this library will be useful.
Note:: The code in this tutorial was written and tested using Microsoft Visual Studio (Visual C++ 2022, Console App project) with Boost version 1.88.0.
== Sample of Regular Expression Parsing
If the text you are parsing is well-formatted then you can use boost:regex[] which we will base our sample on here, rather than a full-blown parser implementation using boost:spirit[].
We'll write a program that scans a string for dates in the format "YYYY-MM-DD" and validates them. The code:
. Finds dates in text
. Validates correct formats (for example, 2024-02-20 is valid, but 2024-15-45 is not)
. Handles multiple dates in a single input string
[source,cpp]
----
#include <iostream>
#include <boost/regex.hpp>
#include <boost/algorithm/string.hpp>
// Function to check if a given date is valid (basic validation)
bool is_valid_date(int year, int month, int day) {
if (month < 1 || month > 12 || day < 1 || day > 31) return false;
if ((month == 4 || month == 6 || month == 9 || month == 11) && day > 30) return false;
if (month == 2) {
bool leap = (year % 4 == 0 && year % 100 != 0) || (year % 400 == 0);
if (day > (leap ? 29 : 28)) return false;
}
return true;
}
// Function to find and validate dates in a text
void find_dates(const std::string& text) {
// Regex pattern: YYYY-MM-DD format
boost::regex date_pattern(R"((\d{4})-(\d{2})-(\d{2}))");
boost::smatch match;
std::string::const_iterator start = text.begin();
std::string::const_iterator end = text.end();
bool found = false;
while (boost::regex_search(start, end, match, date_pattern)) {
int year = std::stoi(match[1]);
int month = std::stoi(match[2]);
int day = std::stoi(match[3]);
if (is_valid_date(year, month, day)) {
std::cout << "Valid date found: " << match[0] << "\n";
} else {
std::cout << "Invalid date: " << match[0] << " (Incorrect month/day)\n";
}
start = match[0].second; // Move to next match
found = true;
}
if (!found) {
std::cout << "No valid dates found in the input text.\n";
}
}
int main() {
std::string input;
std::cout << "Enter a sentence containing dates (YYYY-MM-DD format):\n";
std::getline(std::cin, input);
find_dates(input);
return 0;
}
----
The following shows a successful parse:
[source,text]
----
Enter a sentence containing dates (YYYY-MM-DD format):
Today is 2024-02-19, and tomorrow is 2024-02-20.
Valid date found: 2024-02-19
Valid date found: 2024-02-20
----
And the following shows several unsuccessful parses:
[source,text]
----
Enter a sentence containing dates (YYYY-MM-DD format):
The deadline is 2024-02-30.
Invalid date: 2024-02-30 (Incorrect month/day)
Enter a sentence containing dates (YYYY-MM-DD format):
There are no dates in this sentence.
No valid dates found in the input text.
----
== Add Robust Date and Time Parsing
The clunky date validation in the sample above can be improved by integrating boost:date_time[], which provides functions for handling dates and times correctly.
[source,cpp]
----
#include <boost/regex.hpp>
#include <boost/date_time/gregorian/gregorian.hpp>
namespace greg = boost::gregorian;
// Function to check if a date is valid using Boost.Date_Time
bool is_valid_date(int year, int month, int day) {
try {
greg::date test_date(year, month, day);
return true; // If no exception, it's valid
}
catch (const std::exception& e) {
return false; // Invalid date
}
}
// Function to find and validate dates in a text
void find_dates(const std::string& text) {
boost::regex date_pattern(R"((\d{4})-(\d{2})-(\d{2}))");
boost::smatch match;
std::string::const_iterator start = text.begin();
std::string::const_iterator end = text.end();
bool found = false;
while (boost::regex_search(start, end, match, date_pattern)) {
int year = std::stoi(match[1]);
int month = std::stoi(match[2]);
int day = std::stoi(match[3]);
if (is_valid_date(year, month, day)) {
greg::date valid_date(year, month, day);
std::cout << "Valid date found: " << valid_date << "\n";
}
else {
std::cout << "Invalid date: " << match[0] << " (Does not exist)\n";
}
start = match[0].second; // Move to next match
found = true;
}
if (!found) {
std::cout << "No valid dates found in the input text.\n";
}
}
int main() {
std::string input;
std::cout << "Enter a sentence containing dates (YYYY-MM-DD format):\n";
std::getline(std::cin, input);
find_dates(input);
return 0;
}
----
Note:: The code handles leap years correctly, and invalid dates throw an exception.
The following shows a successful parse:
[source,text]
----
Enter a sentence containing dates (YYYY-MM-DD format):
Today is 2024-02-29, and tomorrow is 2024-03-01.
Valid date found: 2024-Feb-29
Valid date found: 2024-Mar-01
----
Note:: The "Valid date found" output now includes text for the month name.
And the following shows several unsuccessful parses:
[source,text]
----
Enter a sentence containing dates (YYYY-MM-DD format):
The deadline is 2024-02-30.
Invalid date: 2024-02-30 (Does not exist)
Enter a sentence containing dates (YYYY-MM-DD format):
There are no dates in this sentence.
No valid dates found in the input text.
----
== Culturally Aware Date Formatting
Dates are not represented consistently across the globe. Let's use boost:locale[] to format dates according to the user's locale. For example:
* US: March 15, 2024
* UK: 15 March, 2024
* France: 15 mars 2024
* Germany: 15. März 2024
[source,cpp]
----
#include <boost/regex.hpp>
#include <boost/date_time/gregorian/gregorian.hpp>
#include <boost/locale.hpp>
namespace greg = boost::gregorian;
namespace loc = boost::locale;
// Function to check if a date is valid using Boost.Date_Time
bool is_valid_date(int year, int month, int day) {
try {
greg::date test_date(year, month, day);
return true; // If no exception, it's valid
}
catch (const std::exception&) {
return false; // Invalid date
}
}
// Function to format and display dates based on locale
void display_localized_date(const greg::date& date, const std::string& locale_name) {
std::locale locale = loc::generator().generate(locale_name);
std::cout.imbue(locale); // Apply locale to std::cout
std::cout << locale_name << " formatted date: "
<< loc::as::date << date << "\n";
}
// Function to find and validate dates in a text
void find_dates(const std::string& text, const std::string& locale_name) {
boost::regex date_pattern(R"((\d{4})-(\d{2})-(\d{2}))");
boost::smatch match;
std::string::const_iterator start = text.begin();
std::string::const_iterator end = text.end();
bool found = false;
while (boost::regex_search(start, end, match, date_pattern)) {
int year = std::stoi(match[1]);
int month = std::stoi(match[2]);
int day = std::stoi(match[3]);
if (is_valid_date(year, month, day)) {
greg::date valid_date(year, month, day);
std::cout << "Valid date found: " << valid_date << "\n";
display_localized_date(valid_date, locale_name);
}
else {
std::cout << "Invalid date: " << match[0] << " (Does not exist)\n";
}
start = match[0].second; // Move to next match
found = true;
}
if (!found) {
std::cout << "No valid dates found in the input text.\n";
}
}
int main() {
std::locale::global(loc::generator().generate("en_US.UTF-8")); // Default global locale
std::cout.imbue(std::locale()); // Apply to output stream
std::string input;
std::cout << "Enter a sentence containing dates (YYYY-MM-DD format):\n";
std::getline(std::cin, input);
std::string user_locale;
std::cout << "Enter your preferred locale (e.g., en_US.UTF-8, fr_FR.UTF-8, de_DE.UTF-8): ";
std::cin >> user_locale;
find_dates(input, user_locale);
return 0;
}
----
The following shows successful parses:
[source,text]
----
Enter a sentence containing dates (YYYY-MM-DD format):
The meeting is on 2024-03-15.
Enter your preferred locale (e.g., en_US.UTF-8, fr_FR.UTF-8, de_DE.UTF-8): en_US.UTF-8
Valid date found: 2024-Mar-15
en_US.UTF-8 formatted date: March 15, 2024
Enter a sentence containing dates (YYYY-MM-DD format):
Rendez-vous le 2024-07-20.
Enter your preferred locale (e.g., en_US.UTF-8, fr_FR.UTF-8, de_DE.UTF-8): fr_FR.UTF-8
Valid date found: 2024-Jul-20
fr_FR.UTF-8 formatted date: 20 juillet 2024
----
And the following shows an unsuccessful parse:
[source,text]
----
Enter a sentence containing dates (YYYY-MM-DD format):
The deadline is 2024-02-30.
Enter your preferred locale (e.g., en_US.UTF-8, fr_FR.UTF-8, de_DE.UTF-8): en_US.UTF-8
Invalid date: 2024-02-30 (Does not exist)
----
== Local Time
On a similar global vein, when you install the boost:date_time[] library (or all the Boost libraries), a file containing definitions of time zones across the world is available for your use at: `boost_<version>\\libs\\date_time\\data\\date_time_zonespec.csv`.
The following short sample shows how to use the contents of the file. Enter a city and timezone in the IANA format (such as: 'Europe/Berlin' or 'Asia/Tokyo'), and the current date and time will be output.
[source,cpp]
----
#include <boost/date_time/local_time/local_time.hpp>
namespace pt = boost::posix_time;
namespace lt = boost::local_time;
int main() {
try {
//---------------------------------------------
// Load the Boost tz_database from CSV
//---------------------------------------------
lt::tz_database tz_db;
tz_db.load_from_file("<YOUR PATH>\\date_time_zonespec.csv"); // Adjust the path to your Boost installation
// Extract all valid timezone names
std::vector<std::string> valid_timezones;
for (const auto& tz_name : tz_db.region_list()) {
valid_timezones.push_back(tz_name);
}
std::string city;
while (true) {
std::cout << "\nEnter 'city/timezone' (or 'exit' to quit, or 'zones' for list of options): ";
std::getline(std::cin, city);
if (city == "exit") break;
if (city == "zones")
{
std::cout << "Available timezones:\n";
for (const auto& tz : valid_timezones) {
std::cout << tz << "\n";
}
}
else
{
// Find the timezone (case-sensitive, must match CSV)
lt::time_zone_ptr tz = tz_db.time_zone_from_region(city);
if (!tz) {
std::cout << "Invalid timezone! Try again.\n";
continue;
}
// Get current UTC time
pt::ptime utc_now = pt::second_clock::universal_time();
// Convert UTC to local time in the chosen timezone
lt::local_date_time local_now(utc_now, tz);
// Get user's local machine time
pt::ptime user_now = pt::second_clock::local_time();
std::cout << "\nYour local system time: " << user_now << "\n";
std::cout << "Current local time in " << city << ": " << local_now << "\n";
}
}
}
catch (const std::exception& e) {
std::cerr << "Fatal error: " << e.what() << "\n";
return 1;
}
return 0;
}
----
Run the program and test out a few options:
[source,text]
----
Enter 'city/timezone' (or 'exit' to quit, or 'zones' for list of options): America/New_York
Your local system time: 2025-Sep-03 16:38:02
Current local time in America/New_York: 2025-Sep-03 19:38:02 EDT
Enter 'city/timezone' (or 'exit' to quit, or 'zones' for list of options): Antarctica/South_Pole
Your local system time: 2025-Sep-03 16:38:20
Current local time in Antarctica/South_Pole: 2025-Sep-04 11:38:20 NZST
Enter 'city/timezone' (or 'exit' to quit, or 'zones' for list of options): zones
Available timezones:
Africa/Abidjan
Africa/Accra
Africa/Addis_Ababa
Africa/Algiers
Africa/Asmara
Africa/Asmera
Africa/Bamako
Africa/Bangui
Africa/Banjul
Africa/Bissau
Africa/Blantyre
Africa/Brazzaville
Africa/Bujumbura
Africa/Cairo
Africa/Casablanca
Africa/Ceuta
Africa/Conakry
....
----
== Next Steps
If more complex input is required, consider the boost:spirit[] approach to parsing, refer to xref:task-natural-language-parsing.adoc[].
== See Also
* https://www.boost.org/doc/libs/latest/libs/libraries.htm#Miscellaneous[Category: Miscellaneous]
* https://www.boost.org/doc/libs/latest/libs/libraries.htm#Parsing[Category: Parsing]
* https://www.boost.org/doc/libs/latest/libs/libraries.htm#String[Category: String and text processing]
Reference in New Issue View Git Blame Copy Permalink