# Management Commands

- [Management Commands](#management-commands)
  - [`boost_setup`](#boost_setup)
  - [`import_versions`](#import_versions)
  - [`import_archives_release_data`](#import_archives_release_data)
  - [`update_libraries`](#update_libraries)
  - [`import_library_versions`](#import_library_versions)
  - [`import_library_version_docs_urls`](#import_library_version_docs_urls)
  - [`update_maintainers`](#update_maintainers)
  - [`update_authors`](#update_authors)
  - [`import_commits`](#import_commits)
  - [`update_issues`](#update_issues)
  - [`import_beta_release`](#import_beta_release)
  - [`sync_mailinglist_stats`](#sync_mailinglist_stats)
  - [`update_library_version_dependencies`](#update_library_version_dependencies)
  - [`release_tasks`](#release_tasks)
  - [`refresh_users_github_photos`](#refresh_users_github_photos)
  - [`clear_slack_activity`](#clear_slack_activity)

## `boost_setup`

Runs the management commands required to populate the Boost database from scratch with Boost versions, libraries, and other associated data.

Read more about `boost_setup` in [Populating the Database for the First Time](./first_time_data_import.md).

**Example**

```bash
./manage.py boost_setup
```

**Options**

| Options              | Format | Description                                                  |
|----------------------|--------|--------------------------------------------------------------|
| `--token`            | string | GitHub API Token. If passed, will use this value. If not passed, will use the value in settings. |

**Process**: See [Populating the Database for the First Time](./first_time_data_import.md).

## `import_versions`

Imports `Version` objects from GitHub.

**Example**

```bash
./manage.py import_versions
```

**Options**

| Options              | Format | Description                                                                                             |
|----------------------|--------|---------------------------------------------------------------------------------------------------------|
| `--delete-versions`  | bool   | If passed, will delete all Version objects before importing Versions.                                   |
| `--new`              | bool   | Default: 'true'. If 'true', will import only new Version objects. Set to 'false' to import all versions |
| `--token`            | string | GitHub API Token. If passed, will use this value. If not passed, will use the value in settings.        |

**Process**

- Retrieves the tags for the GitHub repo in `BASE_GITHUB_URL`
- Loops through all tags, and discards any that do not match our inclusion logic, by default only versions that haven't already been imported.
- For each successful tag, import it as a `Version` object
- Then, run the command to the release downloads from Artifactory as `VersionFile` objects

## `import_archives_release_data`

*This process is run automatically as part of `import_versions`.*

Import `VersionFile` objects from Artifactory.

**Example**

```bash
./manage.py import_archives_release_data
```

**Options**

| Options    | Format | Description                                                                                                                  |
|------------|--------|------------------------------------------------------------------------------------------------------------------------------|
| `--new`    | bool   | Default: 'true'. If 'true', will import only the newest release data. Set to 'false' to import archive data for all releases |
| `--release` | string   | Format: `boost-1.63.0`. If passed, will import Archive urls for only that release. Overrides --new                        |

**More Information**

- Loops through `Version` objects, by default only the most recent one, and calls the task that retrieves the Archives data with the version information
- Saves the Archives JSON data as `VersionFile` objects

## `update_libraries`

**Purpose**: Import and update `Library` and `Category` objects. Runs the library update script, which cycles through the repos listed in the Boost library and syncs their information. Most library information comes from `meta/libraries.json` stored in each Boost library repo.

**Example**

```bash
./manage.py update_libraries
```

**Options**

| Options              | Format | Description                                                  |
|----------------------|--------|--------------------------------------------------------------|
| `--token`            | string | GitHub API Token. If passed, will use this value. If not passed, will use the value in settings. |

## `import_library_versions`

**Purpose**: Import and update `LibraryVersion` objects.

**Example**

```bash
./manage.py import_library_versions
```

**Options**

| Options              | Format | Description                                                                                                                                                                            |
|----------------------|--------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `--token`            | string | GitHub API Token. If passed, will use this value. If not passed, will use the value in settings.                                                                                       |
| `--release`  | string   | Format: `boost-1.63.0`. If passed, will import Artifactory urls for only that version. Partial versions are accepted (so "1.7" will import libraries for version 1.70.0, 1.71.0, etc.) |
| `--new`    | bool   | Default: 'true'. If 'true', will import data for the newest release. Set to 'false' to import library version data for all releases                                                    |

**Process**

- Loops through `Version` objects based on passed-in options, by default just the most recent one.
- For each `Version`, gets the libraries in that release from the `.gitmodules` file using the GitHub API
- For each library listed in the `.gitmodules` file, get the complete list of libraries from the library's `meta/libraries.json` file (in its GitHub repo) using the GitHub API. (A single library repo might contain information for multiple libraries. Example: Functional also hosts Functional/Factory).
- Save the `LibraryVersion` objects
- Call the task to import documentation urls from data hosted in the S3 bucket


## `import_library_version_docs_urls`

*This process is taken care of automatically as part of `import_library_versions`.*

**Purpose**: Because of uniqueness in Boost library data, it's not possible to consistently format the URL for each Boost library-version. The current process involves hitting the url for the page in the Boost release notes that lists all the libraries and manually copying the URLs from the `<ul>` on that page to each `LibraryVersion` object.

**Example**

```bash
./manage.py import_library_version_docs_urls --version=1.81.0
```

**Options**

| Options              | Format | Description                                                  |
|----------------------|--------|--------------------------------------------------------------|
| `--min-version`            | string | Specify the minimum version for which you want to retrieve documentation URLs. The default is defined in the settings file. |
| `--version`  | string   | Format: `1.81.0`. Specify the version for which you want to retrieve documentation URLs. You can provide a specific version number (example: `1.81.0`) or a partial version number to process all versions that contain the partial version number (example: `1.7` would process 1.70.0, 1.71.0, 1.72.0, etc.). If no version is specified, all active versions will be processed. |

**Process**

- This command cycles through all Versions in the database, or specified versions using the `--version` option.
- For each version, the command calls a celery task that retrieves and stores the library version documentation url paths from S3.
- If a library version's documentation URL cannot be found, the command will skip and continue with the next library version.

## `update_maintainers`

Cycles through all library-versions and uses the `maintainers` element in the `data` JSONField to load the maintainer information from GitHub into the database.

**Example**

```bash
./manage.py update_maintainers
```

**Options**

| Options              | Format | Description                                                  |
|----------------------|--------|--------------------------------------------------------------|
| `--library-name`  | string   | Name of the library. If passed, the command will load maintainers for only this library. |
| `--release`  | string   | Format: `boost-1.63.0`. If passed, will import Artifactory urls for only that version. |

If both the `--release` and the `--library-name` are passed, the command will load maintainers for only that Library-Version.

## `update_authors`

**Purpose**: Cycles through all libraries and uses the `authors` element in the `data` JSONField to load the author information from GitHub into the database.

**Example**

```bash
./manage.py update_authors
```

**Options**

| Options              | Format | Description                                                  |
|----------------------|--------|--------------------------------------------------------------|
| `--library-name`  | string   | Name of the library. If passed, the command will load maintainers for only this library. |


## `import_commits`

**Purpose**: Cycles through all libraries and their library versions to import `Commit`, `CommitAuthor`, and `CommitAuthorEmail` models. Updates `CommitAuthor` github profile URLs and avatar URLs.

**Example**

```bash
./manage.py import_commits
```

**Options**

| Options              | Format | Description                                                  |
|----------------------|--------|--------------------------------------------------------------|
| `--key`  | string   | Key of the library. If passed, the command will import commits for only this library. |
| `--clean`  | boolean   | If passed, will delete all existing commits before importing new ones. |


## `update_issues`

**Purpose**: Cycles through all libraries and imports github Issues for that Library

**Example**

```bash
./manage.py update_issues
```

**Options**

| Options              | Format | Description                                                  |
|----------------------|--------|--------------------------------------------------------------|
| `--key`  | string   | Key of the library. Only update_issues for one library. |
| `--clean`  | boolean   | If passed, will delete the libraries' issues just before running the import. |


## `import_beta_release`

**Purpose**: Imports the most recent beta release

**Example**

```bash
./manage.py import_beta_release
```

**Options**

| Options              | Format | Description                                                  |
|----------------------|--------|--------------------------------------------------------------|
| `--token`  | string   | Pass a GitHub API token. If not passed, will use the value in `settings.GITHUB_TOKEN`. |
| `--delete-versions`  | bool  | If passed, all existing beta Version records will be deleted before the new beta release is imported. |


## `sync_mailinglist_stats`

**Purpose**: Build EmailData objects from the hyperkitty email archive database.

**Example**

```bash
./manage.py sync_mailinglist_stats
```

**Options**

| Options              | Format | Description                                                  |
|----------------------|--------|--------------------------------------------------------------|
| `--clean`  | bool  | If passed, all existing beta EmailData records will be deleted before running the sync. |


## `update_library_version_dependencies`

**Purpose**: Read a boostdep report text file uploaded as an artifact from a github action and update dependencies for LibraryVersion models.

**Example**

```bash
./manage.py update_library_version_dependencies
```

**Options**

| Options              | Format | Description                                                  |
|----------------------|--------|--------------------------------------------------------------|
| `--token`  | string  | Pass a GitHub API token. If not passed, will use the value in `settings.GITHUB_TOKEN`. |
| `--clean`  | bool    | If passed, existing dependencies in the M2M will be cleared before reinserting them. |
| `--owner`  | string  | The repo owner. Defaults to "boostorg", which is correct in most cases but can be useful to specify for testing. |


## `release_tasks`

**Purpose**: Execute a chain of commands which are necessary to run during a release. Imports new versions, beta versions, slack messages, github issues, commits, authors, maintainers, etc... Inspect the management command to see exactly which commands are being run.

For this to work `SLACK_BOT_API` must be set in the `.env` file.

**Example**

```bash
./manage.py release_tasks
```

**Options**

| Options              | Format | Description                                                  |
|----------------------|--------|--------------------------------------------------------------|
| `--user_id`  | int  | If passed, the user with this ID will receive email notifications when this task is started and finished, or if the task raises and exception. |


## `import_ml_counts`

**Purpose**: Import mailing list counts from the mailman archives.

```bash
./manage.py import_ml_counts
```

**Options**

| Options        | Format | Description                                                                                                          |
|----------------|--------|----------------------------------------------------------------------------------------------------------------------|
| `--start_date` | date   | If passed, retrieves data from the start date supplied, d-m-y, default 1998-11-20 (the start of the data in mailman) |
| `--end_date`   | date   | If passed, If passed, retrieves data until the start date supplied, d-m-y, default today                             |

## `link_contributors_to_users`

**Purpose**: Links commit authors to users in the database by setting `user.github_username` for users where no `github_username` value has been set, by matching the commit author email address against a user's account email address.


**Example**

```bash
./manage.py link_contributors_to_users
```

## `refresh_users_github_photos`

**Purpose**: Refresh GitHub profile photos for all users who have a GitHub username. This command fetches the latest profile photo from GitHub for each user and updates their local profile image. This is useful for local dev/testing, isn't used for production where a periodic celery task is used.

**Example**

```bash
./manage.py refresh_users_github_photos
```

**Options**

| Options      | Format | Description                                                                                  |
|--------------|--------|----------------------------------------------------------------------------------------------|
| `--dry-run`  | bool   | Show which users would be updated without actually updating them. Useful for testing.        |

**Usage Examples**

Refresh photos for all users with GitHub usernames:
```bash
./manage.py refresh_users_github_photos
```

Preview which users would be updated:
```bash
./manage.py refresh_users_github_photos --dry-run
```
**Process**

- Calls the `refresh_users_github_photos()` Celery task which queues photo updates for all users with GitHub usernames
- With `--dry-run`, displays information about which users would be updated without making any changes

## `clear_slack_activity`

**Purpose**: Delete all slack activity tracking data from the database. This command removes all records from the `SlackActivityBucket` and `ChannelUpdateGap` tables, and resets the `last_update_ts` field to "0" for all channels. This is useful for resetting the slack activity tracking system to its initial state.

**Example**

```bash
./manage.py clear_slack_activity --confirm
```

**Options**

| Options      | Format | Description                                                                                  |
|--------------|--------|----------------------------------------------------------------------------------------------|
| `--confirm`  | bool   | Required flag to confirm deletion. The command will not execute without this flag.           |

**Usage Examples**

Execute the deletion:
```bash
./manage.py clear_slack_activity --confirm
```

**Process**

- Deletes all `SlackActivityBucket` records (message counts per user per channel per day)
- Deletes all `ChannelUpdateGap` records (tracking of message fetch progress)
- Resets `last_update_ts` to "0" for all `Channel` records
- All operations are performed within a database transaction to ensure atomicity
- Logs the number of records affected in each table

**Warning**: This command permanently deletes all slack activity data. Use with caution.