Merge pull request #10 from r-spacex/prepare-release-060

Release Sub Manager version 0.6.0
r-spacex · Aug 25, 2021 · a45128e · a45128e
2 parents 16ad4cf + 699eb37
commit a45128e
Show file tree

Hide file tree

Showing 13 changed files with 141 additions and 73 deletions.
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
@@ -101,7 +101,7 @@ repos:
 
 # Fix Python
 - repo: https://github.com/asottile/pyupgrade
-  rev: v2.23.3
+  rev: v2.24.0
   hooks:
   - id: pyupgrade
     name: Fix Python with Pyupgrade
@@ -172,9 +172,10 @@ repos:
     name: Check for logger.warn
 
 - repo: https://github.com/Lucas-C/pre-commit-hooks-safety
-  rev: v1.2.0
+  rev: v1.2.1
   hooks:
   - id: python-safety-dependencies-check
+    name: Run Safety check on dependencies
     files: 'requirements.txt'
 
 # Lint Python
@@ -254,7 +255,7 @@ repos:
 
 # Lint YAML
 - repo: https://github.com/adrienverge/yamllint.git
-  rev: v1.26.2
+  rev: v1.26.3
   hooks:
   - id: yamllint
     name: Lint Yaml

diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -11,44 +11,102 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 <!-- START doctoc generated TOC please keep comment here to allow auto update -->
 <!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
 
-- [[0.5.1] - 2021-06-15](#051---2021-06-15)
-  - [Fixed](#fixed)
-- [[0.5.0] - 2021-05-18](#050---2021-05-18)
+- [[0.6.0] - 2021-08-24](#060---2021-08-24)
   - [Added](#added)
   - [Changed](#changed)
-- [[0.4.0] - 2021-03-24](#040---2021-03-24)
+  - [Fixed](#fixed)
+  - [Removed](#removed)
+- [[0.5.1] - 2021-06-15](#051---2021-06-15)
+  - [Fixed](#fixed-1)
+- [[0.5.0] - 2021-05-18](#050---2021-05-18)
   - [Added](#added-1)
   - [Changed](#changed-1)
-- [[0.3.1] - 2021-03-01](#031---2021-03-01)
-  - [Changed](#changed-2)
-  - [Fixed](#fixed-1)
-- [[0.3.0] - 2021-02-08](#030---2021-02-08)
+- [[0.4.0] - 2021-03-24](#040---2021-03-24)
   - [Added](#added-2)
+  - [Changed](#changed-2)
+- [[0.3.1] - 2021-03-01](#031---2021-03-01)
   - [Changed](#changed-3)
-- [[0.2.2] - 2021-02-01](#022---2021-02-01)
-  - [Added](#added-3)
   - [Fixed](#fixed-2)
-- [[0.2.1] - 2021-01-22](#021---2021-01-22)
+- [[0.3.0] - 2021-02-08](#030---2021-02-08)
+  - [Added](#added-3)
+  - [Changed](#changed-4)
+- [[0.2.2] - 2021-02-01](#022---2021-02-01)
   - [Added](#added-4)
   - [Fixed](#fixed-3)
-- [0.2.0 - 2021-01-21](#020---2021-01-21)
+- [[0.2.1] - 2021-01-22](#021---2021-01-22)
   - [Added](#added-5)
-  - [Changed](#changed-4)
   - [Fixed](#fixed-4)
-- [[0.1.2] - 2021-01-20](#012---2021-01-20)
-  - [Changed](#changed-5)
-- [[0.1.1] - 2021-01-20](#011---2021-01-20)
+- [0.2.0 - 2021-01-21](#020---2021-01-21)
   - [Added](#added-6)
-  - [Changed](#changed-6)
+  - [Changed](#changed-5)
   - [Fixed](#fixed-5)
-- [[0.1.0] - 2021-01-01](#010---2021-01-01)
+- [[0.1.2] - 2021-01-20](#012---2021-01-20)
+  - [Changed](#changed-6)
+- [[0.1.1] - 2021-01-20](#011---2021-01-20)
   - [Added](#added-7)
+  - [Changed](#changed-7)
+  - [Fixed](#fixed-6)
+- [[0.1.0] - 2021-01-01](#010---2021-01-01)
+  - [Added](#added-8)
 
 <!-- END doctoc generated TOC please keep comment here to allow auto update -->
 <!-- markdownlint-restore -->
 
 
 
+## [0.6.0] - 2021-08-24
+
+### Added
+
+* Full command-based CLI with many more options, including separate ``start`` and ``run`` commands
+* Command ``get-config-info`` to print information about the config and list endpoints
+* Command ``install-service`` to automatically install a systemd service to run the bot
+* Command ``generate-config`` to generate a user config file with example defaults
+* Command ``validate-config`` to validate the current config file, offline or online
+* Command ``cycle-threads`` to post new versions of the indicated threads
+* ``--debug`` flag, to control whether user-friendly error messages or full tracebacks are printed
+* Config option ``approve_new`` to control if new posted threads are automatically approved (default)
+* ``"auto"`` option for ``pin_mode`` (default) will unpin the old thread and pin the new if currently pinned
+* ``truncate_lines`` option for source and target endpoints, to truncate to a specified number of lines
+* More fillable/replaceable variables in thread titles
+* Comprehensive offline and online functional tests exercising the options of every supported command
+* Exhaustive suite of pre-commit checks to validate correctness, best-practices, conventions and style
+* Tests, linting and analysis all run across a matrix of platforms/versions in CIs via Github Actions
+
+
+### Changed
+
+* Reorganize much of user and dynamic config for consistency and future extensibility
+* Use Platformdirs to install config and state files into platform-appropriate locations
+* Validate config and resync all sources for the first run, to handle any config changes
+* Use Pydantic for config, allowing robust access and user-friendly validation
+* Vastly improved error handing of almost all areas, with much more helpful, user friendly messages
+* Safety run multiple instances via locking, to enable new commands and avoid strange bugs
+* Enable thread redirect in OP and sticky by default (configurable)
+* Inject start and end comments into thread targets, to allow easily adding extra content
+* Formally rename everything to Sub Manager, with consistent naming everywhere
+* Refactor from a monolithic script into a modular package, greatly improving organization
+* Make a Python package for installing from Github or from PyPI and add lint and test extras
+* Package is now fully statically typed, improving correctness, robustness and documentation
+* Add/greatly improve docs including Readme, Contributing Guide, security policy and more
+* Auto-generate pinned requirements files for reproducible dev and production installation
+
+
+### Fixed
+
+* Issues with hierarchical config defaults not getting inherited correctly
+* Not syncing new targets until a change is made in the source
+* Not injecting appropriate line breaks in targets for some sources, causing formatting issues
+* Many other edge-case bugs and issues
+
+
+### Removed
+
+* Config file no longer generated automatically if its not present, use ``generate-config`` instead
+* Drop previous support for token managers, in line with Reddit and PRAW changes
+
+
+
 ## [0.5.1] - 2021-06-15
 
 ### Fixed

diff --git a/RELEASE.md b/RELEASE.md
@@ -2,7 +2,7 @@
 
 A step by step checklist for cutting a new release.
 
-**Note:** This guide uses [``hub``](https://hub.github.com/) branches more easily, but it can be replaced by switching to the ``master``/release branch and running ``git pull upstream <BRANCH-NAME>``.
+**Note:** This guide uses [``hub``](https://hub.github.com/) to sync branches more easily, but it can be substituted with switching to the ``master``/release branch and running ``git pull upstream <BRANCH-NAME>``.
 
 
 <!-- markdownlint-disable -->
@@ -30,23 +30,23 @@ A step by step checklist for cutting a new release.
 0. Check that no release steps need updating
 1. Ensure all issues/PRs linked to Github milestone are closed or moved to next release
 2. Check ``git status`` and ``git diff`` for any untracked local changes and handle as needed
-3. Sync ``upstream`` repo with ``hub sync``/``git pull upstream``, push to ``origin`` and clean up old branches
-4. Create a new branch ``prepare-release-XYZ``, push to ``upstream`` and open a PR
+3. Sync from ``upstream`` repo, push to ``origin`` and clean up old branches: ``git fetch --all``, then ``hub sync``/``git pull upstream``, then ``git push origin``, and finally ``git branch -d <BRANCH>`` and ``git push -d <REMOTE> <BRANCH>`` for any branches
+4. Create a new branch ``prepare-release-XYZ`` and push to ``upstream``: ``git switch -c prepare-release-XYZ`` then ``git push -u upstream prepare-release-XYZ``
 
 
 
 ## Update pinned dependencies
 
 0. Check ``MANIFEST.in`` and ``setup.cfg`` to ensure they are up to date and all data files are included
 1. Check each dependency for new upper bound version and examine changelogs for breaking changes
-2. Commit and push if changes made to run regen deps on Linux machine (RPi)
+2. Commit and push if changes made to run regen deps on Linux machine (RPi), and open PR against base branch
 3. On RPi, activate env and run ``python -X dev tools/generate_requirements_files.py build`` to update build deps
 4. Run ``python -m pip install --upgrade -r requirements-build.txt`` to install updated build deps
 5. Run ``python -X dev tools/generate_requirements_files.py`` to update all reqs files
 6. Run ``pip install --upgrade -r requirements-dev.txt`` install updated dev deps
-7. Run ``pip check`` to verify environment integrity
-8. Run ``pip install -e .`` to ensure package install is up to date
-9. Run ``python -bb -X dev -W error pytest --run-online`` and fix any issues
+7. Run ``pip install -e .`` to ensure package install is up to date
+8. Run ``pip check`` to verify environment integrity
+9. Run ``python -bb -X dev -W error -m pytest --run-online`` and fix any issues
 10. Run ``pre-commit run --all-files`` and fix any issues
 11. Sync back changes to dev machine and fixup prior commit
 12. Push and test on PR and ``git reset --hard`` on Pi
@@ -60,7 +60,7 @@ A step by step checklist for cutting a new release.
 2. Manually check ``additional_dependencies`` for updates and update as needed
 3. Check hook/dep changelogs and add/update any new settings
 4. Run ``pre-commit run --all-files`` and fix any issues
-5. Run ``python -bb -X dev -W error pytest --run-online`` and fix any issues
+5. Run ``python -bb -X dev -W error -m pytest --run-online`` and fix any issues
 6. Commit changes, push & test on PR
 
 
@@ -75,27 +75,27 @@ A step by step checklist for cutting a new release.
 5. Create a fresh, clean virtual environment, e.g. ``deactivate`` then ``python3 -m venv clean-env``
 6. Activate the new environment, e.g. ``source clean-env/bin/activate``
 7. Install/upgrade core install deps in new environment: ``python -m pip install --upgrade pip setuptools wheel``
-8. Install the build in the new environment: ``pip install dist/*.whl[test]``
+8. Install the build in the new environment: ``pip install dist/submanager-X.Y.Z.dev0-py3-none-any.whl[test]``
 9. Check the env with pip: ``pip check``
-10. Test the installed version: ``python -bb -X dev -W error pytest --run-online``
+10. Test the installed version: ``python -bb -X dev -W error -m pytest --run-online``
 11. Fix any bugs, commit, push and retest
 
 
 
 ## Transition new version to production on RPi
 
-0. Disable and stop production service
+0. Disable and stop production service: ``systemctl --user disable submanager`` then ``systemctl --user stop submanager``
 1. Activate production venv (e.g. ``source env/bin/activate``)
 2. Upgrade core install deps ``python -m pip install --upgrade pip setuptools wheel``
 3. Install/upgrade pinned deps from requirements file: ``pip install --upgrade -r requirements.txt``
-4. Install package from built wheel: ``pip install dist/*.wheel``
+4. Install package from built wheel: ``pip install dist/submanager-X.Y.Z.dev0-py3-none-any.whl``
 5. Ensure config/state dir names, locations and structure is up to date
 6. Sync static config and praw.ini from dev machine
 7. Update local dynamic config as needed
 8. Validate config with ``python -b -X dev -m submanager validate-config``
 9. Start running and verify nominal performance: ``python -b -X dev -m submanager start``
-10. Install service with ``python -b -X dev -m install-service``
-11. Enable, start, wait 30 seconds and verify ``status``, ``journalctl`` log output and on sub
+10. Install service with ``python -bb -X dev -W error -m submanager install-service``
+11. Enable and start service, wait 30 seconds and verify ``status``, ``journalctl`` log output and on sub
 
 
 
@@ -112,9 +112,9 @@ A step by step checklist for cutting a new release.
 ## Rebuild and upload to PyPI on RPi/production machine
 
 0. Repeat steps 0 through 4 (inclusive) in "Build and test on RPi" section to rebuild release version
-1. Install built wheel in dev environment: ``pip install dist/*.wheel[lint,test]``
+1. Install built wheel in dev environment: ``pip install dist/submanager-X.Y.Z.dev0-py3-none-any.whl[lint,test]``
 2. Verify version is correct: ``submanager --version``
-3. Run basic tests one last time: ``python -bb -X dev -W error pytest``
+3. Run basic tests one last time: ``python -bb -X dev -W error -m pytest``
 4. Merge PR and wait for checks to pass
 5. Upload to live PyPI: ``twine upload dist/*``
 

diff --git a/ROADMAP.md b/ROADMAP.md
@@ -1,40 +1,44 @@
 # Roadmap
 
-
-## v0.6.0 (June 2021?)
-
+## [0.7.0] - 2021-10
+
+* On-demand automated new thread generation and creation
+* New command to automatically create sources and/or targets
+* New command to generate clean config without examples
+* Built-in support for refresh token setup
+* Re-add support for optionally filling variables in the source/target content
+* Add support for inline source declarations in the source content that get injected in the target
 * Add optional regex support for replace patterns
-* Refactor to modular Python package structure with ``setup.py`` and modular components
-* Use consistent API to run individual manager tasks
 * Improve error handling when one request/module fails
-* Use Python logging module and add many more specific logging messages
-* Add baseline pre-commit and CI checks
-* Publish to Github and PyPI
+* Add YAML support for config
+* Fully document config schema
+* Improve setup/teardown in functional tests and check Reddit results more thoroughly
 
 
 
-## v0.6.0 (July 2021?)
+## [0.8.0] - 2021-12
 
-* Add automod-manager module to pre-process automod config
-* Add serviceinstaller support for automatic service installation
-* Add ability to re/generate popup menu of threads (e.g. in removal dialog)?
+* Refactor validators and managers into modular, extensible class-based architecture with consistent API
+* More comprehensive validation and validator control
+* Add a ``managed-thread`` source/target type, that targets the current thread for a key
+* More unit and integration tests
 
 
 
-## v1.0.0 (???)
+## [1.0.0] - ???
 
-* Add proper tests and CIs
-* Further improve logging and error handling as needed (better logging? verbose mode?)
+* Greatly improve unit and integration test coverage
+* Use Python logging module and add many more specific logging messages
 * Further improve documentation
+* Add ability to re/generate popup menu of threads (e.g. in removal dialog)?
 
 
 
-## v2.0.0 (???)
+## [2.0.0] - ???
 
 * Add more desired features (sticky comments, moderation tools, more automod automation...)
-* Add support for running within the Brokkr framework
+* Add support for running within the Brokkr framework?
     * Create wrappers for plugin(s) for session, wiki sync and megathread sync
     * Create preset(s) for plugins(s)
-* Create example Brokkr system config
-* Add MyPy type annotations
+* Create example Brokkr system config?
 * Add more detailed logging, optionally integrated with Brokkr
diff --git a/SECURITY.md b/SECURITY.md
@@ -7,7 +7,7 @@ The following summarizes the support status of recent Sub Manager versions.
 
 | Version | Supported          |
 | ------- | ------------------ |
-| 0.6.x   | :white_check_mark: |
+| 0.6.x   | :heavy_check_mark: |
 | < 0.6   | :x:                |
 
 

diff --git a/requirements-build.txt b/requirements-build.txt
@@ -18,7 +18,7 @@ click==8.0.1
     # via pip-tools
 colorama==0.4.4
     # via twine
-cryptography==3.4.7
+cryptography==3.4.8
     # via secretstorage
 docutils==0.17.1
     # via readme-renderer
@@ -74,7 +74,7 @@ tomli==1.2.1
     # via
     #   build
     #   pep517
-tqdm==4.62.1
+tqdm==4.62.2
     # via twine
 twine==3.4.2
     # via -r requirements-build.in

diff --git a/requirements-dev.txt b/requirements-dev.txt
@@ -8,7 +8,7 @@ aenum==3.1.0
     # via pyanalyze
 ast-decompiler==0.4.0
     # via pyanalyze
-astroid==2.6.6
+astroid==2.7.2
     # via pylint
 asynq==1.4.0
     # via pyanalyze
@@ -63,6 +63,7 @@ packaging==21.0
     #   submanager (setup.py)
 platformdirs==2.2.0
     # via
+    #   pylint
     #   submanager (setup.py)
     #   virtualenv
 pluggy==0.13.1
@@ -81,7 +82,7 @@ pydantic==1.8.2
     # via submanager (setup.py)
 pygments==2.10.0
     # via asynq
-pylint==2.9.6
+pylint==2.10.2
     # via submanager (setup.py)
 pyparsing==2.4.7
     # via packaging

diff --git a/requirements-lint.txt b/requirements-lint.txt
@@ -8,7 +8,7 @@ aenum==3.1.0
     # via pyanalyze
 ast-decompiler==0.4.0
     # via pyanalyze
-astroid==2.6.6
+astroid==2.7.2
     # via pylint
 asynq==1.4.0
     # via pyanalyze
@@ -55,6 +55,7 @@ nodeenv==1.6.0
     # via pre-commit
 platformdirs==2.2.0
     # via
+    #   pylint
     #   submanager (setup.py)
     #   virtualenv
 praw==7.4.0
@@ -69,7 +70,7 @@ pydantic==1.8.2
     # via submanager (setup.py)
 pygments==2.10.0
     # via asynq
-pylint==2.9.6
+pylint==2.10.2
     # via submanager (setup.py)
 python-dateutil==2.8.2
     # via submanager (setup.py)