Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhance sdk-changelog commands #4040

Merged
merged 60 commits into from
Feb 14, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
60 commits
Select commit Hold shift + click to select a range
0252abb
add script to comment to changelog
GuyAfik Feb 12, 2024
3ce6c07
call script on-push
GuyAfik Feb 12, 2024
07473bf
call pr
GuyAfik Feb 12, 2024
84ec72e
fix issues
GuyAfik Feb 12, 2024
daed917
poetry github
GuyAfik Feb 12, 2024
7f8c267
add changelog
GuyAfik Feb 12, 2024
ad8ef8e
fix bug
GuyAfik Feb 12, 2024
4d7b755
test
GuyAfik Feb 12, 2024
891c588
test
GuyAfik Feb 12, 2024
339adf2
fix bug
GuyAfik Feb 12, 2024
d476e77
try to fix 404
GuyAfik Feb 12, 2024
6bfa676
test
GuyAfik Feb 12, 2024
26083df
remove env
GuyAfik Feb 12, 2024
311c67f
echo
GuyAfik Feb 12, 2024
21be113
echo
GuyAfik Feb 12, 2024
0e0a07e
test
GuyAfik Feb 12, 2024
d22a74c
prints
GuyAfik Feb 12, 2024
4f3effb
int type
GuyAfik Feb 12, 2024
d709b35
create issue comment
GuyAfik Feb 12, 2024
3b0532c
fix
GuyAfik Feb 12, 2024
f3ec472
update message
GuyAfik Feb 12, 2024
34a3d74
test
GuyAfik Feb 12, 2024
45507d5
fix
GuyAfik Feb 12, 2024
95a3169
test
GuyAfik Feb 12, 2024
8df7448
fetch before fetching commit
GuyAfik Feb 12, 2024
a3351f3
test§
GuyAfik Feb 13, 2024
2ae22f8
git log
GuyAfik Feb 13, 2024
d6e6c2e
-n 8
GuyAfik Feb 13, 2024
e13b0b3
add previous commit
GuyAfik Feb 13, 2024
bee045a
fix on-push
GuyAfik Feb 13, 2024
533bc36
from remote = false
GuyAfik Feb 13, 2024
6be9e4b
test
GuyAfik Feb 13, 2024
4581bfb
test
GuyAfik Feb 13, 2024
8acdd4c
refactor logic
GuyAfik Feb 13, 2024
57396d6
test changes in changelog
GuyAfik Feb 13, 2024
f70248a
see what happens without changelog
GuyAfik Feb 13, 2024
17be450
continue on error
GuyAfik Feb 13, 2024
c9a363a
add changelog
GuyAfik Feb 13, 2024
40e6776
pre-commit
GuyAfik Feb 13, 2024
e36fc59
changelog
GuyAfik Feb 13, 2024
b20b8f8
docstring
GuyAfik Feb 13, 2024
2037ba5
get PR number automatically
GuyAfik Feb 13, 2024
d02ba65
reverse if statemetns
GuyAfik Feb 13, 2024
7c8bbd0
get pulls
GuyAfik Feb 13, 2024
9002002
handle better exceptions
GuyAfik Feb 13, 2024
d45ace7
fix message
GuyAfik Feb 13, 2024
7b8279f
changelog
GuyAfik Feb 13, 2024
115eeec
remove ssl and make const
GuyAfik Feb 13, 2024
8493c57
change the changelog
GuyAfik Feb 13, 2024
9296535
remove dot
GuyAfik Feb 13, 2024
dd4d404
cr fixes
GuyAfik Feb 13, 2024
98175c3
code formatting
GuyAfik Feb 13, 2024
9ad2350
add unit-tests for comment method
GuyAfik Feb 13, 2024
538b787
fix comment_changelog_on_pr
GuyAfik Feb 13, 2024
d4a5bb8
pre-commit
GuyAfik Feb 13, 2024
cbff16e
Merge branch 'master' of github.com:demisto/demisto-sdk into enhance_…
GuyAfik Feb 14, 2024
285b41d
fix tests
GuyAfik Feb 14, 2024
345854c
last changes
GuyAfik Feb 14, 2024
66c0a64
fix
GuyAfik Feb 14, 2024
d7504ac
pre-commit
GuyAfik Feb 14, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .changelog/4040.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
changes:
- description: Added a new step in the **validate-changelog** to comment the changelog description in PR comments.
type: internal
- description: Added support to query the pull request number automatically when running **sdk-changelog --init** command.
type: internal
pr_number: 4040
8 changes: 8 additions & 0 deletions .github/workflows/on-push.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,14 @@ jobs:
run: |
echo "Validating changelog for PR: $PR_NUMBER with Name: $PR_TITLE"
poetry run python ./Utils/github_workflow_scripts/changelog_validation_scripts/validate_changelog.py -n "$PR_NUMBER" -t "$PR_TITLE"
- name: Comment Changelog In PR Description
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider making this run only if the changelog validation passed (needs: validate-changelog) instead of continue-on-error

Copy link
Contributor Author

@GuyAfik GuyAfik Feb 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wdym? it already runs only if validate was successful. (its like the behaivor you described, needs: validate-changelog)
I didn't see a reason to fail the whole job in case the comment step didn't work, that's why i use continue_on_error only on the last step.

continue-on-error: true
env:
PR_NUMBER: ${{ github.event.pull_request.number }}
LATEST_COMMIT_SHA: ${{ github.event.pull_request.head.sha }}
run: |
echo "Commenting changelog for PR: $PR_NUMBER"
poetry run python ./Utils/github_workflow_scripts/changelog_comment_scripts/comment_changelog.py -n "$PR_NUMBER" -lt "$LATEST_COMMIT_SHA" -ght ${{ secrets.GITHUB_TOKEN }}

pre-commit-checks:
runs-on: ubuntu-latest
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
import argparse
import sys

from demisto_sdk.commands.common.logger import logger
from demisto_sdk.scripts.changelog.changelog import Changelog


def comment_changelog_on_pr(pr_num: int, latest_commit: str, github_token: str):
try:
Changelog(pr_num).comment(latest_commit, github_token)
sys.exit(0)
except Exception:
logger.exception("Couldn't comment on the changelog.")
sys.exit(1)


def arguments_handler():
"""Validates and parses script arguments.

Returns:
Namespace: Parsed arguments object.

"""
parser = argparse.ArgumentParser(description="")
parser.add_argument(
"-n", "--pr-number", help="The PR number.", required=True, type=int
)
parser.add_argument(
"-lt",
"--latest_commit",
help="The commit number that triggered the workflow.",
required=True,
)
parser.add_argument(
"-ght", "--github_token", help="The token for Github-Api", required=True
)

return parser.parse_args()


def main():
options = arguments_handler()
pr_num = options.pr_number
latest_commit = options.latest_commit
github_token = options.github_token
comment_changelog_on_pr(pr_num, latest_commit, github_token)


if __name__ == "__main__":
main()
49 changes: 49 additions & 0 deletions demisto_sdk/commands/common/git_util.py
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,55 @@ def get_commit(self, commit_or_branch: str, from_remote: bool = True) -> Commit:
commit_or_branch, from_remote=from_remote, exception=e
)

def get_previous_commit(self, commit: Optional[str] = None) -> Commit:
"""

Returns the previous commit of a specific commit.
If not provided returns previous commit of the head commit.

Args:
commit: any commit
"""
if commit:
return self.get_commit(commit, from_remote=False).parents[0]

return self.repo.head.commit.parents[0]

def has_file_changed(
self, file_path: Union[Path, str], commit1: str, commit2: str
) -> bool:
"""
Checks if file has been changed between two commits.

Args:
file_path: file path
commit1: the first commit to compare
commit2: the second commit to compare

Returns:
True if file has been changed between two commits, False if not.
"""
return bool(self.repo.git.diff(commit1, commit2, str(file_path)))

def has_file_added(self, file_path: Union[Path, str], commit1: str, commit2: str):
"""
Checks if a file has been added between two commits.

Args:
file_path: file path
commit1: the first commit to compare
commit2: the second commit to compare

Returns:
True if file has been added between two commits, False if not.
"""
return (
file_path
in self.repo.git.diff(
"--name-only", "--diff-filter=A", commit1, commit2
).splitlines()
)

def read_file_content(
self, path: Union[Path, str], commit_or_branch: str, from_remote: bool = True
) -> bytes:
Expand Down
86 changes: 80 additions & 6 deletions demisto_sdk/scripts/changelog/changelog.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,12 @@

import typer
from git import Repo # noqa: TID251
from github import Github
from more_itertools import bucket
from pydantic import ValidationError

from demisto_sdk.commands.common.files.yml_file import YmlFile
from demisto_sdk.commands.common.git_util import GitUtil
from demisto_sdk.commands.common.handlers import (
DEFAULT_JSON_HANDLER,
DEFAULT_YAML_HANDLER,
Expand All @@ -22,10 +25,11 @@
LogType,
)

DEMISTO_SDK_REPO = "demisto/demisto-sdk"
CHANGELOG_FOLDER = Path(f"{git_path()}/.changelog")
CHANGELOG_MD_FILE = Path(f"{git_path()}/CHANGELOG.md")
RELEASE_VERSION_REGEX = re.compile(r"demisto-sdk release \d{1,2}\.\d{1,2}\.\d{1,2}")

GIT_UTIL = GitUtil(".")
yaml = DEFAULT_YAML_HANDLER
json = DEFAULT_JSON_HANDLER
sys.tracebacklimit = 0
Expand All @@ -48,12 +52,53 @@ def validate(self) -> None:
- checks that the `CHANGELOG.md` file has not changed
- checks that a log file has been added and its name is the same as the PR name
- ensure that the added log file is valid according to the `LogFileObject` model convention

Prints out a comment on how the PR would look like with the rn

"""
if is_release(self.pr_name):
return
else:
_validate_branch(self.pr_number)

""" Comment """

def comment(self, latest_commit: str, github_token: str) -> None:
"""
Comment on the PR

Checks the following:
- If the changelog file has been added in latest commit OR If the changelog file has been
modified between the last two commits.

"""
changelog_path = CHANGELOG_FOLDER / f"{self.pr_number}.yml"

previous_commit = GIT_UTIL.get_previous_commit(latest_commit).hexsha
if GIT_UTIL.has_file_changed(
changelog_path, latest_commit, previous_commit
) or GIT_UTIL.has_file_added(changelog_path, latest_commit, previous_commit):
logger.info(f"Changelog {changelog_path} has been added/modified")

current_changelogs = LogFileObject(
**YmlFile.read_from_local_path(changelog_path)
).get_log_entries()

github_client = Github(login_or_token=github_token)

pr = github_client.get_repo(DEMISTO_SDK_REPO).get_pull(int(self.pr_number))
markdown = "Changelog(s) in markdown:\n"
markdown += "\n".join(
[changelog.to_string() for changelog in current_changelogs]
)
pr.create_issue_comment(markdown)

logger.info(f"Successfully commented on PR {self.pr_number} the changelog")
else:
logger.info(
f"{changelog_path} has not been changed, not commenting on PR {self.pr_number}"
)

""" INIT """

def init(self) -> None:
Expand All @@ -66,13 +111,18 @@ def init(self) -> None:
"This PR is a release (by its name), use the release command instead."
)

if self.pr_number:
pr_num = self.pr_number
else:
pr_num = get_pr_number_by_branch(GIT_UTIL.repo.active_branch)

log = INITIAL_LOG
log["pr_number"] = int(self.pr_number)
log["pr_number"] = int(pr_num)

with (CHANGELOG_FOLDER / f"{self.pr_number}.yml").open("w") as f:
with (CHANGELOG_FOLDER / f"{pr_num}.yml").open("w") as f:
yaml.dump(log, f)

logger.info(f"Created changelog template at .changelog/{self.pr_number}.yml")
logger.info(f"Created changelog template at .changelog/{pr_num}.yml")

""" RELEASE """

Expand Down Expand Up @@ -116,6 +166,30 @@ def release(self, branch_name: str) -> None:
""" HELPER FUNCTIONS """


def get_pr_number_by_branch(branch_name: str):
"""
Get the PR number of the current branch from Github

Args:
branch_name: the branch name that is assosicated with the PR

Returns:
PR number associated with the local branch
"""
try:
error_message = (
"Failed to get PR number from Github, please add the PR number manually"
)
repo = Github(verify=False).get_repo(DEMISTO_SDK_REPO)
branch = GIT_UTIL.repo.active_branch.name
for pr in repo.get_pulls(state="open", head=branch):
if pr.head.ref == branch:
return pr.number
raise Exception(error_message)
except Exception as e:
raise Exception(f"{error_message} error:\n{e}")


def extract_errors(error: str, file_name: Path) -> str:
"""
Extracts the error messages from the json_errors string.
Expand All @@ -128,7 +202,7 @@ def extract_errors(error: str, file_name: Path) -> str:
def is_changelog_modified() -> bool:
return (
"CHANGELOG.md"
in Repo(".").git.diff("HEAD..origin/master", name_only=True).split()
in GIT_UTIL.repo.git.diff("HEAD..origin/master", name_only=True).split()
)


Expand Down Expand Up @@ -270,7 +344,7 @@ def commit_and_push(branch_name: str):
is_flag=True,
)

pr_number = typer.Option(..., "--pr_number", "-n", help="Pull request number")
pr_number = typer.Option("", "--pr_number", "-n", help="Pull request number")

pr_title = typer.Option(
"", "--pr_title", "-t", help="Pull request title (used for release)"
Expand Down
Loading
Loading