Merge branch 'release/1.1.4'

CHANGELOG.rst

@@ -1,6 +1,28 @@
 Changelog
 =========
 
+1.1.4
+-----
+
+- Alter default JIRA query to handle situations in which instances do not use the column names we are expecting. `34d99341e <https://github.com/ralphbean/bugwarrior/commit/34d99341e463cbdedd2ed12493c885c3ec771eec>`_
+- Merge pull request #213 from coddingtonbear/generalize_jira_query `9ef8f17e3 <https://github.com/ralphbean/bugwarrior/commit/9ef8f17e37105cbc10bb79fc9191b5a3da25de19>`_
+- It's a gerund! `5189ef81d <https://github.com/ralphbean/bugwarrior/commit/5189ef81db1d389ecf32e464e13c9fa53c440b9f>`_
+- gitlab: handle pagination `3067b32bc <https://github.com/ralphbean/bugwarrior/commit/3067b32bc341008f8a4fab553cb2a115ae2cec01>`_
+- gitlab: fix documentation typo `a2f1e87c9 <https://github.com/ralphbean/bugwarrior/commit/a2f1e87c96ac40b8237bc630aeb3d445ec69b437>`_
+- gitlab: add a state entry `7790450a3 <https://github.com/ralphbean/bugwarrior/commit/7790450a3cc2eb042325b21f8f3e30eaa2e4a6f0>`_
+- gitlab: fill in milestone and update/create time `a37eff259 <https://github.com/ralphbean/bugwarrior/commit/a37eff2596804e0028a1510468e22e7938b5c08f>`_
+- Merge pull request #214 from mathstuf/gitlab-pagination `befe0ed46 <https://github.com/ralphbean/bugwarrior/commit/befe0ed4666934c4cbdf97e2910c2fee514f36aa>`_
+- Phabricator service is not called phabricator, but phab `df96e346b <https://github.com/ralphbean/bugwarrior/commit/df96e346b70dbc38e65aec68b687da181583d3b6>`_
+- Phabricator service: Adding option to filter on users and projects `584b28fc3 <https://github.com/ralphbean/bugwarrior/commit/584b28fc3f8fafa6ec2ade3680ee7602dd9b50d1>`_
+- Unified filtering handling `29714c432 <https://github.com/ralphbean/bugwarrior/commit/29714c432e7600b8708a89830acb40870ac534c4>`_
+- Fixing a slightly-out-of-date gitlab test. `7174361ab <https://github.com/ralphbean/bugwarrior/commit/7174361ab3bd51ba5e4959dc7d7209cabfa9d1c7>`_
+- Adding the documentation for phabricator filtering options. `15a6a43a0 <https://github.com/ralphbean/bugwarrior/commit/15a6a43a0249dc3441e18a7f8aae401839f1478d>`_
+- Fix link to remove the browser warning of invalid certificate `77f84855b <https://github.com/ralphbean/bugwarrior/commit/77f84855b09da5af213c1ae7638d61a4d9ba34c2>`_
+- Merge pull request #218 from jonan/develop `07ef02dbd <https://github.com/ralphbean/bugwarrior/commit/07ef02dbd7c15026e59780dc743a554b5abf8d59>`_
+- Merge pull request #216 from ivan-cukic/develop `1f1f4f00e <https://github.com/ralphbean/bugwarrior/commit/1f1f4f00e72af4bc734690737d4ef0c9a0ebfd5c>`_
+- Add tests to MANIFEST.in `a4d643234 <https://github.com/ralphbean/bugwarrior/commit/a4d6432343cdcf1862b87a6d5ea381c8fa9e45c8>`_
+- Merge pull request #221 from koobs/patch-1 `42d320a05 <https://github.com/ralphbean/bugwarrior/commit/42d320a0581fe6d7f6cd79cab5460433fac70c1b>`_
+
 1.1.3
 -----
 

MANIFEST.in

@@ -3,3 +3,4 @@ include README.rst
 include LICENSE.txt
 recursive-include docs *
 recursive-include bugwarrior/docs *.rst
+recursive-include tests *

bugwarrior/README.rst

@@ -24,7 +24,7 @@ Documentation
 -------------
 
 For information on how to install and use bugwarrior, read `the docs
-<https://bugwarrior.rtfd.org>`_ on RTFD.
+<https://bugwarrior.readthedocs.org>`_ on RTFD.
 
 Build Status
 ------------

bugwarrior/docs/services/gitlab.rst

@@ -80,7 +80,7 @@ Although you can filter issues using :ref:`common_configuration_options`,
 merge requests are not filtered by default.  You can filter merge requests
 by adding the following configuration option::
 
-    gitlab.filter_merge_requests = False
+    gitlab.filter_merge_requests = True
 
 Provided UDA Fields
 -------------------

bugwarrior/docs/services/jira.rst

@@ -37,10 +37,13 @@ Service Features
 Specify the Query to Use for Gathering Issues
 +++++++++++++++++++++++++++++++++++++++++++++
 
-You can specify the query used for gathering issues by using the
-``jira.query`` parameter.  For example, to select issues assigned to
-'ralph' having a status that is not 'closed' and is not 'resolved', you
-could add the following configuration option::
+By default, the JIRA plugin will include any issues that are assigned to you
+but do not yet have a resolution set, but you can fine-tune the query used
+for gathering issues by setting the ``jira.query`` parameter.
+
+For example, to select issues assigned to 'ralph' having a status that is
+not 'closed' and is not 'resolved', you could add the following
+configuration option::
 
     jira.query = assignee = ralph and status != closed and status != resolved
 

bugwarrior/docs/services/phabricator.rst

@@ -22,13 +22,41 @@ Here's an example of an Phabricator target::
 .. note::
 
    Although this may not look like enough information for us
-   to gather information from Phabricator, 
+   to gather information from Phabricator,
    but credentials will be gathered from the user's ``~/.arcrc``.
 
 The above example is the minimum required to import issues from
 Phabricator.  You can also feel free to use any of the
 configuration options described in :ref:`common_configuration_options`.
 
+Service Features
+----------------
+
+If you have dozens of users and projects, you might want to
+pull the tasks and code review requests only for the specific ones.
+
+If you want to show only the tasks related to a specific user,
+you just need to add its PHID to the service configuration like this::
+
+    phabricator.user_phids = PHID-USER-ab12c3defghi45jkl678
+
+If you want to show only the tasks and diffs related to a specific project or a repository,
+just add their PHIDs to the service configuration::
+
+    phabricator.project_phids = PHID-PROJ-ab12c3defghi45jkl678,PHID-REPO-ab12c3defghi45jkl678
+
+Both ``phabricator.user_phids`` and ``phabricator.project_phids`` accept
+a comma-separated (no spaces) list of PHIDs.
+
+If you specify both, you will get tasks and diffs that match one **or** the other.
+
+If you do not know PHID of a user, project or repository,
+you can find it out by querying Phabricator Conduit
+(``https://YOUR_PHABRICATOR_HOST/conduit/``) --
+the methods which return the needed info are ``user.query``, ``project.query``
+and ``repository.query`` respectively.
+
+
 Provided UDA Fields
 -------------------
 

bugwarrior/services/__init__.py

@@ -40,7 +40,7 @@
     'activecollab':  'bugwarrior.services.activecollab:ActiveCollabService',
     'jira':          'bugwarrior.services.jira:JiraService',
     'megaplan':      'bugwarrior.services.megaplan:MegaplanService',
-    'phabricator':   'bugwarrior.services.phabricator:PhabricatorService',
+    'phabricator':   'bugwarrior.services.phab:PhabricatorService',
     'versionone':    'bugwarrior.services.versionone:VersionOneService',
 })
 

bugwarrior/services/gitlab.py

@@ -19,6 +19,7 @@ class GitlabIssue(Issue):
     REPO = 'gitlabrepo'
     TYPE = 'gitlabtype'
     NUMBER = 'gitlabnumber'
+    STATE = 'gitlabstate'
     UPVOTES = 'gitlabupvotes'
     DOWNVOTES = 'gitlabdownvotes'
 
@@ -59,6 +60,10 @@ class GitlabIssue(Issue):
             'type': 'numeric',
             'label': 'Gitlab Issue/MR #',
         },
+        STATE: {
+            'type': 'string',
+            'label': 'Gitlab Issue/MR State',
+        },
         UPVOTES: {
             'type': 'numeric',
             'label': 'Gitlab Upvotes',
@@ -76,16 +81,18 @@ def _normalize_label_to_tag(self, label):
     def to_taskwarrior(self):
         if self.extra['type'] == 'merge_request':
             priority = 'H'
-            milestone = '' # No milestone
-            created = '' # No creation time
-            updated = '' # No updated time
+            milestone = self.record['milestone']
+            created = self.record['created_at']
+            updated = self.record['updated_at']
+            state = self.record['state']
             upvotes = self.record['upvotes']
             downvotes = self.record['downvotes']
         else:
             priority = self.origin['default_priority']
             milestone = self.record['milestone']
             created = self.record['created_at']
             updated = self.record['updated_at']
+            state = self.record['state']
             upvotes = 0
             downvotes = 0
 
@@ -111,6 +118,7 @@ def to_taskwarrior(self):
             self.NUMBER: self.record['iid'],
             self.CREATED_AT: created,
             self.UPDATED_AT: updated,
+            self.STATE: state,
             self.UPVOTES: upvotes,
             self.DOWNVOTES: downvotes,
         }
@@ -212,7 +220,7 @@ def filter_repos(self, repo):
 
     def _get_notes(self, rid, issue_type, issueid):
         tmpl = 'https://{host}/api/v3/projects/%d/%s/%d/notes' % (rid, issue_type, issueid)
-        return self._fetch(tmpl)
+        return self._fetch_paged(tmpl)
 
     def annotations(self, repo, url, issue_type, issue, issue_obj):
         notes = self._get_notes(repo['id'], issue_type, issue['id'])
@@ -224,10 +232,11 @@ def annotations(self, repo, url, issue_type, issue, issue_obj):
             issue_obj.get_processed_url(url)
         )
 
-    def _fetch(self, tmpl):
+    def _fetch(self, tmpl, **kwargs):
         url = tmpl.format(host=self.auth[0])
         headers = {'PRIVATE-TOKEN': self.auth[1]}
-        response = requests.get(url, headers=headers)
+
+        response = requests.get(url, headers=headers, **kwargs)
 
         if response.status_code != 200:
             raise IOError(
@@ -239,23 +248,39 @@ def _fetch(self, tmpl):
         else:
             return response.json
 
+    def _fetch_paged(self, tmpl):
+        params = {
+            'page': 1,
+            'per_page': 100,
+        }
+
+        full = []
+        while True:
+            items = self._fetch(tmpl, params=params)
+            full += items
+            if len(items) < params['per_page']:
+                break
+            params['page'] += 1
+
+        return full
+
     def get_repo_issues(self, rid):
         tmpl = 'https://{host}/api/v3/projects/%d/issues' % rid
         issues = {}
-        for issue in self._fetch(tmpl):
+        for issue in self._fetch_paged(tmpl):
             issues[issue['id']] = (rid, issue)
         return issues
 
     def get_repo_merge_requests(self, rid):
         tmpl = 'https://{host}/api/v3/projects/%d/merge_requests' % rid
         issues = {}
-        for issue in self._fetch(tmpl):
+        for issue in self._fetch_paged(tmpl):
             issues[issue['id']] = (rid, issue)
         return issues
 
     def issues(self):
         tmpl = 'https://{host}/api/v3/projects'
-        all_repos = self._fetch(tmpl)
+        all_repos = self._fetch_paged(tmpl)
         repos = filter(self.filter_repos, all_repos)
 
         repo_map = {}

bugwarrior/services/jira.py

@@ -126,7 +126,7 @@ def __init__(self, *args, **kw):
             )
 
         default_query = 'assignee=' + self.username + \
-            ' AND status != closed and status != resolved'
+            ' AND resolution is null'
         self.query = self.config_get_default('query', default_query)
         self.jira = JIRA(
             options={

bugwarrior/services/phab.py

@@ -6,7 +6,6 @@
 # This comes from PyPI
 import phabricator
 
-
 class PhabricatorIssue(Issue):
     TITLE = 'phabricatortitle'
     URL = 'phabricatorurl'
@@ -63,6 +62,14 @@ def __init__(self, *args, **kw):
         # These reads in login credentials from ~/.arcrc
         self.api = phabricator.Phabricator()
 
+        self.shown_user_phids = self.config_get_default("user_phids", "").strip().split(',')
+        if self.shown_user_phids[0] == "":
+            self.shown_user_phids = None
+
+        self.shown_project_phids = self.config_get_default("project_phids", "").strip().split(',')
+        if self.shown_project_phids[0] == "":
+            self.shown_project_phids = None
+
     def issues(self):
 
         # TODO -- get a list of these from the api
@@ -74,17 +81,41 @@ def issues(self):
         log.name(self.target).info("Found %i issues" % len(issues))
 
         for phid, issue in issues:
+
             project = self.target  # a sensible default
             try:
                 project = projects.get(issue['projectPHIDs'][0], project)
             except IndexError:
                 pass
 
+            this_issue_matches = False
+
+            if self.shown_user_phids is None and self.shown_project_phids is None:
+                this_issue_matches = True
+
+            if self.shown_user_phids is not None:
+                # Checking whether authorPHID, ccPHIDs, ownerPHID
+                # are intersecting with self.shown_user_phids
+                issue_relevant_to = set(issue['ccPHIDs'] + [issue['ownerPHID'], issue['authorPHID']])
+                if len(issue_relevant_to.intersection(self.shown_user_phids)) > 0:
+                    this_issue_matches = True
+
+            if self.shown_project_phids is not None:
+                # Checking whether projectPHIDs
+                # is intersecting with self.shown_project_phids
+                issue_relevant_to = set(issue['projectPHIDs'])
+                if len(issue_relevant_to.intersection(self.shown_user_phids)) > 0:
+                    this_issue_matches = True
+
+            if not this_issue_matches:
+                continue
+
             extra = {
                 'project': project,
                 'type': 'issue',
                 #'annotations': self.annotations(phid, issue)
             }
+
             yield self.get_issue_for_record(issue, extra)
 
         diffs = self.api.differential.query(status='status-open')
@@ -93,12 +124,41 @@ def issues(self):
         log.name(self.target).info("Found %i differentials" % len(diffs))
 
         for diff in list(diffs):
+
             project = self.target  # a sensible default
             try:
                 project = projects.get(issue['projectPHIDs'][0], project)
             except IndexError:
                 pass
 
+            this_diff_matches = False
+
+            if self.shown_user_phids is None and self.shown_project_phids is None:
+                this_diff_matches = True
+
+            if self.shown_user_phids is not None:
+                # Checking whether authorPHID, ccPHIDs, ownerPHID
+                # are intersecting with self.shown_user_phids
+                diff_relevant_to = set(diff['reviewers'] + [diff['authorPHID']])
+                if len(diff_relevant_to.intersection(self.shown_user_phids)) > 0:
+                    this_diff_matches = True
+
+            if self.shown_project_phids is not None:
+                # Checking whether projectPHIDs
+                # is intersecting with self.shown_project_phids
+                phabricator_projects = []
+                try:
+                    phabricator_projects = diff['phabricator:projects']
+                except KeyError:
+                    pass
+
+                diff_relevant_to = set(phabricator_projects + [diff['repositoryPHID']])
+                if len(diff_relevant_to.intersection(self.shown_user_phids)) > 0:
+                    this_diff_matches = True
+
+            if not this_diff_matches:
+                continue
+
             extra = {
                 'project': project,
                 'type': 'pull_request',

setup.py

@@ -1,6 +1,6 @@
 from setuptools import setup, find_packages
 
-version = '1.1.3'
+version = '1.1.4'
 
 f = open('bugwarrior/README.rst')
 long_description = f.read().strip()

tests/test_gitlab.py

@@ -88,6 +88,7 @@ def test_to_taskwarrior(self):
             'tags': ['feature'],
             issue.URL: self.arbitrary_extra['issue_url'],
             issue.REPO: 'project',
+            issue.STATE: self.arbitrary_issue['state'],
             issue.TYPE: self.arbitrary_extra['type'],
             issue.TITLE: self.arbitrary_issue['title'],
             issue.NUMBER: self.arbitrary_issue['iid'],