Merge pull request #1090 from akvo/991/join

[#991] Join 'develop' with 'feature/rsr_v3'

.gitignore

@@ -16,6 +16,10 @@ akvo/settings/6*.conf
 akvo/settings/backup
 akvo/testing.py
 scripts/dev-vm/
+data/*
+!data/README.md
+!data/dump.sh
+!data/load.sh
 
 # Dev environment
 .vagrant
@@ -26,6 +30,12 @@ scripts/devhelpers/node_modules
 vagrant/vagrant-ssh-conf.*
 
 
+# PG migration
+pg_migrate_conf.yml
+pg_tests.log
+pg_db_password.tmp
+
+
 ## Deployment
 scripts/deployment/builders/osx/config/python_system.config
 scripts/deployment/builders/osx/config/rsr_env.config

README.md

@@ -16,3 +16,37 @@ More information in these files:
 * [AUTHORS](AUTHORS.txt)
 * [LICENSE](LICENSE.md)
 * [INSTALL](INSTALL.md)
+
+
+## Development
+For local development Vagrant is used. The development env is loaded with a small fixture data set with [available accounts](docs/RSR%20Developer%20Documentation/Environment%20Setup/Accounts.md). To get from checkout to running server use these steps:
+
+### Setup local settings
+```shell
+$ cp akvo/settings//66_local.template akvo/settings/66_local.conf
+```
+
+### Boot vagrant environment
+This will download a vagrant box, provision the box with Puppet, download initial data and load that data. Hence this steps takes a while,
+```shell
+$ cd vagrant && vagrant up --provider=virtualbox
+...
+$ cd ..
+```
+Now open a browser and visist [http://rsr.localdev.akvo.org/](http://rsr.localdev.akvo.org/). We're now using the production Gunicorn webserver. For development the Django development server which gives you code reloading is really handy.
+
+### Run a local devserver
+To run the Django devserver with code reloading and getting prints
+```shell
+$ cd scripts/devhelpers
+$ ./supervisord.sh stop rsr && ./manage.sh runserver
+```
+
+### Make sure assets are rebuilt on change (WIP)
+To build assets we're relying on node.js and gulp. So with node and gulp available: 
+```shell
+$ cd scripts/devhelpers
+$ npm install
+$ gulp watch
+```
+

RELEASE_NOTES.md

@@ -7,6 +7,138 @@ Read more about [Akvo Products](http://akvo.org/products/).
 
 --------
 
+Akvo RSR ver 2.6 Conakry
+---
+
+Wednesday 17th December 2014, adriancollier
+
+New & Noteworthy
+---
+
+### PostgreSQL as the Database Engine
+
+In this release we are migrating our underlying database engine from MySQL to PostgreSQL. Following on from this change, we will begin to be able to take advantage of many of the additional pieces of functionality that PostgreSQL offers us including Materialised Views.
+
+Github issue: [#813](https://github.com/akvo/akvo-rsr/issues/813) & [#753](https://github.com/akvo/akvo-rsr/issues/753)
+
+### Minor Visual Updates
+
+We have made some minor changes to the data visualisation within RSR. Nothing large, but these should meet some of the demands we've been receiving.
+
+#### Visualise Identifiers on Project Pages
+
+If a project has additional identifiers, including an IATI ID or an Internal ID obtained from an external system, then these are now displayed on the Project page alongside the RSR ID.
+
+Github issue: [#946](https://github.com/akvo/akvo-rsr/issues/946)
+
+#### Start and End dates
+
+We are accepting 4 types of dates on a project now, start and end with both planned and actual values. We have now implemented a change to show all of these dates within the Funding page for the project.
+
+Github issue: [#936](https://github.com/akvo/akvo-rsr/issues/936)
+
+#### IATI Activity Link in Akvo Pages
+
+This is a minor fix to enhance the IATI Identifier (if existing) on the Funding page to direct a user towards an external source of further information (such as Openaid.nl) to match the functionality within RSR.
+
+Github issue: [#868](https://github.com/akvo/akvo-rsr/issues/868)
+
+#### Partner Name on Akvo Pages
+
+The Partner Name on Akvo Pages was previously only displaying the short name. We have updated this to match the layout and display of both the name and country as displayed in RSR.
+
+Github issue: [#729](https://github.com/akvo/akvo-rsr/issues/729)
+
+
+--------
+
+Akvo RSR ver 2.5.1 Bujumbura Hotfix
+---
+
+Sunday 2nd November 2014, kasperbrandt
+
+Bug Fixes
+---
+
+### Update locations on maps
+
+We noticed a bug that showed project update locations with (0, 0) as longitude and latitude on the maps, these are now not shown anymore.
+
+Github issue: [#837](https://github.com/akvo/akvo-rsr/issues/837)
+
+### Cordaid import
+
+Two bugs in the Cordaid import have been fixed. One causing the import to run slowly due to the 'sync_owner' field, and another that did not import the 'date_start_planned' and 'date_end_planned' fields correctly.
+
+Github issue: [#865](https://github.com/akvo/akvo-rsr/issues/865)
+
+--------
+
+Akvo RSR ver 2.5 Bujumbura
+---
+
+Wednesday 21st October 2014, adriancollier
+
+Improvements
+---
+
+### Filtering of Updates
+
+The API has been modified to allow the list of updates to be filtered on timestamp. This was to ensure that the RSR API is fully compatible with the upcoming release of RSR Up.
+
+Github issue:  [#769](https://github.com/akvo/akvo-rsr/issues/769)
+
+Bug Fixes
+---
+
+### Finalisation of PayPal Donations
+
+We noticed a bug that was preventing PayPal donations from being completed as the callback from the PayPal service was not being correctly read. This has been fixed and we will apply this to all affected donations.
+
+Github issue: [#796](https://github.com/akvo/akvo-rsr/issues/796)
+
+### IATI Export Activity ID
+
+There was a mistake in the IATI export that resulted in an incorrect IATI ID being applied to activities that has now been rectified.
+
+Github issue: [#799](https://github.com/akvo/akvo-rsr/issues/799)
+
+### Project Update Extra API Resource
+
+An issue was discovered that when an organisation has no location, the API resource was returning an error value. While not common, this needed to be resolved for the few instances where this is the case.
+
+Github issue: [#820](https://github.com/akvo/akvo-rsr/issues/820)
+
+--------
+
+Akvo RSR ver 2/4/2 Astana Hotfix II
+---
+
+Tuesday 30th September 2014, zzgvh
+
+Included changes
+---
+
+### Import RAIN IATI Data
+
+Update the import scripts with better logging, use of online source files for the import adn a bugfix for an API call needed by the RAIN import.
+
+Github issue: [#710](https://github.com/akvo/akvo-rsr/issues/710)
+
+### Project exclusion for Pages
+
+Using keywords introduced in 2.3.9 Yam it is now possible to exclude projects for an Akvo Pages website.
+
+Github issue: [#745](https://github.com/akvo/akvo-rsr/issues/745)
+
+### Bugfix for RSR country list
+
+The country list used in RSR also contains a mapping between countires and continents. The mapping for Bonaire, Sint Eustatius and Saba was missing.
+
+Github issue: [#748](https://github.com/akvo/akvo-rsr/issues/748)
+
+--------
+
 Akvo RSR ver 2/4/1 Astana Hotfix
 ---
 

akvo/api/authentication.py

@@ -18,6 +18,7 @@ class ConditionalApiKeyAuthentication(ApiKeyAuthentication):
     """
     def __init__(self, methods_requiring_key=None):
         "All methods require API key if nothing else is specified on object creation"
+        super(ConditionalApiKeyAuthentication, self).__init__()
         if methods_requiring_key is not None:
             self.methods_requiring_key = methods_requiring_key
         else:

akvo/api/resources/project.py

@@ -7,7 +7,7 @@
 
 from decimal import Decimal
 
-from django.contrib.auth import get_permission_codename
+from django.contrib.auth import get_permission_codename, get_user_model
 from django.core.exceptions import ObjectDoesNotExist
 from django.forms.models import ModelForm
 
@@ -16,6 +16,7 @@
 from tastypie.authorization import Authorization
 from tastypie.constants import ALL, ALL_WITH_RELATIONS
 from tastypie.exceptions import NotFound
+from tastypie.http import HttpUnauthorized
 from tastypie.resources import ModelResource
 
 from akvo import settings
@@ -65,10 +66,14 @@ class IATIProjectResource(ModelResource):
         'partnerships', full=True, related_name='project'
     )
 
-    sync_owner = fields.ToOneField(
-        'akvo.api.resources.OrganisationResource',
-        'sync_owner', full=True, related_name='project'
-    )
+    # This field makes the Cordaid import dismally slow, and isn't used for anything else,
+    # so rather than trying to fix it by re-writing the OrganisationResource, we set the sync_owner
+    # to Cordaid in the post_import.py script
+    # This commented code remains as a reminder NOT to add the field as currently defined ;-)
+    # sync_owner = fields.ToOneField(
+    #     'akvo.api.resources.OrganisationResource',
+    #     'sync_owner', full=True, related_name='project'
+    # )
 
     class Meta:
         allowed_methods = ['post', 'put']
@@ -126,7 +131,6 @@ def alter_deserialized_detail_data(self, request, data):
         reporting_iati_org_id = data['partnerships'][0]['reporting_org']
         # Cordaid custom code
         if reporting_iati_org_id == getattr(settings, 'CORDAID_IATI_ID', 'NL-KVK-41160054'):
-            data['sync_owner'] = Organisation.objects.get(iati_org_id_exact='NL-KVK-41160054')
             # Figure out the category for the project from the business unit
             business_unit_categories = {
                 "K6020": dict(cat_name="Children & Education", fa="Education"),
@@ -157,7 +161,6 @@ def alter_deserialized_detail_data(self, request, data):
                     benchmarks.append(new_benchmark)
             data['benchmarks'] = benchmarks
         if reporting_iati_org_id == getattr(settings, 'RAIN_IATI_ID', 'NL-KVK-34200988'):
-            data['sync_owner'] = Organisation.objects.get(iati_org_id__exact='NL-KVK-34200988')
             # remove benchmarks, as they currently have no category
             data['benchmarks'] = []
 
@@ -316,7 +319,12 @@ def get_object_list(self, request):
                 regardless of publishing status
         """
         object_list = super(ProjectResource, self).get_object_list(request)
-        if self._meta.authentication.is_authenticated(request):
+        # The whole point of ConditionalApiKeyAuthentication is to allow some access even for unauthorised requests,
+        # but here we need to figure out if the request contains a name/key pair and if so allow access to unpublished
+        # projects. So we call ApiKeyAuthentication.is_authenticated() (using super() which returns True if there is an
+        # identified user holding an api key, AND is_authenticated() also sets request.user to the User object which we
+        # need to be able to call request.user.has_perm() correctly.
+        if super(ConditionalApiKeyAuthentication, self.Meta.authentication).is_authenticated(request) is True:
             opts = Project._meta
             if request.user.has_perm(opts.app_label + '.' + get_permission_codename('change', opts)):
                 return object_list
@@ -325,7 +333,6 @@ def get_object_list(self, request):
                 return object_list.distinct()
         return object_list.published()
 
-
     def dehydrate(self, bundle):
         """ add thumbnails inline info for Project.current_image
         """

akvo/api/resources/project_update.py

@@ -65,3 +65,89 @@ def dehydrate(self, bundle):
             if isinstance(bundle.data['time'], bool): bundle.data['time'] = None
             if isinstance(bundle.data['time_last_updated'], bool): bundle.data['time_last_updated'] = None
             return bundle
+
+
+class ProjectUpdateResourceExtra(ProjectUpdateResource):
+    class Meta(ProjectUpdateResource.Meta):
+        allowed_methods = ['get',]
+        resource_name = 'project_update_extra'
+
+    def dehydrate(self, bundle):
+        def primary_location_data_for_update(obj):
+            """ We need similar data for both the project and the organisation associated with the update
+            """
+            try:
+                primary_location = dict(obj.primary_location.__dict__)
+            except:
+                return None
+            # remove Django internal field
+            primary_location.pop('_state', None)
+            country_id = primary_location.pop('country_id', None)
+            if country_id:
+                primary_location['country'] = dict(
+                    resource_uri=reverse(
+                        'api_dispatch_detail', kwargs={
+                            'resource_name':'country', 'api_name': 'v1', 'pk': country_id
+                        }
+                    ),
+                )
+                country = Country.objects.get(pk=country_id)
+                country_dict = country.__dict__
+                # remove Django internal field
+                country_dict.pop('_state', None)
+                primary_location['country'].update(country_dict)
+            return primary_location
+
+        def org_data_for_update(organisation):
+            """ return relevant data for the organisation that is linked to an update through the user that created the update
+            """
+            return dict(
+                absolute_url=organisation.get_absolute_url(),
+                long_name=organisation.long_name,
+                name=organisation.name,
+                resource_uri=reverse(
+                    'api_dispatch_detail', kwargs={
+                        'resource_name':'organisation', 'api_name': 'v1', 'pk': organisation.pk
+                    }
+                ),
+            )
+
+        def user_data_for_update(user):
+            return dict(
+                first_name=user.first_name,
+                last_name=user.last_name,
+            )
+
+        def project_data_for_update(bundle):
+            return dict(
+                resource_uri=bundle.data['project'],
+            )
+
+        bundle = super(ProjectUpdateResourceExtra, self).dehydrate(bundle)
+
+        organisation = bundle.obj.user.userprofile.organisation
+        org_dict = org_data_for_update(organisation)
+        user_resource_uri = bundle.data['user']
+        bundle.data['user'] = user_data_for_update(bundle.obj.user)
+        bundle.data['user'].update(resource_uri=user_resource_uri)
+        bundle.data['user']['organisation'] = org_dict
+        bundle.data['user']['organisation'].update(primary_location=primary_location_data_for_update(organisation))
+        bundle.data['project'] = project_data_for_update(bundle)
+        bundle.data['project'].update(primary_location=primary_location_data_for_update(bundle.obj.project))
+
+        return bundle
+
+    def build_schema(self):
+        data = super(ProjectUpdateResourceExtra, self).build_schema()
+        data['fields']['user'] = {
+            'default': "No default provided.",
+            'type': "to_one",
+            'nullable': False,
+            'blank': False,
+            'readonly': True,
+            'help_text': "A custom related resource with parts of data from user and the organisation the user belongs to. "
+                "Includes the fields full_name, organisation and resource_uri of user and absolute_url, long_name, name and resource_uri "
+                "of organisation.",
+            'unique': False,
+        }
+        return data

akvo/api/xml/iati-xslt.xsl

@@ -16,7 +16,9 @@
 
       <xsl:apply-templates select="activity-date[@type='start-actual']" />
       <xsl:apply-templates select="activity-date[@type='end-actual']" />
-
+      <xsl:apply-templates select="activity-date[@type='start-planned']" />
+      <xsl:apply-templates select="activity-date[@type='end-planned']" />
+
       <xsl:apply-templates select="description[@type='1']" /><!-- General -->
       <xsl:apply-templates select="description[@type='2']"/><!-- Objectives -->
       <xsl:apply-templates select="description[@type='3' and @akvo:type='3']"/><!-- Traget groups -->