Snapshot Upload API

In addition to the GraphQL-based read-only API to load snapshots, workspaces and site configurations from the backend, new since version 0.7.0 an Upload API provides the possibility to upload snapshot json files. This entry explores how to use these endpoints on a technical level to implement uploaders into existing workflows and automation systems.

Uploading a Snapshot via the Web User Interface

Uploading a Snapshot via the API

Step 1: Get a CORS session token

This requires a simple GET request to the relevant instance, e.g. sandbox.gemeindescan.ch/account/login/.

If successful the response contains the CORS-Token as a cookie called csrftoken.

In python this could be done with the requests module:

import requests

baseUrl = "https://sandbox.gemeindescan.ch/account/login/" # example cividi Gemeindescan Sandbox endpoint
client = requests.Session() # Set up a session to reuse below

client.get(f'%s/account/login/' % (baseUrl)) # make an HTTP GET request to the endpoint
csrftoken = client.cookies['csrftoken'] # Extract the CORS-Token from the corresponding Cookie

Step 2: Get the user session id

This performs a login on the django backend with a valid user and password to obtain a sessionid.

To do so, a POST request on the same session as above (using the same CORS-Token etc. from earlier) is submitted, adding the username and password as application/x-www-form-urlencoded payload in the request body.

If successful, the response contains a sessionid to use in combination with the CORS-Token from earlier to make subsequent requests that need authentication on the GraphQL (/graphql/) and REST (/api/v1/) endpoints.

# continued from above
# client, csrftoken and baseURL see step 1

import urllib.parse

headers = {
    'Cookie': '; '.join([f'csrftoken=%s' % csrftoken]), # make sure the CORS-Token cookie is set
    'Content-Type': 'application/x-www-form-urlencoded', # set the Content Type to form-urlencoded
    'Referer': f'%s/account/login/' % baseUrl
}
payload = {
    'csrfmiddlewaretoken': csrftoken, # set the the CORS-Token
    'username': '<YOUR GEMEINDESCAN USERNAME>',
    'password': '<YOUR GEMEINDESCAN PASSWORD>',
}

payload = urllib.parse.urlencode(payload) # encode payload as www-form-urlencoded

r = client.post(f'%s/account/login/' % # make the POST request
            (baseUrl), data=payload, headers=headers)

sessionid = client.cookies['sessionid']

Step 3: Get a Snapshot ID and a Workspace ID

To upload a file we need the snapshot ID. This can be either an existing snapshot or creating a new snapshot via SnapshotMutation on GraphQL API.

Look up your Snapshot ID...

The Snapshot ID is part of the URL you use to open a snapshot. E.g. for the URL https://sandbox.gemeindescan.ch/de/7X6LIB/ the snapshot ID is 7X6LIB, if you are in a workspace, e.g. https://sandbox.gemeindescan.ch/de/B5BXT/8OK4XB/ the snapshot ID is 8OK4XB, whereas B5BXT is the Workspace ID.

Alternatively you can create a new snapshot via the GraphQL endpoint (e.g. https://sandbox.gemeindescan.ch/graphql/). This requires authentication by providing a valid CORS-Token and sessionid (see above).

Mutation Query

mutation updatesnapshot($data: SnapshotMutationInput!) {
    snapshotmutation(input: $data) {
        snapshot {
            id
            pk
            title
            topic
            municipality {
                bfsNumber
            }
            datafile
        }
    }
}

Variables to include as SnapshotMutationInput

• wshash: a base64 url encoded version of WorkspaceNode:${Workspace ID} # see above for Workspace ID
• title, topic: Title and topic as shown on the left column
• bfsNumber: A valid BfS number of a Swiss municipality

Example JSON payload

{
  "data": {
    "title": "YOUR TITLE",
    "topic": "YOUR TOPIC",
    "bfsNumber": "BFS NUMBER OF MUNICIPALITY",
    "wshash": "WSHASH"
  }
}

The response contains the data specified in the mutation query above, for example the newly created id which is the Snapshot ID.

Example for creating a new snapshot in python using gql and AIO

# continued from above
# see step 1 and 2 for variables

from gql import gql, Client
from gql.transport.aiohttp import AIOHTTPTransport
import base64

wshash = "B5BXT" # e.g. from https://sandbox.gemeindescan.ch/de/B5BXT/8OK4XB/ -> B5BXT

transport = AIOHTTPTransport(
    url=f"%s/graphql/" % (baseUrl), cookies=client.cookies, headers=client.headers)
client = Client(transport=transport, fetch_schema_from_transport=True)

query = gql(
    """
mutation updatesnapshot($data: SnapshotMutationInput!) {
    snapshotmutation(input: $data) {
        snapshot {
            id
            pk
            title
            topic
            municipality {
                bfsNumber
            }
            datafile
        }
    }
}
"""
)

params = {
    "data": {
        "title": "YOUR TITLE",
        "topic": "YOUR TOPIC",
        "bfsNumber": "YOUR BFS NUMBER",
        "wshash": base64.b64encode(':'.join(['WorkspaceNode', "<YOUR WORKSPACE ID>"]).encode('utf-8')).decode('ascii')
    }
}

result = client.execute(query, variable_values=params)

if(result["snapshotmutation"]["snapshot"]):
    snapshot = result["snapshotmutation"]["snapshot"]
    snapshot["wshash"] = wshash
else:
    raise ValueError("GraphQL API query failed: %s.\nParams: %s" % (result, params)) # check if login above returned a CORS-Token and sessionid

Step 4: Upload snapshot.json to Django REST endpoint

This consists of a PATCH request to /api/v1/snapshots/<SNAPSHOT ID>/, with the X-CSRFToken header set to the CORS-Token from earlier and two cookies

• csrftoken set to the CORS-token (see CORS-token step)
• sessionid set to the user session id (see login step)

Example script in python

uploadUrl = f"%s/api/v1/snapshots/%s/" % (baseUrl, snapshot["pk"])

files = [
    ('data_file', ('<YOUR FILE NAME>', open('<YOUR PATH TO THE FILE>', 'rb'), 'application/json'))
]
headers = {
    'X-CSRFToken': client.cookies["csrftoken"], # CORS-Token from above
}

r = client.request(
    "PATCH", uploadUrl, headers=headers, files=files) # submit the PATCH request