Merge branch 'qa'

AUTHORS.rst

@@ -8,6 +8,7 @@ The list of contributors in alphabetical order:
 - `Adriano Di Florio <https://github.com/AdrianoDee>`_
 - `Alexander Ioannidis <https://github.com/slint>`_
 - `Ana Trisovic <https://github.com/atrisovic>`_
+- `Andrii Verbytskyi <https://orcid.org/0000-0002-3713-8033>`_
 - `Anna Trzcinska <https://github.com/annatrz>`_
 - `Annemarie Mattmann <https://github.com/Kjili>`_
 - `Anxhela Dani <https://github.com/anxheladani>`_
@@ -38,6 +39,7 @@ The list of contributors in alphabetical order:
 - `Osama Sh. Almomani <https://github.com/OsamaMomani>`_
 - `Pamfilos Fokianos <https://github.com/pamfilos>`_
 - `Patricia Herterich <https://github.com/pherterich>`_
+- `Richard Hildebrandt <https://github.com/rdebrand>`_
 - `Sebastian Feger <https://github.com/sefeg>`_
 - `Sergey Dmitrievsky <https://github.com/dmitr25>`_
 - `Silvia Amerio <https://github.com/samerio>`_

DEVELOPING.rst

@@ -50,6 +50,12 @@ Another possibility to develop the CERN Open Data instance locally is to use
 the Podman container technology. This has an advantage that your containers
 will be running in the regular user space, not requiring any superuser access.
 
+If you are using Linux operating system with SELinux, and would like to use the
+developer-oriented installation method with the live code-reload feature, then
+please configure your SELinux to use either "permissive", "minimal" or
+"disabled" policy. You can use the ``getsebool`` command to show your current
+SELinux policy level.
+
 An example of a Podman development session:
 
 .. code-block:: console
@@ -61,6 +67,7 @@ An example of a Podman development session:
        ./scripts/populate-instance.sh --skip-docs --skip-glossary --skip-records
    $ podman exec -i -t opendatacernch_web_1 \
        cernopendata fixtures records --mode insert -f cernopendata/modules/fixtures/data/records/cms-primary-datasets.json
+   $ firefox http://0.0.0.0:5000/
    $ podman-compose -f docker-compose-dev.yml down -v
 
 Note that if you would like to test production-like conditions with Podman, you
@@ -267,6 +274,16 @@ you can do:
    docker exec -i -t opendatacernch-web-1 cernopendata fixtures records -f /code/cernopendata/modules/fixtures/data/records/opera-ecc-datasets.json --mode insert
    docker exec -i -t opendatacernch-web-1 cernopendata fixtures records -f /code/cernopendata/modules/fixtures/data/records/opera-ed-datasets.json --mode insert
 
+Beware when switching between production and development or between different
+version of Python, since this may necessitate to delete all `*.pyc` and similar
+files created during development. The best is to make sure that you don't have
+any non-committed changes to the source code in your workspace and then to
+clean your workspace fully by running:
+
+.. code-block:: shell
+
+   sudo git clean -d -ff -x
+
 Appendix: Git workflow
 ======================
 

cernopendata/config.py

@@ -1,7 +1,7 @@
 # -*- coding: utf-8 -*-
 #
 # This file is part of CERN Open Data Portal.
-# Copyright (C) 2017, 2018, 2020, 2022 CERN.
+# Copyright (C) 2017, 2018, 2020, 2022, 2023 CERN.
 #
 # CERN Open Data Portal is free software; you can redistribute it
 # and/or modify it under the terms of the GNU General Public License as
@@ -337,7 +337,7 @@ def _query_parser_and(qstr=None):
                 ),
             ),
             experiment=dict(terms=dict(field="experiment", order=dict(_key="asc"))),
-            year=dict(terms=dict(field="date_created", order=dict(_key="asc"))),
+            year=dict(terms=dict(field="date_created", size=70, order=dict(_key="asc"))),
             file_type=dict(
                 terms=dict(
                     field="distribution.formats", size=50, order=dict(_key="asc")

cernopendata/modules/fixtures/data/docs/cms-completes-run1-heavy-ion-open-data-collection-2023/cms-completes-run1-heavy-ion-open-data-collection-2023.json

@@ -15,7 +15,7 @@
     ],
     "created": "2023-09-18",
     "experiment": "CMS",
-    "featured": 1,
+    "featured": 2,
     "short_description": {
       "content": "New release of simulations, proton-lead collision data, and proton reference data."
     },

cernopendata/modules/fixtures/data/docs/cms-completes-run1-pp-data-release-2022/cms-completes-run1-pp-data-release-2022.json

@@ -15,7 +15,6 @@
     ],
     "created": "2022-12-05",
     "experiment": "CMS",
-    "featured": 3,
     "short_description": {
       "content": "All proton-proton data collected by the CMS experiment during LHC Run-1 (2010-2012) are now available through the CERN Open Data Portal."
     },

cernopendata/modules/fixtures/data/docs/jade-about/jade-about.json

@@ -0,0 +1,28 @@
+[
+  {
+    "body": {
+      "content": "jade-about.md",
+      "format": "md"
+    },
+    "collections": [
+      {
+        "experiment": "JADE"
+      }
+    ],
+    "experiment": "JADE",
+    "short_description": {
+      "content": "JADE was one of the experiments located at the PETRA e+e- storage ring at DESY in Hamburg, Germany. The experiment took data between 1979 and 1986 in the center-of-mass range between 12 and 46.6 GeV."
+    },
+    "slug": "about-jade",
+    "tags": [
+      "Getting Started"
+    ],
+    "title": "About JADE",
+    "type": {
+      "primary": "Documentation",
+      "secondary": [
+        "About"
+      ]
+    }
+  }
+]

cernopendata/modules/fixtures/data/docs/jade-about/jade-about.md

@@ -0,0 +1,65 @@
+The JADE experiment was one of the experiments located at the PETRA e+e-
+ storage ring at DESY in Hamburg, Germany.  Its scientific
+programme aimed at answering a series of fundamental questions such as:
+
+*   What is the dynamics of the strong interaction that binds quarks inside ordinary matter ?
+You can find more information on the JADE detector and physics on the [JADE website](https://wwwjade.mpp.mpg.de/)
+
+## About JADE History
+The JADE experiment was one of the experiments located at the PETRA e+e-
+ storage ring at DESY in Hamburg, Germany. The experiment took data
+ between 1979 and 1986 in the center-of-mass range between 14 and 45GeV.
+ Since 1997 a group at the University of Aachen, now located at the MPI
+ in Munich, started to re-analyze the data taken during that time period.
+ New theoretical calculations developed during and after the running of
+ the LEP experiments are applied to the data taken between 14 and 45GeV.
+ This allows a unique access to data at that energy range.
+
+## About JADE Data
+
+The following are provided through this portal:
+
+*   Primary datasets
+
+    *   The available [primary JADE datasets](FIXME) contain a all
+     events recorded by JADE. The data format is the BOS
+    (event summary data) format generated by the JADE raw data
+    reconstruction software.
+
+*   Software tools
+
+    *   A downloadable [Virtual Machine (VM)](/docs/jade-virtual-machine)
+    image with the JADE software environment which allows to access the above datasets
+    *   An [analysis example chain](/docs/jade-getting-started) running
+    in the VM, reading a primary dataset using the
+    [JADE analysis framework](/docs/jade-virtual-machine).
+    The sources for the example can be adapted and recompiled by the
+    user to look at different features of JADE data. This exercise
+    requires a minimal understanding of the usage of the analysis framework.
+
+## Primary Datasets formats
+
+Data in the primary datasets are in a format known as
+BOS (Event Summary Data), produce by the
+JADE [reconstruction procedure](/docs/jade-virtual-machine)
+
+*   The BOS contains lists of reconstructed
+tracks/particles and global event properties. BOS files contain the
+information that is needed for the analysis:
+
+    *   All the high-level physics objects (such as muons, electrons, etc.);
+    *   Tracks with associated hits, calorimetric clusters with associated
+     hits, vertices;
+    *   Information about event selection (triggers), data needed for
+    further selection and identification criteria for the physics objects.
+*   An BOS file is not the final event interpretation with a simple list
+of particles, but just its best current understanding that takes into
+account several parameters
+
+## Disclaimer
+
+*   The open data are released under the [Creative Commons CC0 waiver](http://creativecommons.org/publicdomain/zero/1.0/).
+*   Neither JADE nor CERN endorse any works, scientific or otherwise, produced using these data.
+*   All released data samples will have a unique DOI that you are requested to cite in any applications or publications.
+*   Despite being processed, the high-level primary datasets remain complex and selection criteria need to be applied in order to analyse them,
+*   requiring some understanding of particle physics and detector functioning. The data cannot be viewed in simple data tables for spreadsheet-based analyses.

cernopendata/modules/fixtures/data/docs/jade-getting-started/jade-getting-started.json

@@ -0,0 +1,31 @@
+[
+  {
+    "body": {
+      "content": "jade-getting-started.md",
+      "format": "md"
+    },
+    "collections": [
+      {
+        "experiment": "JADE"
+      },
+      {
+        "primary": "VM"
+      }
+    ],
+    "experiment": "JADE",
+    "short_description": {
+      "content": "JADE software comes via a virtual machine image. The only thing you need to install by yourself on your desktop is VirtualBox. Then you just need to download the JADE virtual machine image open it with VirtualBox..."
+    },
+    "slug": "jade-getting-started",
+    "tags": [
+      "Getting Started"
+    ],
+    "title": "Getting Started with JADE Open Data",
+    "type": {
+      "primary": "Documentation",
+      "secondary": [
+        "Guide"
+      ]
+    }
+  }
+]

cernopendata/modules/fixtures/data/docs/jade-getting-started/jade-getting-started.md

@@ -0,0 +1,21 @@
+Learn how to use the JADE virtual machine to have a first look at JADE events and use analysis tools:
+
+1. ["How do I start JADE software?"](#start)
+2. ["What kind of JADE data will I work on?"](#data)
+3. ["I have installed VirtualBox, downloaded JADE VM image and launched it. And now?"](#vbox)
+
+## <a name="start">"How do I start JADE software?"</a>
+
+JADE software comes via a virtual machine image. The only thing you need to
+install by yourself on you desktop is VirtualBox. Then you just need to
+download the JADE virtual machine image open it with VirtualBox. See
+instructions [here](/docs/jade-virtual-machine).
+
+## <a name="data">"What kind of JADE data will I work on?"</a>
+
+The data samples you can download from this portal consists all events
+collected by the JADE.
+
+## <a name="vbox">"I have installed VirtualBox, downloaded JADE VM image and launched it. And now?"</a>
+
+FIXME

cernopendata/modules/fixtures/data/docs/jade-mc-generation/jade-mc-generation.json

@@ -0,0 +1,31 @@
+[
+  {
+    "body": {
+      "content": "jade-mc-generation.md",
+      "format": "md"
+    },
+    "collections": [
+      {
+        "experiment": "JADE"
+      },
+      {
+        "primary": "VM"
+      }
+    ],
+    "experiment": "JADE",
+    "short_description": {
+      "content": "JADE Monte Carlo generation primer"
+    },
+    "slug": "jade-mc-generation",
+    "tags": [
+      "Getting Started"
+    ],
+    "title": "Understanding JADE MC generation",
+    "type": {
+      "primary": "Documentation",
+      "secondary": [
+        "Guide"
+      ]
+    }
+  }
+]

cernopendata/modules/fixtures/data/docs/jade-mc-generation/jade-mc-generation.md

@@ -0,0 +1,64 @@
+This document describes the analysis of MC simulated event samples for JADE.
+
+0. ["Stages of simulations"](#stages)
+1. ["Which generators to use"](#generators)
+2. ["Most important settings"](#setings)
+3. ["Smearing"](#smearing)
+
+## <a name="stages">"Stages of simulation"</a>
+The simulation the events observed in the JADE detector can be split into two large stages: the simulation of the
+hadronic final state and the simulation of the detector responce (detection) of the simulated hadronic final state.
+
+The very first step of the hadronic final state simulation is the simulation of  production of partons in the final
+state, i.e. quarks, gluons or other elementary particles.
+This step can be described by the MC event generators precisely to some fixed order of perturbation theory.
+The next step involves the simulation of radiation by the simulated particles, so called parton shower (algorithm).
+This parton shower helps to improve the overall description of the physics in the final state.
+After the parton shower application the hadronisation process is simulated. At this stage the produced coloured partons
+(quarks and gluons)  are recombined and/or split to
+form in the end colourless particles -- hadrons.
+Afther the simulation of hadronisation process the decays of unstable particles are simulated.
+This is the final stage of the simulation of hadronic final state. As the end result of simulation, a set of stable or relatively
+long-lived particles is produced. Typically these include pions, electrons, muons, photons, protons, neutrons, kaons and
+a tiny fraction of undecayed heavier hadrons.
+
+
+
+The simulation of the detector response in JADE is done via parametric smearing.
+Note that as of 2019 this tecnique is used in HEP only for the "fast" simulation of detector response, e.g. in DELPHES framework.
+
+The basic idea of the tecnique is to establish a relation between  the properties of objects observed in the detector
+(e.g. charged aprticle tracks) and the properties of particles that produces these objects.
+The relation is build via distortion of the relevant parameters of the original particles, e.g. momenta, direction, etc.
+The average size and type of distortion is selected  to provide best match to the data observed in the JADE detector.
+For the individual object the size and type of distortion is selected randomly acording to predefined distributions.
+
+
+## <a name="gen">"Which generators to use?"</a>
+
+The most common  processes in $e^+e^-$ collisions can be simulated by many moders MC event generators.
+E.g. $e^+e^-\rightarrow q\bar{q}$ can be simulated by Pythia6, Pythia8, SHERPA, Herwig6, Herwig7, WHIZARD.
+For more complictaed processes that include a more complicated final states, higher order corrections,
+simultions of physics processes beyond Standard Model specific generators can be used.
+
+
+## <a name="settings">"Most important settings"</a>
+The most important general settings are
+The enegry of the beams and initial and final state radiation.
+The energy of the beams should match the energy of JADE beams during the simulated datataking period.
+The output of the MC event generators should be put in the HepMC2, HePMC3 or HEPEVT formats.
+
+
+## <a name="smearing">"Smearing"</a>
+To Smear the MC generated events, these are fed to the original JADE software.
+The events should be converted forts to the format accepted by the JADE software.
+That is done with the convert utility.
+
+
+
+
+
+
+
+
+

cernopendata/modules/fixtures/data/docs/jade-software/jade-software.json

@@ -0,0 +1,34 @@
+[
+  {
+    "body": {
+      "content": "jade-software.md",
+      "format": "md"
+    },
+    "collections": [
+      {
+        "experiment": "JADE"
+      },
+      {
+        "primary": "Documentation",
+        "secondary": [
+          "Guide"
+        ]
+      }
+    ],
+    "experiment": "JADE",
+    "short_description": {
+      "content": "JADE Software for reconstruction"
+    },
+    "slug": "jade-software",
+    "tags": [
+      "Software"
+    ],
+    "title": "Understading JADE Software",
+    "type": {
+      "primary": "Documentation",
+      "secondary": [
+        "Guide"
+      ]
+    }
+  }
+]