Merge branch 'python' of https://github.com/CMS-HGCAL/rpi-daq into py…

…thon

Makefile

@@ -0,0 +1,73 @@
+# https://gist.github.com/fm4dd/c663217935dc17f0fc73c9c81b0aa845
+
+# 100 ev in 14.700s
+#MYCFLAGS= -O3 -mtune=cortex-a53 -mcpu=cortex-a53 -mfloat-abi=hard -mfpu=neon-fp-armv8 -mneon-for-64bits
+
+# 100 ev in 17.370s
+#MYCFLAGS= -O0
+
+# 100 ev in 14.860s
+#MYCFLAGS= -O2
+
+# 100 ev in 14.670s
+#MYCFLAGS= -O1
+
+# 100 ev in 14.380s
+MYCFLAGS= -Os
+
+# 100 ev in 17.540s
+#MYCFLAGS=
+
+# 100 ev in 14.700s
+#MYCFLAGS = -O2 -mcpu=cortex-a53 -mfpu=neon-fp-armv8 -mfloat-abi=hard -funsafe-math-optimizations 
+
+# Benchmarking command
+# make distclean; make; time (python run_local.py --dataNotSaved > /dev/null)
+
+all: lib/libgpiohb.so
+
+src/gpiohb.o: src/gpiohb.c lib/libbcm2835.so
+	gcc -c -I ./src/bcm2835/src -L ./lib -fPIC $(MYCFLAGS) $< -o $@
+
+lib/libgpiohb.so: src/gpiohb.o
+	mkdir -p lib
+	gcc -shared $< -o $@
+
+lib/libbcm2835.so: src/bcm2835/src/bcm2835.o
+	mkdir -p lib
+	gcc -shared $< -o $@
+
+src/bcm2835/src/bcm2835.o: src/bcm2835/src/bcm2835.c
+	make -C src/bcm2835
+
+src/bcm2835/src/bcm2835.c:
+	if ! dpkg -l | grep html-xml-utils -c >>/dev/null; then sudo apt-get --yes install html-xml-utils; fi
+	mkdir -p src/bcm2835
+	wget -qO - `curl -sL http://www.airspayce.com/mikem/bcm2835 | hxnormalize -x -e | hxselect -s '\n' -c "div.textblock>p:nth-child(4)>a:nth-child(1)"` | tar xz --strip-components=1 -C src/bcm2835
+	cd src/bcm2835 && (./configure CFLAGS=" -fPIC $(MYCFLAGS)")
+
+.PHONY: distclean clean packages testrun
+
+packages:
+	if ! dpkg -l | grep python-bitarray -c >>/dev/null; then sudo apt-get --yes install python-bitarray; fi
+	if ! dpkg -l | grep python-yaml -c >>/dev/null; then sudo apt-get --yes install python-yaml; fi
+	if ! dpkg -l | grep python-zmq -c >>/dev/null; then sudo apt-get --yes install python-zmq; fi
+
+clean:
+	rm -rf lib/*
+	rm -f src/*.o
+	rm -f ./*.pyc
+	if [ -e src/bcm2835/src ]; then make -C src/bcm2835 clean; fi;
+
+distclean: clean
+	rm -rf lib
+	rm -rf src/bcm2835
+	@find ./ $(RCS_FIND_IGNORE) \
+        \( -name '*.orig' -o -name '*.rej' -o -name '*~' \
+        -o -name '*.bak' -o -name '#*#' -o -name '.*.orig' \
+        -o -name '.*.rej' -o -size 0 \
+        -o -name '*%' -o -name '.*.cmd' -o -name 'core' \) \
+        -type f -print | xargs rm -f
+
+testrun: all
+	python run_local.py --externalChargeInjection --channelIds=30 --acquisitionType=const_inj --injectionDAC=3000 --dataNotSaved --showRawData

README.md

@@ -1,45 +1,134 @@
 # rpi-daq
-Software for single-module control using a Raspberry Pi
-
- * Install bitarray python package
- ```
- BITARRAY_DIR=/choose/wherever/you/want
- cd ${BITARRAY_DIR}
- git clone https://github.com/ilanschnell/bitarray.git
- cd bitarray
- sudo python setup.py install
- sudo python setup.py build_ext -i
- ```
-
- * Download this code
- ```
- HOME_DIR=/choose/wherever/you/want
- cd ${HOME_DIR}/
- git clone https://github.com/asteencern/rpi-daq.git
- ```
-
- * Install bcm2835 and create shared library
+Software for taking data from one [hexaboard module](https://edms.cern.ch/ui/#!master/navigator/item?I:1141765299:1141765299:subDocs) with four [`SKIROC2_CMS` chips](http://llr.in2p3.fr/~Geerebaert/CMS/HGCal/CMS_HGCal-SKATE_design_notes.pdf),  using the [test stand board](https://edms.cern.ch/ui/#!master/navigator/document?D:1749508128:1749508128:subDocs).
+
+Two main acquisition programs:
+- `run_local.py` for local debugging.
+- `daq-zmq_client.py`, that spawns a server over the network in the device and processes data off-device. This package then needs to be installed in both machines, though only the python part in the client.
+
+
+Two main sources of configuration:
+- `yaml` file
+- command line options
+
+More details on the `SKIROC2_CMS` readout chip configuration can be found [here](https://cms-docdb.cern.ch/cgi-bin/DocDB/ShowDocument?docid=12963) and [here](https://indico.cern.ch/event/559024/contributions/2261087/attachments/1316812/1972848/20160727_SimuReview.pdf).
+
+## Installation
+
+See [here](#raspberry-pi-3-from-scratch) to install Raspbian on a Pi 3.
+
+Then:
+```bash
+cd $HOME
+git clone https://github.com/CMS-HGCAL/rpi-daq
+cd rpi-daq
+make packages
 ```
-cd ${HOME_DIR}/rpi-daq/RPi_software/bcm2835-1.52/src
-gcc -shared -o libbcm2835.so -fPIC bcm2835.c
-sudo cp libbcm2835.so /usr/local/lib/libbcm2835.so
+
+The following step is only needed in the server or for local running (it is not needed for the client):
+```bash
+make
 ```
 
-* Install python-yaml (if not yet installed)
+Finally, a local test run with charge injection:
+```bash
+make testrun
 ```
-sudo apt-get install python-yaml
-```  
 
-* Compile gpiolib.c and create shared library
+Another example of a local acquisition:
+```
+python run_local.py --showRawData --dataNotSaved
 ```
-cd ${HOME_DIR}/rpi-daq
-gcc -c -I ./RPi_software/bcm2835-1.52/src ./RPi_software/bcm2835-1.52/src/bcm2835.c -fPIC gpiolib.c
-gcc -shared -o gpiolib.so gpiolib.o
-sudo cp gpiolib.so /usr/local/lib/gpiolib.so
-mkdir Data
-```  
-
-* Example of acquisition:
+
+An example of client and server running in the same machine
 ```
-python run_local.py
+python daq-zmq-client.py -e
 ```
+
+## Development
+
+### Code structure
+
+* `rpi_daq.py`: configures the chips and reads events.
+* `src/gpiohb.c`: utilities for low-level communication with the hexaboard; uses the [`bcm2835` library](www.airspayce.com/mikem/bcm2835/).
+* `unpacker.py`: parses (and compresses) raw data.
+* `skiroc2cms_bit_string.py`: utilities for configuration of the readout chips.
+
+* `run_local.py`: simple local running application.
+* `daq-zmq-{client,server}.py`: full-fledged system with data processing offloaded to client side.
+
+### Ideas for contributions
+
+- [ ] 🚀 Optimize timing of `gpiohb.c` I/O routines.
+- [ ] 💡 Develop an ASUS Tinker Board version.
+- [ ] ➕ Add functionality to `skiroc2cms_bit_string.py`.
+- [ ] 🔒 Improve the client-server usage, e.g., streamlining `ssh` sessions (no password, public key).
+
+### Contributing
+
+* Fork this project,
+* Clone your repository when installing,
+* _Your magic happens here_,
+* Push to your repository, and
+* Submit pull requests.
+
+# Installing OS on device
+
+## Raspberry Pi 3 from scratch
+
+- Download [Raspbian with desktop](https://www.raspberrypi.org/downloads/raspbian/).
+- Use [Etcher](https://etcher.io/) to write the image to the SD card.
+- Boot the device and connect a screen, keyboard, and mouse.
+- Connect device to network using desktop tools.   
+   When using wired and wireless network, editing `/etc/wpa_supplicant/wpa_supplicant.conf` may be needed to sort out priorities. See also [this](https://raspberrypi.stackexchange.com/questions/58304/how-to-set-wifi-network-priority).
+- Register device to network, including both wired and wireless MAC addresses that can be obtained using `ifconfig`.
+- Update the system:
+    ```bash
+    sudo apt-get update
+    sudo apt-get --yes dist-upgrade
+    sudo apt-get clean
+    sudo apt --yes autoremove
+    ```
+- `sudo reboot`
+- Change `pi` user password using `passwd`.
+- Enable `VNC` in pi customization GUI from `Desktop menu → Preferences → Raspberry Pi Configuration → Interfaces → VNC: Enabled`.
+- Install some goodies:
+    ```bash
+    sudo apt-get --yes install emacs25 htop iotop nmap liquidprompt ipython elpa-markdown-mode yaml-mode
+    liquidprompt_activate
+    ```
+
+## 🚧 ASUS Tinker Board from scratch 
+
+- Download `TinkerOS-Debian` image from the downloads section in the [Tinker Board page](https://www.asus.com/Single-Board-Computer/Tinker-Board/).
+- Use [Etcher](https://etcher.io/) to write the image to the SD card.
+- Default user and password are `linaro:linaro`. Change it.
+- Register device to network, including both wired and wireless MAC addresses that can be obtained using `ifconfig`.
+- Update the system:
+    ```bash
+    sudo apt-get update
+    sudo apt-get --yes dist-upgrade
+    sudo apt-get clean
+    sudo apt --yes autoremove
+    ```
+- Install some goodies:
+    ```bash
+    sudo apt-get --yes install emacs25 htop iotop nmap liquidprompt ipython elpa-markdown-mode yaml-mode git tree
+    liquidprompt_activate
+    echo heartbeat | sudo tee /sys/class/leds/led1-led/trigger
+    ```
+- Setup the `C` GPIO API (see also the [Tinker Board page](https://www.asus.com/Single-Board-Computer/Tinker-Board/)). Basically:
+    ```bash
+    cd $HOME
+    git clone http://github.com/TinkerBoard/gpio_lib_c.git
+    cd gpio_lib_c/
+    sudo ./build
+    gpio -v
+    gpio readall
+    ```
+- Set up VNC (`sudo tinker-config` allows to start VNC, but not to set it up as a deamon):
+    ```bash
+    # From https://tinkerboarding.co.uk/wiki/index.php?title=Software#Remote_access
+    sudo apt-get --yes install x11vnc
+    echo "@x11vnc -forever -noxrecord" >> ~/.config/lxsession/LXDE/autostart
+    sudo service lightdm restart
+    ```