Merge pull request #64 from epics-containers/dev

Adding some Architectural Decisions

docs/developer/explanations/decisions/0002-switched-to-pip-skeleton.rst

@@ -1,5 +1,5 @@
-2. Adopt epics-containers.github.io for project structure
-=========================================================
+2. Adopt python3-pip-skeleton for project structure
+===================================================
 
 Date: 2022-02-18
 
@@ -11,7 +11,7 @@ Accepted
 Context
 -------
 
-We should use the following `pip-skeleton <https://github.com/epics-containers/epics-containers.github.io>`_.
+We should use the following `pip-skeleton <https://github.com/DiamondLightSource/python3-pip-skeleton>`_.
 The skeleton will ensure consistency in developer
 environments and package management.
 

docs/developer/explanations/decisions/0003-use-substitution-files.rst

@@ -0,0 +1,55 @@
+3. Use of substitution files to generate EPICS Databases
+========================================================
+
+Date: 2023-11-30
+
+Status
+------
+
+Accepted
+
+Context
+-------
+
+There are two proposals for how EPICS Databases should be generated:
+
+1.  At IOC startup ``ibek`` should generate a substitution file that describes the
+    required Databases.
+
+    The IOC instance yaml combined with the definitions from support module yaml
+    controls what the generated substitution file will look like.
+
+    ``ibek`` will then execute ``msi`` to generate the Databases from the
+    substitution file.
+
+2.  The dbLoadRecord calls in the startup script will pass all macro substitutions
+    in-line. Removing the need for a substitution file.
+
+
+Decision
+--------
+
+Proposal 1 is accepted.
+
+Some template files such as those in the ``pmac`` support module use the
+following pattern:
+
+.. code-block::
+
+    substitute "P=$(PMAC):, M=CS$(CS):M1, ADDR=1, DESC=CS Motor A"
+    include "pmacDirectMotor.template"
+
+This pattern is supported by msi but not by the EPICS dbLoadRecord command which
+does not recognise the ``substitute`` command.
+
+
+Consequences
+------------
+
+An extra file ``ioc.subst`` is seen in the runtime directory. In reality this
+is easier to read than a full Database file. So can be useful for debugging.
+
+Finally those developers who are unable to use ``ibek yaml`` for some reason can
+supply their own substitution file and ibek will expand it at runtime. This is
+much more compact that supplying a full Database file and important due to the
+1MB limit on K8S ConfigMaps.

docs/developer/explanations/decisions/0004-autosave-req-files.rst

@@ -0,0 +1,45 @@
+4. How to configure autosave for IOCs
+=====================================
+
+Date: 2023-11-30
+
+Status
+------
+
+Accepted
+
+Context
+-------
+
+There is a choice of supplying the list of PVs to autosave by:
+
+- adding info tags to the Database Templates
+- supplying a raw req file with list of PVs to autosave
+
+Decision
+--------
+
+We will go with req files for the following reasons:
+
+- https://epics.anl.gov/tech-talk/2019/msg01600.php
+- adding info tags would require upstream changes to most support modules
+- default req files are already supplied in many support modules
+- req files are in common use and many facilities may already have their own
+  req files for support modules.
+
+We expect to autogenerate the list of PVs to autosave from the IOC's. We could
+therefore generate a Database override file which adds info tags. But it is
+simpler to just generate a req file.
+
+The mechanism for using req files is that defaults will come from the support
+module or from the generic IOC if the support module does not supply a req file.
+
+Then override files can exist at the beamline level and / or at the IOC
+instance level. These will simply take the form of a req file with the same
+name as the one it is overriding.
+
+Consequences
+------------
+
+Everything is nice and simple.
+

docs/developer/explanations/decisions/0005-python-scripting.rst

@@ -0,0 +1,44 @@
+2. Use Python for scripting inside and outside containers
+=========================================================
+
+Date: 2022-11-30
+
+Status
+------
+
+Accepted
+
+Context
+-------
+
+Inside the container, we use the ``ibek`` tool for scripting. Outside we
+use ``ec`` from ``epics-containers-cli``.
+
+Much of what these tools do is
+call command line tools like ``docker``, ``helm``, ``kubectl``, compilers,
+etc. This seems like a natural fit for bash scripts.
+
+These features were originally implemented in bash but were converted to
+python for the following reasons:
+
+- python provides for much richer command line arguments
+- it is also much easier to provide rich help
+- managing errors is vastly improved with exception handling
+- the unit testing framework allows for 85% coverage in continuous integration
+- complex string handling is a common requirement and is much easier in python
+- there is a clear versioning strategy and packages can be installed with pip,
+  meaning that you can check which version of the script you are running and
+  report bugs against a specific version
+- the code is much easier to read and maintain
+- because the packages can be pip installed they can be used in CI and inside
+  multiple containers without having to copy the scripts around
+
+Decision
+--------
+
+We always prefer Python and keep bash scripts to a minimum
+
+Consequences
+------------
+
+Scripting is much easier to maintain and is more reliable.

docs/user/explanations/introduction.rst

@@ -46,7 +46,7 @@ An important principal of the approach presented here is that an IOC container
 image represents a 'Generic' IOC. The Generic IOC image is used for all
 IOC instances that connect to a given class of device. For example the
 Generic IOC image here:
-`ghcr.io/epics-containers/ioc-adaravis-linux-runtime:2023.10.1
+`ghcr.io/epics-containers/ioc-adaravis-linux-runtime:2023.10.2
 <https://github.com/epics-containers/ioc-adaravis/pkgs/container/ioc-adaravis-linux-runtime>`_
 uses the AreaDetector driver ADAravis to connect to GigE cameras.
 

docs/user/tutorials/dev_container.rst

@@ -109,15 +109,13 @@ used in previous tutorials. Let's go and fetch a version of the Generic IOC
 source and build it locally.
 
 For the purposes of this tutorial we will place the source in a folder right
-next to your test beamline ``bl01t`` folder. We will also be getting a
-specific version of the Generic IOC source so that future changes don't break
-this tutorial:
+next to your test beamline ``bl01t``:
 
 .. code-block:: bash
 
     # starting from folder bl01t so that the clone is next to bl01t
     cd ..
-    git clone --recursive [email protected]:epics-containers/ioc-adsimdetector.git -b 2023.11.1
+    git clone --recursive [email protected]:epics-containers/ioc-adsimdetector.git
     cd ioc-adsimdetector
     ec dev build
 
@@ -162,6 +160,17 @@ You should now be *inside* the container. All terminals started in VSCode will
 be inside the container. Every file that you open with the VSCode editor
 will be inside the container.
 
+.. note::
+
+  Troubleshooting: if you are experiencing problems with the devcontainer you
+  can try resetting your vscode and vscode server caches on your host machine.
+  To do this, exit vscode use the following command and restart vscode:
+
+  .. code-block:: bash
+
+    rm -rf ~/.vscode/* ~/.vscode-server/*
+
+
 There are some caveats because some folders are mounted from the host file
 system. For example, the ``ioc-adsimdetector`` project folder
 is mounted into the container as a volume. It is mounted under

docs/user/tutorials/rtems_ioc.rst

@@ -3,7 +3,7 @@ RTEMS - Deploying an Example IOC
 
 .. Warning::
 
-    This tutorial is out of date and will be updated in November 2023.
+    This tutorial is out of date and will be updated in December 2023.
 
 The previous tutorials walked through how to create a Generic linux soft
 IOC and how to deploy an IOC instance using that Generic IOC.

docs/user/tutorials/rtems_setup.rst

@@ -3,7 +3,7 @@ RTEMS - Creating a File Server
 
 .. Warning::
 
-    This tutorial is out of date and will be updated in November 2023.
+    This tutorial is out of date and will be updated in December 2023.
 
 Introduction
 ------------