ReleaseNotes-2-2.txt

Release Notes for Version 2.2
=============================

.. _Release_Notes_2.2.0:

Release 2.2.0
-------------

This is a new feature release. As with all dot-zero releases, be careful
before using this in production systems.

Built-in Constants
^^^^^^^^^^^^^^^^^^

All built-in constants are now known to *snc* and thus no longer give
"undefined variable" warnings. They are also documented in the reference.

The *snc*-generated C code for built-in constants now uses the name of the
constant, instead of its value. This makes the code a bit more readable and
slightly simplifies code generation.


Foreign Types and Type Casts
^^^^^^^^^^^^^^^^^^^^^^^^^^^^

It is now possible to use foreign types (i.e. types declared in C code, such
as structs, enums, unions, and typedefs) in variable declarations, type
casts, and the special "sizeof" built-in operator. Previously existing
restrictions as to which type expressions are allowed have been lifted, so
that (almost) everything you can say in C is now supported in SNL, too.

There are two notable limitations:

* Using C *type aliases* (defined in C with ``typedef``) is allowed, but you
  must prefix the type name with the new keyword ``typename`` (which I
  borrowed from C++). Supporting typedefs directly would make the
  implementation of the parse *much* more complex, because with typedefs the
  grammar is no longer context free. (C compilers based on parser generators
  must use very dirty and brittle tricks to be able support this).

* There is only very limited support for *defining* your own types (see
  below) and you cannot mix type definition with type usage (for instance in
  a variable declaration) as in C.

The words ``enum``, ``struct``, ``typename``, ``union``, and ``void`` are
reserved words now and can no longer be used as identifiers.


Functions Definitions
^^^^^^^^^^^^^^^^^^^^^

You can now define functions in SNL. The syntax is like in C and you *can*
call the built-in PV functions inside them. This is made possible by passing
the execution context (the state set identifier and the variable block
pointer) as hidden parameters.

You may pass all kinds of variables to such a function, but for channel
("assigned") variables their special "assigned" status gets lost when
passing them, similar as when passing them to a C function. This means you
can call e.g. pvGet(x) in the body of an SNL function, but only if x is a
global variable.

My various attempts to lift this limitation were the main reason it took me
so much longer than I had expected to make this release. The design space is
large, the semantics need careful consideration to preserve consistency of
the language, and the changes in the code base are far-reaching (for
instance, it turned out that I had to add a limited form of type checking to
the compiler). In order not to hold up the release of version 2.2 any
longer, I decided to postpone this feature to a future release.


Type Definitions
^^^^^^^^^^^^^^^^

You can now define you own struct types in SNL (other type formers like
union and enum are not yet supported). As usual the syntax is a (simplified:
no bit fields) variant of the C syntax. This is currently not very useful,
since there is no way to "assign" struct members to PVs. Lifting this
limitation is closely related to allowing PV function arguments and
therefore postponed, too.


Order of Definitions in the Generated C Code
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

In order to make it possible to ``%%#include`` type declarations or define
them in escaped C code (so they appear in the generated C code but not in
the code that SNL has to parse) *and* use them in (the type of) global
variable declarations, the generated variable block is now placed *after*
all other top-level definitions that appear in the program before the first
state set; in particular before escaped C code. In previous versions, this
was the other way around.

Note that this means that escaped C code that appears before the first state
set cannot access global variables declared in SNL, even if the reentrant
option is not in effect. Such C code should now be placed after the last
state set.

Anyway, it is my hope that the possibility to write functions directly in
SNL will make the escape-to-C route mostly obsolete.


Related patches:

* snc: add a name prefix for initialization helper vars
  
  This fixes a bug that crept in when implementing the new variable
  initialization code for variables with global lifetime. The problem
  was that the initialization variable shadowed the global variable
  in non-reentrant mode, which made the initialization a no-op.

* snc: swap order of gen_user_var and gen_defn_c_code

* snc: refactored generation of initializers

  With foreign types we can no longer generate default initializers
  in a portable way, as was done previously (in reentrant mode).
  We now leave off initializers from the UserVar struct and instead
  initialize the members one by one (using memcpy) from statically
  initialized single values.


New Delay Implementation
^^^^^^^^^^^^^^^^^^^^^^^^

While it was always allowed to use arbitrary expressions for the
argument to :c:func:`delay`, this never worked as one would expect if the
expression depended on e.g. monitored variables. The reason is that
previously the delay expression was evaluated once for all delays
appearing in conditions inside a state when the state was entered.
Later checks whether the delay has expired used the cached delay
value.

Now, delay expressions are evaluated like all other parts of the
state transition condition and seq_delay() takes a double argument
instead of a delay id. As a (transparent) side effect, if the
delay has not yet expired, the minimum time to wait for events is
adjusted. This adjustment is now more precise, since we do not store
the delay but rather the (future) time when the wakeup should happen.


Removed PV Library
^^^^^^^^^^^^^^^^^^

The PV library has been almost completely eliminated. What remains is a
thin layer over CA, implemented in C, and offering only the
functionality that is actually needed by the sequencer. The API is similar
(but not identical) to the old C API; particularly, all the pvStat, pvSevr,
and pvType definitions are as before.

The documentation for the PV layer has been removed, too. The only remaining
user relevant part of the interface is contained in pvAlarm.h, see
`Built-in Constants`_ above.

Also removed were the keck examples and the KTL related stuff in other example
directories.


Removed devSequencer
^^^^^^^^^^^^^^^^^^^^

The (broken and ugly) sequencer device support was removed.
Consequently, seqFindProgByName was be removed as it is no longer needed.


Multiple pvSync for multiply assigned arrays
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Now, if you do pvSync(x,ef), where x is an array for which the elements
are individually assigned to a separate PVs, then all the elements get
synced with the event flag, just like if you use the sync clause.


Additional Arguments for pvGetComplete
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The built-in function :c:func:`pvGetComplete` now accepts 2 more (optional)
arguments, just like :c:func:`pvPutComplete`, in order to check for completion
if the variable is an array with elements assigned to different PVs.

Warning: This change is incompatible!

Previously, ``pvGetComplete(x)``
was equivalent to ``pvGetComplete(x[0])``, even for array variables with
multiply assigned elements. That is, ``pvGetComplete(x)`` would check
only for completion of the first array element, whereas now it checks
completion for *all* elements.

Patches:

* enhanced pvGetComplete with extra optional args, like pvPutComplete
  
  This brings asynchronous pvGet to the same standard w.r.t. multi-PV
  arrays as pvPut. Another slight improvement: in non-safe mode, if
  a variable is not assigned to a PV, then return TRUE, so that
  when testing in a condition it does not hang forever.

* documented new features of pvGetComplete


Timeout Arguments for pvGet and pvPut
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

For both :c:func:`pvGet` and :c:func:`pvPut` there is now the possibility to
specify a timeout that differs from the default of 10 seconds. This is done
by giving an extra argument after the ``SYNC`` keyword, as in::

  pvGet(var,SYNC,1.0);

As before, the default behaviour for ``pvGet(var)`` i.e. neither an explicit
``SYNC`` nor ``ASYNC``, is synchronous, unless option :option:`-a` is in effect.
In this case, or if ``SYNC`` is given with no extra argument, the standard
default timeout of 10 seconds is assumed.

This means that the new feature is fully backwards compatible, i.e. existing
programs behave as before.

* added extra optional timeout argument to pvGet and pvPut

  This patch also gets rid of the ugly seq_sync_timeout hack.

* documented extra timeout argument for pvGet and pvPut
* adapted tests to use pvGet/Put timeouts


New Built-in Functions pvGetCancel and pvPutCancel
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

It is now possible to cancel and asynchronous get or put request by calling
:c:func:`pvGetCancel` resp. :c:func:`pvPutCancel`.

.. todo:: sometimes test fails, debug, add test for pvPutCancel

Patches:

* added pvGetCancel and pvPutCancel
* documented pvPutCancel and pvGetCancel
* added test for pvGetCancel


Names of Generated Entities
^^^^^^^^^^^^^^^^^^^^^^^^^^^

These now follow a strict naming convention:

  Generated names start with "_seq" or "_SEQ".

In particular, ``struct UserVar`` became ``struct _seq_var`` (but see
below for a compatibility typedef). The variables (actually function
parameters) ``ssId`` and ``pVar`` have been renamed, too. Since these are
often passed to foreign C procedures, I added similarly named built-in
procedures (``ssId()`` resp. ``pVar``) that return these values.

In order to prevent name clashes, words starting with "_seq" are no longer
valid identifiers in SNL.


External API
^^^^^^^^^^^^

I finally got around to separating the public API (to be called from
e.g. embedded C code) from the internal interface between the compiler
generated code and the runtime system. The latter parts have been moved
from seqCom.h to the new header file seq_snc.h.

In particular, the following changes have been made:

Retained:

- includes of pvAlarm.h and seq_release.h

- enum compType and its members DEFAULT, ASYNC, SYNC

- constants NOEVFLAG and DEFAULT_QUEUE_SIZE
  (but note that DEFAULT_QUEUE_SIZE is now 2 instead of 100)

- typedefs string, EV_ID, VAR_ID, SS_ID,
  seqBool, seqProgram

- all functions except seqRegisterSequencerProgram and seqRegisterSequencerCommands
  but including seq_pvIndex

Added:

- typedef struct _seq_var SEQ_VARS

- builtin "functions" pVar and ssId (like pvIndex these are actually
  implemented as macros); they replace direct uses of generated
  identifiers, which have been renamed (see `Names of Generated Entities`_
  above).

- new builtin functions pvGetCancel, pvPutCancel

- the constant DEFAULT_TIMEOUT

For compatibility I also added:

- typedef SEQ_VARS USER_VAR

Removed:

- DELAY_ID (obsolete, see `New Delay Implementation`_)

Moved to seq_snc.h:

- struct definitions for the static part of the generated program:

  + struct seqChan
  + struct seqState
  + struct seqSS
  + struct seqProgram
  + typedef PROG_ID

- option constants:

  + OPT_DEBUG
  + OPT_ASYNC
  + OPT_CONN
  + OPT_REENT
  + OPT_NEWEF
  + OPT_SAFE

  + OPT_NORESETTIMERS
  + OPT_DOENTRYFROMSELF
  + OPT_DOEXITTOSELF

  User code never needs to use these. It can use seq_optGet instead.

- bitmask operations:

  + NBITS
  + NWORDS
  + bitSet, bitClear, bitTest, optTest

  Like option values, these were never meant to be part of a public API.

- the constants TRUE and FALSE

- typedefs for generated functions; note that these were renamed, too

- the functions seqRegisterSequencerProgram and seqRegisterSequencerCommands

I hope these changes won't break too many programs that use escaped code
and use these internals. If it turns out this is hurting people very
much, I am willing to provide additional compatibility aliases.

My justification for making these potentially breaking changes is that
this is how it should have been done from the start. Mixing self-written
with generated code is fraught with enough hazards as it is, and
accidental name capture, though not very probable, would be very hard to
diagnose for users. Exposing internal data structures where this is not
strictly necessary is similarly ill-advised.

A practical benefit is that this gives me a lot more freedom when
introducing new features or refactoring the way snc generates code.

Related patches:

* make names starting with _seq illegal identifiers in SNL
* renamed some generated identifiers
* renamed all generated identifiers to start with _seq
* renamed function typedefs
* seq_pvIndex is now a macro

* added new builtin functions pVar and ssId

  These are intended for calling C procedures from SNL,
  when the C procedure takes SEQ_VARS* or SS_ID as argument.

* pVar and ssId arguments to generated routines are now const pointers

  This means that user code can mutate the variables and the (opaque)
  contents of the state set control block, but are not allowed to
  invalidate the pointers themselves.

* moved non-API internals from seqCom.h to new seq_snc.h

  Another one of the long overdue re-factorings. The new header file
  seq_snc.h contains all the definitions that are shared between the
  snc generated code and the run-time library. What remains in seqCom.h
  is the public API for C code that wants to interact with an SNL program.

* removed seqRegisterSequencer* from public API

Extra Build Rules
^^^^^^^^^^^^^^^^^

The extra rules in the top level Makefile to produce documentation with
or without generating a pdf file have been changed: 'pdf' and 'docs' are
no longer make variables. Instead there are a number of new targets you
can specify:

- html: build the docs in html format
- docs: additionally build the manual in pdf fromat
- upload etc: these are for my own benefit only, ignore them

Related patches:

* top level Makefile: pdf and docs no longer variables, restructured extra targets


Miscellaneous New Features
^^^^^^^^^^^^^^^^^^^^^^^^^^

* pvAssign now does macro substitution, like assign syntax

* reduce severity of timeouts and user errors from errlogFatal to errlogMajor

* Struct or union members are no longer identified with variables in the
  SNL syntax. Member selection "operators" ('.' and '->') are no longer treated
  as binary operators, but get their own internal representation.

  This finally solves the recurring annoyance of snc issuing
  "undefined variable" warnings for members.

  Declaring members in a foreign declaration is still allowed but
  is now obsolete.


Various Fixes
^^^^^^^^^^^^^

* pv: report CA failure status via msg member of pvVar or pvSys
* option -m is now treated correctly as compile time only
* removed deprecated epicsShareAPI markers
* seq: place program lock around wakeup commands in ca callback

  This prevents a race condition resulting in a crash when the program
  shuts down and deletes mutexes etc before the callback has issued its
  final ss_wakeup call.
* seq: fixed connect and monitor accounting in seq_disconnect and seq_camonitor


Raw Dump of Changes in test
^^^^^^^^^^^^^^^^^^^^^^^^^^^

* added validation test for access to variables of foreign types
* added compiler test for member access
* added tests for foreign types, cast, and sizeof
* renamed and rearranged some tests
* added two compiler tests
* fixed number of errors for misplacedExit
* adapted userfunc to compiler changes, extended the test
* adapted delay.st to better test new delay implementation
* moved ctest.c to test/compiler
* ensure that tests cover seq exit phase

* test/compiler: renamed scripts that test the whole build
* test/compiler: added a test for identifier starting with _seq


Raw Dump of Changes in documentation
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

* updated release notes for 2.2.0
* added 2.2.0 to Installation page
* improved wording of frontpage blurb
* fixed and updated Installation page
* minor cleanups in reference
* added a todo item for delay in action code
* clarified syntax and semantics of option clause
* added versionadded tag to state change command
* superficial change to reference
* removed foreign types from plans
* removed some text that referred to the removed sequencer device support
* distinguish program parameters from CPP macros
* parameter expansion in pvAssign


TODO Before Release
^^^^^^^^^^^^^^^^^^^

* Maybe re-introduce #defines for event flags

* Update reference to document all the changes and new features

What to do about pvPutCompleteArray, see these patches:

* examples: use array version of pvPutComplete
* disabled examples/parallel because pvPutCompleteArray does not yet work

See also pvPutMultiple and pvGetMultiple these are w/o implementation.