diff --git a/documentation/writer_module_ad00_area_detector.md b/documentation/writer_module_ad00_area_detector.md new file mode 100644 index 0000000000000000000000000000000000000000..65336ec93765332df0a820912e1193af70f0d40b --- /dev/null +++ b/documentation/writer_module_ad00_area_detector.md @@ -0,0 +1,36 @@ +# *al00* Alarm information + +## Stream configuration fields + +|Name|Type|Required| Description | +---|---|---|--------------------------------------------------------------------------------------------------------------------| +topic|string|Yes| The kafka topic to listen to for data. | +source|string|Yes| The source (name) of the data to be written. | +writer_module|string|Yes| The identifier of this writer module (i.e. "ad00"). | +cue_interval|int|No| The interval (in nr of events) at which indices for searching the data should be created. Defaults to 100 million. | +chunk_size|int|No| The HDF5 chunk size in nr of elements. Defaults to 1M. | + + +### Example + +Example `nexus_structure`: + +```json +{ + "nexus_structure": { + "children": [ + { + "module": "ad00", + "config": { + "source": "the_source_name", + "topic": "the_topic_name", + "array_size": "$AREADET$", + "dtype": "int16" + } + } + ] + } +} +``` +Typically, the `$AREADET$` placeholder is replaced at runtime by NICOS. +Alternatively, it can contain an array of values corresponding to width and height. diff --git a/documentation/writer_module_al00_alarm.md b/documentation/writer_module_al00_alarm.md new file mode 100644 index 0000000000000000000000000000000000000000..8df17b456fb29af548d775d10ec26bb4a17542cf --- /dev/null +++ b/documentation/writer_module_al00_alarm.md @@ -0,0 +1,32 @@ +# *al00* Alarm information + +## Stream configuration fields + +|Name|Type|Required| Description | +---|---|---|--------------------------------------------------------------------------------------------------------------------| +topic|string|Yes| The kafka topic to listen to for data. | +source|string|Yes| The source (name) of the data to be written. | +writer_module|string|Yes| The identifier of this writer module (i.e. "al00"). | +cue_interval|int|No| The interval (in nr of events) at which indices for searching the data should be created. Defaults to 100 million. | +chunk_size|int|No| The HDF5 chunk size in nr of elements. Defaults to 1M. | + + +### Example + +Example `nexus_structure`: + +```json +{ + "nexus_structure": { + "children": [ + { + "module": "al00", + "config": { + "source": "the_source_name", + "topic": "the_kafka_topic", + } + } + ] + } +} +``` diff --git a/documentation/writer_module_ep00_data.md b/documentation/writer_module_ep01_connection_state.md similarity index 73% rename from documentation/writer_module_ep00_data.md rename to documentation/writer_module_ep01_connection_state.md index e79cdc53ccac622e419315bc38685e97372b6e7d..07bc62fecaa02f633ef61c87ac0a1741d96eb0c8 100644 --- a/documentation/writer_module_ep00_data.md +++ b/documentation/writer_module_ep01_connection_state.md @@ -1,19 +1,16 @@ -# ep00 EPICS connection status writer module - -__Note: This writer module is deprecated and has been replaced by the *pvCn* writer module.__ +# *ep01* EPICS connection status ## Stream configuration fields - |Name|Type|Required|Description| ---|---|---|---| topic|string|Yes|The kafka topic to listen to for data.| source|string|Yes|The source (name) of the data to be written.| -writer_module|string|Yes|The identifier of this writer module (i.e. "ep00").| +writer_module|string|Yes|The identifier of this writer module (i.e. "ep01").| chunk_size|int|No|The HDF5 chunk size in nr of elements. Defaults to 1024.| ## Example -Example `nexus_structure` to write status messages from `ExampleTopic`: +Example `nexus_structure`: ```json { @@ -24,7 +21,7 @@ Example `nexus_structure` to write status messages from `ExampleTopic`: "name": "EpicsConnectionStatus", "children": [ { - "module": "ep00", + "module": "ep01", "config": { "topic": "ExampleTopic", "source": "SIMPLE:DOUBLE" diff --git a/documentation/writer_module_ev44_detector_events.md b/documentation/writer_module_ev44_detector_events.md index b960adc8a46c6ce6211759971530a71a386066ed..780db3cd4b9e5f932dc1269f8c72959d61167261 100644 --- a/documentation/writer_module_ev44_detector_events.md +++ b/documentation/writer_module_ev44_detector_events.md @@ -13,7 +13,7 @@ chunk_size|int|No|The HDF5 chunk size in nr of elements. Defaults to 1M.| ### Example -Example `nexus_structure` to write radiation detector events: +Example `nexus_structure`: ```json { diff --git a/documentation/writer_module_f144_logdata.md b/documentation/writer_module_f144_logdata.md new file mode 100644 index 0000000000000000000000000000000000000000..76daff303730dc8ec80063ed2a19e3c6d003c7bf --- /dev/null +++ b/documentation/writer_module_f144_logdata.md @@ -0,0 +1,52 @@ +# *f144* Logdata + +## Stream configuration fields + +|Name|Type|Required| Description | +---|---|---|--------------------------------------------------------------------------------------------------------------------| +topic|string|Yes| The kafka topic to listen to for data. | +source|string|Yes| The source (name) of the data to be written. | +writer_module|string|Yes| The identifier of this writer module (i.e. "f144"). | +cue_interval|int|No| The interval (in nr of events) at which indices for searching the data should be created. Defaults to 100 million. | +chunk_size|int|No| The HDF5 chunk size in nr of elements. Defaults to 1M. | + + +### Example + +Example `nexus_structure`: + +```json +{ + "nexus_structure": { + "children": [ + { + "module": "f144", + "config": { + "source": "the_source_name", + "topic": "the_kafka_topic", + "dtype": "double" + } + } + ] + } +} +``` + + +## Flatbuffer payloads + +Messages with pulse information (`reference_time` / `reference_time_index`) but +empty `time_of_flight` contain no events and will not be written to file. + + +## Written NeXus structure + +| Description | Dimensions | ev44 name | NXevent_data name | +|---------------------------------------------------------------|------------|----------------------------------|-----------------------| +| Array of offsets from pulse time for each event | `[i]` | `time_of_flight` | `event_time_offset` | +| Array of pixel IDs | `[i]` | `pixel_id` | `event_id` | +| Array of pulse times | `[j]` | `reference_time` | `event_time_zero` | +| Map from each pulse time to the first event of that pulse | `[j]` | `reference_time_index` | `event_index` | +| Array of timestamps for indexing | `[k]` | (configured in JSON `cue_interval`) | `cue_timestamp_zero` | +| Array of event indexes corresponding to each timestamp in cue_timestamp_zero | `[k]` | (configured in JSON `cue_interval`) | `cue_index` | + diff --git a/documentation/writer_module_mdat_metadata.md b/documentation/writer_module_mdat_metadata.md index d23bb2bfb41d615837ca027bfe93c4e877c80302..dd445ddc353bfdb09b73a5f43b5703c7af51b846 100644 --- a/documentation/writer_module_mdat_metadata.md +++ b/documentation/writer_module_mdat_metadata.md @@ -10,7 +10,7 @@ ignored. Currently, it only supports start and stop times. ## Example -Example `nexus_structure` to write start and stop times: +Example `nexus_structure`: ```json { diff --git a/documentation/writer_module_se00_sample_environment.md b/documentation/writer_module_se00_sample_environment.md new file mode 100644 index 0000000000000000000000000000000000000000..74b4799bcf1a6f58ab071c533cdae0333f9a7bda --- /dev/null +++ b/documentation/writer_module_se00_sample_environment.md @@ -0,0 +1,30 @@ +# *se00* Sample environment data + +## Stream configuration fields + +|Name|Type|Required| Description | +---|---|---|--------------------------------------------------------------------------------------------------------------------| +topic|string|Yes| The kafka topic to listen to for data. | +source|string|Yes| The source (name) of the data to be written. | +writer_module|string|Yes| The identifier of this writer module (i.e. "se00"). | +cue_interval|int|No| The interval (in nr of events) at which indices for searching the data should be created. Defaults to 100 million. | +chunk_size|int|No| The HDF5 chunk size in nr of elements. Defaults to 1M. | + +### Example +Example `nexus_structure`: + +```json +{ + "nexus_structure": { + "children": [ + { + "module": "se00", + "config": { + "source": "the_source_name", + "topic": "the_kafka_topic", + } + } + ] + } +} +``` diff --git a/documentation/writer_module_tdct_chopper_tdc_timestamp.md b/documentation/writer_module_tdct_chopper_tdc_timestamp.md index f7616e1fade14f72eec9daab24cfc5bd004c8121..dec1d0e9293399da77a502addd9aeaefeaa5409c 100644 --- a/documentation/writer_module_tdct_chopper_tdc_timestamp.md +++ b/documentation/writer_module_tdct_chopper_tdc_timestamp.md @@ -13,7 +13,7 @@ enable_epics_con_info|bool|No|Enable or disable EPICS connection status writing. ## Example -Example `nexus_structure` to write chopper TDC timestamps: +Example `nexus_structure`: ```json { diff --git a/documentation/writer_modules.md b/documentation/writer_modules.md index f800fc9e9222b530577578b8146dcc4c3f89d583..9ce4fa975580768445ec7d424e7c34a831a87fe6 100644 --- a/documentation/writer_modules.md +++ b/documentation/writer_modules.md @@ -2,8 +2,3 @@ Writer modules for the various FlatBuffer schemas give the file-writer the ability to parse the FlatBuffers and write them to HDF5. - -The actual parsing of the different FlatBuffer schemas and conversion to HDF5 is -handled by modules which register themselves via the `FlatbufferReaderRegistry` -and `HDFWriterModuleRegistry`. For an example, please search for `Registrar` in -`src/schemas/hs00/`. Support for new schemas can be added in the same way.