JSON schema format

Created by: matthew-d-jones

I have been looking at your example JSON, NeXus file schemas and have a few questions and comments.

• Have you considered how to include hard and soft links?
• For "values" sent as part of the schema it looks like you are inferring type, is that correct? Is it always safe to do so, what about if you populate a dataset with some initial values and want to append from a stream?
• Would we send the geometry description as part of this schema? For some instruments it is likely to exceed what we can put in a single message.
• I can't see any example of using attributes for a dataset, only for groups. Given that datasets can also have attributes, is it unambiguous what is a group and what is a dataset in the current schema?
• Using names as keys can be dangerous, it leads to restrictions on valid names where you have to avoid key words, like "FW_stream" and "NX_class". This tends to get worse over time as new features and key words are added. Given that it is not a restriction of NeXus files, should it be a restriction of the JSON schema?
• I don't see any examples of datasets with arrays. Multidimensional arrays are no problem in JSON, but we have to be careful about the differences with HDF5 datasets. JSON arrays can contain mixtures of types and be ragged, datasets cannot.

Here is an example format which would solve many of the potential problems at the expense of greater verbosity

{
  "name": "raw_data_1",
  "type": "group",
  "attributes": [{"NX_class": "NXentry"}],
  "children": [
    {
      "name": "instrument_1",
      "type": "group",
      "attributes": [{"NX_class": "NXinstrument"}],
      "children": [
        {
          "name": "detector_1",
          "type": "group",
          "attributes": [{"NX_class": "NXdetector"}],
          "children": [
            {
              "name": "pixel_shape",
              "type": "group",
              "attributes": [{"NX_class": "NXsolid_geometry"}],
              "children": [
                {
                  "name": "cylinder",
                  "type": "uint32",
                  "values": [0, 1, 2]
                },
                {
                  "name": "vertices",
                  "type": "float",
                  "values": [[-0.001, 0, 0], [0, 0.004, 0], [0.001, 0, 0]],
                  "attributes": [{"units": "m"}]
                }
              ]
            }
          ]
        },
        {
          "name": "detector_2",
          "type": "group",
          "attributes": [{"NX_class": "NXdetector"}],
          "children": [
            {
              "name": "pixel_shape",
              "type": "hardlink",
              "path": "/raw_data_1/instrument_1/detector_1/pixel_shape"
            }
          ]
        }
      ]
    }
  ]
}

For the attention of @zjttoefs @SkyToGround @dominikwerder @michele-brambilla