Skip to content

Issue 107

Jekabs Karklins requested to merge issue_107 into master

Created by: martukas

File dumping to h5 is generalized in the DumpFile template class and relevant classes (Multigrid::Hit, NMX/Hit, NMX/Readout) have been refactored to use it. The template class takes care of chunking setup, buffering, file rotation, and in the future, some format version checking. Data can be pushed as single structs or arrays of structs. The class can also act as a basis for a file reader/streamer, as it does for some nmxgen solutions. The structs intended for being written to file must provide some additional metadata and define an hdf5 TypeTrait that defines how the member variables are mapped. A few macros are provided to make this mapping as painless as possible, and may be candidates for including in h5cpp itself. My hope is that all pipelines can adopt this method of dumping raw data. Since any modifications to the struct's definition will break file compatibility, ample warnings are provided in comments explaining the procedure to be followed. Though this may appear fragile, I believe this dhf5 Compound-type based solution is far safer than us having to write tons of code to create parallel datasets for each member variable, and furthermore buffer incoming data in multiple arrays, etc.. My own experience with the alternative approach shows that it would actually be more difficult to track down problems.

This closes #121 (closed) #41 (closed) #98 (closed) #107 (closed)

Issue reference / description

The branch you merge from should already reference an event-formation-unit github ticket number. You can add a descriptive title, but if an issue is referenced, you don't have to.

Checklist for submitter

  • Check for conflict with integration test
  • Unit tests pass

Nominate for Group Code Review (Anyone can nominate it)

Indicate if you think the code should be reviewed in a Thursday code review session.

  • Recommend for group code review

Also, nominate it on the code_review Slack channel.

Merge request reports