Dear NSRR team,
I am currently working on the sof data and the two sets of annotations, the nsrr and the profusion set. As far as I have seen, the event data (leg movements, respiratory events, etc.) are exactly the same in both sets and the profusion set has additional information regarding scoring and montage settings that are not part of the nsrr xml files.
The difference that is puzzling me regards the sleep stages which are in the form of start and duration of each block of continuous sleep stage for nsrr while it is a list of 30 s epochs in the profusion file. Besides the two lists having different lengths with the profusion one being longer, the onset of sleep i.e. the first epoch that is not wake, and the subsequent scoring seems to be shifted by exactly 30 s with the nsrr information being 30 s earlier compared to the profusion stages. When I checked the two sets against the actually edf traces in some selected cases, according to how I would score the epoch, it seemed that indeed the nsrr stages are 30 s off.
I double-checked my procedures, but it is of course possible that I made an error or missed something obvious, for which I apologize in advance.
So the two questions I have are: is the stage scoring information in the two files really different and possible shifted by 30 s? and regarding the different lengths of the sleep staging does the nsrr file uses some additional information to end the staging (it is not merely the post-sleep wake but also additional sleep that is part of the profusion but not nsrr staging)?
Thanks very much and with kind regards
Thanks for inquiring about this on the forum! It appears you have discovered a bug with the NSRR-translated XML files. The bug exists in the EDF Editor and Translator tool, which appears to not be counting the first epoch (30 seconds) of the recording in the duration of the first staging element (typically Wake).
To answer your questions:
Yes, the staging information appears different between the files, which is incorrect. Your best bet is to use staging information from the Profusion XML files until we implement a fix in the tool and regenerate the NSRR-translated XMLs.
As for your second question about total lengths of sleep staging, I believe the NSRR-translated XMLs drop the last staging group in the file. Hence, if you had two hours of wake (Profusion XML staging = 0) at the end of the recording, this wouldn't be output to the NSRR-translated XML file. I gather this was done for "cropping" purposes to eliminate lots of not-that-useful data from the end of the file and make the hypnogram display better in the accompanying EDF Viewer. That said, I am going to follow-up with the developers of the tool to confirm and ask whether it might be more appropriate to have the total lengths of sleep staging match 1:1 between the two XML files (i.e. output the last staging group).
Thanks again for bringing this to our attention!
Thanks for the detailed information. I wanted to add that if you would like to follow any of the code changes and discussions, we have two pull requests/issues open on GitHub to suggest changes to the NSRR translator tool.
Fixing Missing First Epoch
Discussion on How to Handle Last Sleep Stage
Thanks again, we'll send you an update when these changes go through, and will also update the corresponding files in the NSRR datasets.
As an update -- we have regenerated most of the "NSRR XML" files for our datasets (only MrOS is still pending). We fixed the two issues you brought to our attention. Remo's suggested changes/fixes from above were both incorporated into the translation tool.