I'm working with the raw edf PSG's and profusion annotations in the sleep heart health study data (SHHS1). We have custom matlab code which then merges this data back together for further analysis.
I note that there seems to be a number of studies where desaturation events have only been scored in the the first half or two thirds of the night (or at least according the the .xml file); However, on examination of the SpO2 data, there are clear desaturations associated with reductions in ventilation in thermistor/RIP channels and scored respiratory events.
I have gone back to the raw .xml files to check that this is not an error in our import/analysis code.
Some example studies include:
However, it seems very widespread in the subset I have been looking at.
Has anyone else run into this problem? I'm wondering if the export function has not worked correctly?
Thanks for bringing this issue to the forum. I'll ask someone who is familiar with the SHHS1 sleep studies to take a look and share their thoughts.
Thanks for such a quick response Mike.
I got some input from a couple of our sleep scorers. The SHHS1 desaturation events, as you're finding, are not always present and may not always line up as expected with associated respiratory events. This issue has been recognized by other users of the SHHS1 data (pre-NSRR days) and the explanation points back to the desaturation data from the original scoring having been affected (damaged?) during conversions to newer versions of the Compumedics Profusion scoring software. The SHHS1 studies were first scored in an MS-DOS based package and eventually converted into Windows-based versions of Profusion in the 2000s. Evidently, Compumedics changed the way their scoring software handled the desaturation data and at that point these issues likely crept into our raw data, which then made its way into our EDF and XML files that were generated roughly 8-9 years ago.
Presently, our best suggestion for users is to have them derive new desaturation event data from the raw SaO2 channel. We may pursue doing this ourselves, or perhaps we may publish a tool to assist in this process.
Thanks again for raising the issue here and sorry for the confusion. I added a note to the SHHS documentation regarding the problems with desaturation events.
Any idea if these problems also exist in what I presume to be the original annotation files (i.e. shhs1-200001-nsrr.xml)? Being a .xml I'm guessing these are also post-windows conversion, and are therefore likely to have the same problems.
I've already written a fairly simple desaturation scoring algorithm (currently with validation data in infants but not adults); so I'll use this for my pilot analysis and will continue putting some more work into this.
Once I have something that I'm reasonably happy with (and with some reasonable validation in a reliably scored dataset), I'll get in touch about uploading it here.
There will be some down-stream challenges in re-associating desaturations with the manually scored respiratory events in order to classify respiratory events according to current AASM criteria (my understanding from the documentation is that at present, annotated hypopnoea's are scored as any ventilatory disturbance with no desaturation or arousal criteria).
The Profusion XMLs are the originals - the NSRR versions are translations intended for the EDF Viewer tool. Hence, both XML files will have the same issue.
Glad to hear you've got something up and running. It would be great to have the algorithm posted here once you've had a chance to put it through its paces.