HI, i am on sleep study with CFS & MESA data
on the study i has found critical issue can affect other future works
this issue should be identified promptly.
you know, total sleep time (TST) is can be calculated from sleep stage like equation 1 and the sleep stage can be extracted from annotation xml file
equation1) TST = in bed time - (wake after sleep onset + sleep latency)
Common sense, a TST from the sleep stage and TST from variable csv file are must be same, however several subject are difference
To check the exact value of errors, i ploted figures can express the two values.
i has attached the plots and writed down details, i hope you to deal with this issue promptly
plot figures link
MESA (1899 subjects), unit: min
(subject number, from variable TST, from sleep stage TST)
(1738, 226, 256)
(2852 , 393, 502.5)
(6476 , 403, 411)
CFS (694 subjects), unit: min
(800937, 19, 28)
(800170, 248, 265)
(801121, 269, 276.5)
(802726, 276, 283.5)
(802368, 326, 340)
(802678, 335, 362.5)
(802001, 359, 363.5)
(801376, 380, 383.5)
(800402, 379, 432.5)
Thanks for bringing this to our attention. I will explore some of these discrepancies. Can you please answer a couple questions?
I'm Brian, a colleage of minsooyeo (Minsoo Yeo).
I have gotten the access permission of those data . You can check my account about that.
We're working in the same team. Actually he is sitting beside me. :-)
If there is anything he has to do to use the data, please let us know.
I always appreciate your kind supports.
Thanks for your kindly reply,
I send you information you requested
. about "variable TST"
CFS: "SLPPRDP" variable
MESA: "slpprdp5" variable
. account issue
Yes i has obtained access to data on a different account, one of the our team member got access for data.
If you need to a information about the account, i will inform you that
Thanks to you both. Please have Minsoo submit a data request for MESA , CFS, and whatever other datasets your team is working with. He can use the same sort of language from brianhoonsukbyun's request.
Your findings are not entirely unanticipated. We have encountered such discrepancies ourselves. Right now the NSRR team is undertaking a large-scale effort to review all our datasets for issues exactly like this. We will make corrections whenever possible or otherwise note that the issue exists (and possibly why).
I hope to have a chance to look at a handful of these specifically and report back some initial findings next week.
Hello again - thanks for your patience.
For MESA - would you please re-download and check 2852 again? This study looks OK to us. This study was re-exported/fixed around the time of Brian's original data request, so perhaps you have an old copy.
For the other two MESA studies (1738, 6476) we traced the discrepancy issue back to issues of data loss/corruption with the original scoring files, which caused these unexpected mismatches. We were unable to fix the scoring data at the source to match the data you see in the summary result file (CSV). I will make a note of this issue with the scored data export here: https://sleepdata.org/datasets/mesa/pages/polysomnography-introduction.md
I will look into the CFS issues next.
Thanks for your kindly reply
I has totaly understanded your answers and written down two data requirement forms (cfs, mesa) you asked me
I will re-download data and review one more time
I have confirmed that there is no problem with 2852 subject of mesa data ("slpprdp5" variable --> 393, the TST from new xml files sleep stage --> 393)