NSRR staff
Boston, MA
0000-0002-0506-8368
Top Topics
Recent Topics
Thanks for raising this issue - we have realized this is a limitation in many of our older datasets that were scored in Compumedics Profusion. That software doesn't export lights off/on information into the EDF or default XML annotations.
For some datasets, such as MESA, you can find the lights timing variables in the CSV covariate dataset (in the /datasets folder). For instance:
In SHHS, I only found lights off (https://sleepdata.org/datasets/shhs/variables/stloutp).
In the future we intend to encode lights information inside of harmonized annotation files. Until then, please use the variables in the dataset!
Those variables have the same meaning in the non-randomized dataset as they do for the randomized subjects, so you can refer to the meanings in the variables browser. I don't see a ref6 variable anywhere, however.
https://sleepdata.org/datasets/chat/variables/ref5 (ethnicity)
https://sleepdata.org/datasets/chat/variables/ref8 (BMI)
https://sleepdata.org/datasets/chat/variables/ref9 (gender)
Hope that helps and thanks for using the site!
Hey - thanks for checking out the site! Unfortunately, you are right, we do not have LDL in SHHS. The lipid panel variables were "A variables", which means they weren't collected as part of the SHHS protocol, but rather requested from the parent cohorts separately (to be taken from separate research visits). Evidently, LDL was not in the "A variable" request list.
I pinged the STAGES team for input. I see the description in the MOP you're referencing. Note that the STAGES project ended prematurely so they may not have been able to complete all their original aims.
Thanks for using the site!
I heard back, unfortunately the position signal encoding information was not retained by the STAGES data contributor, so we can't be certain of the different position signal value meanings across the STAGES sites.
Thanks - if you have an algorithm to detect the EKG-related data yourself then I recommend using that. The information on the HRV page (https://sleepdata.org/datasets/mesa/pages/hrv-analysis.md) is all the information we have. I believe only the R-points in sleep were intended to be reviewed.
Thanks for using the site. I'll reach out to our STAGES contact to see if we can get more information on the position channel.
Hey - please try nsrr download mesa-commercial-use - since your data request was commercial in nature we switched you (behind the scenes) to a modified version of the dataset. Thanks for using the site.
Thanks for using the site. You don't need Ruby to download the CSV datasets - it's more useful when downloading thousands of EDF and annotation files at once.
From what I can tell this is likely an encoding issue, perhaps you need to lookup our CSV dataset's encoding and specify that in the read_csv command.
I found some links that might be helpful:
https://stackoverflow.com/questions/18171739/unicodedecodeerror-when-reading-csv-file-in-pandas-with-python https://www.kaggle.com/code/paultimothymooney/how-to-resolve-a-unicodedecodeerror-for-a-csv-file
Hey Tim - thanks for visiting. Unfortunately we don't have any additional information on those SHHS spirometry variables. These were called "A-variables" by the SHHS team and were not collected as part of the SHHS visit protocol, but were rather gathered from the SHHS parent cohorts from other research visits. I think the cohorts were only constrained by time (i.e., provide values within the last X years), but it's conceivable the methods underlying the source spirometry measurements could have differed between SHHS parent cohorts.
I wish I knew more!