Working to improve the lives of sleep apnea patients and researchers one commit at a time.
Thanks for the detailed information. I wanted to add that if you would like to follow any of the code changes and discussions, we have two pull requests/issues open on GitHub to suggest changes to the NSRR translator tool.
Fixing Missing First Epoch
Discussion on How to Handle Last Sleep Stage
Thanks again, we'll send you an update when these changes go through, and will also update the corresponding files in the NSRR datasets.
Hi Stephany, currently there is no method for single file downloads, outside of directly downloading them through the browser. However we have had this requested in the past, and may be adding this in a v0.3.0 of the nsrr gem. I'll send you a message when the functionality has been added. Thanks for the feedback!
edit: v0.3.0 is now available!
Just as another quick reference, here are a couple of nsrr gem commands to download different datasets:
nsrr download shhs
nsrr download shhs/polysomnography/edfs
nsrr download shhs/datasets
nsrr download chat
nsrr download chat/polysomnography/edfs
nsrr download chat/datasets
Also remember to run the command in the same location on your computer, that way the nsrr gem can resume if you already downloaded files from a dataset.
If your current directory in the command prompt or terminal is "~/Desktop" for example, running nsrr download chat/polysomnography/edfs will create a folder in "~/Desktop/chat/polysomnography/edfs". If you then run the full CHAT download command (nsrr download chat) from the "~/Desktop", then it would already see that it has downloaded the EDFs, and continue with "~/Desktop/chat/datasets" and other subfolders.
Glad to hear that you were able to use the nsrr gem!
If you want to only download a portion of a dataset, you can do so by navigating to the appropriate subfolder under the dataset files.
For example, if you only want to download datasets (and not EDFs) for SHHS, you can go to the datasets subfolder. Near the bottom of the page you will see a command listed that will only download that subfolder, in this case nsrr download shhs/datasets.
There are also other parameters you can set that you can read about here on our NSRR gem Documentation page.
If you need any further questions, I'd be happy to clarify further, thanks!
The easiest way is to copy and paste the download token from the website directly into the console when using the NSRR gem. Make sure to include the number before the dash - as well as the rest of the string. The NSRR gem hides the input on the screen, but will still register the token if entered correctly. The gem still allows you to download publicly available files (like forms and data dictionaries) without entering a password which is why you sometimes see the folder structure generated without any files in it. A future version of the gem (v0.2.0+) should provide better feedback if the password was entered correctly or not. Hope that helps and thanks for the feedback!
For others interested in using the NSRR gem for downloading large numbers of files easily, you can view a short tutorial here:
cc @mrueschman can you take a look at the staging files? I took a look at the MD5 hash for the files, and they are indeed identical!
Nice catch Shaun, we'll make this a priority to fix these. Glad you like the nsrr downloader!
After signing in, you can access your download token here: https://sleepdata.org/token
Additionally, the download token will only work for datasets for which you have an approved DAUA.
Hi Robert, The NSRR Review Committee should be meeting again soon and will review a number of recently submitted DAUAs. You'll be notified if your DAUA is approved or sent back for resubmission. Once it's approved, you'll be able to download the CSV datasets listed under the downloads folder for each dataset. Good luck with the computation!
We provide an official download tool called nsrr which is a Ruby gem.
Installation instructions for the gem are here: https://github.com/nsrr/nsrr-gem
In short, once you have Ruby installed, you can do the following two steps:
Install nsrr Ruby Gem
gem install nsrr --no-document
Finally, to download only the EDFs for chat, you can type:
nsrr download chat/polysomnography/edfs
The downloader supports pause and resume, so that it will pick up where it last left off.
The above is a screenshot of the app running through the CHAT folder
Additionally, each page in a dataset's download folder contains the specific nsrr command needed to download that particular folder once you have signed in and filled out a Data Access and Use Agreement for that particular dataset.
Hi Edward, I'm not sure if this is quite what you meant, but I went ahead and generated a treemap of the files hosted on the NSRR website. While the large sections of each dataset are primarily taken up by EDFs, they also include EDF annotations (XML), covariate datasets (CSVs), and Code Book CRFs (PDFs).
Also interesting to note is that the CHAT EDFs are much larger than other EDFs, since they contain many more signal channels for each EDF.
We may make a web version of this in the future, as it's a great way of exploring the datasets. I haven't found a good web-based treemap tool yet, although Google Charts appears to have a rudimentary one available.