What is the process to download a full dataset (e.g., all CHAT edf's)
We provide an official download tool called nsrr which is a Ruby gem.
Installation instructions for the gem are here: https://github.com/nsrr/nsrr-gem
In short, once you have Ruby installed, you can do the following two steps:
Install nsrr Ruby Gem
gem install nsrr --no-document
Finally, to download only the EDFs for chat, you can type:
nsrr download chat/polysomnography/edfs
The downloader supports pause and resume, so that it will pick up where it last left off.
The above is a screenshot of the app running through the CHAT folder
Additionally, each page in a dataset's download folder contains the specific nsrr command needed to download that particular folder once you have signed in and filled out a Data Access and Use Agreement for that particular dataset.
Looks like I need a download token.
After signing in, you can access your download token here: https://sleepdata.org/token
Additionally, the download token will only work for datasets for which you have an approved DAUA.
hello , i followed your instructions as you said,and i type the website in the bowser,which returned the download token. however ,i could type the download token in the console, the only thing i can do is to press the enter button.unfortunately, it can create the folder but any contens.what should i do?
sorry , i could not type the download token in the console.....
The easiest way is to copy and paste the download token from the website directly into the console when using the NSRR gem. Make sure to include the number before the dash - as well as the rest of the string. The NSRR gem hides the input on the screen, but will still register the token if entered correctly. The gem still allows you to download publicly available files (like forms and data dictionaries) without entering a password which is why you sometimes see the folder structure generated without any files in it. A future version of the gem (v0.2.0+) should provide better feedback if the password was entered correctly or not. Hope that helps and thanks for the feedback!
For others interested in using the NSRR gem for downloading large numbers of files easily, you can view a short tutorial here:
Dear NSRR team,
I just got my DAUA approved and a mere 20 minutes later I already downloaded the first data set with the nsrr Ruby Gem without any problems. So, thanks very much for the absolutely clear and easy to follow tutorial!
The only question I would like to ask is whether there is any way to specify a different download path?
Thanks very much!
Glad to hear that you were able to use the nsrr gem!
If you want to only download a portion of a dataset, you can do so by navigating to the appropriate subfolder under the dataset files.
For example, if you only want to download datasets (and not EDFs) for SHHS, you can go to the datasets subfolder. Near the bottom of the page you will see a command listed that will only download that subfolder, in this case nsrr download shhs/datasets.
nsrr download shhs/datasets
There are also other parameters you can set that you can read about here on our NSRR gem Documentation page.
If you need any further questions, I'd be happy to clarify further, thanks!
Just as another quick reference, here are a couple of nsrr gem commands to download different datasets:
nsrr download shhs
nsrr download shhs/polysomnography/edfs
nsrr download chat
nsrr download chat/polysomnography/edfs
nsrr download chat/datasets
Also remember to run the command in the same location on your computer, that way the nsrr gem can resume if you already downloaded files from a dataset.
If your current directory in the command prompt or terminal is "~/Desktop" for example, running nsrr download chat/polysomnography/edfs will create a folder in "~/Desktop/chat/polysomnography/edfs". If you then run the full CHAT download command (nsrr download chat) from the "~/Desktop", then it would already see that it has downloaded the EDFs, and continue with "~/Desktop/chat/datasets" and other subfolders.
got it! As you explained, the files will be downloaded to the current directory in the command prompt at the time of execution. Thanks very much, again. If I may follow up with another question - could you also only download selected edf files with the nsrr gem? I tried some variations of the nsrr download command but got nowhere. Thanks!
Hi Stephany, currently there is no method for single file downloads, outside of directly downloading them through the browser. However we have had this requested in the past, and may be adding this in a v0.3.0 of the nsrr gem. I'll send you a message when the functionality has been added. Thanks for the feedback!
edit: v0.3.0 is now available!
My access for shhs database has been approved. But when i try to download with nsrr command as per the posts above, and copy paste my token, it always gives me this error. "UNAUTHORIZED Public Access Only".
Is there anything that I am missing ?
Thanks in Advance,
Sorry for the inconvenience, I noticed an issue with your access token and have reset the token. Can you give the NSRR gem another try, it should now correctly download the files from SHHS. You token has also changed, so you'll need to visit https://sleepdata.org/token to get the new one.
Now it works. Many thanks Remo.