Dear all,
Thank you as always for your kind support.
I want to analyze the PSG data from NCH Sleep Data Bank using LUNA. I created a sample list from edf and annotation files (.tsv), but LUNA returned the following error message to any analysis. “error : invalid interval (start) : onset duration description”
I’d appreciate it if you would tell me how to deal with it.
Many thanks
These are 'as is' data - we plan to post 'harmonized' versions of this (and all) NSRR datasets soon, with consistent (and Luna-friendly) formatting. In the mean time, given access to the command line, you can make .annot files with a one-liner script. Luna's .annot format (described here: https://zzz.bwh.harvard.edu/luna/ref/annotations/#annot-files ) is designed to be ~easy to convert to from other formats. To make a 3-column .annot file: a) remove header rows, b) order columns label, start, duration, with duration starting to "+" to indicate it is not elapsed seconds from EDF start, and c) a small tweak, but swap out colons in labels (a special character fro class/instance label distinctions) to something else: e.g. something like:
awk -F"\t" ' NR != 1 { print $3 , $1 , "+"$2 } ' OFS="\t" 10012_22912.tsv | tr ':' '_' > 10012_22912.annot
Luna then reads it:
input(s): 10012_22912.edf
output : .
commands: c1 DESC
edffile [10012_22912.edf]
Processing: 10012_22912 [ #1 ] EDF+ [10012_22912.edf] did not contain any time-track: adding... duration 10.48.52, 38932s | time 19.19.06 - 06.07.58 | date 01.01.01
signals: 27 (of 26) selected in an EDF+C file Patient_Event | EOG_LOC_M2 | EOG_ROC_M1 | EMG_Chin1_Chin2 | EEG_F3_M2 | EEG_F4_M1 | EEG_C3_M2 | EEG_C4_M1 EEG_O1_M2 | EEG_O2_M1 | EEG_CZ_O1 | EMG_LLeg_RLeg | ECG_EKG2_EKG | Snore | Resp_PTAF | Resp_Airflow Resp_Thoracic | Resp_Abdominal | SpO2 | Rate | EtCO2 | Capno | Resp_Rate | C_flow Tidal_Vol | Pressure | EDF Annotations extracting 'EDF Annotations' track from EDF+
annotations: ? (x107) | BED_RAILS_UP (x1) | Blink (x2) | Body_Position_Supine (x1) CBG_STICK (x1) | Close_Eyes (x2) | Deep_Breath (x2) | EEG_arousal (x3) Flex_Left_Leg (x2) | Flex_Right_Leg (x2) | Gain_Filter_Change (x8) | HAND_TO_FACE (x1) HOB_FLAT_FLAT_1_PILLOW_PULSE_OX_RIGHT_INDEX_FINGER (x1) | Hold_Breath (x2) | Impedance_at_10_kOhm (x3) | Look_Down (x2) Look_Left (x2) | Look_Right (x2) | Look_Up (x2) | MOM_GIVING_PATIENT_A_DRINK (x2) Montage_Channel_Test_Referential (x2) | Montage_NCH_PSG_STANDARD (x2) | Montage_NCH_PSG_STANDARD,_Ref (x2) | N1 (x6) N2 (x148) | N3 (x155) | Nasal_Breath (x2) | Open_Eyes (x3) Oral_Breath (x2) | Oxygen_Desaturation (x29) | PATIENT'S_MOM_COUGHING_MAY_BE_AN_OVERCAST_TO_PATIENT (x1) | PATIENT_AROUSED_LOOKING_AROUND (x1) PATIENT_BEING_GIVEN_A_DRINK (x1) | PATIENT_COUGHING (x1) | PATIENT_DRINKING_WATER (x1) | PATIENT_REQUESTING_FOR_SNACKS (x1) PATIENT_WANTED_HER_MOTHER_TO_PROVIDE_HER_WITH_A_DRINK (x1) | PATIENT_WILL_WATCH_TV_TILL_2200_HRS (x1) | RSOL_85 (x1) | Recording_Analyzer_ECG (x1) SOL_61.5 (x1) | SPONTANEOUS_AROUSALS (x1) | Start_Recording (x2) | Started_Analyzer_Data_Trends (x2) Started_Analyzer_ECG (x3) | Started_Analyzer_Sleep_Events (x2) | Stopped_Analyzer_Data_Trends (x1) | Stopped_Analyzer_Sleep_Events (x1) Swallow (x2) | TECH_IN_BATHROOM_IN (x1) | TECH_IN_PATIENT_REQUESTING_ASSISTANCE (x1) | TECH_IN_TO_FIX (x1) TECH_IN_TO_FIX_BELT (x1) | TECH_OUT (x4) | TECH_OUT_BATHROOM_OUT (x1) | TENSION_ON_C3_AND_O1. (x1) THORACIC_AND_ABDOMINAL_BELTS_NOTED._FIXING_SIGNAL (x1) | UNRESOLVED_TECH_BACK_IN (x1) | Video_Recording_ON (x2) | W (x653) chew_gum (x2) | edf_annot (x0) | lights_off (x2) | lights_on (x1) snore (x1)
variables: airflow=Resp_Airfl... | ecg=ECG_EKG2_EKG | eeg=EEG_F3_M2,... | effort=Resp_Thora... emg=EMG_Chin1_... | eog=EOG_LOC_M2... | generic=Patient_Ev... | id=10012_22912 | leg=Rate,Resp_... oxygen=SpO2 | snore=Snore .................................................................. CMD #1: DESC options: sig=* EDF filename : 10012_22912.edf ID : 10012_22912 Header start time : 19.19.06 Last observed time: 06.07.58 Duration : 10:48:52 38932 sec
Signals : Patient_Event[256] EOG_LOC_M2[256] EOG_ROC_M1[256] EMG_Chin1_Chin2[256] EEG_F3_M2[256] EEG_F4_M1[256] EEG_C3_M2[256] EEG_C4_M1[256] EEG_O1_M2[256] EEG_O2_M1[256] EEG_CZ_O1[256] EMG_LLeg_RLeg[256] ECG_EKG2_EKG[256] Snore[256] Resp_PTAF[256] Resp_Airflow[256] Resp_Thoracic[256] Resp_Abdominal[256] SpO2[256] Rate[256] EtCO2[256] Capno[256] Resp_Rate[256] C_flow[256] Tidal_Vol[256] Pressure[256]
...processed 1 EDFs, done.
To make for all .tsvs, if using bash you can script a simple loop (obviously changing folder location):
for f in ls /data/nsrr/datasets/nchsdb/sleep_data/*.tsv | xargs -n 1 basename
do
echo "$f"
fannot=echo $f | sed 's/\.tsv/\.annot/g'
awk -F"\t" ' NR != 1 { print $3 , $1 , "+"$2 } ' OFS="\t" /data/nsrr/datasets/nchsdb/sleep_data/${f} | tr ':' '_' > ${fannot}
done
Alternatively, we can upload all these new .annot files to sleepdata.org, if you look back in a day or so.
Cheers, --Shaun
The .annot files are now available here: https://sleepdata.org/datasets/nchsdb/files/sleep_data
Thanks for using the site!