We use cookies and other tools to enhance your experience on our website and to analyze our web traffic.
For more information about these cookies and the data collected, please refer to our Privacy Policy.

Analyze NCH PSG (with .tsv) Using LUNA

4 posts
Was this reply useful? Learn more...
 
[-]
CookieMonster +0 points · 11 months ago

Dear all,

Thank you as always for your kind support.

I want to analyze the PSG data from NCH Sleep Data Bank using LUNA. I created a sample list from edf and annotation files (.tsv), but LUNA returned the following error message to any analysis. “error : invalid interval (start) : onset duration description”

I’d appreciate it if you would tell me how to deal with it.

Many thanks

11 posts
Was this reply useful? Learn more...
 
[-]
shaunpurcell +0 points · 11 months ago

These are 'as is' data - we plan to post 'harmonized' versions of this (and all) NSRR datasets soon, with consistent (and Luna-friendly) formatting. In the mean time, given access to the command line, you can make .annot files with a one-liner script. Luna's .annot format (described here: https://zzz.bwh.harvard.edu/luna/ref/annotations/#annot-files ) is designed to be ~easy to convert to from other formats. To make a 3-column .annot file: a) remove header rows, b) order columns label, start, duration, with duration starting to "+" to indicate it is not elapsed seconds from EDF start, and c) a small tweak, but swap out colons in labels (a special character fro class/instance label distinctions) to something else: e.g. something like:

 awk -F"\t" ' NR != 1 { print $3 , $1 , "+"$2 } ' OFS="\t" 10012_22912.tsv | tr ':' '_' > 10012_22912.annot 

Luna then reads it:

$ luna 10012_22912.edf annot-file=10012_22912.annot -s DESC

+++ luna | v0.28.0, 10-Apr-2023 | starting 09-May-2023 12:41:10 +++

input(s): 10012_22912.edf output : . commands: c1 DESC
edffile [10012_22912.edf]


Processing: 10012_22912 [ #1 ] EDF+ [10012_22912.edf] did not contain any time-track: adding... duration 10.48.52, 38932s | time 19.19.06 - 06.07.58 | date 01.01.01

signals: 27 (of 26) selected in an EDF+C file Patient_Event | EOG_LOC_M2 | EOG_ROC_M1 | EMG_Chin1_Chin2 | EEG_F3_M2 | EEG_F4_M1 | EEG_C3_M2 | EEG_C4_M1 EEG_O1_M2 | EEG_O2_M1 | EEG_CZ_O1 | EMG_LLeg_RLeg | ECG_EKG2_EKG | Snore | Resp_PTAF | Resp_Airflow Resp_Thoracic | Resp_Abdominal | SpO2 | Rate | EtCO2 | Capno | Resp_Rate | C_flow Tidal_Vol | Pressure | EDF Annotations extracting 'EDF Annotations' track from EDF+

annotations: ? (x107) | BED_RAILS_UP (x1) | Blink (x2) | Body_Position_Supine (x1) CBG_STICK (x1) | Close_Eyes (x2) | Deep_Breath (x2) | EEG_arousal (x3) Flex_Left_Leg (x2) | Flex_Right_Leg (x2) | Gain_Filter_Change (x8) | HAND_TO_FACE (x1) HOB_FLAT_FLAT_1_PILLOW_PULSE_OX_RIGHT_INDEX_FINGER (x1) | Hold_Breath (x2) | Impedance_at_10_kOhm (x3) | Look_Down (x2) Look_Left (x2) | Look_Right (x2) | Look_Up (x2) | MOM_GIVING_PATIENT_A_DRINK (x2) Montage_Channel_Test_Referential (x2) | Montage_NCH_PSG_STANDARD (x2) | Montage_NCH_PSG_STANDARD,_Ref (x2) | N1 (x6) N2 (x148) | N3 (x155) | Nasal_Breath (x2) | Open_Eyes (x3) Oral_Breath (x2) | Oxygen_Desaturation (x29) | PATIENT'S_MOM_COUGHING_MAY_BE_AN_OVERCAST_TO_PATIENT (x1) | PATIENT_AROUSED_LOOKING_AROUND (x1) PATIENT_BEING_GIVEN_A_DRINK (x1) | PATIENT_COUGHING (x1) | PATIENT_DRINKING_WATER (x1) | PATIENT_REQUESTING_FOR_SNACKS (x1) PATIENT_WANTED_HER_MOTHER_TO_PROVIDE_HER_WITH_A_DRINK (x1) | PATIENT_WILL_WATCH_TV_TILL_2200_HRS (x1) | RSOL_85 (x1) | Recording_Analyzer_ECG (x1) SOL_61.5 (x1) | SPONTANEOUS_AROUSALS (x1) | Start_Recording (x2) | Started_Analyzer_Data_Trends (x2) Started_Analyzer_ECG (x3) | Started_Analyzer_Sleep_Events (x2) | Stopped_Analyzer_Data_Trends (x1) | Stopped_Analyzer_Sleep_Events (x1) Swallow (x2) | TECH_IN_BATHROOM_IN (x1) | TECH_IN_PATIENT_REQUESTING_ASSISTANCE (x1) | TECH_IN_TO_FIX (x1) TECH_IN_TO_FIX_BELT (x1) | TECH_OUT (x4) | TECH_OUT_BATHROOM_OUT (x1) | TENSION_ON_C3_AND_O1. (x1) THORACIC_AND_ABDOMINAL_BELTS_NOTED._FIXING_SIGNAL (x1) | UNRESOLVED_TECH_BACK_IN (x1) | Video_Recording_ON (x2) | W (x653) chew_gum (x2) | edf_annot (x0) | lights_off (x2) | lights_on (x1) snore (x1)

variables: airflow=Resp_Airfl... | ecg=ECG_EKG2_EKG | eeg=EEG_F3_M2,... | effort=Resp_Thora... emg=EMG_Chin1_... | eog=EOG_LOC_M2... | generic=Patient_Ev... | id=10012_22912 | leg=Rate,Resp_... oxygen=SpO2 | snore=Snore .................................................................. CMD #1: DESC options: sig=* EDF filename : 10012_22912.edf ID : 10012_22912 Header start time : 19.19.06 Last observed time: 06.07.58 Duration : 10:48:52 38932 sec

signals : 26

EDF annotations : 1

Signals : Patient_Event[256] EOG_LOC_M2[256] EOG_ROC_M1[256] EMG_Chin1_Chin2[256] EEG_F3_M2[256] EEG_F4_M1[256] EEG_C3_M2[256] EEG_C4_M1[256] EEG_O1_M2[256] EEG_O2_M1[256] EEG_CZ_O1[256] EMG_LLeg_RLeg[256] ECG_EKG2_EKG[256] Snore[256] Resp_PTAF[256] Resp_Airflow[256] Resp_Thoracic[256] Resp_Abdominal[256] SpO2[256] Rate[256] EtCO2[256] Capno[256] Resp_Rate[256] C_flow[256] Tidal_Vol[256] Pressure[256]


...processed 1 EDFs, done.

...processed 1 command set(s), all of which passed

+++ luna | finishing 09-May-2023 12:41:11 +++

To make for all .tsvs, if using bash you can script a simple loop (obviously changing folder location):

for f in ls /data/nsrr/datasets/nchsdb/sleep_data/*.tsv | xargs -n 1 basename do echo "$f" fannot=echo $f | sed 's/\.tsv/\.annot/g' awk -F"\t" ' NR != 1 { print $3 , $1 , "+"$2 } ' OFS="\t" /data/nsrr/datasets/nchsdb/sleep_data/${f} | tr ':' '_' > ${fannot} done

Alternatively, we can upload all these new .annot files to sleepdata.org, if you look back in a day or so.

Cheers, --Shaun

11 posts
Was this reply useful? Learn more...
 
[-]
shaunpurcell +0 points · 11 months ago

wow, pls ignore the intense formatting of the prior reply, didn't realise it renders as markdown ;-)

442 posts
bio
Was this reply useful? Learn more...
 
[-]
mrueschman +0 points · 11 months ago

The .annot files are now available here: https://sleepdata.org/datasets/nchsdb/files/sleep_data

Thanks for using the site!

4 posts
Was this reply useful? Learn more...
 
[-]
CookieMonster +0 points · 11 months ago

Thank you so much for your quick and kind reply! I truly appreciate it.