Count Apneas in EDF Annotations

Count Apneas in EDF Annotations

In this tutorial, you will learn how to:

  1. Count the number of apneas in an EDF annotation file and export the result to a CSV spreadsheet.

The Requirements for running this tutorial are described after the discussion of the tutorial script.

The script

require 'rubygems'
require 'colorize'
require 'csv'
require 'xmlsimple'
require 'nsrr'
require 'nsrr/commands/download'

# Take the XML files in this folder `/annotation-testing` and answer two
# questions:
#   1) How many epochs/minutes was the subject asleep?
#   2) How many obstructive apnea events were marked on the recording?
# Asleep is defined as SleepStage = 1,2,3,4,5

AWAKE_STAGES = [0]
SLEEP_STAGES = [1, 2, 3, 4, 5]
NREM_STAGES = [1, 2, 3, 4]
REM_STAGES = [5]

# Simplifies filtering an array of objects by attributes.
class Array
  def where(filters)
    results = dup
    filters.each do |method, value|
      if value.is_a?(Array)
        results.select! { |object| value.include?(object.send(method)) }
      else
        results.select! { |object| object.send(method) == value }
      end
    end
    results
  end
end

# Represents a single scored event in an EDF, along with information present in
# the signal at the location of the event.
class ScoredEvent
  attr_accessor :name, :lowest_spo2, :desaturation, :start, :duration, :input,
                :stage

  def initialize(hash = {})
    @name = hash[:name]
    @lowest_spo2 = hash[:lowest_spo2]
    @desaturation = hash[:desaturation]
    @start = hash[:start]
    @duration = hash[:duration]
    @input = hash[:input]
    @stage = nil
  end

  def set_stage(sleep_stages, epoch_length)
    return if @start.nil?
    index = (@start / epoch_length).floor
    @stage = sleep_stages[index]
  end
end

# Represents a set of Annotation of an EDF, including sleep stages, epoch
# length, and scored events.
class Annotation
  attr_accessor :filename, :xml, :epoch_length, :sleep_stages, :scored_events

  def initialize(filename)
    @filename = filename
    @xml = XmlSimple.xml_in(filename)
    @epoch_length = parse_epoch_length
    @sleep_stages = parse_sleep_stages
    @scored_events = parse_scored_events
  end

  private

  def parse_epoch_length
    @xml['EpochLength'][0].to_i
  end

  def parse_sleep_stages
    @xml['SleepStages'][0]['SleepStage'].collect(&:to_i)
  end

  def parse_scored_events
    @xml['ScoredEvents'][0]['ScoredEvent'].collect do |hash|
      hash = {
        name: extract(hash, 'Name'),
        lowest_spo2: extract(hash, 'LowestSpO2', convert: :to_f),
        desaturation: extract(hash, 'Desaturation', convert: :to_f),
        start: extract(hash, 'Start', convert: :to_f),
        duration: extract(hash, 'Duration', convert: :to_f),
        input: extract(hash, 'Input')
      }
      scored_event = ScoredEvent.new(hash)
      scored_event.set_stage(@sleep_stages, @epoch_length)
      scored_event
    end
  end

  def extract(hash, key, convert: nil)
    value = hash[key] ? hash[key][0] : nil
    convert && value ? value.send(convert) : value
  end
end

def annotations(recursive: true)
  return enum_for(:annotations, recursive: recursive) unless block_given?
  annotation_paths(recursive: recursive).each do |file_path|
    yield Annotation.new(file_path)
  end
end

def annotation_paths(recursive: true)
  path = "#{'**/' if recursive}*.xml"
  Dir.glob(path, File::FNM_CASEFOLD)
end

Nsrr::Commands::Download.run(%w(download learn/polysomnography/annotations-events-profusion/learn-nsrr01-profusion.xml --token=public))
Nsrr::Commands::Download.run(%w(download learn/polysomnography/annotations-events-profusion/learn-nsrr02-profusion.xml --token=public))
Nsrr::Commands::Download.run(%w(download learn/polysomnography/annotations-events-profusion/learn-nsrr03-profusion.xml --token=public))

total_annotations = annotation_paths.count
csv_path = 'annotations.csv'

CSV.open(csv_path, 'wb') do |csv|
  csv << [
    'Annotation', 'Asleep (min)', 'Recording Time (min)',
    'OAs', 'REM OAs', 'NREM OAs',
    'CAs', 'REM CAs', 'NREM CAs',
    'HYs', 'REM HYs', 'NREM HYs'
  ]
  annotations.each_with_index do |annotation, index|
    asleep_stages = annotation.sleep_stages.where(itself: SLEEP_STAGES)
    minutes_asleep = asleep_stages.count * annotation.epoch_length / 60.0
    minutes_total = annotation.sleep_stages.count * annotation.epoch_length / 60.0

    oa_events = annotation.scored_events.where(name: 'Obstructive Apnea')
    rem_oa_events = oa_events.where(stage: REM_STAGES)
    nrem_oa_events = oa_events.where(stage: NREM_STAGES)

    ca_events = annotation.scored_events.where(name: 'Central Apnea')
    rem_ca_events = ca_events.where(stage: REM_STAGES)
    nrem_ca_events = ca_events.where(stage: NREM_STAGES)

    hy_events = annotation.scored_events.where(name: 'Hypopnea')
    rem_hy_events = hy_events.where(stage: REM_STAGES)
    nrem_hy_events = hy_events.where(stage: NREM_STAGES)

    # puts message
    print "\rFile #{index + 1} of #{total_annotations}: #{annotation.filename}         "

    csv << [
      annotation.filename, minutes_asleep, minutes_total,
      oa_events.count, rem_oa_events.count, nrem_oa_events.count,
      ca_events.count, rem_ca_events.count, nrem_ca_events.count,
      hy_events.count, rem_hy_events.count, nrem_hy_events.count
    ]
  end
end

puts "\n\nAnnotation apnea counts exported to " + "#{csv_path}".colorize(:white) + "."

puts "\nCongrats! You have completed your fifth tutorial!".colorize(:green).on_white

download script

Breaking down the script

The beginning of the script sets up classes for ScoredEvents and Annotations. Starting at the Nsrr::Commands::Download.run method, the script downloads annotation files from the NSRR.

csv_path = 'annotations.csv'
CSV.open(csv_path, 'wb') do |csv|
  ...
end

The script then opens up a CSV file for writing the resulting counts of apneas found in the EDFs. Next, the script writes the column headers to the CSV:

csv << [
  'Annotation', 'Asleep (min)', 'Recording Time (min)',
  'OAs', 'REM OAs', 'NREM OAs',
  'CAs', 'REM CAs', 'NREM CAs',
  'HYs', 'REM HYs', 'NREM HYs'
]

After this, each annotation file is loaded, and the scored_events are filtered by their type using the where method.

oa_events = annotation.scored_events.where(name: 'Obstructive Apnea')
rem_oa_events = oa_events.where(stage: REM_STAGES)
nrem_oa_events = oa_events.where(stage: NREM_STAGES)

The where method accepts a hash of key value pairs that represent the ScoredEvent attribute and value being filtered. For example, where(name: 'Obstructive Apnea', filters the scored_events to only include those that are marked as "Obstructive Apnea". The value can also be an array, in the case of where(stage: NREM_STAGES), where NREM_STAGES is equal to the array [1, 2, 3, 4].

Multiple key value pairs can be added to a single where method call, and where method calls can also be chained. The two following lines create identical results.

annotation.scored_events.where(name: 'Obstructive Apnea').where(stage: NREM_STAGES)
annotation.scored_events.where(name: 'Obstructive Apnea', stage: NREM_STAGES)

Once the filtering of the scored_events is done, the count method is passed to the filtered array, and the result is written to the CSV, along with a few additional attributes.

csv << [
  annotation.filename, minutes_asleep, minutes_total,
  oa_events.count, rem_oa_events.count, nrem_oa_events.count,
  ca_events.count, rem_ca_events.count, nrem_ca_events.count,
  hy_events.count, rem_hy_events.count, nrem_hy_events.count
]

When the script is finished, the annotations.csv can be opened to compare counts across the annotation files that were loaded from the NSRR.

Try These

  1. Modify the script to output annotations that occur across two different sleep stages in their own column.

If you completed this tutorial, please leave a review.

This tutorial concludes the introductory tutorials aimed at teaching you how to access and work with EDFs and annotations in Ruby.

Requirements

This tutorial requires that you have Ruby 2.2 or above installed, preferably Ruby 2.3 or higher.

Extract

Extract the script to your desktop to a folder of the same name. Open your console (Command Prompt on Windows or Terminal on Mac/Linux) and navigate to your tutorial folder.

MacOS/Linux

cd ~/Desktop/tutorial_05

Windows

cd %USERPROFILE%\Desktop\tutorial_05

Notes

  • cd stands for "Change Directory"
  • ~, and %USERPROFILE% represents your user home directory

Installing the required gems.

This script requires you to have the colorize, nsrr, and xml-simple gems installed. The colorize gem is used to color command line output. The nsrr gem is used to easily download sets of files from the National Sleep Research Resource (https://sleepdata.org). The xml-simple gem is used to read and EDF annotation files using Ruby.

In your console type:

gem install colorize nsrr xml-simple --no-document

This command will get you the latest stable release of the colorize, nsrr, and xml-simple gem from RubyGems.org. The --no-document flag specifies to skip the installation of the documentation for the gems.

Running the script.

In your console and from the tutorial_05 directory, type:

ruby tutorial_05.rb

Questions?

Remember, if you have any questions on this script, please let us know on the NSRR forum here: https://sleepdata.org/forum, or email us at support@sleepdata.org.