BrainBrowser

Neurological Data Visualization
Using HTML5 and WebGL

Tarek Sherif

McGill Centre for Integrative Neuroscience

What is BrainBrowser?

What is BrainBrowser?

  • Open-source JavaScript library exposing web-based tools for neuroimaging
  • Key technologies:
    • HTML5 Canvas 2D
    • WebGL
    • Web Workers

What is BrainBrowser?

  • Volume Viewer
    • Explore volumetric data through orthogonal slices

What is BrainBrowser?

  • Surface Viewer
    • Explore geometry data in real-time 3D

Background: Neuroimaging

  • Neuroimaging is a fairly young field
    • Became widespread in the 1990s
  • Magnetic Resonance Imaging (MRI) became the most common technique
    • Low invasiveness
    • No exposure to radiation

Background: Neuroimaging

Typical Workflow

  • Acquisition
    • Volume of voxels representing intensity of the MR signal
    • Intensity can represent structural, activation or diffusion patterns
  • Processing
    • Extract features of interest (e.g. surfaces, measurements)
  • Visualization
    • Explore the data
    • Quality control

Background: Technology

HTML5 Canvas 2D

  • High-performance, scriptable 2D drawing surface
  • Pixel-level image processing

Background: Technology

WebGL

  • The browser talks directly to the graphics card
  • High-performance 3D graphics

Background: Technology

Web Workers

  • JavaScript is single-threaded by design
  • Web Workers allow for true multi-threading
  • Compute-intensive tasks can be done in the browser without blocking UI or rendering

Volume Viewer

  • Display orthogonal 2D slices of a 3D or 4D volume
  • Uses HTML5 Canvas 2D for image processing and display

Volume Viewer

Demo

Volume Viewer

Code

          
  BrainBrowser.VolumeViewer.start("visualization-div", function(viewer) {
    viewer.render();
    
    viewer.loadDefaultColorMapFromURL("color-map.txt");
    
    viewer.loadVolume({
      type: "minc",
      header_url: "volume.mnc.header",
      raw_data_url: "volume.mnc.raw"
    });
  });
        
      

Volume Viewer

Workflow

Volume Viewer

Workflow

  • Loading data
    • Data can be loaded over the network or from the local file system
      • AJAX for remote data
      • FileReader API for data

Volume Viewer

Workflow

  • Volume data
    • 3 or 4D array of intensity values
    • Header describing the volume
    • Each voxel contains a scalar intensity value

Volume Viewer

Workflow

  • Colour maps
    • Array of colours
    • Used to colourize scalar data

Volume Viewer

Workflow

  • Displaying a slice
    • Pull out slice along one axis of the volume
    • Colourize intensity values based on the colour map
    • Write to canvas as an image

Surface Viewer

  • Real-time display of 3D surfaces or tractography
  • Visualize various data maps on loaded surfaces
  • WebGL for rendering using the three.js library
  • Web Workers used for data parsing

Surface Viewer

Demo

Surface Viewer

Code

          
  BrainBrowser.SurfaceViewer.start("visualization-div", function(viewer) {
    viewer.render();

    viewer.loadColorMapFromURL("models/spectral.txt");
    
    viewer.loadModelFromURL("models/brain.obj", {
      format: "mniobj",
      complete: function() {
        viewer.loadIntensityDataFromURL("models/cortical-thickness.txt");
      }
    });

  });
        
      

Surface Viewer

Workflow

  • Load geometry
  • Load intensity data

Surface Viewer

Workflow

  • Loading data
    • Data can be loaded over the network or from the local file system
      • AJAX for remote data
      • FileReader API for data

Surface Viewer

Geometry Workflow

Surface Viewer

Geometry Workflow

  • Geometry data
    • Vertices: points in 3D space
    • Indices: array of indices into the vertex array to create triangles
    • Sometimes also normals and vertex colours for colouring

Surface Viewer

Geometry Workflow

  • De-index
    • Two ways to describe geometry in WebGL
      • Two arrays: vertices and indices
      • One array: vertices in drawing order
    • Core WebGL 1.0 spec limits indices to 16 bits
      • Maximum of 65536 vertices for indexed geometry

Surface Viewer

Geometry Workflow

  • De-index
    • Neurological geometry can get much bigger
      • DTI demo model contains 560674 vertices
    • De-index Web Worker "unrolls" the geometry indices
    • Converts from indexed geometry to unindexed

Surface Viewer

Geometry Workflow

  • Geometry loaded

Surface Viewer

Intensity Data Workflow

Surface Viewer

Intensity Data Workflow

  • Intensity data
    • List of per-vertex scalar values
    • Can represent various types of information:
      • Measurements (e.g. cortical thickness)
      • Brain regions
      • Connectivity

Surface Viewer

Intensity Data Workflow

  • Map colours
    • Map per-vertex scalar values to colours
    • Apply colours to vertices

Surface Viewer

Intensity Data Workflow

  • Intensity data applied

Thanks!

tsherif@gmail.com

http://tareksherif.ca

@thsherif