BrainBrowser

Neurological Data Visualization
Using HTML5 and WebGL

Tarek Sherif

McGill Centre for Integrative Neuroscience

What is BrainBrowser?

What is BrainBrowser?

• Open-source JavaScript library exposing web-based tools for neuroimaging
• Key technologies:
  • HTML5 Canvas 2D
  • WebGL
  • Web Workers

What is BrainBrowser?

• Volume Viewer
  • Explore volumetric data through orthogonal slices

What is BrainBrowser?

• Surface Viewer
  • Explore geometry data in real-time 3D

Background: Neuroimaging

• Neuroimaging is a fairly young field
  • Became widespread in the 1990s
• Magnetic Resonance Imaging (MRI) became the most common technique
  • Low invasiveness
  • No exposure to radiation

Background: Neuroimaging

Typical Workflow

• Acquisition
  • Volume of voxels representing intensity of the MR signal
  • Intensity can represent structural, activation or diffusion patterns
• Processing
  • Extract features of interest (e.g. surfaces, measurements)
• Visualization
  • Explore the data
  • Quality control

Background: Technology

HTML5 Canvas 2D

• High-performance, scriptable 2D drawing surface
• Pixel-level image processing

Background: Technology

WebGL

• The browser talks directly to the graphics card
• High-performance 3D graphics

Background: Technology

Web Workers

• JavaScript is single-threaded by design
• Web Workers allow for true multi-threading
• Compute-intensive tasks can be done in the browser without blocking UI or rendering

Volume Viewer

• Display orthogonal 2D slices of a 3D or 4D volume
• Uses HTML5 Canvas 2D for image processing and display

Volume Viewer

Demo

• https://brainbrowser.cbrain.mcgill.ca/volume-viewer

Volume Viewer

Code

          
  BrainBrowser.VolumeViewer.start("visualization-div", function(viewer) {
    viewer.render();
    
    viewer.loadDefaultColorMapFromURL("color-map.txt");
    
    viewer.loadVolume({
      type: "minc",
      header_url: "volume.mnc.header",
      raw_data_url: "volume.mnc.raw"
    });
  });
        
      

Volume Viewer

Workflow

Volume Viewer

Workflow

• Loading data
  • Data can be loaded over the network or from the local file system
  • AJAX for remote data
  • FileReader API for data

Volume Viewer

Workflow

• Volume data
  • 3 or 4D array of intensity values
  • Header describing the volume
  • Each voxel contains a scalar intensity value

Volume Viewer

Workflow

• Colour maps
  • Array of colours
  • Used to colourize scalar data

Volume Viewer

Workflow

• Displaying a slice
  • Pull out slice along one axis of the volume
  • Colourize intensity values based on the colour map
  • Write to canvas as an image

Surface Viewer

• Real-time display of 3D surfaces or tractography
• Visualize various data maps on loaded surfaces
• WebGL for rendering using the three.js library
• Web Workers used for data parsing

Surface Viewer

Demo

• https://brainbrowser.cbrain.mcgill.ca/surface-viewer

Surface Viewer

Code

          
  BrainBrowser.SurfaceViewer.start("visualization-div", function(viewer) {
    viewer.render();

    viewer.loadColorMapFromURL("models/spectral.txt");
    
    viewer.loadModelFromURL("models/brain.obj", {
      format: "mniobj",
      complete: function() {
        viewer.loadIntensityDataFromURL("models/cortical-thickness.txt");
      }
    });

  });
        
      

Surface Viewer

Workflow

• Load geometry
• Load intensity data

Surface Viewer

Workflow

• Loading data
  • Data can be loaded over the network or from the local file system
  • AJAX for remote data
  • FileReader API for data

Surface Viewer

Geometry Workflow

Surface Viewer

Geometry Workflow

• Geometry data
  • Vertices: points in 3D space
  • Indices: array of indices into the vertex array to create triangles
  • Sometimes also normals and vertex colours for colouring

Surface Viewer

Geometry Workflow

• De-index
  • Two ways to describe geometry in WebGL
    • Two arrays: vertices and indices
    • One array: vertices in drawing order
  • Core WebGL 1.0 spec limits indices to 16 bits
    • Maximum of 65536 vertices for indexed geometry

Surface Viewer

Geometry Workflow

• De-index
  • Neurological geometry can get much bigger
    • DTI demo model contains 560674 vertices
  • De-index Web Worker "unrolls" the geometry indices
  • Converts from indexed geometry to unindexed

Surface Viewer

Geometry Workflow

• Geometry loaded

Surface Viewer

Intensity Data Workflow

Surface Viewer

Intensity Data Workflow

• Intensity data
  • List of per-vertex scalar values
  • Can represent various types of information:
    • Measurements (e.g. cortical thickness)
    • Brain regions
    • Connectivity

Surface Viewer

Intensity Data Workflow

• Map colours
  • Map per-vertex scalar values to colours
  • Apply colours to vertices

Surface Viewer

Intensity Data Workflow

• Intensity data applied

Thanks!

tsherif@gmail.com

http://tareksherif.ca

@thsherif