The BioDigital Human

Opportunities and Challenges of WebGL

Tarek Sherif

BioDigital

BioDigital's Goals

  • Use interactive 3D to make anatomical content engaging
  • Move beyond the anatomy atlas
    • Tell stories
    • Bring content to life
    • Map data to and from 3D models
  • Make it all widely accessible through the Web

The BioDigital Human

  • 3D content library
    • 5000+ anatomy objects
    • 500+ health conditions
  • Rendering engine built on SceneJS
  • User-created annotations and custom views
  • API for embedding Human content into external websites

The BioDigital Human

Key Clients
  • Medical device manufacturers
  • Pharmaceutical
  • Educational
  • Medical students

What is WebGL?

  • A low-level JavaScript API that provides access to the GPU
  • Allows for high-performance, interactive 3D graphics on the Web

Why WebGL?

Pros
  • It's everywhere!
  • Use HTML/CSS/JavaScript for easy UI
  • Integrate into client web sites
  • Leverage existing web services
  • (Relatively) easy distribution
  • (Relatively) easy deployment of updates

In a nutshell...

  • Leverage the entire infrastructure of the Web as a platform

Why WebGL?

Cons
  • Web UX expectations
    • It should be fast
    • It should be easy
    • It should just work

Why WebGL?

Cons
  • JavaScript!!!
    • Memory management is hard
    • Common JS patterns are bad for performance
          
            object.transform({
              rotate: {
                axis: vec1.cross(vec2).normalize(),
                angle: Math.PI / 4
              } 
            })

The BioDigital Human

Key Components
  • Content
  • 3D Rendering
  • Integration and API

Types of Content

Content Challenges

SIZE!
  • Artists want it beautiful: big textures, detailed geometry
  • Engineers want it to not crash on an iPhone
  • Tension between anatomical detail and application stability

Rendering

  • Navigating anatomical content is difficult
    • Deeply nested
    • Hierarchical
    • Scale (from cells to whole body)

Rendering

  • Focus on giving users ways to view, emphasize, interact with information of interest:
    • Highlight object
    • Highlight region (defined by texture)
    • Dissect (remove an object)
    • Isolate (remove all other objects)
    • Transparency
    • Annotations
    • (+ text, audio, video, etc.)

Example

The Human Heart

Rendering Challenges

MOBILE!
  • Rendering performance and memory can be severely limited on mobile devices
  • Possible to query some properties:
    • GL variable limits
    • Shader precision
    • Available extensions
  • Not others:
    • GPU memory

Rendering Challenges

ARGGGH... APPLE!!!
Source: http://codeflow.org/entries/2014/jun/08/some-issues-with-apples-ios-webgl-implementation/

Human API

  • Embed Human content in an external web page through an iFrame
  • Basic interactions built-in
    • Mouse movement
    • Dissection, highlight, annotate, etc.
  • UI can be customized through URL parameters

Human API

  • For more customized interactions, use the JavaScript API
    • Communicates with iFrame using the window messaging API
          
              var human = new HumanAPI.Human("iFrameID"); 

              human.camera.flyTo({
                  eye: { z: -25 },
                  velocity: 20
              });

              human.pick.on("picked", function(e) {
                console.log(e.worldPos);
              });

Example

API Challenges

  • Want the average web developer to understand it all
    • 3D terminology and concepts
    • 3D navigation
    • Anatomical concepts
    • Architecture of the Human

Solution

  • Documentation
  • Tutorials
  • Support

Get involved!

Thanks!