HoloLens Blueprints电子书

Title Page

Copyright

HoloLens Blueprints

Credits

About the Authors

Acknowledgments

About the Reviewer

Disclaimer

Disclaimer

Customer Feedback

www.PacktPub.com

Dedication

Preface

What this book covers

What you need for this book

Who this book is for

Conventions

Reader feedback

Customer support

Downloading the example code

Downloading the color images of this book

Errata

Piracy

Questions

Digital Reality - Under the Hood

Virtual Reality – what it is?

Realizing Virtual Reality

Virtual Reality in the field

Augmented Reality – what it is?

Visualization techniques for Augmented Reality

Screen-based

Augmented Reality head-mounted globally displays

Projection-based

Augmented Reality in the field

Mixed Reality – what it is?

Mixed Reality in the field

Digital Reality and the current market place

Summary

HoloLens – The Most Natural Way to Interact

A closer look at HoloLens hardware

Inside the HoloLens device

Cameras and sensors

The Lenses

Speakers

Microphones

The processing units

Inertial Measurement Unit

Vent

Connectivity

Control buttons

LED indicator

Putting things together

Turning your device on/off

Cortana on HoloLens

The Mixed Reality world - HoloLens shell

Holograms in reality

HoloLens, holograms and your real world

Hologram and the Windows Holographic platform

Recording holograms – Mixed Reality capture

HoloLens interaction and application model

Interacting with holograms

Gaze

Gesture

Voice

Spatial Audio

Spatial Mapping

HoloLens app life cycles

HoloLens Clicker

Controlling your HoloLens remotely

Key differentiators

Immersive with surroundings

Independent standalone device

No screen staring

Mixed not virtual

Broad scope

Exploring the possibilities with HoloLens

Virtual collaboration

Remote expert guidance

People-centric applications

Development process and team building

Building up the team

Designers

3D Artists

Developers

Audio designers

Leads

Development process overview

Requirement Envisioning

Storyboarding

Planning, Designing and Prototyping

Planning and Design

Prototyping

Develop

Deploy and Test

Summary

Explore HoloLens as Hologram - Scenario Identification and Sketching

Envisioning

Training scenario for HoloLens

Features

Collaboration

Scenario prioritization

Scenario elaboration

Primary View

Lenses

Speakers

Cameras

Volume control buttons

Brightness control buttons

Sketching the scenarios

Sketching - interaction with speakers

Assets design and development

3D Modeling

3D Modeling tools

3D Modeling workflow

Build your 3D Model

Model sheet - base reference images

Scene creation

Importing model sheet

Import side image

Import front image

Import top image

3D Model construction with cube

Edge loop

Vertex adjustment

Mirroring

Texturing workflow

UV Unwrapping of model

Create texture in Photoshop

Export as Filmbox (FBX)

Assembling assets in Unity3D

Things to consider while using the 3D Asset Pipeline

Maintaining quads

Checking Surface Normals

Maintaining grouping

Optimizing the scene file

Setting up a development environment

Prerequisites

Installation checklist

Using HoloLens emulator

Summary

Explore HoloLens as Hologram - Developing Application and Deploying on Device

Getting started – creating a new project

The Unity3D basics

The Core interface

Scene View

Game View

Object Hierarchy

Project Assets

Inspector

The Toolbar

The Core concepts

Game Object

Components

Prefabs

Tags

Coordinate system

Setting up a project for holographic application

Setting up the HoloToolkit - Unity

Get The HoloToolkit - Unity

Preparing the package

Import the package

Applying project settings

Behind the scenes

Updating the initial scene

Importing 3D Assets

Exploring the 3D Object

Saving the scene

Build, run and test the initial setup

Giving life to object – scripting

Highlighting the objects - enabling Gaze

The Gaze manager

The Cursor

Adding the Box Collider

See Gaze in action

Behind the scene

Adding actions to objects - applying gesture

Adding Air Tap on Lenses

Air Tap on Lenses - see it in action

Adding Air Tap on speaker

Air Tap in speaker – see it in action

Behind the scenes

Interaction using Voice Command

Enabling the Microphone capabilities

The Keyword Manager

Defining Keywords

Implementing actions for Keywords

Mapping with Keyword Manager, Keyword and actions

Voice Command – see it in action

Behind the scenes

Interaction using text to speech

Setting up text to speech Manager

Integration of text to speech

Text to speech in application launch

Text to speech in Air Tap

Text to speech - see it in action

Behind the scenes

Deploying your app

Deploy using Visual Studio

Deploy using Build Automation Window

Deploy using Device Portal

Load the package

Start the app

Deploy using Unity remoting

Configure HoloLens device for remoting

Holographic emulation – remoting from Unity

Running the app on a device

Testing your application

Lighting conditions

Space

Angle of views

View distance

Audio or visual hints

Ambient sounds

Summary

Remote Monitoring of Smart Building(s) Using HoloLens - Scenario Identification and Sketching

Envisioning

Solution scope overview

Devices and Sensors

Event Processing

Visualization

Real-time monitoring

Analyse and Act

Scenario prioritization

Scenario elaboration

Internet of Things – Remote Monitoring

Devices

Cloud Gateway (Azure IoT Hub)

Steam Processing (Azure Stream Analytics)

Data Store (Azure Storage)

Machine Learning

Real-time visualization through HoloLens

APIs

Holographic application

Sketching the scenarios

Sketching - Interaction with Rooms and Sensors

3D Model – structure and components

3D Model of building

3D Model of sensors

Summary

Remote Monitoring of Smart Building(s) Using HoloLens - Developing Application and Deploying on Device

Solution development

Logical design diagram

Setting up the backend Infrastructure

Create the Azure Event Hub

Create the Azure Cosmos DB

Create the Stream Analytics Job

Building up the Azure IoT solution

Connecting Device with Azure Event Hub – simulator

Developing the Web API

Setting up a project for a holographic application

Getting started – creating a new project

Import the package

Applying Project Settings

Updating the initial scene

Importing 3D Assets – building model

Exploring the building's 3D Objects

Save the scene

Build, run and test the initial setup

Tracking building floors from 3D Model

Adding Gaze

Adding the cursor

Adding the Box Collider

Add tag to each floor for mapping

Tracking Gaze for each floor

See Gaze in action

Verify the Gazed floor

Connecting a holographic app with Azure

Defining data models

Enabling the InternetClient capabilities

Building Azure Bridge

Getting the data

Attaching the Azure Bridge

Binding data with sensor indicators

See the live data in action

Enabling sensor binding for all floors

Adding actions to floors – applying gesture

Adding Air Tap on floors

Displaying detailed information

Designing the Panel layouts

Grouping UI elements – adding Canvas Group

Binding Data with Panel

Invoking the Panel

Closing the Panel

Air Tap on Lenses - see it in action

Deploying your app onto a device

Extending the current solution

Adding Voice Command

3D Model placement as table top mode

Additional interaction with services

Summary

Build End-to-End Retail Solution - Scenario Identification and Sketching

Envisioning

Scenario elaboration

Catalog and merchandising

Visualization

Order placement

Sketching the scenarios

Sketching - interaction for catalog and items

Sketching - interaction with 3D Model

3D Model

Summary

Build End-to-End Retail Scenario - Developing Application and Deploying on Device

Getting the 3D Assets ready

Unity Project and Asset Bundle setup

Importing 3D retail assets

Assets Bundles generation

Solution development

Logical design diagram

Building the backend Azure Solution

Creating the Azure Blob Storage

Creating the Azure Cosmos DB

Developing the Web API

Setting up the holographic project

Importing the main menu 3D Model

Exploring the main menu

Building , running and testing the initial solution

Making the main menu Intractable

Enabling Gaze with cursor

Attaching the script with menu items

Enabling Air Tap with menu items

API connector - connecting with Web API

Defining the application data model

Enabling the IntenetClient capabilities

API Connector

Getting the data

Attaching the API connector

Getting holograms on the fly

Handling the Air Tap on menu item

Downloading assets and rendering holograms on the fly

Associating toolbars with holograms

The toolbar controls

Instantiating toolbars on the fly

Showing controls based on types

Dynamic holograms in action

Adding action to the toolbar controls

Remove product

Add to Cart

Changing pictures for Wall Paints

Placing the holograms - spatial mapping

Deploying your app on a device

Extending the current solution

Dynamic menu for products

Adding Voice Command

Application and storage access security

Rotating the individual product

Summary

Possibilities

HoloLens and Microsoft Bot Framework

Solution

Client authentication

Start conversation

Send message

Receive message

HoloLens in healthcare

Solution

Search data from HealthVault

HoloLens and live streaming

Solution

Setting up media services for live streaming

Consume live streaming within holographic application

Handling secure keys with Azure Key Vault

Summary

Microsoft HoloLens in Enterprise

Microsoft HoloLens Commercial Suite

HoloLens device management using Microsoft Intune

Manual registration

Automatic registration

Installing applications on HoloLens

Device Portal for application deployment

Business Store for application deployment

Summary