Why Apple’s Vision Pro headset might actually take AR mainstream

by Scott Montgomerie | Jul 26, 2023 | Blog, Export

And how enterprise will make it happen.

Apple’s new headset is a massive step forward for the AR industry. As Apple has done with the iPhone and Apple Watch, the company has revolutionized how we interact with our world digitally. 

Why very few companies are positioned to make augmented reality devices ubiquitous

Augmented reality (AR) has been challenging for hardware companies to build well, because the technical challenges of producing a device require sophisticated vertical integration techniques. To build a great experience, you need tight integration of the cameras, silicon, memory, displays, form factor and UX. Alongside Microsoft and Google, Apple is one of the only companies with the degree of vertical integration capable of producing such a complex device as an AR headset. 

However, Apple stands apart in one respect — a laser focus on user experience. AR glasses require a plethora of new technologies to work, but technology is useless unless it’s usable by humans. Apple’s complete vertical integration across these many complex (and ground-breaking) technologies, combined with its ability to use these technologies to focus on the user experience, is what will finally bring AR into the mainstream.
While Qualcomm is trying to do something similar for the AR ecosystem to what Android did for smartphones, they have vertical integration for most of the stack — but not all of it. Qualcomm’s Snapdragon Spaces platform vertically integrates the silicon, memory and displays, but they’re relying on their OEMs for the form factor and fighting with Android’s capabilities on the UX. A similar dynamic played out years ago with Google: At first, Google allowed OEMs to produce their hardware but quickly realized that in order to adequately compete with Apple, they needed to vertically integrate to produce the Pixel, and it wasn’t until many years after Android launched that its user experience finally caught up to the iPhone.

What sets the Vision Pro apart

But let’s get back to the Apple Vision Pro. I’m cautiously optimistic that this is the device we’ve been waiting for. The early reviews indicate that the passthrough AR is almost seamless, and with 23 million pixels and 12 milliseconds latency, that certainly seems plausible. I’m also optimistic they’ve solved the UX with highly fine-tuned gestures. Previous hand gesture implementations resulted in “gorilla arm” or inaccuracy and a frustrating experience, but early reviews of Vision Pro state that it finally nails “lazy” gestures. If the combined eye tracking/hand tracking mechanism works, it will be revelatory.

Adoption depends on enterprises using augmented reality for training

Ultimately, what this platform really needs is a killer app. Is watching movies in 4K ultra HD a killer app? Probably not. Is immersive FaceTime? Probably not. But could I see myself wearing these on a plane to watch a movie or as a second screen? Yes, definitely. And although I’m not a gamer, I could see how one might be able to have an amazing gaming experience wearing the Vision Pro.

Altogether, these consumer use cases make it almost compelling for someone with enough disposable income to buy a Vision Pro. But the true value lies in the enterprise. HoloLens 1 was focused on consumers, and quickly HoloLens 2 pivoted to an enterprise play. Magic Leap, same thing. Qualcomm, same thing. Every one of these companies is attracted to the giant consumer market, and yet penetration for XR will only succeed with enterprise first.

This brings me back to the killer app. Every major newspaper has written something about Apple needing a killer app. But none of the use cases showcased in the keynote is it.

What is? Training — specifically, enterprise training.

Companies continue to struggle with skills gaps. In a survey of more than 39,000 global companies, The Manpower Group found that 77% of employers report difficulty filling roles — which is a 17-year high. And with technology shifting faster than workforces can adapt, as well as workers changing jobs too quickly to become experts, companies are short on the critical knowledge and skills that make up their DNA. 

But AR technology allows companies to “create instant experts.” The ability to see something in your native environment, in 3D — to walk around it and experience it and interact with it — that is the killer app because it solves a real business problem, really well. Scope AR’s Worklink customers routinely see 50% better retention, higher job satisfaction, and a plethora of other results using this technology.

Why the cost of the Vision Pro is justified (for now)

So, why would an organization be willing to shell out $3500 for an Apple Vision Pro specifically, rather than $499 for a Meta Quest Pro? 

A number of issues have so far hampered the adoption of VR: VR training has limited use cases, and it’s difficult to scale; and the uncanny valley phenomenon means that you need to replicate an environment very well to ensure the user is comfortable. Combine that with the social stigma and isolating experience, and it becomes clear why VR has experienced lackluster adoption. 

But the Vision Pro has gone to great lengths to solve these issues with two-way passthrough video. The presence of the eye passthrough feature is evidence enough of this. The cost to the bill of materials for this feature is astounding — two cameras facing the eyes, the equivalent of two iPhone displays externally showing the eyes, not to mention the computing power you’re allocating to render the user’s eyes. That alone must account for $500-1000 of the price. Apple likely believed that the social stigma and isolation problem was worth that cost to address (although I predict that those features will be dropped in future versions). But if doing so overcomes the social stigma and the uncanny valley to finally hit mainstream adoption, it will be well worth the effort and expense.

Our vision: Augmented reality might finally be ready to go mainstream

So, with the combination of tepid consumer use cases and a strong focus on the device as a workforce-scale training utility, is it possible that this is finally the AR device we’ve been waiting for? If organizations can be convinced to purchase them at scale for workforce training purposes, with the benefit being that the headsets can double as a productivity device similar to a laptop, then maybe. Just maybe.

A Surprising Solution to the Manufacturing Skills Gap: Augmented Reality

by Scott Montgomerie | Apr 11, 2023 | Blog, Export

As a manufacturing training professional, you might sometimes feel you’ve been asked to perform the impossible, like filling a growing gap of open positions with highly skilled workers — especially production workers, engineers, and tradespeople — quickly and cost-effectively. Job openings in manufacturing remain at historic high levels, with the US Census Bureau estimating 2.1 million unfilled positions in the sector by 2030 due to a skills shortage. Older workers are retiring at record rates, taking expert and institutional knowledge with them.  

Traditional training strategies are increasingly falling short in effectiveness, usage, and the ability to measure success. According to the 2022 Training Industry Report, the highest priorities for training are increasing program effectiveness (32 percent vs. 31 percent last year); increasing learner usage (21 percent vs. 19 percent last year); and measuring the impact of training programs (21 percent vs. 17 percent last year). 

Maybe it’s time for a change. A recent study by Deloitte hits the nail on the head: “Overall, training models for construction and manufacturing are not in sync with the twenty-first century needs of these industries, contributing to growing worker shortages.” One of the solutions Deloitte recommends is delivering “new models of learning…that leverage technology to help workers develop the new mix of skills required in the age of automation.”

A surprising solution: Augmented reality

You might be surprised to learn that augmented reality (AR) — a technology that combines digital and physical worlds, allowing interactions in real time — can address many of these concerns in a highly effective and efficient way. AR lets users overlay digital information onto the real world, creating an immersive experience that can enhance the learning process. Imagine if instead of having a manual, you could see how to fix a piece of equipment by simply pointing an iPad at it and following illustrated on-screen instructions overlayed directly on the image to repair it.

Using a phone, tablet, or headset, workers can manipulate computer-generated objects in a 3D space – whether in the classroom or on the shop floor – seeing everything as if it were in front of them. They can zoom inside objects, work alongside a remote teammate who sees what they see and do, or follow written or video instructions for repairs and other tasks. 

Here are seven ways in which AR can reduce the cost of training while boosting its effectiveness and speed.

Reduced equipment needs. One of the biggest misconceptions around AR is that it requires costly equipment. This is largely due to confusion with virtual reality (VR), in which users wear a headset that leaves them oblivious to and unable to see the real world. By contrast, augmented reality works within the real world – and it can operate on familiar devices such as smartphones and tablets as well as visors that project objects onto a user’s vision. In fact, best-in-class AR platforms are device-agnostic, making them far more flexible and future-ready than proprietary VR equipment. 

At the same time, AR’s ability to create digital versions of real-world objects can eliminate the need for even traditional training equipment. This could include machinery, specialized tools or expensive software programs. For example, instead of a company purchasing and maintaining expensive machinery for training purposes, employees can use AR to simulate the experience of using the machinery. This not only saves on equipment costs, but also reduces the risk of accidents during training.

Just-in-time training. The best time to learn new information or a new skill is when you need it most. A just-in-time training approach is one in which you create a library of e-learning courses that learners can complete at a time that best suits them. Often this will be when they have an immediate need for the information. AR is ideally suited for this type of training, and can even be used by seasoned professionals to tackle a new task. Need to repair an unfamiliar piece of equipment? Get written, video, or even remote live help while actually performing the work.  

Lower travel costs. Training programs often require employees to travel to classroom settings. This can be a significant expense for companies, especially if the training takes place in a different city or country. Similarly, an experienced professional will often need to go onsite to support technicians. With AR, training can take place remotely, eliminating the need for employees to travel. This not only saves on travel costs, but also saves time as employees can complete the training without having to take time off work to travel.

Faster learning. AR can also make training programs more cost-effective by speeding up the learning process. With AR, employees can learn at their own pace and in their own time. This means that they can complete the training more quickly, reducing the amount of time and resources required to get them up to speed. Additionally, AR can make training more engaging and interactive, which can improve learning outcomes.

Fewer training staff required. Another significant cost associated with training programs is staffing. Companies need to hire and train the trainers to effectively deliver training programs. AR can provide employees with a self-guided training experience, allowing them to learn on their own without the need for a trainer. This not only saves on staffing costs, but frees up trainers to focus on more complex tasks.

Improved retention. AR can make training programs more cost-effective by improving retention rates. Traditional training methods can be dry and dull, making it difficult for employees to retain the information they have learned. AR, however, provides a more engaging and interactive and visual experience, which can improve retention rates. When employees retain information more effectively, they’re less likely to require additional training in the future, reducing the overall cost of training programs.

Performance tracking. As a digital delivery system, AR allows employers to monitor employees’ progress and performance during training sessions in real-time. This can identify problem areas and the provision of immediate feedback, leading to better learning outcomes.

Augmented reality has the potential to revolutionize training programs, making them more cost-effective and efficient. With Scope AR, the time to train and certify your staff, customers, and technicians on new technologies can be dramatically reduced – by as much as 90% or more. 

ScopeAR’s WorkLink Platform gives workers the information they need to do their best work by providing expert guidance at any moment, and on any device. WorkLink is the first solution to combine AR work instructions and remote AR assistance into one enterprise-ready platform.

Scope AR revolutionized the way enterprises work and collaborate by offering a visual “knowledge base” solution that provides effective and efficient knowledge-sharing to conduct complex remote tasks, employee training, product and equipment assembly, maintenance and repair, field and customer support, and more.

To learn more about the benefits AR can bring to your organization, visit this quick guide to get started.

Building Scalable AR: The High Price of Going It Alone

by Scott Montgomerie | Mar 1, 2023 | Blog, Export

Recently, I had the opportunity to visit the Silicon Valley HQ of a prospective customer, a Fortune 500 enterprise technology conglomerate with a name most people would recognize. They had expressed an interest in adopting the Scope AR platform, and we walked into the meeting assuming that would be the main topic of conversation.

Most of the people at the meeting, however, weren’t the usual folks who make software buying decisions. Instead, I found myself sitting across from a group of the company’s own software engineers. For the next two hours, they asked detailed questions about how Scope AR had solved some of our toughest technical problems.

I knew right away what they were up to: They wanted to build their own AR solution from the ground up. The real purpose of the meeting, it turned out, was to pick my brain for the technical know-how they needed to do the job.

I could have walked out then and there, but I didn’t. I sat through the interrogation, and I actually answered most of their questions. 

Build your own: The alluring illusion of AR development

This might seem crazy: CEOs don’t earn their keep wandering the globe, spilling trade secrets to anyone who asks. But I’ve seen this before, and I know how this story ends. Sooner or later, I had a feeling we would reconnect—and this time, they would understand why so many enterprises choose to work with Scope AR rather than trying to build their own AR applications.

I also get why so many companies have to learn this lesson the hard way. Our team at Scope AR started down the same path, for many of the same reasons, a decade ago. We know why a DIY approach looks so alluring at first, and we also know why it’s easy to pour resources into what looks like a winning investment.

We also know why these projects, almost without exception, turn into the software engineering version of quicksand: the harder you work to stay afloat, the deeper you sink.

Going deeper: inside the AR development process

Let’s look more closely at why AR development looks at first like such a good fit for a DIY approach—and at why the truth is usually very different.

First, it’s important to understand that AR applications are a very real source of value across a wide range of industries and use cases. Scope AR customers, for example, include industry leaders in aerospace and defense, medical device development, knowledge and learning management, and advanced manufacturing, among others. They work with Scope AR because we give them a scalable, cost-effective AR platform that delivers the potential they already recognized in the technology.

In many cases, that romance with AR technology started with an internal proof of concept (PoC) project. These are intimate affairs by enterprise IT standards: funding for a couple of developers, often by way of a company’s technology innovation or R&D budget, and some enterprise-licensed seats for Unity, a real-time 3D development environment that has emerged as a de facto industry standard for building AR applications. Unity excels at prototyping cross-platform AR/VR applications. It has a solid toolset. It natively supports development in C#, which is a language most enterprise developers can pick up pretty easily. And while Unity isn’t free for enterprise use, it’s not very expensive, either.

Throw all of this into the enterprise IT blender, wait six months, and there’s a good chance you’ll get a proof of concept for an enterprise AR use case that’s popular with users and shows real value potential to the business, and even might interface with a corporation’s internal systems.

Before a successful PoC gets promoted to a production environment, however, it will need to do a number of things nobody asked it to do the first time around.

This is where reality starts intruding on the romance.

The many dimensions of running AR applications at scale

While there’s a lot to love about Unity as an AR prototyping environment, it’s a lot less lovable as a conduit for building scalable, production-ready AR applications. A team’s second time through the AR development process for a second use case will probably cost about as much and take about as long as the first one did. So will its third AR project . . . and its fourth.

What’s worse is that none of these applications are going to scale in any meaningful way. 

When we talk about scalability, the conversation can focus on a number of topics. Is your AR application configured and optimized for its intended deployment? Can your provisioning workflow scale to hundreds or even thousands of endpoints for use cases involving front line workers? Has your application been tested and validated across every platform and device type in your company’s inventory, including those being purchased in the future? What about security—especially when your AR application handles data subject to PCI or HIPAA compliance?

Then there’s the development side of the equation. Is your home-grown AR application going to incorporate tools and UI/UX elements that allow non-developers to adapt it for new use cases? Or will it turn into yet another ongoing project, and another source of technical debt, for an overworked IT department?

But for now, let’s focus here on just one facet of scalability: your content.

The content conundrum for do-it-yourself AR: one company’s story

Most enterprises that adopt AR applications already manage a significant amount of content. And in one way or another, content will play a decisive role in the success or failure of an AR project.

One of the most vivid examples I can think of involves a global aerospace firm whose name you might recognize. This firm happens to be a prospective Scope AR client; our conversations over the past several months included a fascinating deep dive into the company’s technical documentation library. This content serves a mission-critical purpose: documenting every aspect of the company’s aircraft assembly, testing, maintenance, and repair processes, right down to the smallest individual components. 

This company also happens to maintain a custom-built AR platform designed to help employees learn and perform a wide range of product assembly and repair tasks. Importing and integrating the company’s technical content will be a critical step towards making the system useful to its assembly-line and service teams—and to getting value from its AR development investments. 

Unfortunately, that’s exactly where the company is running into one of its biggest AR development challenges: most of its technical documentation currently resides in a legacy product lifecycle management (PLM) system. The vendor of that system had already confirmed that it could facilitate the process of accessing, converting, and integrating that content with the company’s custom-built AR application—but only at a cost that even one of the world’s biggest aerospace companies found prohibitively expensive.

This story will have a happy ending: Scope AR has already created a tool that can perform exactly the same task: moving relevant content out of a legacy PLM system and into the Scope AR platform, where it will be far more useful and accessible to front-line workers. It’s just one of many similar ways that we leverage some very hard-earned lessons about building and scaling AR solutions—allowing customers to enjoy the benefits minus the cost, complexity, and pain that comes with any do-it-yourself AR effort.

Here’s the thing. There are countless ways for content to add cost, complexity, and risk to an in-house AR project. Most organizations don’t have just one content repository: they maintain multiple data warehouses, database systems, content management systems, and repositories for audiovisual, geospatial, and other specialized data types. Each will require a separate integration workflow and bring its own concerns over versioning, data security, compliance, and so on.

And then we still have to talk about the complexities of converting dozens of data formats into something that your AR application can ingest, manipulate, and display to end users. It’s an exacting process that will require endless experiments, false starts, and expensive mistakes—along with the ever-present risk that you might simply run out of time, money, or developer bandwidth before you see one dollar of ROI.

The best mistakes are the ones you never have to make

I mentioned earlier that I have watched, over and over, as companies try to scale AR projects that won rave reviews in their PoC stage. And nearly 100 percent of the time, I watch as those projects implode under the weight of technical debt, or stumble on issues integrating content and other enterprise applications, or attempt to scale by soaking up IT resources like a gigantic sponge.

And of course, our team at Scope AR, myself included, got the best seats in the house for one of these contests—because we were the ones making the mistakes. We learned hard lessons about scaling and succeeding with enterprise AR application development, and we slowly built the tools and methodologies required to deal with them, and built the most scalable platform for the many many use cases we’ve encountered over the years.

It was a slow, painful, often expensive process. And the only reason it made sense for us to solve these problems is because it’s how Scope AR creates value for our customers.

Like I said, I get it: Some companies only learn from the mistakes they make themselves. But when you’re talking about AR applications, trying to stay with a go-it-alone approach can be especially painful and costly.  And with Scope AR, we’re doing everything we can to give enterprises a better way to work with AR applications.

Interested in building AR applications more easily? Reach out to us to get started.                                                                                                                                                                              

AR Training and the “Mean Time to Mastery”

by Aaron DeYonker | Feb 27, 2023 | Blog, Export

As we all know, technology is moving at a breathtaking pace.  A mere 10 years ago, Facebook launched its mobile app (in earnest) – a milestone many believe marks the true shift to the age of mobile computing, the largest digital transformation since the rise of the Internet in the 90s.  Just as impactful as the mobile digital transformation, we are entering the decade of spatial computing.  Everything from self-driving cars to advanced autonomous systems will rely on spatial computing, the theoretical apotheosis of industrial and digital transformations.

As we sit on the precipice of our next big digital step forward, we need to ask ourselves how to prepare for an explosion of complexity in our daily lives. 

Spatial computing, particularly Augmented Reality, has the potential to force-multiply the power of human intelligence.  In many respects, AR is our best way to contend with the inherent complexity of our future.  And while AR has limitless applications, one area it will help meet the challenges of our complex future is through training and knowledge acquisition.

AR has already demonstrated huge benefits to employee training.  Inherently on demand, AR training can reduce costs significantly without the need to travel to seminars and classes, especially if implemented on existing hardware platforms like mobile phones and tablets. It can also help job training where safety might be a concern.  But most importantly, AR training provides more effective learning and retention by removing cognitive barriers, creating stronger emotional connection to content, and narrowing focus to core subject materials in ways that were not previously possible at scale.

In his book, UX for XR (Design Thinking), Cornel Hillman cites studies that show AR participants learn an average of “four times faster, with 3.5 times higher emotional connection while being 2.5 times more confident and four times more focused.”  What this study highlights goes beyond mere rote memorization.  Emotional connection and focus are primary indicators of true knowledge retention.  In situ spatial interaction, gestures and speech all elevate digital engagement in ways that have profound impact on cognition.

Bloom’s Taxonomy of Learning is an interesting framework to consider as we evaluate the impact of augmented reality on knowledge acquisition.  Originally published as the ‘Taxonomy of Educational Objectives’ in 1956 by Benjamin Bloom, the framework has undergone various iterations to get it to its current state.  The framework is meant to help structure objectives and learning goals, organize the objectives in a way that makes sense, and ensure that instruction and assessment are aligned with objectives.  Bloom’s work has served as a cornerstone in structuring curriculum at all levels of education from pre-K to post graduate and professional instruction.

With Bloom’s Taxonomy, we can evaluate the impact of training with the use of AR.  One could argue that AR helps redistribute cognitive load from the base levels of Bloom’s taxonomy to the higher levels of cognition focused on evaluation, adaptive thinking, judgment, and decision-making.  We can also look at practical applications in training around smart systems and medical equipment to help illustrate the value of AR with respect to Bloom’s Taxonomy.

At its base, we find ‘Remember’, or the ability to recall facts and concepts.   Any deficiency in this base level of knowledge impedes efficiency in every level above it.  Much of our superficial and factual knowledge has been offloaded to digital sources like Google and Wikipedia.  The ability to project factual knowledge into our field of vision, as needed, will supercharge our ability to understand, apply, analyze and evaluate problems and solutions like never before.  Rather than ‘right-click -> Look up’ on your 2D screen, a simple gesture or voice command can pull up relevant information in spatial proximity to your focused gaze, accelerating knowledge application in a subtle but paradigm shifting way.

Beyond the level of ‘remembering’ in Bloom’s Taxonomy, we progress to higher levels of cognition.  Because AR projects information into the real world, and because AR experiences can optimize around gaze and gestures, learning can accelerate at all levels of the taxonomy with supercharged focus and engagement.  Removing a litany of impediments related to focus and context switching will have profound impact on engagement, comprehension and retention.  At the ‘Understand’ level of the taxonomy, we want the trainee to identify, locate, select, and classify.  At the ‘Apply’ level of the taxonomy, we want the trainee to solve, demonstrate and operate to reflect competency from the lower levels of the hierarchy.  Again, in situ information projected into the real world against real objects fast-tracks this level of comprehension. 

Imagine the simple case of AR training for maintenance of a robotic arm.  The goal is to diagnose and fix a problem related to restricted movement of the equipment.  With a simple voice command, device instrumentation data can appear overlaid on top of the arm with various status readings and error codes. A voice command can bring up details for a particular error code.  AR has helped at the “Remember” level.  The AR instruction could then walk the trainee through a procedure, step by step, while she is working with the physical equipment.  Having all of this information overlaid in context accelerates the trainees ability to identify and classify a mechanical issue with the robotic arm.  AR has helped the trainee ‘Understand’ more efficiently than ever before.

Because augmented reality accelerates the ability to recall facts and understand basic concepts in novel and effective ways, cognitive load shifts away from areas of the brain devoted to memory retention and moves toward the area of the brain responsible for critical thinking.  With the limitless simulations possible through AR, the trainee is engaged at a level that promotes emotional connection with the content, which then fosters deeper retention of the material.  Quick visual feedback loops at the point of execution accelerates richer application, analysis and evaluation – concepts at the higher end of Bloom’s hierarchy.  

As an example of real-world application of AR integrated into an overall learning experience, consider the assembly of test kits for medical labs.  These kits require careful placement of various test tubes and solution vials into precise locations depending on a particular SKU.  AR content can help with repetitive selection and placement of the parts into the correct position in the kit.  Once the trainee achieves a certain level of comfort in remembering and understanding the general kit construction, AR can start introducing variants of the kit on the fly, including incorrect part numbers and assembly ordering.  The trainee is now engaged at the application, analysis and evaluation levels of our hierarchy.  The number of hours required for a trainer to prepare and execute these scenarios in the real world makes this type of dynamic training nearly impossible without AR.

As we conclude our application of Bloom’s Taxonomy to AR training, we consider the top of the hierarchy: Create.  This level of the taxonomy reflects true mastery of subject matter.  As we look at the next technology wave, we see a human capital resource gap.  The demand for mastery cannot be met with traditional forms of knowledge acquisition.  Curriculum development and execution, especially for the medical devices and smart systems of our future, faces too many impediments to be truly effective without augmented reality.  Costs of training prep, lack of access to heavy equipment and safety concerns present practical problems that AR can solve.  Beyond cost and logistics, AR offers a revolutionary opportunity to accelerate mastery.  Given the growing complexity of technology, we need the AR revolution to help get us there.

References

Armstrong, P. (2010). Bloom’s Taxonomy. Vanderbilt University Center for Teaching. Retrieved on September, 2021 from https://cft.vanderbilt.edu/guides-sub-pages/blooms-taxonomy/