Welcome to this course on augmented virtual and mixed reality, specifically how to develop for these kinds of exciting new technologies using Webex AR, Unity, and Unreal. So this is really the course that is looking at these technologies from a more technological angle. This is only one of the courses we have as part of the xMOOC specialization. It is the third course, to be precise. It is the one that kind of assumes you have the foundational knowledge that I otherwise established in the first course. Really about the concepts and technologies in both VR and AR, and then also the key issues. And so in this course here, specifically, we're going to learn how to develop applications. And we're going to learn about other principles. And I'm going to walk over the main ingredients of the course. Some of the highlights. Let's look at this course together. And I hope you will join me in this journey off developing AR, VR, and mixed reality applications. This course is really structured into four main modules, starting from development approaches. Like really, what are the different ways of creating AR, VR experiences? What are the paths to being an XR creator, given your background and experience. And an introduction and overview off Webex AR, Unity, and Unreal. That's something you will find in here. Also making the jump from 2D to 3D to really set the foundation for what is to come. We will then focus on VR development. We're starting from design. We're learning about navigation and menus in VR. And we're also going to learn about object selection and manipulation. We're going to learn about creating basic, and then immersive VR experiences, that do require more effort and more technology. We will also then learn about AR development. We're going to look specifically at marker-based, and also markless approaches. So marker-based, usually those are some of the popular markers, Vuforia, and this is AR Toolkit, and AR.js, so part of Webex AR. We're going to do all these things, and we are going to in the end cover what I call here special topics. So that would be advanced techniques. I'm going to talk about XR research. And I'm going to also show a little bit off the making off off this MOOC. Because it'll showcase a few techniques that I've used to actually bring these technologies closer to learners in remote settings. I hope I was successful. I hope you enjoy the continent I put together. Trust me, it was a lot of work. And I hope it is visible and that you enjoy the content here, and that you feel like you're learning a lot. That would be my dream. So in terms of learning, we then have quizzes. So you have to assess your learning and making sure, well, for you to make sure that you feel like you're on track. And so we have quizzes after each of the main modules. We also have a lot of content as part of the Honors track. Specifically, exercises that help you, really, develop a systematic approach to developing 3D, VR, and then AR scenes. We have a few major examples throughout this MOOC. One is a case study focused on virtuality design. This one is modeled after an example that I started creating in one of my online office hours that I streamed on Twitch. It was modeled after the Detroit Zoo, and it became really this project that was then developed further by one of my students, Cara Daly. She really helped create bringing this to life and adding all these things, and making it a night safari, including a petting and feeding area, which we can only do in virtual reality. So in the Honors track you will then create your own 3D scene. Could look like something like this initially, and then you could take it further and make it a virtual reality scene. So [LAUGH] here we have our giraffe. You probably don't notice, but one of the main things I adjusted here is scale, obviously provided an environment. And also have support for virtuality controllers. So we'll learn all about how to handle interaction and navigation in virtuality. We're also going to learn about turning VR scenes into AR experiences. So here you, for example, see how I bring a, well, maybe life sized giraffe into my living room using a markerless AR approach. And I'm going to show you a marker-based version off this as well. So here's a Giraffe and so here is the Marker based version. So obviously what we learn about is differences when it comes to marker based and Marker AR Scene. How tracking works, how we should design differently, how interactions work differently. So throughout the course, you will learn about the extra process, we will learn about not so much need finding, brainstorming, story boarding and prototyping. These are key ingredients of the exit process, but they are in the focus of the second course is part of the specialization. This course focuses on development and testing. It does really go deep there, and deployment and analytics is really also something that is kind of like covered in the second course. So the way you should look at this is this moot really goes deep here, and it's embedded into the larger process that is ultimately covered by this entire specialization. I also care really about my students, and I always think about different ways for them to really get the most out of this content. So how do we make this content accessible? And a lot of us may have a web design development background. Maybe not even some of us may already be familiar with game design and development. So then tools like unity and unreal feel like a good fit. And some of us may have previously designed mobile applications are. And now we are very interested in How do we translate this knowledge? So for the web developers, we're going to talk in this movie. I have lots of examples in the buybacks, our space. I cover all these platforms, we do mention native SDKs, but this is not really the focus of this movie. So we cover WebEx our unity and unreal for each of these WebEx our unit on riel. We're going to talk about popular platforms like a frame energia, steam VRMRT cable for your air foundation XR interaction. So these are popular examples. We also learn about popular AR and VR devices. So here are listed VR devices here are just a our devices, including the webcam. If you think of the webcam as an air device, given that it is the means to do market based AR and, we'll also learn about how to assess some of these technologies, how to map these two kids to platforms and to the specific devices. And then with that knowledge, we're really going to learn about the process. And in this course, we're trying to make this jump into three D so that we can then so as quickly as possible, actually in week one, so that we can then actually explore learning about VR and AR by doing projects in that space, at least this part of the honest track and then topics wise, I cover all the important key elements that help us create immersive. VR experiences and then we learn also about how to do marker based markeless AR experiences for handheld and then also had worn. In the first module, we're going to spend quite a bit of time to make the jump from two D to three D. So we learn about coordinate systems. We learned about how rendering works a perspective camera. The fact that light is required, the fact that there are three access or x, y and z. We're going to learn about object transforms. We're going to learn about all these concepts and principles in a way that is too diagnostic, and then you will be given the opportunity to obviously learn it in your tool of choice. So, we will cover things like the perspective camera so that you know how and three D C is rendered, how virtual cameras work. And then how that knowledge translates into VR and to AR. You kind of start with all the key things we're going to learn about Ray casting and hit testing as a way of determining where people are pointing, enabling objects, election enabling object manipulation in VR, you kind of learn about different kinds of menu designs here and show, here I show one of the latest examples a hand menu. It's pretty cool, implemented on the Oculus quest so it's in the Oculus browser has done in WebEx are we're going to learn about passed through a little bit and how hand tracking is now possible and becoming a standard ingredient in VR devices. We focus most of our attention on hand held AR, but we're also going to learn about had worn a are and the additional capabilities that device like the Holland to I would give us. So here, for example, I'll show the special mesh in real time as it is recognized and how it is used for inclusion and how it can be used for physics so that these cubes can then actually really land on, for example, my table there. So devices why is obviously we will cover a lot of the devices and so but it's too many out there to go over each of them. You see some on the background, so I do work. Work, primarily in this course, I work primarily with the HoloLens 2 and the Oculus Rift S, but I realized that not everybody has access to these devices and that is fine. We will also work a lot with cardboard. So that would require a mobile phone going in there. And that's something we will do and then we can actually destroy cardboard to have the camera exposed and then we can also do augmented reality with cardboard. We can also obviously use some more specialized cardboard AR headsets like the hollow kid. If you haven't seen this, I like this one it's actually pretty good and we're going to learn more about AR, we have a whole case study on AR contributed by one of my students Shader Veteran. And this is a Kepler's laws of planetary motion example, where we're using marker based AR and I think in a really cool way because we re making use of the paper. We also talk about the transition from marker based to marker ALS and vice versa. So for augmented reality that changes quite a bit in terms of the interaction, in terms of the user experience. Users have to care less or pay less attention to where the camera is and go more intuitively about the experience. For example, we're going to go back in this course to going really low level sometimes to really understand how does this stuff really work behind the scenes? So how does markup based tracking actually work? I'll walk you through the pipeline. We're going to also learn about obviously some of the latest toolkits out there. So on top of Air Corps and Air Corps in unity, you have a our foundation here. I show an example of how they read the environment, and so that is something that we learn about how Michael SAR works. We also going to learn about the affordances of AR technology. Obviously challenges of blending in with the environment. One of the things that used to be really hard is rendering occlusion here, I place a few objects behind this chair, and they are indeed occluded and this is possible with the depth API on AR core. And then finally some of the latest featuresreally allow us to make sense of the scene and the content that the device sees. So here I show a virtual Vets and, it should be accrued by my hand. And then when I'm behind it, it should actually occlude my hand. And so that is very interesting. And ultimately in the end, I want to show you one of the cooler AR experiences that I have tried out, right here in my studio, which is mission AR it's an unreal project for the HoloLens 2. And it was really, it is actually really quite immersive, even though we would normally speak more of immersiveness when it comes to virtual reality, but this is really quite well done for Augmented Reality. And also at the end cover some more advanced topics like procedural generation, which I think is pretty cool for more open ended virtual reality applications. We also going to learn more about advanced topics in Augmented Reality, like 3D Reconstruction, which then creates a spatial mesh, reading the environment. And then also doing object recognition like this table here, which requires semantic classification and labels, so that we actually know we're looking at a table. Once we do, we can actually do interesting things and these ideas are now becoming more or less a standard in the latest toolkits. Finally, I'm going to talk about XR Research. So we're going to explore a few topics, at least the ones that I think are popular and going to be interesting for the next few years. From an HCI, Human Computer Interaction perspective, I talked about skills, skills that I think students will need to participate in XR Research, which include obviously user experience and design. So human computer interaction, research skills, obviously AR VR technological skills and programming is one of those important ones. And then actually the ability to do research and because of the cutting edge or let's say really bleeding edge nature of AR VR technologies, this is really the part where all these skills come together, so a research approach to this whole field of XR is actually a very good approach. Finally, we're going to talk about good and bad questions to ask when it comes to XR Research. At least my opinion. And throughout this MOOC and all my MOOCs, we are going to emphasize, not always sufficiently addressed but at least emphasize really important design issues like social and ethical concerns as these technologies are becoming mainstream more pervasive and making it into our daily lives. To some extent, yes, really changing daily life. Accessibility in equity and equity is really a big issue. We will probably notice there is a lot also in this course just for practical reasons, such as you not have having access to, for example, the hololens toe or even some Markle S A R solutions like Air Kid and Air Corps or a virtual reality headset. Accessibility is a pretty big concern, and it is actually really poorly address so far. Finally, we are going to talk about privacy and security. But again, I would refer to the first course in this example specialization, which really emphasizes these issues and provides a platform for discussion about these issues. In this course, we are going to be so busy working with these technologies. I mean putting this on your head and giving it them oil and learning. Learning about how this works is really one thing, and it is fundamental And that is like where this course really sets the emphasis and the focus. But what does it mean that this device sees everything we do in the environment in which we use it, including people that maybe around and so it is important? While this course focuses on all these technological issues and how to get things running. We will just be glad if things are running, but it cannot end there, even though the course obviously is. That is the scope of the course. But please, then,add to your portfolio by also considering taking the other courses that are really important, focused on design, thinking and design doing and a lot of the issues. So let us get started in this course. Hopefully, it is an interesting course for you. It is a different approach to learning about these technologies by doing and really doing getting our hands really dirty this time doing X R.
