[MUSIC] Let's now turn our attention from control and planning to sensing and estimation. In everything you have seen thus far we've conducted experiments with the quadrotor in a laboratory environment. In this laboratory environment we had motion capture cameras that are able to measure different features in the environment with precision and repeatability. Here we have reflective markers on the quadrotor, allowing the cameras to estimate the position of the quadrotor to within several millimeters, and the orientation of the quadrotor to within a fraction of a degree. These updates can be obtained at rates as high as 200 hertz. So, in a sense, what you have is an indoor GPS like system, but with a degree of precision that's several orders of magnitude higher, and the ability to get this information much faster. But we like to operate in environments like this, where there are no motion capture camera systems. Robots must have the ability to estimate their state, their position, the orientation, and the velocities. And they must be able to do it onboard. This is the problem of onboard state estimation. We can do this by equipping a robot with camera-like sensors. In this example, you have a laser scanner that allows the robot to measure distances to obstacles in the environment. IT's also equipped with a RGBD camera, and this is available with the Microsoft XBox system, it's called a Kinect. This sensor is able to project infrared patterns in the environment and then observe how those patterns get deformed by three-dimensional features. Both these sensors, the laser scanner and the Kinect, allow the robot to measure three-dimensional features in the environment. This allows the vehicle to operate in unstructured environments without a motion capture camera system. Here's another example of a vehicle that is able to operate indoors as well as outdoors. It has a GPS on the top. It also has two forward facing cameras and a downward facing camera that you cannot see in this picture. It also has a laser scanner. And it's equipped with an onboard inertial measurement unit. With all these sensors, it's able to track three-dimensional features in the environment. And use this information to track its own position and orientation as it flies through the environment. How does it do that? Well, the basic technique is called simultaneous localization and mapping, and it's sometimes also called structure from motion. Here's the basic problem. Imagine you have the vehicle at position x0. And imagine it's able to, at some point in time, measure features in the environment. f0, f1, f2, etc. Let's say at position x0 it measures the first three features f0, f1, and f2. After moving to a new position, x1, it measures a subset of those features, f1 and f2. It moves to another position, and now it measures a new feature, f3, which was not part of its feature set. At x3 it now measures four different features. Some overlapping with its existing feature set and some contributing to new features. So this goes on and on. The key idea is that you have a graph with two different types of edges. The first set of edges correspond to measurements made by onboard sensors, cameras, or laser scanners. The second set of edges has to do with movements that the robot has made. Based on its applied inputs, and based on the time elapsed, it's able to estimate how far it's moved. In other words, it's able to estimate the difference between x1 and x0, x2 and x1, x3 and x2, and so on. Each of these types of edges corresponds to information gleaned from sensors. This information is noisy. But at the end of the day you have a big graph with two types of edges. And you have equations that describe these edges. So you end up with a big optimization problem that you have to solve. If you're able to solve it, not only can you recover the positions of these features, but also the displacements from x0 to x1, x1 to x2, etc. So you're about to map the features as well as localize yourself as you fly through the environment. This is the simultaneous localization and mapping problem, also called SLAM. Now putting all of these together involves integration of information from different types of sensors. So on the left side you have GPS, laser scanners, pressure altimeters, stereo cameras, a downward camera, and an IMU. Each of these sensors gives you information that's of a different type. This information also comes at different rates. The fastest sensors usually are the IMUs, that provide data at 100 to 200 Hz, while GPS only works around 10 Hz. We combine information from all these sensors using a filter. And that filter obtains state estimates at 100 to 200 Hz, and this allows us to drive the controller that we discussed earlier. In the videos I showed you earlier, this state information was obtained by motion capture camera systems. In this picture, and in what follows we're going to get the same kind of information at similar rates but from onboard sensors. In addition to getting the state information, we're able to poll the information from the sensors to create a map and this is the SLAM problem. If you have a user that's interacting with the map, he or she can specify goals or intermediate weight points for the quadrotor. And guide the vehicle through a complex environment with a map that's being built by the robot without actually being in the same location as the robot. You will see this basic setup and the algorithms in operation in a set of indoor and outdoor experiments. In this video clip, the robot does not have any information about the interior of the building that it enters. But as you can see, when the vehicle enters the building, it starts building a map. The map is registered to positions and orientations that it also obtains using a SLAM-like algorithm. The vehicle successfully builds the map of this four-story building starting from the ground floor all the way up to the fourth floor. In this video, the vehicle is traversing a variety of environments. Indoors and outdoors, it goes in the shadow of tall buildings where it loses GPS, it goes under trees where it might not be able to use GPS. And in fact when you go inside buildings it might be too dark and cameras might not work the way you expect them to. On the top left you can see the legend that tells you based on the color of the trajectory segment what sensors are being used for state estimation. The ellipsoid being shown at the center of the screen is the robot's position, along with an estimate of its uncertainty. The ellipsoid has been scaled up by a factor of 100 in order to help you visualize the actual uncertainty. Here you see examples of high resolution maps that can be built by robots like this. It's flying through an indoor environment. And as it enters the area at the bottom, which is where our lab is, you will see its ability to build maps with five centimeter resolution. You can see the three dimensional structure being mapped in front of you. You can also see the texture of the objects that it sees being laid down in front of you. I want to remind you of a basic fact. Robots like this generally burn roughly 200 watts per kilo of payload. The laser scanner that this robot carries weighs 370 grams. The stereo camera rig weighs about 80 grams. The Intel processor and board weigh about 220 grams. Each of these modules, the processors and the sensors, contribute the total weight of the platform and the total amount of power being consumed. This is one of the drawbacks when you go to onboard state estimation. You have to make sure the vehicle has requisite sensors and the capability for processing the information. And that drives up the weight of the platform. This platform weighs 1.75 kilos. The larger vehicles that we build are definitely more capable because they have better sensors and better processors. They can also fly longer missions because they can carry bigger batteries. However, they lack the advantages that smaller vehicles have. Smaller vehicles can navigate in more complex indoor environments. And they're also inherently more agile and maneuverable.
