Space Navigation's Dark Secret: How We See the Unseen!

next lightning talk is going to be given by Eric Elias. He's a master student currently in our um aerospace controls lab with professor John How. And uh introducing I had to double check this. Um but Eric has three bachelor's degrees. Where are you Eric? Check me if this is correct. Um, so he's a triple major, EECS, physics, and economics. So, let's give him a warm welcome. Yeah, the triple major wasn't here, but that would have made it more impressive. Uh, hello, I'm Eric Elias. I'm a

secondyear grad student here in Aereroastro. Uh I am in the aerospace controls lab and also working with Dave Miller in the space systems lab. And today I'm going to talk to you about my work in navigation around unknown space objects using visible thermal image fusion. So here's a video of SpaceX's Dragon as

it's approaching the International Space Station. You can see that it's uh fully illuminated by the sun. Uh, but just a few minutes later, it goes into Earth's shadow and it goes into eclipse and a normal camera isn't able to see it anymore other than for the navigation lights which aren't on most objects in space. Uh, and it turns out that this period of eclipse in low Earth orbit can last for up to 40% of each orbit. Uh, so imagine this spacecraft was some unknown object you're trying trying to inspect uh with an inspector spacecraft. Uh the

question I'm looking at is how do you figure out your own inspector's relative position to this unknown object and how do you do that in these Eclipse conditions? Uh and this has like you may be wondering how to navigate around an unknown object if you don't know how it looks. Um this has already been done on spheres which Dave talked about which flew on the ISS but uh the assumptions were that illumination conditions were pretty consistent. Uh in reality that's not true as I showed you. Uh so the

first thing these navigation algorithms do is it first does feature tracking which is shown on the left here where high contrast corners are tracked from frame to frame. Uh and then those features are estimated in 3D space which are the white points on the right. Uh and the red line represents the estimated trajectory of the of the uh inspector in the navigation algorithm. And if I play this video here, you can see that um eclipse happens pretty quickly and the map which are those white points kind of disappear during eclipse and this red trajectory estimate which is supposed to perfectly repeat itself accumulates this large error because of the eclipse. Uh so I'm looking at how do we reliably estimate this camera's relative pose about around this unknown target even during these eclipse conditions. So, I've already hopefully shown you how a visible camera is not very usable in these conditions.

Uh, you may be wondering about LAR, and LAR is illumination robust, but they're often very bulky and use a lot of power. So, they're not uh able to be used on many systems because of these constraints. Um, so what I look about

here, what I look at here is how to use a thermal camera. And you can see that it is it is illumination robust. Uh but it it comes with its own challenges. It's lower resolution than the visible camera. It has a lot more noise and it also has a lot less texture. Um but what

we can do is combine the visible and the thermal imagery into this fused composite which takes advantage of the best aspects of each camera and overcomes the challenges that the visible and the thermal have. Um, so I've already shown you this first

scenario, but our goal is to run these three different types of sensors through the navigation algorithm and compare how it performs. So we first run five more scenarios. Each of these is running uh real orbital dynamics. Uh each has its own unique set of illumination and trajectories. And so running these six scenarios through our navigation algorithm, uh, averaging out the the position error of the inspector over all of these scenarios, we see how much better the fused imagery is. Um, so

it's, you know, almost an order of magnitude better navigation performance. Um, and that just shows how, you know, the fused is able to take the texture and the high resolution from the visible camera while also retaining the illumination robustness from the thermal camera and which is why we see these uh these great results here. So, now that

I've laid the foundation for uh more robust and reliable navigation around these unknown space objects, uh what I'm looking at currently is how to improve that error even further. Um turns out with a thermal camera, uh just by thresholding these pixel values, you can get this silhouette of this target you're looking at. And so if we compare the silhouette in the current frame against all the past silhouettes in an efficient and a fast way that can run in real real time, we can determine when they match. And so these silhouettes match when the uh position and the attitude of the inspector spacecraft is the same between those two points in time. If we can bring those two matches into our navigation algorithm, then we can even further drastically reduce the navigation errors that we see. And uh I'm currently working on quantifying these results to show how much uh the performance actually improves. But overall, what you've seen is

illumination in space is a huge challenge. Uh visible cameras are not able to handle this robustly. Liar is able to do that, but it comes with its own, you know, bulkiness and power limitations. But thermal cameras are robust and when coupled with visible cameras, that gives us a really reliable way uh on smaller platforms to do this kind of navigation. So in order to you know be aware of

what's happening in space in Earth orbit and beyond we have to be able to depend on multiple types of sensors and multisspectral imagery to do that for things like on orbit servicing manufacturing to uh active debris removal to keep our our orbit clean and even outside of Earth's sphere of influence to do asteroid exploration. Thank you.