NASA ARSET_ Introduction to Full-Waveform Lidar

Speaker 1: Hi everyone, Welcome to our RSET training series spaceborne lightar for monitoring vegetation, structure and biomass using JEDI. Today is part one of our training series, Introduction to full waveform LIDAR.

Speaker 1: My name is Savannah Cooley. I'm a researcher at NASA, AIMES research center with the Bay Area Environmental Research Institute in Mountain View, California, and I'm a trainer with the URSET Ecological Conservation Team. Before we begin today's training, I'll cite a few quick words about the RSET program. The NASA Applied Remote Sensing Training or RSET program, provides cost free training on the use of remote sensing observations, analysis methods, and tools. We provide training in several thematic areas, including agriculture, climate and resilience, disasters, ecological conservation, health and air quality, water resources, and wildland fires.

Speaker 1: Our SET provides trainings both online or in person. Our online trainings are delivered in two formats, live and instructor led like today's training, or asynchronous and self pacede like NASA's free and accessible data. All of our trainings are offered at no cost. We try to offer trainings in more than one language whenever we can, and we only use no cost and open source software and data. We offer our trainings at a range of levels, so you can find a training series that fits your level of experience and interest.

Speaker 1: Please visit us at our website to learn more. We'll start today by giving an overview of this training series. Spaceborne lightar for monitoring vegetation, structure and biomass using JEDI. One fundamental question that many of us are interested in is thinking about forest canopies and what's happening specifically beneath the canopy, which from many traditional satellites we don't actually know. Because a lot of optical remote sensing instruments can tell us about, for instance, greenness of the canopy, but can't actually tell us any structural information or information about the understory.

Speaker 1: Spaceborn Lighter can do this. Spaceborn Lighter instruments such as NASA's Global Ecosystem Dynamics Investigation known as JEDI, measures forest structure across over twenty billion locations on Earth today. You'll learn to use this data. By the end of this training, you'll be able to first identify the fundamental physical principles of light our remote sensing, including the uses and limitations for terrestrial applications. Second, access and retrieve full wave form light our data from public massive repositories to visualize vegetation structure.

Speaker 1: Third, identify key JEDI products including elevation, canopy height, vegetation structure metrics, and biomass across different processing levels and spatial resolutions. Fourth, access and visualized JEDI data related to elevation, height, vegetation structure, and biomass for specific areas of interest using openly available tools. Fifth, utilize JEDI standard biomass products at footprint and one kilometer skills to map forest resources. Sixth access the open source OBIE Application Programming Interface API to generate estimates of biomass change in areas of interest.

Speaker 1: Finally, seventh identify how obi wan can be used to create baseline scenarios for forest carbon accounting projects. The only prerequisite for this intermediate level course is the our set training on Fundamentals of Remote Sensing. Here we provide three other suggested trainings, as well as a resource from the Land Processes doc which has a number of data tutorials for JEDI and the other three trainings are our set trainings. All of them are available on our training web page. There are three parts in this training.

Speaker 1: In Part one, today we'll introduce the full waveform lighter with theoretical background and two hands on exercises using Jedi Level one and level two data. Parts two and three will cover additional spaceborne lighter products from the Jedi miss as well as by a mass estimation. The homework opens on the last day of the training on November six, twenty twenty five. It is due two weeks later on November twentieth. A certificate of completion will be awarded to participants who attend all live sessions and complete the homework assignment before the given due date, and with that will begin Part one, Introduction to full Waveform Lightar.

Speaker 1: We have two instructors for Part one, myself my name is Savannah Cooley, and our second trainer is Nayara Pinto, who is a research scientist in the Suborbital Radar Science and Engineering Section of NASA Jet Propulsion Laboratory. I would also like to acknowledge my fellow colleagues in the ARSET Ecological Conservation Team who have supported the creation of this training, Sativa Cruz, Juan Torres Perez and Justini. By the end of the session, participants will be able to first recognize essential physical principles of light our remote sensing.

Speaker 1: Second, identify different types of light our sensors. Third, assess the uses and limitations of full waveform light our technology for terrestrial ecosystem management and conservation applications. Fourth, access public NASA repositories to retrieve full waveform light our observations to visualize and quantify vegetation structure. If you have questions during today's training, you can put them in the Q and A box within WebEx at any time. At the end of the training, we will try to address as many questions as we can and will post these questions and written answers to the training page within a week after the training.

Speaker 1: Right, let's dive into section one, which focuses on the fundamentals of LIGHTAR remote sensing. Remote sensing systems can be classified into two fundamental categories, passive and active sensors. Passive sensors detect natural electromagnetic radiation reflected or emitted from Earth's surface, such as sunlight reflected from vegetation or thermal radiation from the ground. Examples include optical satellites like Lansat and thermal infrared sensors. In contrast, active sensors generate their own electromagnetic energy and measure the signal returned from the target.

Speaker 1: LIGHTAR, which is short for Light Detection and Ranging, is an active remote sensing technology that uses laser pulses to measure distances and properties of objects and surfaces. LIGHTER operates on the principle of laser altimetry, measuring the time it takes for a laser pulse to travel from the sensor to a target and back. The full equation governing light ART distance measurement is that distance equals the speed of light, which is a constant multiplied by the time of flight divided by two. The system emits short laser pulses, typically in the near infrared spectrum at one thy sixty four nanometers, and precisely measures the time delay of the return signal.

Speaker 1: This time of flight measurement, combined with the sensor's position and orientation, allows for the calculation of three dimensional coordinates of the reflecting surface. Lighter's ability to penetrate vegetation canopies and provide three dimensional structure information makes it sensitive to several biophysical parameters, elevation and topography is the first that I'll talk about where lighter can create high resolution digital elevation models referred to often as dems. By measuring ground returns beneath vegetation canopies.

Speaker 1: The difference between the highest vegetation return and the ground surface provides direct measurements of canopy height, which is another key parameter often used for forest biomass estimation. Lighter captures the three dimensional distribution of vegetation elements, revealing canopy layers, understory density and structural complexity that cannot be observed with passive optical sensors. These structural measurements support forest reapplications, wildfire applications, and conservation applications.

Speaker 1: Lighters focused on terrestrial studies operate at either green or near infrared wavelengths. The frequencies of these wavelengths, unlike microwave frequencies, cannot penetrate through clouds. Lighter for terrestrial applications uses the near infrared lasers like Jedi's one thousand, sixty four nanimeter laser because it reflects strongly off of vegetation. In contrast, bathymetric LIGHTAR uses a green light laser because that wavelength effectively penetrates water. On this side, you can see a list of some of the available lightar instruments that include both spaceborne and airborne instruments firing in the near infrared as well as in the green portion of the electromagnetic spectrum.

Speaker 1: Light our systems can be deployed on various platforms, each offering distinct advantages. Ground based systems, known as terrestrial lightar provide extremely high resolution measurements up to the even millimeter accuracy for detailed studies of forest structure, archaeology, and engineering applications. These systems are ideal for small scale intensive studies, but are limited in their spatial coverage. There are also so aircraft and unmanned aerial vehicles or UAV platforms offer a balance between resolution and coverage.

Speaker 1: Airborne lighter typically achieves point densities of one to fifty points per square meter and is widely used for topographic mapping, forced inventory, and hydrological studies. Third, satellite based systems like NASA's JEDI and ISAT two provide nearly global coverage, but with lower spatial resolution. These systems are essential for understanding large scale patterns in vegetation, structure, ice sheet dynamics and global carbon cycles. Modern light our systems employ different technologies to capture and process laser returns.

Speaker 1: Discrete return light our systems record individual return pulses when the laser encounters reflecting surfaces. A single laser pulse may generate multiple returns as it interacts with different canopy layers and eventually the ground. Discrete return systems typically record about one to five returns per pulse and are common in airborne applications. Single photon counting light our systems, which include NASA's ISAT two mission, detect individual photons rather than complete pulses, while providing less energy per measurement.

Speaker 1: These systems can operate from much higher altitudes, including space, and offer unique capabilities for ice sheet and vegetation studies. Instead of recording discrete returns or individual photons, full waveform light our systems capture the complete energy profiles called waveforms of the return laser pulse. This provides more detailed information about the vertical distribution of reflecting surfaces and is particularly valuable for vegetation studies. The entire waveform can be analyzed to extract multiple metrics to describing canopy structure.

Speaker 1: Our training today will focus on this ladder category of full waveform lighter. First, we'll talk a little bit more about discrete LIGHTAR. For discrete lighter systems, return pulses are classified into one or more discrete returns. These returns are recorded when the intensity exceeds a pre defined system threshold. That intensity is shown here on the x axis as returned energy. The last returns are especially important for detecting the ground, where you can see there may be one to five returns, and here in this schematic we can see that the fourth return here is corresponds to what's called the ground return.

Speaker 1: Now let's discuss some differences in terms of the spatial sampling patterns of discrete versus full waveform lightar. The decision to choose one system over the other involves a trade off between the level of vertical detail captured and the achievable spatial coverage for a given project. In essence, full waveform lighter excels in providing intricate vertical information at the cost of reduced spatial coverage, while discrete lighter can often allow for prioritizing a high density of samples due to less detailed vertical information.

Speaker 1: Here is an illustration from a study in Taiwan of the spatial coverage differences between a full waveform data set from JEDI which has the rich vertical profile data, and discrete return airborne system which has as you can see, nearly wall to wall coverage. So the JEDI footprints correspond to the light green I set two observations correspond to the black dots and the airborne laser scanning data or the ALS data, which is which was used for validation purposes for this study and is the discrete system is shown in the and the brown point.

Speaker 1: Instead of just recording peaks as in the case of discrete lighter point clouds, full waveform light our systems capture and digitize the entire profile of the return laser energy. This results in a continuous waveform that represents the vertical distribution of all surfaces the laser pulse encountered. For the example shown here in a forest, the shape of the waveform reveals the height of the canopy, its density, and the location of the ground beneath it. This detailed information is exceptionally valuable for complex vegetation and biomass studies.

Speaker 1: Now we will move to section two, which focuses on full waveform light art from the JEDI mission. NASA's Global Ecosystem Dynamics Investigation or JEDI, produces high resolution full waveform lighter observations of three D structure of the Earth. JEDI is deployed on the Japanese Experiment Module Exposed Facility. The highlighted box shows the location of JEDI on this part of the International Space Station.

Speaker 1: This diagram shows how jedi's three lasers work. One laser is split into four coverage beams shown in blue, that create four ground tracks. The other two full power lasers shown in orange are not split and create two ground tracks each. In total, this creates eight parallel tracks of data collection on the Earth's surface, separated by about six hundred meters. JEDI science data products include footprint and gridded data sets that describe the three D features of the Earth. These data products are assigned different levels, which indicate the amount of processing that the data has undergone after collection.

Speaker 1: All products are publicly available, with the lower level products level one and two from NASA's Land Processes Distributed Active Archive Center or lp DOCK, and the higher level level three and four are from the Oakridge National laboratory dock data are initially transferred to the Jedi Mission Operations Center at the Goddard Space Flight Center that deploys acquisition planning on a weekly basis, and then processed through the Science Operations Center to distribute science data products to the docks.

Speaker 1: The physical theories, mathematical procedures, and model assumptions that are used in the creation of these data products are described in what are called the algorithm theoretical basis documents. These documents can be found at the link provided in the slide here, which in the slides will be posted again on the training website. In today's training we are focused on the level one B and Level two A data products. So the level one B data are the geolocated waveforms and the level two A include the ground elevation, the canopy top height, and what are called relative height metrics, which we will get into in the next slide.

Speaker 1: This is a representation of a waveform which contains information about both the vegetation canopy and the underlying topography. A near infrared pulse in the case of Jedi, is fired towards the surface of the Earth, where it is reflected by leaves and branches within a nominal twenty five meter diameter footprint, which is shown and on the right. The returned waveform, which is shown on the left, is processed to find ground topography cannot be height and relative height metrics. Relative height metrics quantify the vertical structure of vegetation by calculating the height at which a specific percentage or quantile of the total laser energy returned to the sensor is reached relative to the ground surface.

Speaker 1: So in this case, what you can see here from this waveform is although there is some small amount of energy here, we would consider that noise, and we'll talk more about noise in the in the demo. But from this waveform the top the height at which all of the energy of the waveform is returned to the sensor occurs here, and that's known as relative height one hundred, So one hundred percent of the energy of the waveform has been returned by this height. Similarly, relative height fifty is the height of median energy.

Speaker 1: In other words, this is the height at which fifty percent so this amount of energy has been this lower part of the waveform has been returned to the sensor, but still fifty percent of the remainder of the waveform energy has not been returned right, So RH fifty is by definition lower than our one hundred. So now let's take a tour of a real world example level one B waveforms from JEDI that we will be looking at in more depth in the hands on demo after this presentation.

Speaker 1: Here is an example of a waveform from Jedi's Level one B data set plotted in Bankhead National Forest in Alabama. We are looking at an area of the forest where some of the trees in the stand appear brown, indicating damage and possible mortality of the trees, which is likely due to the presence of Dendroctenus from tullis the southern pine beetle, and the Southern pine beetle is currently widespread in this region. The trees within the highlighted yellow example footprint in the forest appear mostly green.

Speaker 1: Looking at the corresponding waveform plot to the right of the map, the first peak of the waveform that you see circled and browned corresponds to the laser energy returns that reached all the way to the ground. Since these correspond to the lowest elevation peak in this case, there is a range of high amplitude above ground returns starting around elevation one hundred and eighty meters all the way up to about two hundred and ten meters. The observed wide distribution and multi peaks of returns along this elevation profile could indicate the presence of some healthy trees and intact branches in the shot, despite widespread beetle infestation in the immediate surrounding area to the north.

Speaker 1: Here is another example of JEDI level one B waveform plotted in Bankhead National Forests, so in the same general region, but a slightly different track of data. From the waveform plot, we can see very low amplitude signal in the above ground return The lack of high amplitude returns higher up on the elevation profile could possibly arise because of low tree cover and or tree mortality within the footprint of the shot. You might be wondering though, that, at least from the satellite image base map, it seems like at least most of the area with within this twenty five meter diameter footprint the trees appear green, although in the satellite every worldview bas map that we were using here we see some green trees it is important to note that there might be a temporal disconnect between the satellite imagery base map that we're using for reference and the Jedi shot we are looking at here.

Speaker 1: The imagery for the Ezvery Worldview base map is stitched together from various different acquisition dates, and the exact acquisition date of this image could be any time between twenty nineteen and twenty twenty four. Meanwhile, the Jedi shot here is from twenty nineteen, so there may have been some tree recovery and or regrowth captured in the optical satellite image that is not what's shown and represented by the time slice that the Jedi observation reflects. Here is a third example of Jedi Level one B data.

Speaker 1: The waveform shows a single ground return with no above ground signal whatsoever. In this case, we can see from the base map that this actually makes sense where this footprint is in a sparse vegetation area featuring grass and small shrubs, but no trees in the twenty five meter diameter area, so it's very possible that the small amount of grass or very small shrubs were not picked up by the waveform signal at all. Let's look at a final example In this case, we see multiple peaks of the waveform, but it is unclear from visual analysis at least what portions might correspond to the ground versus what portions of the waveform correspond to vegetation above the ground.

Speaker 1: When you explore these data in an area of interest of your own, using, for instance, the Python notebook that we will be looking at in the upcoming demo, you will notice that some waveforms show clear, sharp ground returns with minim activity below. However, other shots will show more complex patterns, such as the one we are looking at here, where we can see multiple peaks in what we think is the ground to turn or the initial part of the vegetation signal. The good news for us is that in the level two data processing, shots like this would likely be would be clarified through an additional set of processing where there are six different algorithms that all contribute to identifying where the actual ground is, and if these six algorithms don't agree within two meters, they are generally excluded from subsequent analysis.

Speaker 1: The most common reason for a complex or unclear ground signal could be due to noisy measurements, either with from system noise or from thermal noise. Another possibility is that more heterogen there may be more heterogeneous terrain within those footprints. On sloped terrain, the twenty five meter Jedi footprint may sample across elevation gradients. Combined with Jedi's geolocation uncertainty, which we'll mention in a later slide, the assumed ground elevation may not represent the true ground within the foot And then a third potential reason is that different vegetation density might be affecting scattering, where laser photons can scatter multiple times within vegetation before reaching the detector, arriving later than expected and appearing at lower elevations in the actual ground.

Speaker 1: So you may remember seeing this figure before when we first the idea introduced the idea of a waveform. Now we are seeing a third plot which shows the cumulative energy returned at each height as a percentage of the total energy of the waveform. So this is called the cumulative return energy profile with presentage with percentages converted to cumulative probabilities that range from zero to one. So the rh metrics that that we already defined were well that we're circling back to now quantify the vertical structure of vegetation by calculating the height at which a specific percentage of the total laser energy returned to the sensor is reached relative to the ground surface.

Speaker 1: So as you can see here RH ninety five. Canopy height is often defined as RH ninety five, but can also be up to RH one hundred, depending on the particular application. Here's an example of canopy height data from Jedi in the region of Ukayali, Peru. The rectangular areas are oil palm plantations, and the surrounding areas are a mix of mature forest, secondary forest, actively used pastures, and abandoned pastures. In this case, cannapy height is quantified as RH ninety five, although as I mentioned, RH ninety eight or some other similar upper RH metric could also have been used.

Speaker 1: Notice that there are some points along the tracks of data that are excluded. That is because we applied a standard filtering procedure that removed any observations with a low quality flag. Low quality data could be instances of observations that either did not meet the beam sensitivity threshold of point nine or that showed a deviation as I mentioned before, greater than two meters. Among the six algorithms used to identify the ground elevation. This graph shows the mean and standard error of relative RH metrics in the cumulative energy profiles for different forest classes in Ukayali, Peru.

Speaker 1: We can see the mature forest class shown in blue, the secondary forest class in dark green, and the young vegetation regrowth which is generally in areas of abandoned pasture lands that is shown in light green. We see that mature forests have the highest canopy heights and the lowest variance in canopy height relative to secondary and young vegetation regrowth. This is pretty consistent across all urge metrics, not just these upper metrics which correspond to canopy height. Meanwhile, URGE fifty corresponds to the height of median energy as we as we learned in the previous slide, and this metric our age fifty tells us about the distribution of understory vegetation, which also varies by forest type.

Speaker 1: Among the forest classes. Again, we see that mature forests have the highest R fifty values relative to secondary and young vegetation regrowth. RH metrics can be compared across land uses. Here is that same data set that we saw on the previous slide from Ukayali, Peru showing the median and intercortile range of relative height metrics of interest over tropical lowland forest regeneration classes which we saw on the last slide, including mature forests, secondary forest, and young vegetation regrowth typically after abandoned pasture abandonment.

Speaker 1: And then we also see some agricultural classes including oil palm plantations as well as cacao plantations, including both monocrop and agroforestry. Cowsystems. Despite their power for capturing detailed vertical structure spaceborne full waveform lighter systems like JEDI face numerous operational limitations. One limitation is geolocation error. JEDI has positional uncertainty of each footprint of about plus or minus ten for one sigma error. This requires careful filtering and the use of buffers, especially in heterogeneous areas where a footprint might incorrectly sample an adjacent land cover type.

Speaker 1: A best practice is to filter JEDI observations to only include footprints with homogeneous land covered types of interest within a diameter of forty five meters say twenty five meters of the original diameter area of the beam, and plus or minus ten meters additional buffer to account for gelocation error, as shown in the figure here. Another limitation is the inability to fully penetrate clouds, which, unlike other active sensors that are radar systems, light our systems aren't able to penetrate clouds.

Speaker 1: Another limitation is that in exceptionally dense canopies such as tropical forests that we saw in the Peruvian Amazon, the signal can be entirely attenuated, in other words, absorbed or scattered before it reaches the forest floor, resulting in an inability to detect the ground. And as a part of the level to filtering that we did in the examples that we showed that I showed here, this limitation was accounted for by the filtering that we applied. Another limitation is that the data is collected in distinct footprint tracks rather than a continuous map, leaving large spatial gaps.

Speaker 1: This comes back to that difference that we were talking about earlier between many airborne discrete light our systems versus spaceborne full waveform light our systems. So essentially this makes the data unsuitable for applications requiring wall to wall coverage of a region. Another related limitation is that of limited temporal resolution. Due to the spatial sampling of the Jedi mission, and that the sparse spatial sampling, it is very rare for any of the Jedi footprints to be repeated and observed by a second time.

Speaker 1: There are some cases where this happens, but there are not very many over the globe. Finally, another limitation is slope induced waveform contamination. In areas with steep topography, the large laser footprint can hit the ground at different elevations, simultaneously distorting the shape of the return waveform and complicating the identification of the true ground service. Before we move into section three, which will focus on practical force ecosystem applications with Jedi full waveform data, we now have a fifteen minute break for questions.

Speaker 1: I will start by addressing question one, which I believe was asked during the conversation about discrete versus full waveform lighter and airborne versus spaceborn lighter. If I'm correct, and the person who asked this question can can let me know if this was I'm appropriately addressing it. So the question being does the platform and find resolution of airborne lighter sensors is what I'm referring here guarantee accurate biomass estimates in temperate and boreal forests. So the short answer is no.

Speaker 1: While airborne lighter provides more accurate estimates than spaceborne systems, it does not guarantee accuracy due to numerous fundamental and uncertainties. So airborne lighter is more accurate than spaceborn for a few reasons. One is the reduced geolocation error as we talked about, The other is the higher point density, and then finally there's generally better ground penetration, so lower altitude and denser sampling increase the likelihood of being able to detect ground returns, especially in dense canopies, which I guess more applies to tropical forests than you are specifically asking about temperate and boreal forests, where essentially, with denser canopies, space borne systems make experience complete signal attenuation.

Speaker 1: So that is the first question. And then question two, why do we use RH ninety eight for canopy height and not say RH one hundred for example, right, So, and then then the second part of that is what is the possible application of RH fifty? Does RH zero respond to ground elevation? So great questions. So let's start with the question about canopy height. This relates to the fact that there is often noise as a part of the Jedi well the spaceborn light oar waveform that causes RH one hundred to be an artificially high estimate of canopy height.

Speaker 1: And this does vary shot by shot. So that's why there's no single RH metric that across the board is used as canopy height. Right. It can be R ninety five, it can be R ninety eight. Generally, I haven't seen for most spaceborne lighters audies. I haven't seen folks use RH one hundred just because of that, the reality that a lot of times the noise is present and not fully processed out in level too data processing of Jedi data.

Speaker 2: And if I can add something to what you're saying, Savannah, this is sure. Sometimes RH one hundred doesn't capture the canopy surface. You know, depending on the resolution that you want to study, Say you need to characterize the canopy surface with thirty meter resolution, right, and you might have a Judai shot that is centered on one very tall pine tree that has a sticking top, and that is going to drive your RH one hundred up right, that that's going to be your RH one hundred essentially, So that doesn't really represent the canopy surface at the resolution you're trying to study.

Speaker 2: So that might be another reason in addition to noise. It might be like a real return, but it's coming from an outlier tree and it's not necessarily how we want to quantify cannot be height.

Speaker 1: Yeah, thanks for that edition, Nirah. And let's let's go into the and unless there's follow ups about that, we can we can go into the question about our H one hundred, which is the height of median energy of the waveform. So because because these RhE metrics, you know, they reflect the and the way for itself reflects the amount of the distribution of biomass along the vertical profile. When when there when we look at our age fifty, for example, we can learn about out the understory in a way that can tell us about how much vegetation is is how it's distributed.

Speaker 1: Right, So for instance, in a forest that has a since someone did ask about temperate forests, maybe we could use that as an example. If there's a temperate forest that has a denser understory with which is allowed you know, can often happen when this when the hot taller trees are more separate, and that allows for light to get through to the understory that will be detected by higher RH fifty values. And and whereas if there was less of an understory, maybe those RhI values would be lower. So that's just one example.

Speaker 1: Naira, I'm not sure if you'd like to add to that explanation as well. No, if I do it, thank you, okay, great. And the question about RH zero is is a good one because actually I wonder in in the in the follow up, when we when we post this question and answers document to the to the course website, I can add a visual that might help explain or kind of speak to this question about RH zero because it some RH metrics often arch metrics in the level two data DEEDI data are negative, meaning that they're part of the ground return energy.

Speaker 1: But as you saw in the in the one of the previous you know slides that was showing kind of this waveform showing the multiple waveform examples, you'll notice that a lot of the energy that reaches the ground due to a couple of factors, and I could get into them, but I also want to be mindful of time that essentially some of those returns will end up being registered as below zero ground surface. So sometimes RH zero is negative and sometimes it is around zero, and sometimes it's slightly and so I think it basically it varies on the topography and on the level to processing algorithms that are where there's basically six different algorithms that are used at least in the JEDI data processing pipeline, all of which are designed to identify the ground and have slightly different ways of doing that.

Speaker 1: Anything to add, Niaria, No, nothing to add, thank you? Sure? So what is the global availability of this data? Great? So we already have kind of an explanation here where for JEDI in particular, which is on board the International Space Station and due to this, due to its placement on the ISS and the ISS orbit, there is a bounding of latitudes between fifty one point six degrees north and south, so that is that's where the data are available. And then, as of course mentioned in the presentation there this is a sampling mission, right, so it's not wall to wall coverage, and neither is our other light our space and light our missions from NASA, right, So I SAT is also not wall to wall covers, so sampling, they're both sampling missions.

Speaker 1: So question for since lightar is an active data collection platform, why is it limited by cloud cover? This is Yeah, this is a good question, and so there I think maybe another another thing to think about is that there are different types of active sensors and Niar's expertise is actually in radar remote sensing, So Niara, maybe you could also speak to this because radar radar systems are able to penetrate clouds, but JEDI, which fires in the near infrared portion of the electromagna spectrum, that wavelength essentially often does not penetrate through clouds.

Speaker 1: And so that's the short answer, and I r if you want to elaborate more and kind of explain how how radar systems are able to penetrate clouds, I think that would be really helpful for the audience.

Speaker 2: Yeah, Essentially, the active has to do with whether the instrument makes its own signal or not, and then the issue of signal penetration depends on the wavelength. So you have liedre that can't penetrate through clouds, and radar you also have different wavelengths and some of them can penetrate through canopies, and some of them can't. So that's why we usually talk about the wavelength when you characterizing those systems. So the issue of sensitivity has a lot to do with how the signal penetrates in the front media.

Speaker 2: So this is a very good question.

Speaker 1: Question five. You stated the green band penetrates water. Can it be used to study water pollution from an industry like mining? So that is an interesting question. I I'm not aware of applications. It's not my area of expertise, and I wonder if we could do a follow up on this question to kind of look a little a little more. I mean, in terms of lighter, it's I don't think there would be an application for water quality. I think the water applications of for instance, I SAT which which fires in the green, is more for detecting quantities of water and changes in ice.

Speaker 1: So for lighter specifically, I'm not sure that there would be an application for water water pollution the way that you know optical sensors are used. But I'm I'm learning things every day, so you know, it's very it could be. So I think that I'll leave that as my answer for now, and if anyone else wants to chime in, Niara or yeah, we will follow up.

Speaker 2: The first I will follow up and look for some references. M hm.

Speaker 1: Is there any filtering applied to filter out shots with clouds? So this will depend for Jedi specifically. This will depend on the level of processing. I believe in and Nyara's demo actually focuses on level two data, so maybe you're more refreshed up on the exact filtering steps in the in the different levels. But for level one B, which is what the demo the upcoming demo will focus on, I believe there's no filtering. It's up to us essentially to apply the filters, so there's no pre filtering when you when you request the data, all of it will come back.

Speaker 1: But then there are two different quality flags which will learn to use and and then those will help with with the removal of observations that may that may be contaminated from clouds.

Speaker 2: Sometimes you also see a cannot be height that is unusually high, So you see our age ninety five, that's like two hundred meters, and that might be an indication that that's an area with clouds. So one simple thing that you can do is to threshold the RAH ninety five and say, you know, just consider our shots where our age ninety five is the maximum cannot be high in your study area, then you would exclude buildings as well. So I'm assuming we're interested in vegetation and land cover but not buildings.

Speaker 1: That's a great point. How can did I be used to estimate or identify deforestation? So JEDI can be used to jedi? Can you know, tell us about the presence of trees or lack thereof right by by the the waveform you know, returns that that we see. And so if if we only see actually and in one of the examples I gave, we saw various you know, a sparse area which I think was mostly grasped with a few shrubs. That all there was in the in the waveform signal was a single ground return, right, And so that's an indication that there's no forest right when when when the you know, when there's no amplitude above the noise threshold above the ground return questioning, are there some types of forest that Jedi is not able to fully penetrate and thus not reaching the ground.

Speaker 1: If so, how is that detected? And how is the structure analyzed great question. So tropical forests are the type of forests that are most where this's where this happens most most often, right where there's such a dense canopy and understory that it is possible that some of that none of the energy of the laser poles reaches the ground, and in that case, the the level two algorithms I believe that are applied, and we will follow up just to provide more details on this. But essentially, like I said, there's six algorithms that are part of the standard Jedi processing pipeline that their job is to identify where the ground is.

Speaker 1: And so there's a number of different ways that we could like identify cases like this where there is no ground return identified. And one of the ways is by looking at these six different ground algorithms, and if there's greater than two meters disparity between them in terms of their calculation of where the ground is relative to the reference goid, then that will that will be a flag right and and and excluded as from from the in the in the quality filtering. So for now, maybe we can go into the demos and then we'll have a second Q and a session after the demos and we'll do our best to address as many questions there and then those that aren't addressed, we will answer all of them and post the Q and a document on the course website.

Speaker 1: We are now starting section three, which focuses on practical forest ecosystem applications with JEDI full waveform data. Exercise one has two objectives, First to enable participants to access, filter and visualize JEDI Level one B full waveform data using NASA Earth Data Application programming interfaces, and two to teach interpretation of waveform characteristics for vegetation structure analysis. In this hands on demo, we will start by authenticating with NASA Earth Data Login credentials. Hopefully everyone has their Earth Data account already set up, and then we will search for JEDI Level one B granules using the Earth Access Python library.

Speaker 1: We will download JEDI data for the specified area of interest, apply quality assurance filter using the quality flag and the degrade flag fields, so there's two different fields that we'll be searching for. Then we will plot raw waveforms for multiple footprints. We will identify and interpret ground return signals versus canopy returns, and we will analyze waveform orphology to assess vegetation vertical structure. This will be over the same reason that we saw in the slides that have been presented, which is focused on Binkhad National Forest, Alabama in the USA.

Speaker 1: The default latitude and longitude bounding box can be changed to your own area of interest if you clone the GitHub repository and modify your local copy of the Google Collab notebook. So for this demo, we will be using the default AOI, but I encourage you all to modify to your own area of entest either now or in the future. With the code, we will focus on the entire set of observations collected to date from the Jedi mission, starting in April twenty nineteen all the way to up and through October twenty second of this year.

Speaker 1: All right, let's get right into the Python notebook. Now, I'm going to walk you through a Python notebook for processing NASA's JEDI Level one B full waveform light our data. As a reminder, JEDI collects a detailed three D information about Earth's forests and topography from the International Space Station. So this notebook demonstrates how to search for, download quality, filter and visualize the JEDI Level one B waveform data for a specific area of test. This is the read me file for the repository and we will go ahead, and it does list all the steps of setup, but we will just walk through each of these steps together in the Google collab environment.

Speaker 1: So we will just click on this button here on the top of the read me file and this will bring us to the Python notebook shown here. We'll start with step one, which is to clone the GitHub repository and this will allow you to access this notebook in any supplemental files with this step, so we'll press play here that allows the cell to run. You can just click run anyway for the warning because it was not off. The script wasn't authored by Google, and this is what the output would look like. Once it says one hundred percent done, you can go into the next cell here, which involves installing the required libraries.

Speaker 1: So there's a few necessary Python libraries. We will use PIP to do that. So earth Access is a library that we are using for authentic heating with NASA and searching the data archives. H five PI is used for reading Jedi's HDF five file format. Geopanda's library is used for spatial data operations. Mattplotlib and seaburn are both used for creating visualizations, and then finally Fullium is used for interactive mapping. So let's see how.

Speaker 3: We're doing here.

Speaker 1: Once we see at the bottom of the output that all libraries were imported successfully, we can move on to the next step, which is to mount your Google Drive. We can run that cell, and in order to do this, you need to be signed into a Google Drive account or have an account that you can sign into where you will connect to Google Drive and choose the account that you want to use, pressing continue. You might have to do this twice. That happened for me during one of my installation setups. It will run for a bit.

Speaker 1: When that's done successfully, it'll say mounted at and then I'll say content slash drive. In this next part, we will authenticate with NASA Earth Data. You'll need to enter your credentials when prompted, and again these are free and can be obtained from Earth Data dot Nasa dot gov. And actually, before I run the cell, I also just want to highlight in the code here that we are using the default AOI, which the area of interest is in William Bankhead National Forest and the surrounding area in Alabama if you would like during this demo, either now or there will be time after we go through this first default AOI example, to update this bounding box.

Speaker 1: Note that there's then you will be able to do that. And note that there are four coordinates that you need to enter here to change from the default to whatever area of interest that you want. And so what you would need to enter is the minimum longitude, the minimum latitude, the maximum longitude, and the maximum latitude of your area of interest, separated by commas. So I'm gonna go ahead and run this cell where you'll see that you're prompted to enter our Earth data log in username and password,

Speaker 1: and once that happens, the rest of the cell will run.

Speaker 1: I'll also point out that we have a temporal range defined here as all available Jedi data available to date, and then the search for the data starts by focusing only on full power beams, and this is to increase the chance of having high quality data.

Speaker 4: So you in the data with when we apply the filtering that we apply a lot of the data from coverage beam lasers will gets.

Speaker 1: Removed, and so we want to just focus on the power beam for now. And so what this function does is the search JEDI level one be function takes the input bounding box that we provided, the temporal range of the of the data that we are wanting to search for, and then also a maximum results list just to put a cap on how much processing time we're going to take to retrieve some of these data. Once those files are returned, there is a step of extracting the data, and so we will go ahead and just look at the output here in the interest of time.

Speaker 1: This is what the top of the output looks like, where we can see our longitude and latitude range and the focus on just the power beams which are listed here. Here are the granules that were retrieved. It was again a maximum of twenty that we're searched for for this time and for this code, for the purposes of the demon stration I, it will stop at the first granule that has some quality data. So if if this first granule,

Speaker 1: which is this is the name the full the full name of the file here the HDF file. But if this were to have no data, then the code would automatically try a the next file down on the list, and this can take a little while. The queuing well the specifically the processing step of processing the data, because each each file can be several gigabytes, so in this case, the HDF five five file that we're looking at is four point one to nine gigabytes, so each one takes a while to process. And for the purposes of this demo, there were several files on this list that that didn't pass the quality flag filtering and didn't have quality data, and we're skipped, And so I wrote something into the code that if it's the specific bounding box that we're using for the demo, it will skip straight to this fifth file on the list.

Speaker 1: So for those who might be paying close attention to the code, I might have noticed that. I just wanted to say that in general, if you put a different area of interest, it will go in order, file by file until it finds one with at least some quality data to analyze. Further, here is a summary of each of the beams how many shots were retained after the quality filtering step, and as you can see here all in all of these full power beams, one hundred percent of the data were retained, so overall we have three hundred and thirty five shots across all beams.

Speaker 1: This section of the notebook generates four different visualizations. So Plot one shows is the shot locations map. This shows where each Jedi shot landed, color coded by elevation using a terrain color scheme. This helps us understand the topographic variation in our study area. Plot two to the right here shows an elevation histogram, so this displays the distribution of elevations. The shape tells us about the local topography. A narrow peak suggests flat terrain, while a broad distribution indicates varied terrain.

Speaker 1: And as we can see here, most of our shots fall within two hundred and fifty and three hundred meters of elevation, so that's a pretty narrow range. Plot three shows shots per beam. This is a bar chart where it indicates data availability across the full power beams, so an uneven distribution is normal due to Jedi's orbital geometry and data quality. And then finally we can see a pie chart here, which is the processing results PRIE chart, summarizing how many files were tried in their outcomes successful no spatial data or no quality data.

Speaker 1: And as I mentioned before, many files have don't have one hundred percent quality data, but for the purposes of this demo, I'm using an example that does moving on to level one B waveform visualization. We will first produce an interactive map where you will be able to look at the AOI boundary, zoom in and pan around the study area, and then click on individual and click on individual points to see shot details. You'll also be able to distutingush between different beams by color and then understand the spatial patterns of deadI coverage.

Speaker 1: So let's go ahead and run this cell. So it will take a little while to run. Our output will be shown down here.

Speaker 1: First we see a quick summary of the user configuration. We'll note that there is a function at is called select shots by elevation deviation and in this code segment, and this function selects shots based on standard deviations from mean elevation that allows the user to define different standard deviation values that you'd like to focus on in the plots. So there's just ten shots of the three hundred and thirty five in this case total shots that are going to be plotted, and so which of those ten shots are plotted is up to the user to define as based on what elevation they would like to focus on.

Speaker 1: In the default of the code is currently written. Here, I'll just show that we have negative zero point five standard deviations. So, in other words, the ten shots that we're plotting in in the waveforms will be will have slightly lower than average elevation among the three hundred and thirty five shots total that of that were extracted from the HGF file. So here we can see the interactive map where the lowest elevation shots are shown in dark blue and blue, and the medium to high elevation shots are shown in green and red.

Speaker 1: And as I mentioned, we're showing slightly lower than average elevation shots. These ten shots highlighted on the map, and all of those are in the blue colored is well, are in the blue category, but are highlighted in yellow, so that we can see spacially where each of these coincide into and then you can pan in and out whatever zoom level you'd like. And let's look into an example that the same example that we saw from one of the slides in the presentation, which is shot too, where you can see by zooming in, I believe this is the maximum zoom level that's allowed here on this map.

Speaker 1: But we can see that basically this is a pretty sparse area. There aren't any trees within this shot, and there's what looks like either grass or small shrubs in this footprint. So now let's take a look at the waveform that corresponds to this particular footprint that's highlighted, and in order to do that, we'll need to put the cursor inside of this cell and then scroll down to see the plots below. So these are the ten different plots that were selected again as a based on the standard deviation that we put in, and they're numbered here.

Speaker 3: Where we have plot number one corresponding to the footprint one and footprint.

Speaker 1: Two is the one that we zoomed in on above here in the map, which has a lot of bare ground and very short vegetation and grass, which basically shows up as looks like a single ground return without any very minimal indication of above ground vegetation. On the other hand, we can see this waveform to the right, or also we could look at to the left. Let's look at both of these. So shot number three to the right is showing a ground return looks like a pretty clear ground return here and then some evidence of above ground vegetation with most of the waveform amplitude right above the ground return, but then some non noise return of the waveform.

Speaker 1: Returns of the waveform appear a bit higher higher up in elevation from that initial second pulse. So let's take a look here on the map of for looking for footprint three.

Speaker 3: So if we zoom out here and look into footprint three, and again if we click on the footprint, we can see some of the information about the beam and shot number as well as the elevation latitude, longitude, and so what we can see from.

Speaker 1: The base map is it looks like there are some healthy trees within this footprint, which makes sense given that we saw above ground returns. And if we zoom out a little bit, you can see.

Speaker 3: That darker shades in the surrounding neighboring area looks could be evidence of beetle infestation.

Speaker 1: And we know from.

Speaker 3: Colleagues that we that we collaborate with who study.

Speaker 1: These regions that these beetle infestations, the south pine beetle have been very prevalent in recent years in these forests.

Speaker 1: So if we zoom out again and this time look into waveform number one, we can see that there that the forest does begin right after two right, there's looks like there's an edge here where there's more of a thickly dense canopy. And by.

Speaker 3: This first footprint here or footprint number one here, we can guess that there will be at least some.

Speaker 1: Above ground vegetation returns. And like I mentioned in the presentation, we want to be very and a cognizant of the fact that the base map that's used here is from the EZRI Worldview Satellite Imagery composite, which comes from different dates and based on you know, best optimal cloud free acquisitions, and is kind of stitched together that way, and so we don't have a way of knowing exactly, at least not with the script written here, what the temporal difference is between what we can see on the base map versus what when the Jedi wave form was plotted, which, as you can see from the file name, twenty nineteen was the year that.

Speaker 3: Was plotted, and then this is a Julian date.

Speaker 1: So here we see waveform footprint one where there's a clear ground return and also indication of above ground vegetation that has as shown by the kind of multiple peaks of higher elevation returns that are above the waveform or above the ground return rather and below the kind of remaining noise of the signal past around two hundred and ten meters or so. So we looked at a couple of different waveform plots and they're corresponding locations on the map here. I would encourage you to go back into the code and modify the standard deviation that you use to select the ten waveforms that you'd like to plot.

Speaker 1: So for this example, I'm going to increase this to two standard deviations above the mean and then press and then run the cell. And rerunning this cell is going to produce a different set of results that will take a look at.

Speaker 4: It.

Speaker 3: We'll take a little bit to rerun.

Speaker 2: I will.

Speaker 1: I would like to say that for those of you looking at areas of interest that are different than this default AOI, you might notice that changing the elevation, especially if it's high higher elevation shots, these might be areas that are these might be more noisy or more noisy shots or shots that waveforms are, or maybe more difficult to interpret. In this case, there's some elevational gradient, but not a ton. So with the ten different examples that we're looking at here, If we scroll down again to the ten waveforms, we can see that in all of these cases there seems to be a clear ground return except for this one, which I believe was the same example I provided in the slides.

Speaker 1: And in areas that have even more complex topography than the area that we're looking at here, I would anticipate higher elevation areas to be associated with waveforms that are you know, challenging to interpret, such as such as this one. And so for now, in the interest of time, I will conclude this demo, but please feel free to note any questions that you have and share any ideas for future improvements with the code. And yeah, so all your your feedback is is always welcome. And with that I will pass to Nayara to look and to walk us through exercise.

Speaker 2: To thank you Savana. So we are going to do exercise processing JEDI Level two A data sets using the NASA Harmony tool, and our goal is to display relative height metrics for vegetation structure analysis. And the basic steps are to order the data sets to do a spatial subsetting, and then we're going to import the results into a Google Collab notebook. We're going to do some filtering based on existing quality metrics and then display in QGIS. So we're going to get started by looking at the ripo and opening the Collab notebook.

Speaker 2: So let's get started. We start by navigatading to the vegmap or repot, and when you get there, let's go to the Jedi folder. That's where the routines for processing Jedi data sets are located. So we click here, and we want to use the Jedi l to a underscore our set. So we click on this notebook and you it should have a button that says open and Collab, and if you are not signed into Google at this moment, it will prompt you to sign for Google. We're using your Google account. So once you get in the Google Collab notebook, the first thing to do is to install the packages and the dependencies.

Speaker 2: So we're just going to run these two cells without changing the inputs, and it will take a minute. But

Speaker 2: Google Collab has already many of the libraries that we need and we don't need to install them, so let's wait a minute. You can disregard the error messages here and let's just move to the next cell where we import the packages that we need and The main one to keep in mind is the Harmony package, which gives us an API to connect to the NASA deck and request the spatial subsetting of the Jedi orbits. These are really low large files, so we're trying to do the special subsetting in the cloud, and so we just download the smaller files.

Speaker 2: Now we need to authenticate with earth Data. If you haven't signed up for Earth Data, you can do it. It's free. So I'm going to put my credentials here and if your credentials are up to date, it will give you a green check mark. Then the next thing to do is to upload my area of interest, and I'm using a geojason file which I'm going to import here into this area. So I'm going my local files.

Speaker 1: And importing.

Speaker 2: Into You just drag the file into collaboratory and now when we press play, it's going to request the path to this file, and you can click on those three dots here, copy the path and then paste here. So essentially it's in a photo called content and this is the file name. So check make sure that the jeo Jason is correct. You might have an issue if your shape file or if your jeo Jason has too many vertices, in which case you have to simplify it. But what we have is just a square, so it was accepted with no issues.

Speaker 2: I can show you briefly where the site is and how big this area is. This is Brazil and the site that we're working with is in the Atlantic Rainforest in the southeast part of the country. And I picked this site because it has a very nice variability in structural signatures that will help us gain an intuition for what the relative height metrics are giving us. So you have here some water, some forest fragments. We also have some agriculture here in some urban areas, and we have a ton of bare ground.

Speaker 2: So the expectation here is that the top height is going to show us the differences between forest fragments tall versus short, and also between forest fragments and the farms. So for step four, we're going to define the date rain for the subsetting, and we're also going to define where we want to save the results. So to run this cell, I'm going to put the entire range of Jedi observations from twenty nineteen to the present day, and I'm going to define a folder to save my results. So I just want to save here in this content folder.

Speaker 2: So again I go to the three dots, I copy my path and wasted here. So that's where the files are going to be saved. Next step, I'm calling the Harmony client, and for this size of air of interest is going to take up ten minutes, and in interest of time, I'm going to bringk the resulting files here. But when you run yourselves, please wait ten minutes, and you're going to see the files being generated. So for step six, we're just saving the file. And because I already have my file, I'm not going to run this cell.

Speaker 2: But if you run the Harmony cell and you waited ten minutes, you have a file there, and you can run the cell to save the files in your folder, and it's going to look like this. You're going to have a text file and if I open it, it doesn't have the Jedi data sets themselves. Those are just links and each one of them is file in H five format. So for the next step, we're going to define how we're going to further filter these files. And I'm going to use some criteria that are commonly applied to this level two a data set.

Speaker 2: So if I run this, one of the criteria that's commonly used is a sensitivity, and the sensitivity is the maximum cannot be cover through which the lighter can detect the ground with ninety percent probability. So because I'm working a forest, a tropical forest, I'm going to set this to zero point nine and then the minimum Marhe ninety five value. This is if you want to, for example, excludata is where you have cannot be a height equals zero, So I usually put one and then two or false. Do you want night only?

Speaker 2: I'm doing false just to get as many shots as possible, but one way of getting data sets with more signal to noise ratio is to select night only. But for now I'm just going to put false, so we will get shots from daytime and nighttime. So I selected my inputs and then they got checked here and we can go to the next step. So now we're running the main function that does all the filtering. It's going to download all the H five files from the file list, and one by one it's going to apply the filtering using the criteria specified on step seven, and then everything is going to be combining the end in a single CSV file.

Speaker 2: So for this cell, we don't need to do anything. We're just running to get the function. Okay, So at this point, we're just running the function and we're just going to choose the location of the input file. So to get this file, you just click on the three dots here and copy path and then paste in this box. And now we're going to choose where we want to save the resulting CSV file. So what I'm going to do is to copy the path for this folder and then add the file name that I want to use. So it's up to you and it will take about ten minutes to do the processing.

Speaker 2: I do have this file generated already, so what we're going to do is to move to QGS to explore the data. And to get these too QGES, you would click on the three dots and download the file and then we can bring it into q GINS as a vector layer. So let's do that. So we're going to add this layer to QGES. Just choose a limited text layer and we just navigate to location of your file and you just will try to find the latitude and longitude columns for you. So just get test and it was correct. So just say ed and clothes.

Speaker 2: So here are my shots. Now I want to just change the display to show our age ninety five. So instead of a single symbol, I'm going to have graduated symbol based on our age ninety five. And let's change the collor here and they can classify. So this is the range in my data. It goes basically from one. Remember we asked to be to filter out things that are shots that are lower than one and maximum cannot be height here is about twenty four meters. You do have some outliers. That's not uncommon. I would filter out shots where our each ninety five is over sixty meters or eighty meters depending on your forest.

Speaker 2: So let's apply. So now looking at this landscape, the first thing to notice is that you have somewhat irregular sampling of the landscape. Right, you don't have a wall to wall image, but it's still giving you very important and interesting information. And you can see here that the each one of the dots has a value which has the maximum cannot be height for my forest fragment. And the areas of bare ground look orange and red as expected because they have lower cannot be heighted and the force that areas have a higher value.

Speaker 2: And you can also see some variation between forest fragments. Some of them are taller than others, and you would need to make some assumptions about the spatial variability in this landscape to be able to derive metrics at the patch size right, because as you can tell here, we're not covering the entire patch. This area here is an agricultural zone, is a sugarcane, so the height is between three and six point two meters, which makes sense.

Speaker 1: You have.

Speaker 2: Also shots in urban areas. This is an area that with not a lot of buildings here, so about six meters for houses. And again the red low values for a bare ground, so it's always good to have some bare ground as you're trying to get used to this data set to get a reality check. But of course your base map can have a temporal properties that don't match up with the Juedi shots, so we need to keep that in mind as well. And the other thing to keep in mind is that sometimes you don't have shots in the forest.

Speaker 2: You can almost see like the beam is being interrupted, and this is happening because we filter them out, either because sensitivity was slow or because

Speaker 2: you might select nighttime only and the being was acquired during daytime. So one thing that can be done for each one of your sites is to experiment with the filtering variables, and I know for other modules of this tutorial this is going to be discussed in more detail. So thank you everyone for your attention and available to answer any questions that you might have. So with this, I'm going to send us back to Savannah and Savannah you can summarize the discussion.

Speaker 1: Thank you very much, Niara. Now we will move into the summary of Session one, Introduction to Full Waveform Light R.

Speaker 1: Let's review the key concepts from part one. First, we covered light our fundamentals. Lighter is an active sensor that measures distance by timing how long laser pulses take to return. This gives us detailed information about elevation, canopy height, and the three dimensional structure of vegetation. We learned that full waveform light are, unlike discrete return systems, records the complete energy profile of each pulse. This captures the entire vertical distribution of vegetation within jedi's twenty five meter diameter footprints, providing much richer structural information.

Speaker 1: The JEDI mission operates from the International Space Station with eight parallel laser tracks. It covers latitudes between plus and minus fifty one point six degrees and produces both footprint level data at twenty five meters and gridded products at one kilometer resolution, which we'll talk more about in the upcoming session two. A major focus of today's session was relative height metrics. R ninety five gives us canopy height RH fifty tells us about understory characteristics, and the ratios between these reveal canopy layering and structural complexity.

Speaker 1: These metrics are powerful for distinguishing forest types and land uses. We also discussed important limitations. Jedi has about ten meters of geolocation uncertainty. The instrument also can't see clouds, It provides discrete sampling rather than complete coverage, and has limited temporal resolution for any given location. Finally, you gained hands on experience accessing NASA Earth data, downloading and filtering Jedi data, and interpreting waveforms to understand vegetation structure. These are the foundational skills you'll build on in parts two and three.

Speaker 1: Looking ahead to Part two of this training, you will be able to describe the general characteristics, strengths, and limitations of the Jedi mission in its data, identify JEDI data products and their characteristics for elevation, canopy height, vegetation structure, and biomass. Access and visualized jedidata data sets including elevation, cannopy height, vegetation structure, and biomass for an area of interest using openly available tools. This is a list of resources that were cited throughout today's presentation and also could be useful future reading after this session.

Speaker 1: I'd like to remind everyone that there is one homework assignment for this three part series course and it will open on the day of session three on November six, twenty twenty five. The d date for the homework will be two weeks later, which is on the twentieth of November. You can access the homework from the training page and note that your answers must be submitted via the Google forms. If you attend all three live webinars and complete the homework assignment by the deadline, you will receive a certificate via email about two months after completion of this course.

Speaker 1: Here is my contact information as well as the contact information for Nayia Vinto and also linked on this slide is the our Set website and the our Set YouTube channel. But that I thank you all so much for your participation today and look forward to seeing you next session. In part two, and now we will move into the second round of Q and A. All right, thank you all for adding more questions into the document. Here. I will start by talking through some of the questions that came up that were specific to the demos, and we'll see how far we get in the fifteen minutes that we have.

Speaker 1: So let's go with Part one B. Question one was a question about how can Jedi level one be data, which is an HDF five format, be converted into a shape file using code or what is an alternative method to achieve this conversion. So the demo script does extract the coordinate data and then the attributes from the HDF five file into a panda's data frame, so that's in what's called result dot underscore DF and then to convert that to a shape file, you can add a few lines of code following the data extraction.

Speaker 1: And in order to do that, the library we recommend using is Geopanda, So you me to import geopandas, install and import and then convert the pandas data frame to a geodata frame. And then this is the code that you could use to export that geopandas the geodata frame into a shape file. But that said, I do want to highlight alternative formats that are vector based, you know formats, but have some advantages. Specifically, GeoJSON is a great option, which is great for web mapping, and there's no file size limits.

Speaker 1: The GeoPackage format has no attribute name length restrictions which shape files do. And then and then we also kind of talk about CSV as another as another option. So because of the limitations of shape files, both the character field name limit of ten characters and having multiple files as opposed to just one, that's one of the key advantages of these other alternatives that we're listing here, you could consider, you know, replacing the shape file with one of these options. Okay, let's move into question two.

Speaker 1: Is r H ninety eight available with the level to be data or only with level two A data? So

Speaker 1: all rh metrics, including ninety eight are available in level two A only level to be are, which which the next training actually will will cover more. Right, So this training actually didn't didn't discuss level two Beta two BE data that that is can be covered in vertical profile metrics. So basically more processing is used on the r H metrics to create derived products, such as the plant area index, which is essentially analogous to leaf area index. I'm not sure if you all are familiar with l A I, but that is the area the leaf area plus the woody vegetation, the woody biomass area per unit ground area.

Speaker 1: That's that's the definition. And then fully height diversity is always another example of such derived products in the level to B and so so the next session we'll discuss more in depth all of these derived products. And so the RH metrics, though specifically, are just the level to a data question three, Can I do the analysis in a heterogeneous urban environment? So technically, you know you can obtain data from JADI over urban areas, right, So the NASA APIs will not prevent you from doing this, but there are very big challenges and important caveats to think about.

Speaker 1: So JEDI was designed and optimized for vegetated ecosystems, particularly forests, and so starting with the plus anus ten meter location uncertainty combined with the heterogeneous land cover of urban areas means that a JETI footprint right with a twenty five meter diameter footprint might sample a tree, a building, a street, or a mix of all of these within that that footprint area. So a best practice, which was discussed in slide thirty eight of training, is to filter observations to areas with homogeneous land cover within a forty five meter ish or so diameter right, And so that accounts for the twenty five meter footprint plus a twenty ish meter buffer for geolocation error, and the ten plus minus ten meters comes from kind of the one sigma error.

Speaker 1: So if you want to be more conservative, you know, you can add a slightly larger buffer, or you could be you know, less conservative if you want to be able to have smaller kind of buffer zone. So you know, that's up to the individual researcher and research project needs, but that's the general rule of thumb. Right. So in urban areas, you would want to apply this strict spatial filtering to ensure that these footprints are sampling vegetation only. So for example, maybe you could study a large park in an urban area, or a street that has lots of trees and it doesn't have many buildings that are right up against the street, right, So to be able to covered that large of the buffer area, and then you could essentially exclude shots in your buildings and carefully interpret the results since the waveforms may be contaminated by non vegetation returns.

Speaker 1: So hopefully that explanation provides some you know, is useful as you as you think about urban areas and just really thinking carefully about the caveats before before moving forward with that question four. And Naiara, please chime in at any point if you'd like to add on to any of these explanations. So question four is there looks like there was a warning, and so in my club environment, I'm getting the following dependency conflicts when running the PIP install code cell. So dependency resolver does not currently take into account all the packages that are installed.

Speaker 1: This behavior is the source of the following dependency conflicts. So these dependency conflict warnings are common in collab environments, and usually your code will be able to run even with these dependency conflicts. So if the notebook ran successfully, then that basically means that the version differences in these dependencies won't break functionality, and in that case, I would just recommend you can ignore the warnings. There are a couple other ways to address this. So one is to use the is to restart the run time in collab

Speaker 1: and then kind of rerunning the cells from the from the top of the script. So this can resolve some issues.

Speaker 1: And then there's also a way to kind of specify the the versions of of what PIP install, uh, what PIP is installing. So here's an example of a way to do that, which I'm highlighting here. And then there's also a way to run Yeah, there's also a way to run the PIP install command with with to specify no dependencies, so that's dash dash no dash depths flag so and you can do that if you know other dependencies are are satisfied. So essentially, in summary for training purposes, if your code and uh, your code runs and you can authenticate and downlo Jedi data successfully, you could safely proceed despite these warnings.

Speaker 1: They're primarily informational and won't affect the Jedi data processing functions. All right, let's get into question five, which asks why do you think Plot six in the two center divations example had a much more complicated structure than visually similar plots like Plot one. Is this potentially a factor of slope or likely just a more complex structure underneath the canopy. So this is really interesting question. And just to kind of elaborate a little bit more about what I think this first part of the question is is meaning is talking about?

Speaker 1: So the much more complicated structure that we saw was specifically where it was unclear just from visual inspection of that waveform where the ground return was and where the ended and where the vegetation began, right, and so and then and then it's it's it's great that this that this participant, you know, thought of of a couple of possibilities as to why this is happening, right, So one one question was could it be slope or could it be more complex structure underneath, And the the answer I have is that both factors could contribute to this complex waveform structure that you're seeing.

Speaker 1: So we discussed this on slide thirty eight, where slope induced wave wave from contamination is a known limitation. And so you can imagine we have a twenty five meter footprint, right, and then if there's a lot of steep topography within that those twenty five meters, the the laser pulses will hit the ground at different elevations simultaneously distorting the waveform shape and creating multiple peaks, So that could be happening right and then, but you can you can also think about ways to determine which of these factors is dominant by checking the elevation bin zero values and examining whether there's a significant kind of terrain variation within the area.

Speaker 1: And then you can also look at the degrade flag for the shot. So if the six ground finding algorithms disagreed significantly so more than two meters, then it would be flagged and might indicate slope contamination. That said, complex waveforms can also genuinely reflect real vertical structural complexity, where if we have multiple cannonbu layers, dent especially dense understory, or heterogeneous vegetation in the footprint. So the example on side thirty three showed how the ambiguous waveform with the multiple peaks can result from either train effects or actual vegetation complexity.

Speaker 1: So if the particular area and the footprint has relatively flat terrain based on a digital elevation model that we data set that we could look at, then the complex structure is more likely of a possible reason why there was real candid you know, the real canopy architecture with multiple layers close to the ground could could be happening great. So question six asks, is it possible to access the Jedi data and use the functions shown in the exercises using the OUR software package our software practice?

Speaker 1: Is there an equivalent of Harmony for R to allow spatials upsetting of the orbits prior to download. So yes, our users can access and work with Jedi data using the R Jedi package, which was developed by the Jedi Science team specifically for processing Jedi data, and OUR provides functions to download dead I data from the land processes, DAC, read HDF five files, extract metrics, and perform spatial and temporal filtering. For Harmony like spatial subsetting capabilities in OUR, you could try the HARP R package, which provides an R interface to NASA's Harmony services and should support spatial subsetting of DEADID granules before download.

Speaker 1: And this I haven't actually you. It's been numerous years since I've used R for this kind of functionality with deadI data, so I'm not sure you know what the up to date versions look like and what's been developed in the past couple of years. But I think that harp R should still be working. And there are also other kind of standard R spatial packages. Our HD five for reading HG five files and SF for spacial operations and HERA for raster operations can also you know, be work work well for this.

Speaker 2: And one thing to add here is that, in addition to the language, another thing to keep in mind is if you're running this in your local laptop or in the cloud. So one critical step here is to download files from the repository right from NASA DAK and then you're doing the subsetting. And we did this in collab because we wanted to do this in the cloud, so you're avoiding downloading very large files into your laptop. Yeah, to do the subsetting or to do the filtering, I should say, so Harmony is doing the subsetting for you.

Speaker 2: So that's that's another thing to consider. And in collab, of course you can you can implement things in our as well.

Speaker 1: So yeah, so we're coming close to the top of the hour here, so we'll answer I'll answer question eight. So it looks like question seven was a repeat of five, so you can see that explanation true covered question eight. Is there a legend planned for the future. That explains the colors, line types, and access labels RH twenty five, R fifty or seventy five a peer as labels, but without any explanation of what they mean. And then there was an example here of RH fifty equal is is the height below which fifty percent of the reflected energy was received, meeting height of vegetation.

Speaker 1: So that is actually in my explanation here, I'm going to address that because that's actually not quite correct interpretation of RH fifty. So let's start actually just with the overall point. You know, I think is that for lay people, you know, and it sounds like this this participant is specifically in public health applications. You know, just looking at a graph without much interpretation of a waveform is not you know, there's not it's not interpretable. And that's I think, you know, an important point about thinking about how we can make JEDI data and visualizations more accessible to interdisciplinary audiences.

Speaker 1: So I really appreciate that the question and the intention. I do want to quickly clarify a distinction that. So the first part here, so our age fifty is is the height at which fifty percent of the cumulative energy return has been received. So that's the height of median energy of the waveform, not necessarily the median height of vegetation. I just want to clarify that and so, and just to further elaborate a little bit, is it's just that the energy distribution reflects complex interactions between laser penetration and canopy density throughout the vertical profile.

Speaker 1: So it's probably related and close to median height, but not not exact, not exactly, and it could it could vary. So the training addresses wayform interpretation through annotated examples that we provided in slides thirty through thirty three, right, so we identify We added interpretations directly onto the figures where we were circling where the ground returns were versus where above ground vegetation returns were, explaining how with multiple peaks there's likely an understory as well as you know, a canopy height at corresponding to one of the top you know, the tallest elevation peak.

Speaker 1: And then we talked about, you know, the cases where there's just a single ground return and how generally that corresponds to areas where there aren't any trees in the footprint, the sparse vegetation or bare ground. However, hard coding these interpretations into the automatic plotting of the code would create issues in terms of broad applicability and reproducibility across different forest types and conditions and across the different you know, standard deviations even right that we that we plot.

Speaker 1: So we wanted to make the code flexible and versatile for adjusting the areas of interest, adjusting the the standard deviations that we're using to select which waveforms we wanted to plot. So that's why we kept the demo code plotting functions just generalizable and and basic in terms of the axes that we that we have, and not refraining from labeling the waveforms directly, you know, but for other interdiscplinary uses there developing materials for non remote sensing audiences, you know, I think would would certainly benefit from adding custom annotations to plots with for example, matt plotlib and creating supplementary documentation or you know, on your own, you can copy this code from the demo and then modify the code and the plotting functions to include text boxes that you know, do explain the key features, if that's something that you'd like to do.

Speaker 1: All right, we are now a few minutes past the hour, so I think that's going to be the last question that we'll answer in real time here, but look for the Q and a document where we will fill out answers to all the questions that we receive. And I just want to thank everyone again for your participation and looking forward to seeing you all next week for Session two.

Speaker 2: Thank you everyone, See you next week.

Speaker 4: M

The Story