Advances in 3D graphics have made movies and video games more realistic, but can also have an impact on science. Associate Professor Eugene Zhang and Associate Professor (Senior Research) Yue Zhang describe their research to help medical doctors better target cancerous tumors by using 3D modeling and simulation.
Transcript
[MUSIC: Oceanside Drive, Ethereal Delusions, used with permission of the artist.]
ROBERTSON: Hey folks, just a quick note to let you know that we are trying out a new format for Engineering Out Loud. Instead of pairing two related stories in a longer podcast we are creating shorter podcasts that feature just one story. We’d like to hear from you about what you think. You can email us at engineeringoutloud@oregonstate.edu or send us a note to our Facebook account. Now, onto the show.
[MUSIC: The Ether Bunny, Eyes Closed Audio, used with permissions of a Creative Commons Attribution License]
NARRATOR: From the College of Engineering at Oregon State University, this is Engineering out Loud.
ROBERTSON: When you think of engineering do you think of human and environmental health? Maybe not so much. So, for this season we are focusing on stories of how research in engineering at Oregon State can impact broad areas such as cancer treatment, food contamination and the detection of nuclear weapons tests. We are going to start with the topic of 3D modeling and learn how it can advance science in ways that you might not have imagined.
[AUDIO CLIP: from Toy Story]
Video games and movies are what we conjure up when we think of 3D modeling. But it’s also a tool that can help medical professional better target cancerous tumors. Today we’ll start by talking to Eugene Zhang, a professor of computer science at Oregon State University and an expert in computer graphics and data visualization. And, of course, I start with the most important question first.
ROBERTSON: As far as a good example of animation, what is your favorite?
EUGENE ZHANG: Well my favorite is the Toy Stories, as well as Star Wars, I like Star Wars because, you know, my kids like them so I started to really get into them as well.
ROBERTSON: The Star Wars that Eugene is talking about is the animated TV series The Clone Wars from Lucasfilm Animation that ran from 2008 to 2014.
[MUSIC: Oceanside Drive, Ethereal Delusions, used with permission of the artist.]
One of the planets in this show, called Mustafar, is a volcanic planet that has constant eruptions flowing into rivers of lava. You’ll see why I mention that in a minute. I asked Eugene what has changed to take us from very rudimentary 3D wire-frame graphics, such as the depiction of the Death Star plans in the first Star Wars movie in 1977, to the very sophisticated 3D images you see today.
E. ZHANG: Computer animation has matured as a field of research over the years and now we are talking about technologies that have been developed in the last 10-20 years with a strong focus on numerical simulation. Like simulation of fluids. Flows, like the lava flows you see in Star Wars. There have been a lot of techniques to speed up the simulation that would have been impossible before the GPU era.
ROBERTSON: The GPU era which stands for graphics processing unit that Eugene mentions had a breakthrough in 1995 when the first 3D add-in cards came out on the market.
E. ZHANG: With the increased computation capability we are now able to produce a movie at a much faster rate than say 30-40 years ago, for instance it might take 6 months to make another episode for Star Wars.
ROBERTSON: So, how is what you do with 3D modeling related to what people see in the movies?
E. ZHANG: So, one of the things that a lot of people do not necessarily realize is that even day one computer graphics animation, 3D modeling has been an integral part of that. People see these fascinating shapes, motions, but there has to be a way for them to be represented in the computer so that the artist can manipulate these shapes make them move, make them change the form and the more efficient the representation, the easier it is for the artist to work with them. And also for scientific simulation like the simulation of lava it involves very sophisticated mathematical modeling techniques that require very well-designed meshes.
ROBERTSON: Okay, so I’m going to jump in here and explain that a mesh made up of points in a 3D space called vertices, there are lines that connect these points, called edges. And the face is the area between the edges.
E. ZHANG: So, in fact if you think about a simple case. Let’s say you are talking about a cube. A cube has 8 corners, and these corners will be the so called vertices and there are 12 edges separating the 6 faces of the cube. So the surface of the cube consists of 6 faces would be exactly a mesh. And another example is to look at Spiderman. Spiderman has this very interesting sort of mesh like pattern on the cloth. You can see points and lines and these lines intersect at a right angle and they form these rectangular patterns on the cloth of the Spiderman. That’s a simple version of a mesh.
ROBERTSON: The reason we are spending some time with meshes here is that from what I understand from Eugene, the mesh is really, really important in 3D modeling. The mesh is the mathematical bones onto which everything else is applied. For applications like animations or art or architecture a good mesh will allow you to create more realistic images. But, perhaps, more importantly when 3D modeling is used for scientific research such as simulations of tornados, earthquakes and tsunamis the results will be more accurate.
E. ZHANG: In fact, it's known in the community that 90 percent of the time that simulation researchers spend on performing a simulation was spent on generating a really good mesh. And only 10 percent of the time was to actually run the simulation.
ROBERTSON: So, since Eugene works on perfecting these meshes, his work is fundamental to all 3D modeling and can be applied to any field. Even the modeling of complex internal organs. My next guest will explain why she and Eugene would want to do that.
YUE ZHANG: My name is Yue Zhang. My research area is in numerical simulations and scientific visualizations.
ROBERTSON: So, Yue’s background is in mathematics, and along the way she discovered that what she really enjoys is collaborating with others to find mathematical solutions to real world problems.
Y. ZHANG: Looking at fast algorithms and also complex mathematical modeling is very interesting, very challenging. On the other hand the real-life problems can enrich these models even further, because in real life there are always factors that need to be considered in the mathematical modeling.
ROBERTSON: So, that’s why she took the time to attend a mixer between researchers from Oregon State University and Oregon Health & Science University. The connections she and Eugene made there eventually led to a collaboration with Dr. Wolfram Laub in the Department of Radiation Medicine. The problem that Laub wanted help with is better targeting radiation therapy for tumors associated with prostate cancer. Yue explains why this is important.
Y. ZHANG: The difficulty is from the clinical side there is a safety margin because the radiation has toxicity and you can hurt the neighboring healthy organs. The larger the safety margin, the more harmful it is for the neighboring organs and to reduce this safety margin so that only the tumor portion is treated with the right amount of dosage is what we are trying to help with.
ROBERTSON: Next Yue tells us why targeting the tumor at the right location can be trickier than you might think.
Y. ZHANG: Because the organs are actually moving, so the organ shapes and positions are very important, are critical because the patients can have some movement the organs and the geometry can change.
ROBERTSON: So, people might not understand that, so why ... usually we think of our organs as fairly stable so why are they moving?
Y. ZHANG: Breathing, breathing and other bodily functions.
ROBERTSON: Think about that cup of coffee you had this morning. What’s going to happen to during a radiation therapy treatment that could take hours. Your bladder is going to fill up. And if there is a tumor next to it, the location of the tumor will change as the bladder pushes it out of the way. And so, to create a simulation of how the organs might move and change the location of the tumor, they first have to start with a 3D model constructed from the medical scans.
Y. ZHANG: The scans are 2D and they are taken at different slices on the human body so that we can construct a volume with a volume then we put mesh on this volume. Like what Eugene was saying we put node, edges and cells and faces on this volume. That now we have…
[record scratch]
ROBERTSON: Okay, I’m going to stop the tape there a second to bring in a conversation with Eugene where he talks about adding volume to the mesh.
E. ZHANG: We have an additional element called the cells, for instance in this case I would go back to the cube example, in addition to the 8 corners, 12 edges and 6 faces you can also think of the interior of the cube being a cell and if you put a number of cubes together you would get what is called a hexahedral mesh which means the mesh is made mostly of cubes, and in that case the number of cubes would also be an indicator how complicated the mesh is. For instance, for the simulation that Yue deals with there are usually millions of cells that were involved to generate a realistic simulation.
ROBERTSON: And now back to Yue
Y. ZHANG: Now we have tools to describe material properties on these cells, so if it’s an organ that doesn’t move much it has a little bit rigid than we describe one material property, but if it is something like the bladder, it’s very flexible, stretches a lot we use a different property to describe it.
[MUSIC: Oceanside Drive, Ethereal Delusions, used with permission of the artist.]
ROBERTSON: Now this is getting way more complex that a cool 3D animation. And it brings in another specialty of Eugene’s, called field processing which they are using to add information about the material properties of the organs.
E. ZHANG: Field processing is a very new subfield of geometry processing. Instead of modeling a shape, now we are modeling things that are on the shape. It's one thing to model the shape of the earth, to model the mountains and the ocean and so on and so forth, but it's another to model the magnetic field or the global ocean current flows on the earth and these are vector fields on the surface that can provide a lot of insight into things such as the air stability, pollution and climate change. And tensors are an extension of vector fields whereas a vector field has a direction a tensor field has multiple directions. So a tensor field is more complicated than the vector field and they can be only described mathematically by a matrix. So, you can sort of see the additional complexity here.
ROBERTSON: Indeed, so now you have millions of cells with an additional layer of information about the material properties of the organs. So, you can see why it might take a couple weeks for the simulation to run. But what field processing allows them to do is find the critical points where there is uncertainty in the model that can indicate change. Yue describes why this important for treatment.
Y. ZHANG: By looking at the simulation results the doctor can see a little bit more about what could be the changes to the prostate through the treatment period. Currently the doctors from the clinical side they scan the patient on the first day, so they have an initial scan which is a MRI -- very detailed. Then at that point they develop a treatment plan which includes the direction and the dosage of the radiation. But that's a static method. It's not dynamic it's not adaptive, so using our simulation we hope the doctors can have some predictive knowledge of where the organs could be and why the organs could be at one have one shape during the treatment. In addition, we like to track the how the material is changing through the radiation period.
ROBERTSON: Specifically, they would like to see if the tumor is changing… hopefully shrinking. And if other organs are being affected by the treatment.
Y. ZHANG: What we are hoping to achieve is we will get adaptive treatment plan and individualized for each patient. No two patients are the same. Cancer development varies from patient to patient, their ages, their health conditions, their family histories, all different. What we are trying to do here that is novel is we want to include bio mechanical modeling the simulations we want to include the tensor visualization on the material stress tensors.
E. ZHANG: We believe that tensor field visualization and analysis is key to the medical applications that we are talking about here as well as many other application going back to earthquake, tsunami analysis. This is a new direction for the graphics community but I would want to go bigger than that say it's actually something that faces the whole scientific community -- is to look at ways of modeling everything including the shape and the materials properties on the object. However, it is very challenging as Yue has mentioned, the fact that we are not doing biopsies on people then we only have the scans so we are sort of limited to extracting information about material properties through geometry information, like pixels the color of pixels and there is a lot of guessing work, so I'm hoping that there will be other ways, but non-invasive still to help us in order to model the material properties like the tumors.
[MUSIC: Oceanside Drive, Ethereal Delusions, used with permission of the artist.]
ROBERTSON: 3D modeling is inherently cool, and know you may know a little bit more about how complex it is. In conclusion I asked Eugene what it is about this project that sparked his interest.
E. ZHANG: It's interesting to reflect why I'm very interested in the problem. I guess growing up I had always been sort of interested in abstract things like mathematics geometry. I have also been very interested in science, but I have always considered them separate and unconnected. And this project is one of the projects that I finally feel like the two sides of my interests have started to converge. Where we are doing mathematically motivated research but with real impact where we really want to help patients to survive, to overcome cancer. I'm really hopeful that the techniques we are developing or the tools we are building will useful not only to architects or artists in Disney or Pixar but also available to scientists, doctors that could actually save lives and overcome all these diseases including cancer, various forms of cancer, and AIDS.
ROBERTSON: Okay, my friends, that concludes our first episode on human and environmental health stay tuned for more episodes on how researchers in engineering are working to improve our lives. And remember we want your feedback on the new format for Engineering Out Loud. You can email us at engineeringoutloud@oregonstate.edu or send a message to our Facebook account.
[MUSIC: The Ether Bunny, Eyes Closed Audio, used with permissions of a Creative Commons Attribution License]
This episode was produced by me, Rachel Robertson, with additional editing by Miriah Reddington. Our intro music is The Ether Bunny by Eyes Closed Audio on SoundCloud and used with permission of a Creative Commons attribution license. The other music in this episode was Oceanside Drive by Ethereal Delusions (who is an Oregon State music production student) you can find him on Twitter @quixoticBPMoose. The sound effect was used with permission of Creative Commons. Links for additional materials can be found on our website. For more episodes visit engineeringoutloud.oregonstate.edu or subscribe by searching “Engineering Out Loud” on your favorite podcast app.
