Skip to main content
Move the World.
Removing Water From Underwater Photography

Disclaimer: Sea-thru is protected by a registered patent owned by Carmel Ltd the Economic Co. of the Haifa University and its subsidiary SeaErra. Any use of this technology or other material presented here is prohibited without a prior written approval by the owners.

Derya Akkaynak remembers clearly the day she saw the sea without water.

A late summer rain had forced her to abandon her ritual of a Sunday motorcycle ride, and, consigned to the inside, she decided to tweak the algorithm she had been working on for the last three and a half years. She input the changes that had been rattling around her mind — it just hadn't worked so far — and the image she had spent months staring at, an image made muted, alien, by the real-world Instagram filter of the ocean, suddenly burst into life.

Stripped by her algorithm of the water and its warping effects — bane of underwater photography — the corals along the bottom of the image became a dazzling array of pinks and purples, blues pied by yellows, a breathtaking display that causes excitement in her voice even a year after that rainy August day in 2018.

"My first thought," Akkaynak says, "was why is everything so colorful?'"

Underwater Images Without the Water

Underwater photography and videography is at the mercy of the water. The underwater world can look vastly different from place to place. Your underwater photos could be shot in the ethanol-clear waters of Hawaii, the grey haze of the Atlantic, or in the iced tea opacity of a rushing river. Even underwater images from the same location will be maddeningly variable.

One day, silt could create a small wall of particles, cutting off your vision; the next, an algae bloom could change your world from blue to green. The time of day will have an impact, as the angle of the sun changes; even the angle of the camera. The water interacts with the light in unusual ways, muting colors and warping perceptions. All of this makes underwater pictures and videos harder to analyze for scientists.

Growing up in Turkey, Akkaynak — her first name means "ocean" — spent months of her childhood in the small fishing village of Cesmealti on the Aegean Sea. There, she would spend her days swimming around and poking at the marine life, trying to understand how and why they do what they do.

Derya Akkaynak standing on a sailboat on the coast of the Aegean Sea, flanked by her older sister and their neighbor. Growing up in the small fishing village of Cesmealti, Derya—whose first name means “ocean”—was enamoured with marine life and the sea. Photo courtesy of Derya Akkaynak.

Derya Akkaynak standing on a sailboat on the coast of the Aegean Sea, flanked by her older sister and their neighbor. Growing up in the small fishing village of Cesmealti, Derya—whose first name means “ocean”—was enamoured with marine life and the sea. Photo courtesy of Derya Akkaynak.

After a stint as an aerospace engineer, she realized she needed to return to the ocean. While studying the remarkable camouflage abilities of octopuses, she became frustrated with the quality of the underwater images she was working with.

"Any image you take underwater, it's just degraded by this layer of green or blue or, if you're in a murky place, brown," Akkaynak says. It's hard enough to pick out an octopus underwater. Akkaynak was vexed by the water's impact and wished she could eliminate it.

A few years later Sea-thru was born, an imaging algorithm that makes underwater images appear like they would on land. Developed in the Marine Imaging Lab of Dr. Tali Treibitz at the University of Haifa in Israel — the university is currently in the process of commercializing it — Sea-thru does more than just color correct.

A coral reef in the Red Sea, Israel. Matan Yuval is a doctoral student building 3D models of coral reefs, at scales ranging from meters to kilometers, to study different questions about their ecology. Image by Matan Yuval, Marine Imaging Lab, University of Haifa.

A coral reef in the Red Sea, Israel. Matan Yuval is a doctoral student building 3D models of coral reefs, at scales ranging from meters to kilometers, to study different questions about their ecology. Image by Matan Yuval, Marine Imaging Lab, University of Haifa.

"Many people say, 'But I can do this in Photoshop!'' Akkaynak laughs. In addition to being performed by an algorithm rather than a person, so it is more objective and efficient than Photoshop, her physics-based algorithm does not need a color chart, and it does not simply layer adjustments on top of the image until it pleases a subjective eye test. It uses the physical properties of water and light to reverse engineer the impact of the water on the raw image data.

Essentially, the algorithm understands what water conditions are doing to the light, and, by working backwards, it reverses those conditions. The end result is as if the water was never there.

Beyond Making Underwater Photography Look Better

Sea-thru's images are arresting. Colors pop, the movements of marine life become clearer, edges and shapes take on sharper focus. But the algorithm was designed to do more than just make underwater photography and videography more pleasing to the eye; it was designed to make these underwater images easier for scientists to analyze. By automatically adjusting, cleaning, and standardizing imaging data, Sea-thru may be able to save scientists a ton of time.

"That was my goal," Akkaynak says. "To help speed up the pace of marine science." Additionally, Sea-thru may solve the fundamental problem of recovery color and contrast (and find those damn octopuses).

Alex Jordan has been putting Sea-thru to the test. A researcher at the Max Planck Institute of Animal Behavior, he studies how social behavior evolves and is impacted by the environment. Jordan does this by studying populations of social spiders (your author's new nightmare) and cichlids in Lake Tanganyika.

The fish in the African Great Lakes are famous for their incredible diversity, making them perfect for Jordan's research. One type of cichlid in the lake will be incredibly social, and another, closely related type, a lone wolf, allowing for Jordan and his team to analyze and compare. They do this by filming hours and hours and hours of video footage.

That footage is cleaned up then fed to a machine learning algorithm, which will help pick out the little tics, motions, and nuances of the cichlid's behavior. With its superb pattern finding ability, the algorithm can then help Jordan better understand what's happening in cichlid's social lives.

Machine learning requires clean, consistent data to learn from — which underwater images are not. If you want the algorithm to track this fish and not that fish, the water conditions can make it tough. Say your underwater images one day are of a black and white fish in a brown world; the algorithm can learn that. Then the next day an algae bloom hits, and your fish is now green and gray in a yellow world!

"Remember that dress illusion that went around a year or two ago?" Jordan says. "It's kind of exactly like that. You're trying to train a model to detect a difference, but then day to day to day those differences themselves are different. And it just causes us incredible headaches."

Sea-thru standardizes that data and eliminates the water conditions that make underwater images difficult for algorithms and humans to analyze.

Marine biologists aren't the only researchers who need their underwater images clear as day. Archeologists rely on underwater photography to document and research important finds, and use video to survey and capture artifacts, shipwrecks, and other remnants of humanity scattered under the sea.

A coral reef in Lembeh Strait, Indonesia. Derya Akkaynak is a mechanical engineer and oceanographer whose research focuses on problems of imaging and vision underwater. Image by Dr. Derya Akkaynak, Harbor Branch Oceanographic Institution, Florida Atlantic University.

A coral reef in Lembeh Strait, Indonesia. Derya Akkaynak is a mechanical engineer and oceanographer whose research focuses on problems of imaging and vision underwater. Image by Dr. Derya Akkaynak, Harbor Branch Oceanographic Institution, Florida Atlantic University.

Zoë J. Wood is a professor of computer science at Cal Poly whose research focuses on computer graphics and modeling. Working with engineer Christopher Clark at Harvey Mudd College and archeologist Timmy Gambin from the University of Malta, Wood helped develop algorithms for identifying and mapping shipwrecks off Malta's coast. Two years ago, they found a rare British plane from World War II, now home to crustaceans and baby fish.

The researchers want to not only excavate ancient wrecks, but also be able to compile high-resolution, 3D models of wrecks and use that information to keep tabs on their condition year after year. Using autonomous underwater vehicles (AUVs) — swimming drones — they take underwater images, which can then be stitched together to create models of the ship.

The images Wood is working with face the same challenges as Jordan's; the light coming through the water means the images are not as detailed and accurate, leading to less accurate modeling.

"We have hours and hours of blurry blue video," Wood laughs.

Sea-thru could help their research "massively," Gambin says. It could improve the 3D models and reduce image processing time.

The algorithm may make another intersection of humanity and the sea clearer, as well. Jordan sees potential in combining Sea-thru with underwater photography and videography to create environmental awareness. Remove the polluted waters from a coral reef, for example, and the true beauty of what could be may shine through.

A coral reef in the Red Sea, Israel. Researcher Alex Jordan hopes Sea-thru will also inspire a new wave of environmental awareness. Image by Matan Yuval, Marine Imaging Lab, University of Haifa.

A coral reef in the Red Sea, Israel. Researcher Alex Jordan hopes Sea-thru will also inspire a new wave of environmental awareness. Image by Matan Yuval, Marine Imaging Lab, University of Haifa.

A Different View of the Sea

Seeing his cichlids and their environment clearly for the first time was a special moment, Jordan says. There is red sand at the bottom of Lake Tanganyika, which always comes up in his boots. Watching the fish swim against this red backdrop for the first time was moving.

"It's really nice to see the fish in their true color in their environment," Jordan says. It was like capturing absolutely perfect water conditions.

While Akkanyak's goal was to help scientists analyze their data better, freeing them up for more research, the eerie beauty of the underwater images Sea-thru creates cannot be denied.

She returned to the sea, the setting of her youth, and has made it disappear.

Up Next

Social Entrepreneurs
Austin Is Facing Food Insecurity. This Student is on a Mission to Solve That.
food insecurity
Social Entrepreneurs
Austin Is Facing Food Insecurity. This Student is on a Mission to Solve That.
A med student thinks the key to reducing food insecurity is through transportation. He launched Good Apple, a healthy food delivery service for families in need.

A med student thinks the key to reducing food insecurity is through transportation. He launched Good Apple, a healthy food delivery service for families in need.

The Future Explored
Mojo Vision Wants to Give Our Eyes Superpowers
augmented reality contact lenses
The Future Explored
Mojo Vision Wants to Give Our Eyes Superpowers
AR contact lenses will project the digital world into our retinas.

AR contact lenses will project the digital world into our retinas.

Robotics
Meet Snatcher, the Chameleon Lizard Robot With an Ultra-Fast Tongue
chameleon lizard robot
Robotics
Meet Snatcher, the Chameleon Lizard Robot With an Ultra-Fast Tongue
The chameleon lizard has a highly specialized tongue. Now, a team of engineers created a quick-tongued robot.

The chameleon lizard has a highly specialized tongue. Now, a team of engineers created a quick-tongued robot.

Transportation
First Autonomous Air Taxi Terminal to Open in 2020
air taxi
Transportation
First Autonomous Air Taxi Terminal to Open in 2020
Chinese autonomous aerial vehicle company EHang plans to build the world’s first air taxi terminal in Hezhou, China, before the end of 2020.

Chinese autonomous aerial vehicle company EHang plans to build the world’s first air taxi terminal in Hezhou, China, before the end of 2020.

Future of Cities
Paving the Way With Recycled Roads
Paving the Way With Recycled Roads
Future of Cities
Paving the Way With Recycled Roads
The world is facing a massive build up of waste. But this solution of recycled roads may pave the way for a cleaner future.

A recycled road has been paved with asphalt that contains the equivalent of hundreds of thousands of plastic bags, along with thousands of glass bottles and printer cartridges’ worth of waste toner. In addition to the sheer amount of recycled materials the process will divert away from landfills, these longer-lasting roads also help to reduce the carbon footprint of construction.

Making a Difference
Can People with Autism Help Create Next-Generation AI?
Can People with Autism Help Create Next-Generation AI?
Watch Now
Making a Difference
Can People with Autism Help Create Next-Generation AI?
Daivergent is a new startup that hires people with autism to train artificial intelligence - and helps them start independent careers.
Watch Now

Meet Bryan Dai, the founder of Daivergent--a startup that hires people with autism to train artificial intelligence and helps them start independent careers. His journey began when his mother passed away, and he knew that he would be responsible for helping support his brother with autism. After people with autism turn 21, they often encounter the “support cliff,” after which they stop receiving many forms of government...

Computer Science
An Address for Everywhere on Earth
An Address for Everywhere on Earth
Watch Now
Computer Science
An Address for Everywhere on Earth
Can three simple words change how we find each other?
Watch Now

We take addresses for granted - but billions of people and places don’t have them, and it’s a big problem. Whether it’s voting, disaster relief, or pinpointing a spot on festival grounds, not having an address makes things that should be simple difficult. Enter Chris Sheldrick, who coordinated events in the music industry where he was frustrated by address-related problems. He created What3Words, a method of dividing the...

Challengers
Failure is Inevitable, But It Doesn't Have to Ruin Your Life
Failure is Inevitable, But It Doesn't Have to Ruin Your Life
Challengers
Failure is Inevitable, But It Doesn't Have to Ruin Your Life
Why learning to suck at something is the only way to get good at it.
By Mike Riggs

Why learning to suck at something is the only way to get good at it.