Distortion Correction Algorithms Adapt Video in Real-Time

Imaging technology company Immervision said it is now able to offer real-time video distortion correction software algorithms to remove stretched bodies, change lines, objects and face proportions and adjust scenes in real-time on images and videos captured on smartphones.

Patrice Roulet Fontani
Patrice Roulet Fontani

The company will license the new algorithms directly to mobile OEMs through its distribution partner and lead investor CEVA. In an interview with EE Times, vice president and co-founder of Immervision Patrice Roulet Fontani said this new real-time, on-the-fly distortion correction wasn’t really available before. “Some things were there, but to be able to run on a portable device without draining the battery, and providing the appropriate correction, you didn’t have that before.”

He explained the background. “Smartphones are having wider screens, but up to now, whenever you look at a video, it’s still in a 4 by 3 type of ratio. If you want to fill the screen you need to process the image to crop and then split and then increase the size of the image on your sensor. Now with cinematographic and wide-angle lenses, they allow you now to fill the screen, but not without having some artefacts or some distortions. Correcting the picture is one thing, because you can snap the picture, maybe get half a second to process it and modify it. But with a 30-120 fps video, that’s another story.”

“To bring that distortion correction, we have introduced a new algorithm that not only corrects the distortion in terms of one setting, but you can also have multiple types of distortion correction, depending on the situation. And it can be done in real time, correcting areas for example where there are people, we might want to have the people look more normal. Where you have panoramas, you maybe want to provide a feeling of being a wide, immense area.”

Immervsion distortion correction
With skyrocketing demand for video recording on phones, software is needed to create the ideal framing for high-quality videos. Immervision said its new algorithms help deliver “20/20 vision” in a range of smartphones, correcting video distortions in real-time to improve the captured to display experience, without compromising the field of view. (Image: Immervision) 

“What you might see in existing devices today is that they do not correct the distortion at all. There is distortion correction for images, and videos, but they are focused very much on the straight line. And they correct only one distortion correction feed that feeds all the applications scenarios. What we notice with this is, you cannot have one processing that corrects the straight line, and at the same time preserve the body or face proportions. So, in most smartphones today, the body and face proportions as you go towards the edge of the FOV [field of view] are very much distorted. So, they are stretched where the face or the body appear wider than they are.”

“Instead of having one processing feed for all scenarios, we provide smartphone OEMs different video distortion correction processing, depending on the scenarios. So, if you want to capture a landscape, it would be a different processing to if you have a group of people or portrait image. We developed video distortion correction plus what we call face protection, to make the faces appear as they are in reality and not distorted and not stretched. In addition, this process is performed in real time. Our commitment to our customer is to enable what you see is what you get.”

In many newer phones, the wider field of view creates more apparent distortion. Immervision said its new software algorithms help remove stretched bodies, change line, object and face proportions in real-time adjusting to the scene, and reduce editing time, all in one solution.  The software provides different levels of correction, varied projections and real-time adaptation to the scene. This gives smartphone OEMs the capability to offer different options to its users – for example, to leave preset configurations as they are, or be able to fully customize, or let end-users make the decision, as well as allow phone orientation to dictate or leverage machine learning to control the experience.

Fontani said being able to enable distortion correction in real-time is important. “Because when you are filming, there are two things happening. One you are recording, or you are broadcasting, that needs to be done as it goes. You cannot record and then process after.”

Immervision said with the skyrocketing demand for video recording on phones, software is needed to create the ideal framing for high-quality videos. Having worked with its customers for 20 years, it said its hardware and software experts have been working closely with OEMs to fine tune the algorithms to meet specific needs. The new algorithms, it added, help deliver “20/20 vision” in a range of smartphones, correcting video distortions in real-time to improve the captured to display experience, without compromising the field of view.

Adding an off-the-shelf 125 degrees wide-angle lens
In a second announcement, Immervision also introduced what it claims to be the highest-performing off-the-shelf 125-degree wide-angle lens, pre-configured on popular sensors without need for camera customization, and aimed at tier two and tier three mobile OEMs.

Wide-angle lens reduces the number of cameras and costs, while improving image quality for landscape, group, and close-up pictures and videos taken with mobile phones.  While the majority of tier one OEMs have wide-angle lens in their phones, tier two and tier three mobile brands have yet to adopt. Immervision’s technology has been preconfigured on popular sensors including Sony, Omnivision and Samsung.

“Tier two and tier three mobile OEMs are challenged with delivering the same level of image quality of leading-edge mobile phones,” said Immervision’s executive vice president, operations and chief commercial officer, Alessandro Gasparini. “Immervision’s 125 wide-angle lens integrates all the industry-leading metrics into one lens. In the past, these best-in-class metrics were found in multiple phones. Now they are available in one lens with ready-to-use software, reducing camera customization and integration time, and providing the same image quality as in well-recognized mobile phone brands.”

“This is a new optical design in which we achieved the best balance between different parameters to reach the widest possible FOV that doesn’t compromise the f-number, the relative illumination, and without any drop in resolution up to the edge, up to 21 megapixels. It will allow tier two and tier three phone vendors to be able to offer a complete experience going from wide angle to traditional imaging with one single camera. In other words, having two different features on the same camera.”

He added, “What we are doing is making this technology accessible, since it’s usually only in flagship big brand phones. This will be the best lens in terms of performance achieved at wide angle.”

The post Distortion Correction Algorithms Adapt Video in Real-Time appeared first on EETimes.

Read more »

What Really Does and Doesn’t Work for Fair Use in the DMCA

On July 28, the Senate Committee on the Judiciary held another in its year-long series of hearings on the Digital Millennium Copyright Act (DMCA). The topic of this hearing was “How Does the DMCA Contemplate Limitations and Exceptions Like Fair Use?

We’re glad Congress is asking the question. Without fair use, much of our common culture would be inaccessible, cordoned off by copyright. Fair use creates breathing space for innovation and new creativity by allowing us to re-use and comment on existing works. As Sherwin Siy, lead public policy manager for the Wikimedia Foundation, said in his testimony: “That fair uses aren’t rare exceptions to the exclusive rights of copyright law but a pervasive, constantly operating aspect of the law. Fair use not only promotes journalism, criticism, and education, it also ensures that our everyday activities aren’t constantly infringing copyrights. Especially now that so much of our lives are conducted on camera and online.”

Unfortunately, the answer to Congress’s question is: not enough. The DMCA, in particular, by design and as interpreted, doesn’t do enough to protect online fair uses. This is the case in both Section 1201 of the DMCA—the “anti-circumvention” provision which bans interference on technological restrictions on copyrighted works—and Section 512—the provision which immunizes platforms from liability for copyright infringement by their users so long as certain conditions are met.

Fair Use and Notice and Takedown

The DMCA was meant to be a grand bargain, balancing the needs of tech companies, rightsholders, and users. Section 512 embodies a carefully crafted system that, when properly deployed, gives service providers protection from liability, copyright owners tools to police infringement, and users the ability to challenge the improper use of those tools. Without Section 512, the risk of crippling liability for the acts of users would have prevented the emergence of most of the social media outlets we use today.

But Congress knew that Section 512’s powerful incentives could result in lawful material being censored from the Internet, without prior judicial scrutiny, much less advance notice to the person who posted the material, or an opportunity to contest the removal. For example, users often making fair use of copyrighted works in all kinds of online expression. That use is authorized by law, as part of copyright’s “built-in First Amendment accommodations.” Nonetheless, it is often targeted for takedown under the DMCA. 

For Section 512, user protections are supposed to be located in sections 512(g) and 512(f). In practice, neither of these sections have worked quite as intended.

Section 512(g) lays out the requirements for counternotifications. In theory, if a takedown is issued against a work making fair use, then the target can send a counternotice to get the work restored. The counternotice contains personal information of the creator and an agreement to be subject to a court case. If the person or organization doesn’t respond to the counternotice with a legal action within two weeks, the work goes back up. In practice, very few counternotices are sent, even when the original takedown was flawed.

512(f) was supposed to deter takedowns targeting lawful uses by giving those harmed the ability to hold senders accountable. Once again, in practice, this has done little to actually prevent abusive and false takedowns.

Columbia Law Professor Jane C. Ginsburg agreed, saying that these parts of 512 “may not always have worked out as intended.” She highlighted that automated takedown notice systems don’t take fair use into account and that there are relatively few counternotices. She allowed that “fear or ignorance” would cause users not to take advantage of counternotices, a point backed up by cases of trolling and the intimidating nature of the counternotices.

Evidence of how users avoid the process was given by Rick Beato, a musician who also has a popular YouTube channel that teaches music theory. He noted that he has made 750 YouTube videos, of which 254 have been demonetized and 43 have been blocked or taken down. Beato noted that he’s never disputed anything – it’s too much trouble.

Several witnesses urged the creation of some sort of “alternative dispute resolution” to make taking down and restoring content easier. We disagree. Section 512 already makes takedowns far too easy. The experience of the last 22 years shows just how much of the fundamental right to freedom of expression is harmed by extrajudicial systems like the DMCA. The answer to the DMCA’s failures cannot be yet another one.

As for the European model, there is no way to square the Copyright Directive with fair use. The European Union’s Copyright Directive effectively requires companies to ensure that nothing is ever posted on their platforms that might be infringing. That incentivizes them to over-remove, rather than take fair use into account. And to handle that need, filters become necessary. And so it creates a rule requiring online service providers to send everything we post to the Internet to black-box machine learning filters that will block anything that the filters classify as "copyright infringement." And, as Beato testified to in the hearing, filters routinely fail to distinguish even obvious fair uses. For example, his educational videos have been taken down and demonetized because of a filter. And he is not alone.

Witnesses also suggested that fair use has expanded too far. This is a reassertion of the old bogeyman of “fair use creep,” and it assumes that fair use is set in stone. In fact, fair use, which is flexible by design, is merely keeping up with the changes in the online landscape and protecting users’ rights.

As witness Joseph Gratz put it:

Nobody likes to have their word, or their work, or their music used in ways that they can’t control. But that is exactly what fair use protects. And that is exactly what the First Amendment protects. Whether or not the copyright holder likes the use, and indeed, even more so where the copyright holder does not like the use, fair use is needed to make sure that free expression can thrive.

Fair Use, Copyright Protection Measures, and Right to Repair

On balance, Section 512 supports a great deal of online expression despite its flaws. The same cannot be said for Section 1201.  Section 1201 makes it illegal to circumvent a technological protection on a copyrighted work, even if you are doing so for an otherwise lawful reason.

Sound confusing? It is. Thanks to fair use, you have a legal right to use copyrighted material without permission or payment. But thanks to Section 1201, you do not have the right to break any digital locks that might prevent you from engaging in that fair use. And this, in turn, has had a host of unintended consequences, such as impeding the right to repair.

The only way to be safe under the law is to get an exemption from the Copyright Office, which grants exemptions to classes of uses every three years. And even if your use is covered by an exemption, that exemption must be continually renewed. In other words, you have to participate in an unconstitutional speech licensing regime, seeking permission from the Copyright Office to exercise your speech rights.

Nevertheless, Christopher Mohr, the Vice President for Intellectual Property and General Counsel of the Software and Information Industry Association, called Section 1201 a success because it supposedly prevented the “proliferation of piracy tools in big box stores.” And Ginsburg pointed to the triennial exemption process as a success. She said it “responds effectively to the essential challenge” of balancing the need for controls with the rights of users.

That’s one way of looking at it. Another is that even if you have an exemption allowing you access to material you have a Constitutional right to use, you can’t have someone with the technological know-how to do it for you and no one is supposed to provide you a tool to do it yourself, either. You have to do it all on your own.

So if you are, for example, one of the farmers trying to repair your own tractor, you now have an exemption allowing you to do that. But you still can’t go to an independent repair store to get an expert to let you in. You can’t use a premade tool to help you get in. This is a manifestly absurd result.

We’re glad Congress is asking questions about fair use under the DMCA. We wish there were better answers.

Read more »

Turning Lead to Gold with FPGA

We have an exciting announcement: SparkFun Electronics is now producing all Alchitry FPGA boards! Two new FPGA options are available, with the Xilinx Artix 7-equipped Au, and the Lattice iCE40 HX-equipped Cu boards. We also have two shield-like boards called "Elements" that support each of the FPGA's inherently strong capabilities and logic cells.

Don't forget that you can get a free SparkFun Qwiic Pro Micro BoogieBoard with any purchase of $75 or more using promo code "BOOGIEBOARD20" (some restrictions apply).

Now onto our new products!

The gold standard of FPGA!

Alchitry Au FPGA Development Board (Xilinx Artix 7)

added to your cart!

Alchitry Au FPGA Development Board (Xilinx Artix 7)

In stock DEV-16527

The Alchitry Au is the gold standard for FPGA development boards and it's possibly one of the strongest boards of its type on…

$99.95

The Alchitry Au Development Board is the "gold" standard for FPGA development boards, and it's one of the strongest boards of its type on the market. The Au FPGA features a Xilinx Artix 7 XC7A35T-1C FPGA with over 33,000 logic cells and 256MB of DDR3 RAM. This board is a fantastic starting point into the world of FPGAs as the heart of your next project. Now that this board is built by SparkFun, we added a Qwiic connector for easy I2C integration!


Alchitry Cu FPGA Development Board (Lattice iCE40 HX)

added to your cart!

Alchitry Cu FPGA Development Board (Lattice iCE40 HX)

In stock DEV-16526

If you are not needing a lot of power to start your FPGA adventure, or are looking for a more economical option, the Alchitry…

$49.95

If you don't need a lot of power to start your FPGA adventure or are looking for a more economical option, the Alchitry Cu FPGA Development Board might be the perfect option for you! The Alchitry Cu is a "lighter" FPGA version than the Alchitry Au but still offers something completely unique. The Alchitry Cu uses the Lattice iCE40 HX FPGA with 7680 logic cells and is supported by the open source tool chain Project IceStorm, as well as the SparkFun Qwiic Connect System. The Cu truly exemplifies the trend of more affordable and increasingly powerful FPGA boards arriving each year.


Alchitry Io Element Board

added to your cart!

Alchitry Io Element Board

In stock DEV-16525

The Alchitry Io Element Board is the perfect way to get your feet wet with digital design.

$24.95

The Alchitry Io Element Board is the perfect way to get your feet wet with digital design. The Io features four 7-segment LEDs, five momentary push buttons, 24 basic LEDs, and 24 DIP switches. All these features lend themselves to fantastic beginner tutorials that will walk you through the basics of FPGAs.


Alchitry Br Prototype Element Board

added to your cart!

Alchitry Br Prototype Element Board

In stock DEV-16524

The Alchitry Br Element Board is a prototyping periphery for the Au or Cu FPGA Development Boards.

$14.95

The Alchitry Br Element Board is a prototyping periphery for the Au or Cu FPGA development boards. The Br breaks out all the signals on the four headers running from your Au or Cu, and has a large prototyping area with a 0.1" pin grid for custom circuits.

There are also female headers (sold separately) available that can be soldered into the prototyping area, turning the Br Element into a breadboard so you can test out new circuits without making them a permanent resident!


RGB LED Clear Lens Common Cathode (5mm)

added to your cart!

RGB LED Clear Lens Common Cathode (5mm)

In stock COM-16911

These 5mm units have four pins - one for each color and a common cathode (the longest pin).

$2.05

These 5mm LEDs have four pins - one for each color and a common cathode (the longest pin). Use this LED for three status indicators, or pulse width modulate all three and get mixed colors!


That's it for this week! As always, we can't wait to see what you make! Shoot us a tweet @sparkfun, or let us know on Instagram or Facebook. We’d love to see what projects you’ve made!

Never miss a new product!

hbspt.forms.create( portalId: "2224003", formId: "e963ae12-71f6-46d7-bb00-126dbfef8636" );

comments | comment feed

Read more »