3D Ken Burns Effect from a Single Image
The Ken Burns effect allows animating still images with a virtual camera scan and zoom. Adding parallax, which results in the 3D Ken Burns effect, enables significantly more compelling results. Creating such effects manually is time-consuming and demands sophisticated editing skills. Existing automatic methods, however, require multiple input images from varying viewpoints. In this paper, we introduce a framework that synthesizes the 3D Ken Burns effect from a single image, supporting both a fully automatic mode and an interactive mode with the user controlling the camera. Our framework first leverages a depth prediction pipeline, which estimates scene depth that is suitable for view synthesis tasks. To address the limitations of existing depth estimation methods such as geometric distortions, semantic distortions, and inaccurate depth boundaries, we develop a semantic-aware neural network for depth prediction, couple its estimate with a segmentation-based depth adjustment process, and employ a refinement neural network that facilitates accurate depth predictions at object boundaries. According to this depth estimate, our framework then maps the input image to a point cloud and synthesizes the resulting video frames by rendering the point cloud from the corresponding camera positions. To address disocclusions while maintaining geometrically and temporally coherent synthesis results, we utilize context-aware color- and depth-inpainting to fill in the missing information in the extreme views of the camera path, thus extending the scene geometry of the point cloud. Experiments with a wide variety of image content show that our method enables realistic synthesis results. Our study demonstrates that our system allows users to achieve better results while requiring little effort compared to existing solutions for the 3D Ken Burns effect creation.
NurtureToken New!

Token crowdsale for this paper ends in

Buy Nurture Tokens

Authors

Are you an author of this paper? Check the Twitter handle we have for you is correct.

Simon Niklaus (edit)
Long Mai (add twitter)
Jimei Yang (edit)
Feng Liu (edit)
Ask The Authors

Ask the authors of this paper a question or leave a comment.

Read it. Rate it.
#1. Which part of the paper did you read?

#2. The paper contains new data or analyses that is openly accessible?
#3. The conclusion is supported by the data and analyses?
#4. The conclusion is of scientific interest?
#5. The result is likely to lead to future research?

Github
User:
None (add)
Repo:
None (add)
Stargazers:
0
Forks:
0
Open Issues:
0
Network:
0
Subscribers:
0
Language:
None
Youtube
Link:
None (add)
Views:
0
Likes:
0
Dislikes:
0
Favorites:
0
Comments:
0
Other
Sample Sizes (N=):
Inserted:
Words Total:
Words Unique:
Source:
Abstract:
None
09/12/19 06:04PM
12,459
3,350
Tweets
njanakiev: Interesting paper on the 3D Ken Burns effect from a single image using a semantic-aware neural network for depth prediction by @simon_niklaus et al. https://t.co/tyEzlgfKZg https://t.co/baUZ22sUhm
arxiv_pop: 2019/09/12 投稿 1位 CV(Computer Vision and Pattern Recognition) 3D Ken Burns Effect from a Single Image https://t.co/oEqJavXyUH 18 Tweets 80 Retweets 382 Favorites
poxyu_was_here: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
therealjpittman: [R] 3D Ken Burns Effect from a Single Image https://t.co/1nuzKfpMi9 #MachineLearning
SMankovski: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
reddit_ml: [R] 3D Ken Burns Effect from a Single Image https://t.co/M0W8ai8KY1
michaeljdobler: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
arphp: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
seburath: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
Gabriel_Oguna: RT @arxiv_org: 3D Ken Burns Effect from a Single Image. https://t.co/eNCHZBTCBJ https://t.co/fNn8BlU75Q
munkelen: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
vipermu_ai: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
danasday: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
simon_niklaus: @wblut There are a few examples with explanation at the end of the paper: https://t.co/yWdURg42je
saiedhosni: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
nizkroberts: RT @arxiv_org: 3D Ken Burns Effect from a Single Image. https://t.co/eNCHZBTCBJ https://t.co/fNn8BlU75Q
NonLocalityGuy: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
pelayoarbues: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
dendisuhubdy: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
PerthMLGroup: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
fmeister23: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
apolaine: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
cighos: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
juantomas: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
spazmnkee: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
briandixn: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
joachimgoedhart: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
dbtwr: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
manorlaboratory: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
alxndrkalinin: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
iamknighton: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
heyscam: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
Tropion: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
datafab: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
movito: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
bhargavbardipur: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
sarahbadr: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
subhobrata1: RT @arxiv_org: 3D Ken Burns Effect from a Single Image. https://t.co/eNCHZBTCBJ https://t.co/fNn8BlU75Q
EvaEzekwem: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
xuetal: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
__nggih: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
maxymvs: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
xsteenbrugge: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
devcsar: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
robinwoern: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
MishakinSergey: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
straybrid: RT @arxiv_org: 3D Ken Burns Effect from a Single Image. https://t.co/eNCHZBTCBJ https://t.co/fNn8BlU75Q
kuronekodaisuki: RT @arxiv_org: 3D Ken Burns Effect from a Single Image. https://t.co/eNCHZBTCBJ https://t.co/fNn8BlU75Q
round: RT @arxiv_org: 3D Ken Burns Effect from a Single Image. https://t.co/eNCHZBTCBJ https://t.co/fNn8BlU75Q
arxiv_org: 3D Ken Burns Effect from a Single Image. https://t.co/eNCHZBTCBJ https://t.co/fNn8BlU75Q
NosgothSpirit: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
AndyBettisworth: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
NSNPERSA: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
whyboris: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
FradSer: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
shimoke4869: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
fijaap: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
peteskomoroch: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
DavidBeniaguev: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
I99666: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
RexDouglass: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
JPSmithNL: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
FrancoVarriano: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
phongsathron1: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
CasePolen: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
holiprk: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
AssistedEvolve: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
TeddyWahle: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
7voprosov: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
Gabriel__Lewis: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
jgerity: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
round: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
ceobillionaire: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
joshbuchea: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
Saigarich: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
nova77t: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
jknoll: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
Miles_Brundage: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
brigade_light: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
arxiv_cscv: 3D Ken Burns Effect from a Single Image https://t.co/TOzF4geWrx
MassBassLol: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
aviopene: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
NEOkeitaro: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
RossRochford: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
santiagoitzcoat: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
barbolo: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
hidden17: RT @jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and…
jonathanfly: 3D Ken Burns Effect from a Single Image paper: https://t.co/icUggWsx7d project: https://t.co/Ua2T4FbdGn Very smooth and minimal artifacts -- nice improvement over the monocular depth mapping tests I experimented with. https://t.co/T4Fc3sxm4Y https://t.co/tp51NngJo6
arxiv_csgr: 3D Ken Burns Effect from a Single Image https://t.co/nTwR4V9YqA
arxiv_cscv: 3D Ken Burns Effect from a Single Image https://t.co/TOzF4geWrx
arxiv_csgr: 3D Ken Burns Effect from a Single Image https://t.co/nTwR4V9YqA
arxiv_cscv: 3D Ken Burns Effect from a Single Image https://t.co/TOzF4geWrx
arxiv_csgr: 3D Ken Burns Effect from a Single Image https://t.co/nTwR4V9YqA
arxiv_cscv: 3D Ken Burns Effect from a Single Image https://t.co/TOzF4geWrx
arxiv_cscv: 3D Ken Burns Effect from a Single Image https://t.co/TOzF4fXl2X
arxiv_csgr: 3D Ken Burns Effect from a Single Image https://t.co/nTwR4V9YqA
arxiv_cscv: 3D Ken Burns Effect from a Single Image https://t.co/TOzF4geWrx
arxivml: "3D Ken Burns Effect from a Single Image", Simon Niklaus, Long Mai, Jimei Yang, Feng Liu https://t.co/aifkxu3JdO
arxiv_csgr: 3D Ken Burns Effect from a Single Image https://t.co/nTwR4V9YqA
arxiv_cscv: 3D Ken Burns Effect from a Single Image https://t.co/TOzF4geWrx
arxiv_cscv: 3D Ken Burns Effect from a Single Image https://t.co/TOzF4geWrx
simon_niklaus: Papers can contain more than just static images. Why not embrace videos and animations to better convey our ideas? Paper: https://t.co/yWdURg42je https://t.co/j0lhAECYOy
arxiv_csgr: 3D Ken Burns Effect from a Single Image https://t.co/nTwR4V9YqA
BrundageBot: 3D Ken Burns Effect from a Single Image. Simon Niklaus, Long Mai, Jimei Yang, and Feng Liu https://t.co/WKiulUZZJj
arxiv_csgr: 3D Ken Burns Effect from a Single Image https://t.co/nTwR4USn20
Images
Related