Variational Autoencoders and Nonlinear ICA: A Unifying Framework
The framework of variational autoencoders allows us to efficiently learn deep latent-variable models, such that the model's marginal distribution over observed variables fits the data. Often, we're interested in going a step further, and want to approximate the true joint distribution over observed and latent variables, including the true prior and posterior distributions over latent variables. This is known to be generally impossible due to unidentifiability of the model. We address this issue by showing that for a broad family of deep latent-variable models, identification of the true joint distribution over observed and latent variables is actually possible up to a simple transformation, thus achieving a principled and powerful form of disentanglement. Our result requires a factorized prior distribution over the latent variables that is conditioned on an additionally observed variable, such as a class label or almost any other observation. We build on recent developments in nonlinear ICA, which we extend to the case with noisy, undercomplete or discrete observations, integrated in a maximum likelihood framework. The result also trivially contains identifiable flow-based generative models as a special case.
NurtureToken New!

Token crowdsale for this paper ends in

Buy Nurture Tokens

Authors

Are you an author of this paper? Check the Twitter handle we have for you is correct.

Ilyes Khemakhem (edit)
Diederik P. Kingma (add twitter)
Aapo Hyvärinen (add twitter)
Ask The Authors

Ask the authors of this paper a question or leave a comment.

Read it. Rate it.
#1. Which part of the paper did you read?

#2. The paper contains new data or analyses that is openly accessible?
#3. The conclusion is supported by the data and analyses?
#4. The conclusion is of scientific interest?
#5. The result is likely to lead to future research?

Github
User:
None (add)
Repo:
None (add)
Stargazers:
0
Forks:
0
Open Issues:
0
Network:
0
Subscribers:
0
Language:
None
Youtube
Link:
None (add)
Views:
0
Likes:
0
Dislikes:
0
Favorites:
0
Comments:
0
Other
Sample Sizes (N=):
Inserted:
Words Total:
Words Unique:
Source:
Abstract:
None
07/10/19 06:02PM
8,750
2,281
Tweets
therealjpittman: [R] Variational Autoencoders and Nonlinear ICA: A Unifying Framework https://t.co/DnxjPuzxO4 #MachineLearning
tak_yamm: RT @StatsPapers: Variational Autoencoders and Nonlinear ICA: A Unifying Framework. https://t.co/6Hd1HeM3v0
ballforest: RT @StatsPapers: Variational Autoencoders and Nonlinear ICA: A Unifying Framework. https://t.co/6Hd1HeM3v0
hillbig: Nonlinear ICA is known as unidentifiable with data only. They show that VAE conditioned with additional observation (e.g., time index, class label) can identify true components up to trivial transformation, showing principled disentanglement ability of VAE https://t.co/8YO7PcoHzz
hillbig: 非線形独立成分分析(ICA)は観測だけからでは因子を同定不可能である。観測xに加えて追加の観測u(時刻、時系列の直前の観測、クラスラベルなど)も得られ、uが変わると因子zも変わるという条件を満たす場合、uで条件付したVAEによる最尤推定によって因子を同定できる。https://t.co/8YO7PcoHzz
nacim_belkhir: RT @StatsPapers: Variational Autoencoders and Nonlinear ICA: A Unifying Framework. https://t.co/6Hd1HeM3v0
jd_mashiro: RT @StatsPapers: Variational Autoencoders and Nonlinear ICA: A Unifying Framework. https://t.co/6Hd1HeM3v0
prabhuiitdhn: RT @StatsPapers: Variational Autoencoders and Nonlinear ICA: A Unifying Framework. https://t.co/6Hd1HeM3v0
reddit_ml: [R] Variational Autoencoders and Nonlinear ICA: A Unifying Framework https://t.co/pNFigXZf1B
_e_evans: RT @smnlssn: An interesting new paper on identifiability in deep latent variable models https://t.co/lCSe7IyIJi
dizzy_my_future: RT @StatsPapers: Variational Autoencoders and Nonlinear ICA: A Unifying Framework. https://t.co/6Hd1HeM3v0
arxivml: "Variational Autoencoders and Nonlinear ICA: A Unifying Framework", Ilyes Khemakhem, Diederik P. Kingma, Aapo Hyvär… https://t.co/XJmZMLlCSw
StatsPapers: Variational Autoencoders and Nonlinear ICA: A Unifying Framework. https://t.co/6Hd1HeM3v0
smnlssn: An interesting new paper on identifiability in deep latent variable models https://t.co/lCSe7IyIJi
arxiv_cs_LG: Variational Autoencoders and Nonlinear ICA: A Unifying Framework. Ilyes Khemakhem, Diederik P. Kingma, and Aapo Hyvärinen https://t.co/mJfDQ9m7dB
BrundageBot: Variational Autoencoders and Nonlinear ICA: A Unifying Framework. Ilyes Khemakhem, Diederik P. Kingma, and Aapo Hyvärinen https://t.co/G3htUNxuXd
Images
Related