Functional Variational Bayesian Neural Networks
Variational Bayesian neural networks (BNNs) perform variational inference over weights, but it is difficult to specify meaningful priors and approximate posteriors in a high-dimensional weight space. We introduce functional variational Bayesian neural networks (fBNNs), which maximize an Evidence Lower BOund (ELBO) defined directly on stochastic processes, i.e. distributions over functions. We prove that the KL divergence between stochastic processes equals the supremum of marginal KL divergences over all finite sets of inputs. Based on this, we introduce a practical training objective which approximates the functional ELBO using finite measurement sets and the spectral Stein gradient estimator. With fBNNs, we can specify priors entailing rich structures, including Gaussian processes and implicit stochastic processes. Empirically, we find fBNNs extrapolate well using various structured priors, provide reliable uncertainty estimates, and scale to large datasets.
NurtureToken New!

Token crowdsale for this paper ends in

Buy Nurture Tokens

Authors

Are you an author of this paper? Check the Twitter handle we have for you is correct.

Shengyang Sun (edit)
Guodong Zhang (edit)
Jiaxin Shi (edit)
Roger Grosse (add twitter)
Ask The Authors

Ask the authors of this paper a question or leave a comment.

Read it. Rate it.
#1. Which part of the paper did you read?

#2. The paper contains new data or analyses that is openly accessible?
#3. The conclusion is supported by the data and analyses?
#4. The conclusion is of scientific interest?
#5. The result is likely to lead to future research?

Github
User:
None (add)
Repo:
None (add)
Stargazers:
0
Forks:
0
Open Issues:
0
Network:
0
Subscribers:
0
Language:
None
Youtube
Link:
None (add)
Views:
0
Likes:
0
Dislikes:
0
Favorites:
0
Comments:
0
Other
Sample Sizes (N=):
Inserted:
Words Total:
Words Unique:
Source:
Abstract:
None
03/14/19 06:04PM
12,504
3,489
Tweets
AdriGarriga: @decodyng Perhaps take a look at https://t.co/oVP2z4rnlH : it's about variational inference in Bayesian neural networks, but in function space instead of the usual weight space. Might make inference a lot easier.
ElectronNest: "Functional Variational Bayesian Neural Networks" https://t.co/DPQPXwMD8I
svk_jpn: RT @arxiv_pop: 2019/03/13 投稿 1位 LG(Machine Learning) Functional Variational Bayesian Neural Networks https://t.co/Yhq3dRoDq7 9 Tweets 53 R…
arxiv_pop: 2019/03/13 投稿 1位 LG(Machine Learning) Functional Variational Bayesian Neural Networks https://t.co/Yhq3dRoDq7 9 Tweets 53 Retweets 273 Favorites
PerthMLGroup: RT @ssydasheng: Proud to announce our paper "Functional Variational BNNs" https://t.co/uafvwsrwFy. Here we introduce functional variational…
AssistedEvolve: RT @ssydasheng: Proud to announce our paper "Functional Variational BNNs" https://t.co/uafvwsrwFy. Here we introduce functional variational…
gabeibagon: RT @ssydasheng: Proud to announce our paper "Functional Variational BNNs" https://t.co/uafvwsrwFy. Here we introduce functional variational…
Hyon0930: RT @ssydasheng: Proud to announce our paper "Functional Variational BNNs" https://t.co/uafvwsrwFy. Here we introduce functional variational…
RohithBhandaru: RT @ssydasheng: Proud to announce our paper "Functional Variational BNNs" https://t.co/uafvwsrwFy. Here we introduce functional variational…
tsukamotogame96: RT @ssydasheng: Proud to announce our paper "Functional Variational BNNs" https://t.co/uafvwsrwFy. Here we introduce functional variational…
KyleCranmer: RT @ssydasheng: Proud to announce our paper "Functional Variational BNNs" https://t.co/uafvwsrwFy. Here we introduce functional variational…
brandondamos: RT @ssydasheng: Proud to announce our paper "Functional Variational BNNs" https://t.co/uafvwsrwFy. Here we introduce functional variational…
Montreal_AI: RT @ssydasheng: Proud to announce our paper "Functional Variational BNNs" https://t.co/uafvwsrwFy. Here we introduce functional variational…
Creepazoid2: RT @ssydasheng: Proud to announce our paper "Functional Variational BNNs" https://t.co/uafvwsrwFy. Here we introduce functional variational…
anoobian: RT @ssydasheng: Proud to announce our paper "Functional Variational BNNs" https://t.co/uafvwsrwFy. Here we introduce functional variational…
cezzo_sw: RT @ssydasheng: Proud to announce our paper "Functional Variational BNNs" https://t.co/uafvwsrwFy. Here we introduce functional variational…
riashatislam: RT @ssydasheng: Proud to announce our paper "Functional Variational BNNs" https://t.co/uafvwsrwFy. Here we introduce functional variational…
shubh_300595: RT @arxiv_org: Functional Variational Bayesian Neural Networks. https://t.co/oroXAe4Oh4 https://t.co/oIiYUHh87f
cournape: RT @ssydasheng: Proud to announce our paper "Functional Variational BNNs" https://t.co/uafvwsrwFy. Here we introduce functional variational…
lostanlen: RT @ssydasheng: Proud to announce our paper "Functional Variational BNNs" https://t.co/uafvwsrwFy. Here we introduce functional variational…
vnfrombucharest: RT @ssydasheng: Proud to announce our paper "Functional Variational BNNs" https://t.co/uafvwsrwFy. Here we introduce functional variational…
jie_song: RT @arxiv_org: Functional Variational Bayesian Neural Networks. https://t.co/oroXAe4Oh4 https://t.co/oIiYUHh87f
desertnaut: RT @ssydasheng: Proud to announce our paper "Functional Variational BNNs" https://t.co/uafvwsrwFy. Here we introduce functional variational…
TerencePlizga: RT @ssydasheng: Proud to announce our paper "Functional Variational BNNs" https://t.co/uafvwsrwFy. Here we introduce functional variational…
bhagirathl: RT @ssydasheng: Proud to announce our paper "Functional Variational BNNs" https://t.co/uafvwsrwFy. Here we introduce functional variational…
hai_t_pham: RT @ssydasheng: Proud to announce our paper "Functional Variational BNNs" https://t.co/uafvwsrwFy. Here we introduce functional variational…
qiming82: RT @arxiv_org: Functional Variational Bayesian Neural Networks. https://t.co/oroXAe4Oh4 https://t.co/oIiYUHh87f
coolboi95: RT @ssydasheng: Proud to announce our paper "Functional Variational BNNs" https://t.co/uafvwsrwFy. Here we introduce functional variational…
osageev: RT @ssydasheng: Proud to announce our paper "Functional Variational BNNs" https://t.co/uafvwsrwFy. Here we introduce functional variational…
ceobillionaire: RT @ssydasheng: Proud to announce our paper "Functional Variational BNNs" https://t.co/uafvwsrwFy. Here we introduce functional variational…
maggie_albrecht: RT @ssydasheng: Proud to announce our paper "Functional Variational BNNs" https://t.co/uafvwsrwFy. Here we introduce functional variational…
andybaoxv: RT @ssydasheng: Proud to announce our paper "Functional Variational BNNs" https://t.co/uafvwsrwFy. Here we introduce functional variational…
EricSchles: RT @ssydasheng: Proud to announce our paper "Functional Variational BNNs" https://t.co/uafvwsrwFy. Here we introduce functional variational…
diegovogeid: RT @ssydasheng: Proud to announce our paper "Functional Variational BNNs" https://t.co/uafvwsrwFy. Here we introduce functional variational…
vdeepak13: RT @ssydasheng: Proud to announce our paper "Functional Variational BNNs" https://t.co/uafvwsrwFy. Here we introduce functional variational…
SiavashSakhavi: RT @arxiv_org: Functional Variational Bayesian Neural Networks. https://t.co/oroXAe4Oh4 https://t.co/oIiYUHh87f
gaialive: RT @arxiv_org: Functional Variational Bayesian Neural Networks. https://t.co/oroXAe4Oh4 https://t.co/oIiYUHh87f
mvaldenegro: RT @arxiv_org: Functional Variational Bayesian Neural Networks. https://t.co/oroXAe4Oh4 https://t.co/oIiYUHh87f
arxiv_org: Functional Variational Bayesian Neural Networks. https://t.co/oroXAe4Oh4 https://t.co/oIiYUHh87f
kastnerkyle: RT @ssydasheng: Proud to announce our paper "Functional Variational BNNs" https://t.co/uafvwsrwFy. Here we introduce functional variational…
udmrzn: RT @ssydasheng: Proud to announce our paper "Functional Variational BNNs" https://t.co/uafvwsrwFy. Here we introduce functional variational…
Yugioh1984: RT @ssydasheng: Proud to announce our paper "Functional Variational BNNs" https://t.co/uafvwsrwFy. Here we introduce functional variational…
cghosh_: RT @ssydasheng: Proud to announce our paper "Functional Variational BNNs" https://t.co/uafvwsrwFy. Here we introduce functional variational…
jeandut14000: RT @ssydasheng: Proud to announce our paper "Functional Variational BNNs" https://t.co/uafvwsrwFy. Here we introduce functional variational…
RogerGrosse: RT @ssydasheng: Proud to announce our paper "Functional Variational BNNs" https://t.co/uafvwsrwFy. Here we introduce functional variational…
vyousDS: RT @ssydasheng: Proud to announce our paper "Functional Variational BNNs" https://t.co/uafvwsrwFy. Here we introduce functional variational…
letranger14: RT @ssydasheng: Proud to announce our paper "Functional Variational BNNs" https://t.co/uafvwsrwFy. Here we introduce functional variational…
arowan_ml: RT @ssydasheng: Proud to announce our paper "Functional Variational BNNs" https://t.co/uafvwsrwFy. Here we introduce functional variational…
geoffroeder: RT @ssydasheng: Proud to announce our paper "Functional Variational BNNs" https://t.co/uafvwsrwFy. Here we introduce functional variational…
cliff_watkins: RT @ssydasheng: Proud to announce our paper "Functional Variational BNNs" https://t.co/uafvwsrwFy. Here we introduce functional variational…
Guodzh: RT @ssydasheng: Proud to announce our paper "Functional Variational BNNs" https://t.co/uafvwsrwFy. Here we introduce functional variational…
heghbalz: RT @ssydasheng: Proud to announce our paper "Functional Variational BNNs" https://t.co/uafvwsrwFy. Here we introduce functional variational…
ssydasheng: Proud to announce our paper "Functional Variational BNNs" https://t.co/uafvwsrwFy. Here we introduce functional variational inference, which enables us to specify structured priors and perform inference in function space. Gif shows BNN predictions under a Periodic prior. https://t.co/KkePt57rhn
arxivml: "Functional Variational Bayesian Neural Networks", Shengyang Sun, Guodong Zhang, Jiaxin Shi, Roger Grosse https://t.co/0s9JhxggWq
arxiv_cs_LG: Functional Variational Bayesian Neural Networks. Shengyang Sun, Guodong Zhang, Jiaxin Shi, and Roger Grosse https://t.co/A5mBx0XUiS
BrundageBot: Functional Variational Bayesian Neural Networks. Shengyang Sun, Guodong Zhang, Jiaxin Shi, and Roger Grosse https://t.co/bAy0roEgsU
StatsPapers: Functional Variational Bayesian Neural Networks. https://t.co/nLEC0LFlSS
Images
Related