M3D-GAN: Multi-Modal Multi-Domain Translation with Universal Attention
Generative adversarial networks have led to significant advances in cross-modal/domain translation. However, typically these networks are designed for a specific task (e.g., dialogue generation or image synthesis, but not both). We present a unified model, M3D-GAN, that can translate across a wide range of modalities (e.g., text, image, and speech) and domains (e.g., attributes in images or emotions in speech). Our model consists of modality subnets that convert data from different modalities into unified representations, and a unified computing body where data from different modalities share the same network architecture. We introduce a universal attention module that is jointly trained with the whole network and learns to encode a large range of domain information into a highly structured latent space. We use this to control synthesis in novel ways, such as producing diverse realistic pictures from a sketch or varying the emotion of synthesized speech. We evaluate our approach on extensive benchmark tasks, including image-to-image, text-to-image, image captioning, text-to-speech, speech recognition, and machine translation. Our results show state-of-the-art performance on some of the tasks.
NurtureToken New!

Token crowdsale for this paper ends in

Buy Nurture Tokens

Ask The Authors

Ask the authors of this paper a question or leave a comment.

Read it. Rate it.
#1. Which part of the paper did you read?

#2. The paper contains new data or analyses that is openly accessible?
#3. The conclusion is supported by the data and analyses?
#4. The conclusion is of scientific interest?
#5. The result is likely to lead to future research?

Github
User:
None (add)
Repo:
None (add)
Stargazers:
0
Forks:
0
Open Issues:
0
Network:
0
Subscribers:
0
Language:
None
Youtube
Link:
None (add)
Views:
0
Likes:
0
Dislikes:
0
Favorites:
0
Comments:
0
Other
Sample Sizes (N=):
Inserted:
Words Total:
Words Unique:
Source:
Abstract:
None
07/10/19 06:06PM
7,050
2,181
Tweets
arxiv_pop: 2019/07/09 投稿 3位 CV(Computer Vision and Pattern Recognition) M3D-GAN: Multi-Modal Multi-Domain Translation with Universal Attention https://t.co/Nn7rtRmggl 15 Tweets 21 Retweets 34 Favorites
machine_ml: RT @4indata: M3D-GAN: Multi-Modal Multi-Domain Translation with Universal Attention Ma et al.: https://t.co/fcKjDu99uc #4InData #MachineL…
msarozz: RT @4indata: M3D-GAN: Multi-Modal Multi-Domain Translation with Universal Attention Ma et al.: https://t.co/fcKjDu99uc #4InData #MachineL…
4indata: M3D-GAN: Multi-Modal Multi-Domain Translation with Universal Attention Ma et al.: https://t.co/fcKjDu99uc #4InData #MachineLearning #DeepLearning #ArtificialIntelligence
GeeksDataa: M3D-GAN: Multi-Modal Multi-Domain Translation with Universal Attention Ma et al.: https://t.co/Z7vmHbeBof #GeeksData #MachineLearning #DeepLearning #ArtificialIntelligence
DeanPlbn: RT @roadrunning01: M3D-GAN: Multi-Modal Multi-Domain Translation with Universal Attention pdf: https://t.co/P7AovUAheK abs: https://t.co/7N…
alsombra7: RT @roadrunning01: M3D-GAN: Multi-Modal Multi-Domain Translation with Universal Attention pdf: https://t.co/P7AovUAheK abs: https://t.co/7N…
machinelearn_d: RT @rezamahmooudi: M3D-GAN: Multi-Modal Multi-Domain Translation with Universal Attention https://t.co/griFGeq3iE #MachineLearning #Deep…
rezamahmooudi: M3D-GAN: Multi-Modal Multi-Domain Translation with Universal Attention https://t.co/griFGeq3iE #MachineLearning #DeepLearning #ArtificialIntelligence #AI
PerthMLGroup: RT @roadrunning01: M3D-GAN: Multi-Modal Multi-Domain Translation with Universal Attention pdf: https://t.co/P7AovUAheK abs: https://t.co/7N…
SythonUK: RT @arxiv_cscl: M3D-GAN: Multi-Modal Multi-Domain Translation with Universal Attention https://t.co/Gp0AheJjWL
UkiwhY: RT @roadrunning01: M3D-GAN: Multi-Modal Multi-Domain Translation with Universal Attention pdf: https://t.co/P7AovUAheK abs: https://t.co/7N…
ryo_masumura: RT @arxiv_cscl: M3D-GAN: Multi-Modal Multi-Domain Translation with Universal Attention https://t.co/Gp0AheJjWL
AnalyticsFr: M3D-GAN: Multi-Modal Multi-Domain Translation with Universal Attention pdf: https://t.co/HX5LOzt0hm … abs: https://t.co/axSh17iixp https://t.co/ObeiGwePjC - https://t.co/u4Kjwa540h #Analytics thanks @RichardEudes
DataScientistFr: M3D-GAN: Multi-Modal Multi-Domain Translation with Universal Attention pdf: https://t.co/PuruDUrrHy … abs: https://t.co/LSXa7LC70N https://t.co/BpmnZKHcAg - https://t.co/VA8kl8sNoe #datascience thanks @RichardEudes
ceobillionaire: RT @roadrunning01: M3D-GAN: Multi-Modal Multi-Domain Translation with Universal Attention pdf: https://t.co/P7AovUAheK abs: https://t.co/7N…
arxiv_cscl: M3D-GAN: Multi-Modal Multi-Domain Translation with Universal Attention https://t.co/Gp0AherJ5d
arxiv_cscv: M3D-GAN: Multi-Modal Multi-Domain Translation with Universal Attention https://t.co/2SB6PQ74RX
AssistedEvolve: RT @roadrunning01: M3D-GAN: Multi-Modal Multi-Domain Translation with Universal Attention pdf: https://t.co/P7AovUAheK abs: https://t.co/7N…
hjguyhan: RT @roadrunning01: M3D-GAN: Multi-Modal Multi-Domain Translation with Universal Attention pdf: https://t.co/P7AovUAheK abs: https://t.co/7N…
arxivml: "M3D-GAN: Multi-Modal Multi-Domain Translation with Universal Attention", Shuang Ma, Daniel McDuff, Yale Song https://t.co/KfL98aPWEI
Pol09122455: RT @roadrunning01: M3D-GAN: Multi-Modal Multi-Domain Translation with Universal Attention pdf: https://t.co/P7AovUAheK abs: https://t.co/7N…
jp_axs4ll: RT @roadrunning01: M3D-GAN: Multi-Modal Multi-Domain Translation with Universal Attention pdf: https://t.co/P7AovUAheK abs: https://t.co/7N…
Veqtor: RT @roadrunning01: M3D-GAN: Multi-Modal Multi-Domain Translation with Universal Attention pdf: https://t.co/P7AovUAheK abs: https://t.co/7N…
dr_levan: RT @roadrunning01: M3D-GAN: Multi-Modal Multi-Domain Translation with Universal Attention pdf: https://t.co/P7AovUAheK abs: https://t.co/7N…
pbaylies: RT @roadrunning01: M3D-GAN: Multi-Modal Multi-Domain Translation with Universal Attention pdf: https://t.co/P7AovUAheK abs: https://t.co/7N…
KageKirin: RT @roadrunning01: M3D-GAN: Multi-Modal Multi-Domain Translation with Universal Attention pdf: https://t.co/P7AovUAheK abs: https://t.co/7N…
Ron_castillo: RT @Memoirs: M3D-GAN: Multi-Modal Multi-Domain Translation with Universal Attention. https://t.co/xR2JJROpnb
Memoirs: M3D-GAN: Multi-Modal Multi-Domain Translation with Universal Attention. https://t.co/xR2JJROpnb
JidinDinesh: RT @roadrunning01: M3D-GAN: Multi-Modal Multi-Domain Translation with Universal Attention pdf: https://t.co/P7AovUAheK abs: https://t.co/7N…
roadrunning01: M3D-GAN: Multi-Modal Multi-Domain Translation with Universal Attention pdf: https://t.co/P7AovUAheK abs: https://t.co/7NRuVgaaHX https://t.co/zI01CgzSs4
BrundageBot: M3D-GAN: Multi-Modal Multi-Domain Translation with Universal Attention. Shuang Ma, Daniel McDuff, and Yale Song https://t.co/tt4tmJyWEq
arxiv_cscl: M3D-GAN: Multi-Modal Multi-Domain Translation with Universal Attention https://t.co/Gp0AherJ5d
arxiv_cscv: M3D-GAN: Multi-Modal Multi-Domain Translation with Universal Attention https://t.co/2SB6PQ74RX
Images
Related