Language models, being at the heart of many NLP problems, are always of great
interest to researchers. Neural language models come with the advantage of
distributed representations and long range contexts. With its particular
dynamics that allow the cycling of information within the network, `Recurrent
neural network' (RNN) becomes an ideal paradigm for neural language modeling.
Long Short-Term Memory (LSTM) architecture solves the inadequacies of the
standard RNN in modeling long-range contexts. In spite of a plethora of RNN
variants, possibility to add multiple memory cells in LSTM nodes was seldom
explored. Here we propose a multi-cell node architecture for LSTMs and study
its applicability for neural language modeling. The proposed multi-cell LSTM
language models outperform the state-of-the-art results on well-known Penn
Treebank (PTB) setup.

more |
pdf
| html
BrundageBot:
Multi-cell LSTM Based Neural Language Model. Thomas Cherian, Akshay Badola, and Vineet Padmanabhan https://t.co/DMSHIlcOuO

arxivml:
"Multi-cell LSTM Based Neural Language Model",
Thomas Cherian, Akshay Badola, Vineet Padmanabhan
https://t.co/X72rz8OgJR

Soul:
Multi-cell LSTM Based Neural Language Model. https://t.co/bJ8vFVU5Nn

rdzeniu:
RT @Soul: Multi-cell LSTM Based Neural Language Model. https://t.co/bJ8vFVU5Nn

None.

None.

Sample Sizes : None.

Authors: 3

Total Words: 5001

Unqiue Words: 1592

Deep Learning is moving to edge devices, ushering in a new age of distributed
Artificial Intelligence (AI). The high demand of computational resources
required by deep neural networks may be alleviated by approximate computing
techniques, and most notably reduced-precision arithmetic with coarsely
quantized numerical representations. In this context, Bonseyes comes in as an
initiative to enable stakeholders to bring AI to low-power and autonomous
environments such as: Automotive, Medical Healthcare and Consumer Electronics.
To achieve this, we introduce LPDNN, a framework for optimized deployment of
Deep Neural Networks on heterogeneous embedded devices. In this work, we detail
the quantization engine that is integrated in LPDNN. The engine depends on a
fine-grained workflow which enables a Neural Network Design Exploration and a
sensitivity analysis of each layer for quantization. We demonstrate the engine
with a case study on Alexnet and VGG16 for three different techniques for
direct quantization: standard fixed-point, dynamic...

more |
pdf
| html
BrundageBot:
QUENN: QUantization Engine for low-power Neural Networks. Miguel de Prado, Maurizio Denna, Luca Benini, and Nuria Pazos https://t.co/Oud0xgj7vP

nmfeeds:
[O] https://t.co/Jn4eHM2TDR QUENN: QUantization Engine for low-power Neural Networks. Deep Learning is moving to edge devi...

nmfeeds:
[NE] https://t.co/Jn4eHM2TDR QUENN: QUantization Engine for low-power Neural Networks. Deep Learning is moving to edge dev...

Soul:
QUENN: QUantization Engine for low-power Neural Networks. https://t.co/Hw3EN8NZSY

None.

None.

Sample Sizes : None.

Authors: 4

Total Words: 7712

Unqiue Words: 2381

Networks are fundamental building blocks for representing data, and
computations. Remarkable progress in learning in structurally defined (shallow
or deep) networks has recently been achieved. Here we introduce evolutionary
exploratory search and learning method of topologically flexible networks under
the constraint of producing elementary computational steady-state input-output
operations.
Our results include; (1) the identification of networks, over four orders of
magnitude, implementing computation of steady-state input-output functions,
such as a band-pass filter, a threshold function, and an inverse band-pass
function. Next, (2) the learned networks are technically controllable as only a
small number of driver nodes are required to move the system to a new state.
Furthermore, we find that the fraction of required driver nodes is constant
during evolutionary learning, suggesting a stable system design. (3), our
framework allows multiplexing of different computations using the same network.
For example, using a binary...

more |
pdf
| html
net_science:
"Controllability, Multiplexing, and Transfer Learning in Networks using Evolutionary Learning. (arXiv:1811.05592v1 [https://t.co/wj7sSzFbWK])" https://t.co/1JceQva4F2

BrundageBot:
Controllability, Multiplexing, and Transfer Learning in Networks using Evolutionary Learning. Rise Ooi, C. -H. Huck Yang, Pin-Yu Chen, Vìctor Eguìluz, Narsis Kiani, Hector Zenil, David Gomez-Cabrero, and Jesper Tegnèr https://t.co/Ad6XEaaAhW

BioPapers:
Controllability, Multiplexing, and Transfer Learning in Networks using Evolutionary Learning. https://t.co/xP96WbsqBV

None.

None.

Sample Sizes : None.

Authors: 8

Total Words: 6113

Unqiue Words: 2058

Transfer learning is a powerful tool to adapt trained neural networks to new
tasks. Depending on the similarity of the original task to the new task, the
selection of the cut-off layer is critical. For medical applications like
tissue classification, the last layers of an object classification network
might not be optimal. We found that on real data of human corneal tissues the
best feature representation can be found in the middle layers of the
Inception-v3 and in the rear layers of the VGG-19 architecture.

more |
pdf
| html
None.

None.

Sample Sizes : None.

Authors: 8

Total Words: 1404

Unqiue Words: 631

In this paper, a new meta-heuristic algorithm, called beetle swarm
optimization algorithm, is proposed by enhancing the performance of swarm
optimization through beetle foraging principles. The performance of 23
benchmark functions is tested and compared with widely used algorithms,
including particle swarm optimization algorithm, genetic algorithm (GA) and
grasshopper optimization algorithm . Numerical experiments show that the beetle
swarm optimization algorithm outperforms its counterparts. Besides, to
demonstrate the practical impact of the proposed algorithm, two classic
engineering design problems, namely, pressure vessel design problem and
himmelblaus optimization problem, are also considered and the proposed beetle
swarm optimization algorithm is shown to be competitive in those applications.

more |
pdf
| html
None.

None.

Sample Sizes : None.

Authors: 3

Total Words: 6738

Unqiue Words: 2578

Cardiopulmonary resuscitation (CPR) is alongside with electrical
defibrillation the most important treatment for sudden cardiac arrest, which
affects thousands of individuals every year. In this paper, we present a robust
sinusoid model that uses skeletal motion data from an RGB-D (Kinect) sensor and
the Differential Evolution (DE) optimization algorithm to dynamically fit
sinusoidal curves to derive frequency and depth parameters for cardiopulmonary
resuscitation training. It is intended to be part of a robust and easy-to-use
feedback system for CPR training, allowing its use for unsupervised training.
The accuracy of this DE-based approach is evaluated in comparison with data
recorded by a state-of-the-art training mannequin. We optimized the DE
algorithm constants and have shown that with these optimized parameters the
frequency of the CPR is recognized with a median error of 2.55 (2.4%)
compressions per minute compared to the reference training mannequin.

more |
pdf
| html
None.

None.

Sample Sizes : None.

Authors: 6

Total Words: 7936

Unqiue Words: 2561

This paper proposes a hybrid basis function construction method (GP-RVM) for
Symbolic Regression problem, which combines an extended version of Genetic
Programming called Kaizen Programming and Relevance Vector Machine to evolve an
optimal set of basis functions. Different from traditional evolutionary
algorithms where a single individual is a complete solution, our method
proposes a solution based on linear combination of basis functions built from
individuals during the evolving process. RVM which is a sparse Bayesian kernel
method selects suitable functions to constitute the basis. RVM determines the
posterior weight of a function by evaluating its quality and sparsity. The
solution produced by GP-RVM is a sparse Bayesian linear model of the
coefficients of many non-linear functions. Our hybrid approach is focused on
nonlinear white-box models selecting the right combination of functions to
build robust predictions without prior knowledge about data. Experimental
results show that GP-RVM outperforms conventional methods, which...

more |
pdf
| html
None.

None.

Sample Sizes : None.

Authors: 3

Total Words: 7532

Unqiue Words: 2545

Many-objective evolutionary algorithms (MOEAs), especially the
decomposition-based MOEAs, have attracted wide attention in recent years.
Recent studies show that a well designed combination of the decomposition
method and the domination method can improve the performance ,i.e., convergence
and diversity, of a MOEA. In this paper, a novel way of combining the
decomposition method and the domination method is proposed. More precisely, a
set of weight vectors is employed to decompose a given many-objective
optimization problem(MaOP), and a hybrid method of the penalty-based boundary
intersection function and dominance is proposed to compare local solutions
within a subpopulation defined by a weight vector. A MOEA based on the hybrid
method is implemented and tested on problems chosen from two famous test
suites, i.e., DTLZ and WFG. The experimental results show that our algorithm is
very competitive in dealing with MaOPs. Subsequently, our algorithm is extended
to solve constraint MaOPs, and the constrained version of our algorithm...

more |
pdf
| html
jastner109:
RT @Soul: A Many-Objective Evolutionary Algorithm Based on Decomposition and Local Dominance. https://t.co/3WgmAeXeQs

None.

None.

Sample Sizes : None.

Authors: 2

Total Words: 9421

Unqiue Words: 2765

Many applications in machine learning require optimizing a function whose
true gradient is unknown, but where surrogate gradient information (directions
that may be correlated with, but not necessarily identical to, the true
gradient) is available instead. This arises when an approximate gradient is
easier to compute than the full gradient (e.g. in meta-learning or unrolled
optimization), or when a true gradient is intractable and is replaced with a
surrogate (e.g. in certain reinforcement learning applications, or when using
synthetic gradients). We propose Guided Evolutionary Strategies, a method for
optimally using surrogate gradient directions along with random search. We
define a search distribution for evolutionary strategies that is elongated
along a guiding subspace spanned by the surrogate gradients. This allows us to
estimate a descent direction which can then be passed to a first-order
optimizer. We analytically and numerically characterize the tradeoffs that
result from tuning how strongly the search distribution is...

more |
pdf
| html
None.

deeplearning4j:
Guided evolutionary strategies: escaping the curse of dimensionality in random search https://t.co/hRclVnzIFY #machinelearning

arxiv_flying:
#NIPS2018 Guided evolutionary strategies: escaping the curse of dimensionality in random search. (arXiv:1806.10230v1 [cs\.NE]) https://t.co/6aMztzDLrX

nmfeeds:
[NE] https://t.co/dqBWiokhRC Guided evolutionary strategies: escaping the curse of dimensionality in random search. Many a...

nmfeeds:
[O] https://t.co/dqBWiokhRC Guided evolutionary strategies: escaping the curse of dimensionality in random search. Many ap...

reddit_ml:
[R] Guided evolutionary strategies: escaping the curse of dimensionality in random search (Google Brain) https://t.co/0kz46NRhO3

hereticreader:
[1806.10230] Guided evolutionary strategies: escaping the curse of dimensionality in random search https://t.co/gJN7DfiFlT

ahammami0:
Guided evolutionary strategies: escaping the curse of dimensionality in random search
Paper : https://t.co/SFKiAoCix3
Code : https://t.co/hQbvvCIhN8
#artificialintelligence #evolutionary #strategies https://t.co/sEjvUnwoqP

AdaptToReality:
RT @deeplearning4j: Guided evolutionary strategies: escaping the curse of dimensionality in random search https://t.co/hRclVnzIFY #machine…

Klevis_Ramo:
RT @deeplearning4j: Guided evolutionary strategies: escaping the curse of dimensionality in random search https://t.co/hRclVnzIFY #machine…

Guided Evolutionary Strategies

Stargazers: 151

Subscribers: 12

Subscribers: 12

Forks: 12

Open Issues: 0

Open Issues: 0

None.

Sample Sizes : None.

Authors: 4

Total Words: 9028

Unqiue Words: 2442

As edge applications using convolutional neural networks (CNN) models grow,
it is becoming necessary to introduce dedicated hardware accelerators in which
network parameters and feature-map data are represented with limited precision.
In this paper we propose a novel quantization algorithm for energy-efficient
deployment of the hardware accelerators. For weights and biases, the optimal
bit length of the fractional part is determined so that the quantization error
is minimized over their distribution. For feature-map data, meanwhile, their
sample distribution is well approximated with the generalized gamma
distribution (GGD), and accordingly the optimal quantization step size can be
obtained through the asymptotical closed form solution of GGD. The proposed
quantization algorithm has a higher signal-to-quantization-noise ratio (SQNR)
than other quantization schemes previously proposed for CNNs, and even can be
more improved by tuning the quantization parameters, resulting in efficient
implementation of the hardware accelerators for...

more |
pdf
| html
nmfeeds:
[AI] https://t.co/GoG5LlyMWl Convolutional Neural Network Quantization using Generalized Gamma Distribution. As edge appli...

nmfeeds:
[NE] https://t.co/GoG5LlyMWl Convolutional Neural Network Quantization using Generalized Gamma Distribution. As edge appli...

nmfeeds:
[O] https://t.co/GoG5LlyMWl Convolutional Neural Network Quantization using Generalized Gamma Distribution. As edge applic...

None.

None.

Sample Sizes : None.

Authors: 5

Total Words: 6149

Unqiue Words: 1690

Assert is a website where the best academic papers on arXiv (computer science, math, physics), bioRxiv (biology), BITSS (reproducibility), EarthArXiv (earth science), engrXiv (engineering), LawArXiv (law), PsyArXiv (psychology), SocArXiv (social science), and SportRxiv (sport research) bubble to the top each day.

Papers are scored (in real-time) based on how verifiable they are (as determined by their Github repos) and how interesting they are (based on Twitter).

To see top papers, follow us on twitter @assertpub_ (arXiv), @assert_pub (bioRxiv), and @assertpub_dev (everything else).

To see beautiful figures extracted from papers, follow us on Instagram.

*Tracking 57,756 papers.*

Sort results based on if they are interesting or reproducible.

Interesting

Reproducible