Questions tagged [perceptron]
An early example of neural network without any hidden layers and with a single (possibly nonlinear) output unit.
165 questions
0
votes
0
answers
38
views
How to include contextual information along with input to a multi layer perceptron
Let $\mathbf{x}_k\in \mathrm{R}^{n\times 1}$ be an $n$-dimensional input to multi-layer perceptron(MLP) at time $t = k$. The output is $\mathbf{x}_{k+1}\in \mathrm{R}^{n\times 1}$ at time $t = k+1$. ...
- 121
1
vote
0
answers
47
views
Why is the threshold term incorporated into the weight vector in linear classifiers?
In the context of linear classifiers, such as the perceptron or logistic regression, I understand that the decision boundary is defined by a linear combination of input features and weights, plus a ...
- 121
1
vote
2
answers
136
views
ADALINE simple implementation with 2 features bug
I am reading Machine Learning with PyTorch and Ski-kit learn book by Sebastian Raschka
While plotting the decision boundary (a line in this case, since the number of features considered = 2) I can't ...
- 21
5
votes
2
answers
247
views
Confusion regarding the criteria for defining a ML model as a linear model
I am confused about the criteria which determines whether a model is linear or not. As far as I understand, the following statements are equivalent :
A model is linear
Output class label is a linear ...
- 53
0
votes
1
answer
105
views
Is it possible to predict non-classifiable label using single layer perceptron and sigmoid function? (without using any perceptron library)
Imagine predicting BMI index like 1,2,3,4,5 and having weight and height as input. I know it can be easily done with other method. Also I have to use sigmoid function and I am really new to this. I ...
4
votes
1
answer
135
views
Why there are two different versions for batch perceptron algorithm?
In the book "Understanding Machine Learning, S. David Ben et al.", the authors describe the Batch Perceptron Algorithm as follows:
However, in the book "Python Machine Learning, ...
- 670
2
votes
0
answers
142
views
Proving Perceptron algorithm mistake bound is tight [closed]
How would I prove the Perception mistake bound is tight. Avrim Blum’s lecture notes claim that the upper bound for mistakes is $\frac{R}{\gamma}^2$, but I don’t understand how to prove this is mistake ...
- 21
1
vote
1
answer
90
views
Does it matter which variable I assign 1 or -1 in a perceptron machine learning algorithm
I am using perceptron machine learning to solve the binary classification problem A vs B. For this I have to assign the actual values of A and B to either 1 or -1 to be able to use perceptron. Does it ...
- 11
1
vote
0
answers
35
views
Can Perceptron and Naive Bayes classifier create a vertical decision boundary in a two-dimensional graph?
A decision boundary like in the picture.
2
votes
1
answer
665
views
Is bias nothing but perceptron threshold value?
I was revisiting neural network basics from this post. The perceptron follows below equation:
$$\begin{align}
y & = 1 & \text{if } \sum_{i=1}^n w_i\times x_i \geq \theta \\
& = 0 & \...
- 151
3
votes
0
answers
92
views
What's the unified definition of Tikhonov regularization
I met with "Tikhonov regularization" in two textbooks. The first is "Pattern Recognition and Machine Learning" by Christopher M. Bishop. In page 267 of his book, the regularized ...
- 333
1
vote
1
answer
1k
views
What are the mathematical differences between the Perceptron and the MP-Neuron?
I try to understand the differences between the MP Neuron and the Perceptron. Is my understanding right that the MP Neuron mathematically only differences in the activation function. I.e. the MP ...
- 159
5
votes
2
answers
284
views
Understanding Weight Updates in a Perceptron
I am learning about perceptrons and how they work. I read that each weight $w_j$ is updated based on the equation:
$\begin{equation}
w_j:=w_j+\Delta w_j
\end{equation} $
Where:
$\begin{equation}
\...
- 51
1
vote
0
answers
197
views
How can I explain the difference in decision boundary and its reason in relation to the dimension of the hidden layer?
I got these 3 different plots of decision boundaries using 3 different parameters for hidden_layer_sizes of the MLPClassifier from sklearn on XOR gate.
...
- 21
3
votes
1
answer
185
views
Explanation of Equation 5.80 in Pattern Recognition and Machine Learning - Bishop
How the equation 5.80 in _Pattern Recognition and Machine Learning_ by Bishop is derived?
- 133
3
votes
1
answer
320
views
Does the n >> p holds also for minibatches
It's pretty known that when dealing with models (without regularization) the main assumption is $n >> p$ where $p$ is the number of features in the dataset
Let's suppose that we have 1.000.000 ...
- 1,561
4
votes
1
answer
200
views
How do I check if the weights of my perceptron/step activation function are correct
I am new to stack overflow and deep learning so I hope I am doing this the right way. I tried to find the solution myself but it has not been successful so I am seeking some help. This is the ...
- 43
1
vote
0
answers
1k
views
Neural network for imbalanced data
I have an imbalanced data (n = 600, about 97% majority and 3% minority) with 20 features and a binary outcome. The data has been split into a training set and a test set (80%/20%). I used H2o autoML ...
- 45
2
votes
0
answers
84
views
What are the differences between the perceptron and linear regression hypothesis set?
While studying machine learning I have known 2 learning models: linear regression and perceptron.
I know the difference between the Learning algorithm they use, but the hypothesis set look the same to ...
- 81
1
vote
0
answers
99
views
Differentiating output of layer with respect to its input [duplicate]
Say we have a relationship $ z = Wx$ for a multi layer perceptron where $z$ and $x$ are $n$ dimensional vectors. When we find $\frac{dz}{dx}$ , I would assume this would just be $W$, not $W^T$.
I was ...
- 11
1
vote
1
answer
687
views
What is the interpretation of sklearn's linear perceptron coefficients?
I'm stumped as to why this example doesn't do a better job fitting the data, I suspect it has to do with my interpretation of the perceptron object's coefficients. Note that I'm interested in the <...
- 161
1
vote
1
answer
327
views
Different results for Logistic Regression (wrong) and Perceptron (correct)
To help me with some understanding, I'm trying to learn the Logical AND and Logical OR using Linear Regression trained over the following data:
...
- 219
0
votes
0
answers
78
views
Why do the training and validation accuracy drop after tuning?
I thought my MLP (multi-layer perceptron)'s accuracy will increase after tuning. However, the accuracy dropped.
Then someone told me that I should add Dropout layers with 50% dropping.
I did that. ...
- 2,077
1
vote
1
answer
833
views
Rank of gradient-of-loss with respect to layer weights in an MLP
The paper: https://arxiv.org/abs/2110.11309, makes the following claim at the end of page 3:
The gradient of loss $L$ with respect to weights $W_l$ of an MLP is a rank-1 matrix for each of B batch ...
- 13
2
votes
1
answer
370
views
Single-layer perceptron mathematical formulation
I'm trying to btter understand the formalism under the following compact formulation of a single-layer perceptron. If we consider $V=\mathbb{R}^d$, then
$$\hat{f}(x_1, \dots, x_d) = \sum_{i=1}^Nc_i\...
- 669
0
votes
1
answer
247
views
How to show that the gradient of the smoothed surrogate loss function leads to perceptron update?
This is about the contents of section 1.2.1 and 1.2.1.1 of the book "Neural Networks and Deep Learning: A Textbook". The link to the sections is here. The question arises from the following ...
- 333
1
vote
1
answer
241
views
Did Hinton introduce the concept of distributed representation?
From Goodfellow et al.'s Deep Learning book:
Several key concepts arose during (...) the 1980s that remain central to today’s deep learning. One of these concepts is that of distributed ...
- 189
1
vote
1
answer
299
views
Stop criterion is Infinitive in Perceptron in Sklearn
I read code in book "Hand-on Machine Learning in Sklearn and TensorFlow" by Aurelien Geron
...
- 113
22
votes
1
answer
7k
views
Why pure exponent is not used as activation function for neural networks?
The ReLU function is commonly used as an activation function in machine learning, as well, as its modifications (ELU, leaky ReLU).
The overall idea of these functions is the same: before ...
- 323
2
votes
1
answer
285
views
How to resolve the perceptron dilemma for binary classification?
I have a following thought problem involving perceptron and binary classification that I wonder if anyone has thought about before. This is not from any textbook or reference, although I doubt I'm the ...
- 744
0
votes
1
answer
43
views
In AI/ML, using the Perceptron model, would it ever make sense to have both negative weights and data?
I understand the math but I want to make sure I understand the mapping back to real world scenarios. Thinking about it logically, I cannot think of a real world scenario where you would have a ...
- 255
1
vote
1
answer
1k
views
In what way are SGD and the Perceptron learning algorithm very similar?
I'm reading Hands-On Machine Learning and the author states that:
You may have noticed the fact that the Perceptron learning algorithm strongly resembles Stochastic Gradient Descent. In fact, Scikit-...
- 151
1
vote
0
answers
161
views
One-hot encoding in Keras for R [closed]
I am trying to build a binary classifier using a MLP with the Keras package in R.
My question is, why does the package require the labels to be a one-hot vector? For example, the value 1 will be the ...
- 111
3
votes
1
answer
206
views
Best approach for energy demand forecasting
I am trying to predict the amount of energy demand(Wh) of the next two weeks per hour. The dataset I have, contains each hour of each day since 2019 of the energy demand, is something like this:
...
- 53
1
vote
1
answer
970
views
How to draw the single perceptron decision boundary when weights and bias are 0?
I've been following an algorithm described on a book called Knowledge Discovery with Support Vector Machines by Lutz H. Hamel. In the book, there is this learning algorithm for a single perceptron ...
- 145
3
votes
1
answer
159
views
About Multi-layer Perceptrons
I've always been a bit confused when it comes to Deep Learning terminology.
Is the definition of the perceptron, whether single layer or multi layer, associated with a specific type of activation ...
- 101
1
vote
0
answers
120
views
Rule of thumb for Data Requirements when Designing a Neural Network for Deep Learning
I'm designing an MLP classifier and I've been noticing that:
Using a very shallow network, or one whose at least one layer has a small number of neurons yields bad performance
Using a deep network ...
- 111
1
vote
1
answer
601
views
Perceptron as a Logistic Regression
If by following way single perceptron is made to work like Logistic Regression.
How much correct is it to say that I made perceptron to work as Logistic Regression.
Question came to mind as ...
- 185
1
vote
0
answers
116
views
Why do we use Matrix in Perceptron instead of Functions?
Matrices are good objects to store connections between dimensions/entities. However, matrix computation is often time consuming and sometimes wasteful if matrix is too sparse. Also thinking about the ...
- 111
2
votes
0
answers
196
views
0
votes
0
answers
87
views
Why in gated recurrent unit gates are controlled by only one layer perceptrons?
Why don't I see a GRU anywhere with more than one layer of perceptrons inside, it's pretty obvious to try to put more layers in there, but I don't see anyone doing that
- 1
2
votes
1
answer
913
views
Whats the different between Logistic regression and perceptron?
In a binary classification problem, if both logistic regression and a single preceptron uses Sigmoid function, what's the difference in
classification results, since they will have the same decision ...
- 21
1
vote
1
answer
433
views
Can we express CNNs in terms of a MLP?
I have been wondering whether a convolution can be represented in terms of an MLP. We can say that in convolution we have shared parameters between different neurons. But how to express this ...
8
votes
1
answer
2k
views
Why aren't neural networks used with RBF activation functions (or other non-monotonic ones)?
In most work I've seen, MLPs (multilayer perceptron, the most typical feedforward neural network) and RBF (radial basis function) networks are compared as distinct models, where
MLP neuron outputs $\...
1
vote
1
answer
2k
views
Neural network regression with constraint
I am training a neural network for a regression task, where the dependent variable varies in the range from $0$ to $10$. Unsurprizingly, with the test data set, I obtain the predictions that fall ...
- 5,091
2
votes
1
answer
911
views
Can a 4D perceptron be plotted in 2 dimensions?
I am wondering if it is at all possible to plot a 4D perceptron line in 2D.
Obviously, it would be impossible to observe it with all of its original information, but is there a way for me to observe ...
- 23
2
votes
1
answer
430
views
Can we use perceptron training algorithm to train a single neuron with sigmoid activation?
The perceptron training algorithm is summarized as:
Apply the inputs and calculate the output $ y $
Compare with the desired output yd and calculate error $e = y-y_d$
Update the weights based on the ...
- 119
5
votes
2
answers
2k
views
SciKit Learn: Multilayer perceptron early stopping, restore best weights
In the SciKit documentation of the MLP classifier, there is the early_stopping flag which allows to stop the learning if there is not any improvement in several ...
- 777
0
votes
1
answer
853
views
Weight update rule for Rosenblatt's Perceptron
I'm wondering if anybody can explain how Rosenblatt reached his formula for updating the weights of his Perceptron:
$\textbf{w}_{t+1} = \textbf{w}_{t} +\eta ( y_j - \hat{y}_j ) \textbf{x}_j$
It seems ...
1
vote
1
answer
253
views
Use a multilevel logistic regression and cross validation
I want to use a multilevel logistic regression for a double purpose, estimating the value of coefficients to explain a phenomenon. At the same time, I want to split the data through cross-validation ...
