Can you use bivariate analysis in a multiple regression problem
shiya43
Answered question
2022-10-29
I have one response variable and around 10 potential explanatory variables. I am looking for simple ways to visualize and explore my data before modelling. As well as carrying out PCA and looking for correlations between my explanatory variables I wanted to look at bi-variate scatter plots of the response with each explanatory variable - but I have been told this wont work.
This is they way I am picturing it -
Imagine you have n independent, uncorrelated explanatory variables.
You propose to use these variables in a multiple regression. You want to simplify the analysis by not including more variables than is necessary.
So , you want to know if the explanatory variables have any correlation with the response variable to decide whether to include the variable in the regression.
My feeling is 1) that you can look at bi-variate plots for the response variable with each explanatory variable to see if there is a correlation.
2) If a single explanatory variable has no correlation with a the response in bi-variate correlation, it cannot have any affect on the response if included in the multiple regression.
(The only way this variable could affect the response variable would through an interaction with another explanatory variable - but it has already been determined that all the explanatory variables are uncorrelated.)
I have been told that the above is incorrect and a variable showing no correlation in a bi-variate plot can affect the response variable in a multiple regression.
Could someone help me see where I am going wrong? Many Thanks.
Answer & Explanation
Jimena Torres
Beginner2022-10-30Added 20 answers
for all , and let for the sake of simplicity each vector be normalized to a unit "ball", i.e.,
The OLS vector is given by
Namely,
meaning that the coefficient of the th explanatory variable does not depend on other variables. Note that in the simple linear regression of on the OLS coefficient will be
namely, the same as in the multiple regression.
PCA in this case will result in orthogonal eigenvectors that are the initial axes and all the eigenvalues will equal
New Questions in Inferential Statistics
Which of the following statements is not correct for the relation R defined by aRb, if and only if b lives within one kilometre from a?
A) R is reflexive
B) R is symmetric
C) R is not anti-symmetric
D) None of the aboveA line segment is a part of a line as well as a ray. True or False
Which characteristic of a data set makes a linear regression model unreasonable?
Find the meaning of 'Sxx' and 'Sxy' in simple linear regression
In the least-squares regression line, the desired sum of the errors (residuals) should be
a) zero
b) positive
c) 1
d) negative
e) maximizedCan the original function be derived from its order Taylor polynomial?
Should the independent (or dependent) variables in a linear regression model be normal or just the residual?
What is the relationship between the correlation of two variables and their covariance?
What kind of technique is to be adopted if I have to find an equation or model for say, depends on , changes for a set of , which changes for different .
Correlation bound
Let x and y be two random variables such that:
Corr(x,y) = b, where Corr(x,y) represents correlation between x and y, b is a scalar number in range of [-1, 1]. Let y' be an estimation of y. An example could be y'=y+(rand(0,1)-0.5)*.1, rand(0,1) gives random number between 0, 1. I am adding some noise to the data.
My questions are:
Is there a way where I can bound the correlation between x, y' i.e. Corr(x,y')?I mentioned y' in light of random perturbation, I would like to know what if I don't have that information, where I only know that y' is a estimation of y. Are there any literature that cover it?What is the benefit of OU vs regression for modeling data, say data in the form of () pairs?
Can you determine the correlation coefficient from the coefficient of determination?
How can one find the root of sesquilinear form with positive definite matrix?
From numerical simulation and regression analysis I discovered that the root-mean-square amplitude of white noise with bandwidth is proportional to . How can this be derived mathematically ?
In a Simple Linear Regression analysis, independent variable is weekly income and dependent variable is weekly consumption expenditure. Here % confidence interval of regression coefficient, is .
