Chapter 3 Linear regression, Q12

(a)

$\beta = \frac{\sum xy}{\sum x^{2}} = \frac{\sum yx}{\sum y^{2}} \\ \sum x^{2} = \sum y^{2}$

For the coefficient estimates of linear regression of Y onto X and X onto Y to be equal, the sum of the squares of X and Y should be equal.

(b)

In [2]:
set.seed(1)
x = rnorm(100,mean=0,sd=1)
eps = rnorm(100,mean=0,sd=sqrt(0.25))
y = 4*x+eps
plot(x,y)
In [7]:
lm.model1 = lm(y~x+0)
summary(lm.model1)

Call:
lm(formula = y ~ x + 0)

Residuals:
     Min       1Q   Median       3Q      Max 
-0.95768 -0.32358 -0.08853  0.25279  1.15545 

Coefficients:
  Estimate Std. Error t value Pr(>|t|)    
x  3.99694    0.05324   75.08   <2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.4793 on 99 degrees of freedom
Multiple R-squared:  0.9827,	Adjusted R-squared:  0.9826 
F-statistic:  5636 on 1 and 99 DF,  p-value: < 2.2e-16
In [8]:
lm.model2 = lm(x~y+0)
summary(lm.model2)

Call:
lm(formula = x ~ y + 0)

Residuals:
     Min       1Q   Median       3Q      Max 
-0.27588 -0.05931  0.02032  0.07818  0.23263 

Coefficients:
  Estimate Std. Error t value Pr(>|t|)    
y 0.245873   0.003275   75.08   <2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.1189 on 99 degrees of freedom
Multiple R-squared:  0.9827,	Adjusted R-squared:  0.9826 
F-statistic:  5636 on 1 and 99 DF,  p-value: < 2.2e-16

We can see that the coefficient estimates of the linear regression of Y onto X and X onto Y are different.

(c)

In [11]:
x = rnorm(100,mean=0,sd=1)
y = sample(x,100)
In [12]:
#output True if both are equal
sum(x^2)==sum(y^2)
TRUE
In [13]:
lm.model1 = lm(y~x+0)
summary(lm.model1)

Call:
lm(formula = y ~ x + 0)

Residuals:
    Min      1Q  Median      3Q     Max 
-1.9420 -0.6263 -0.1302  0.5447  2.1924 

Coefficients:
  Estimate Std. Error t value Pr(>|t|)
x -0.07043    0.10025  -0.703    0.484

Residual standard error: 0.9563 on 99 degrees of freedom
Multiple R-squared:  0.00496,	Adjusted R-squared:  -0.005091 
F-statistic: 0.4935 on 1 and 99 DF,  p-value: 0.484
In [14]:
lm.model2 = lm(x~y+0)
summary(lm.model2)

Call:
lm(formula = x ~ y + 0)

Residuals:
    Min      1Q  Median      3Q     Max 
-1.9785 -0.7097 -0.1747  0.5529  2.2622 

Coefficients:
  Estimate Std. Error t value Pr(>|t|)
y -0.07043    0.10025  -0.703    0.484

Residual standard error: 0.9563 on 99 degrees of freedom
Multiple R-squared:  0.00496,	Adjusted R-squared:  -0.005091 
F-statistic: 0.4935 on 1 and 99 DF,  p-value: 0.484

The coefficients of both models are equal when the sum of square of X and Y are equal.

In [ ]: