Lab 6: Model fitting

Not graded, just practice

1 Model fitting

1. True or false, gradient descent and orinary least squares are both iterative optimization algorithms.

True False

2. What cost function have we been using to perform our gradient descent?

standard deviation bootstrapping sum of squared error absolute error

3. True or false, when performing gradient descent on the model given by the equation \(y = w_0 + w_1x_1 + w_2x_2\), we might arrive at a local minimum and miss the global one.

True False

4. Which of the following would work to estimate the free parameters of a nonlinear model?

gradient descent ordinary least squares solution both work

5. True or false, in gradient descent, we search through all possible parameters in the parameter space.

True False

2 Model fitting in R

Questions 6-9 refer to the code and output below, performing gradient descent with optimg:

optimg(data = data, par = c(0,0), fn=SSE, method = "STGD")

$par
[1] 3.37930046 0.06683237

$value
[1] 959.4293

$counts
[1] 6

$convergence
[1] 0

6. How many steps did the gradient descent algorithm take?

7. What was the sum of squared error of the optimal paramters?

8. What coefficients does the algorithm converge on?

3.37930046, 0.06683237 0, 0 959.4293 6, 0 all of the above

9. What parameters were used to initialized the algorithm?

3.37930046, 0.06683237 0, 0 959.4293 6, 0

Questions 10-12 refer to the output below from lm():

Call:
lm(formula = y ~ x, data = data)

Coefficients:
(Intercept)            x  
    3.37822      0.06688  

10. Use R notation to write the model specification.

y ~ x  # this works (implicit intercept)

y ~ 1 + x # this also works (explicit intercept)

11. Given the model is specified by the equation \(y = w_0+w_1x_1\), what is the parameter estimate for \(w_0\) = and \(w_1\) = .

12. True or false, for this model, optimg() with gradient descent would converge on the same parameter estimates?

True False