Linear Regressions and Confidence Intervals
Are regressions a valid way of analyzing longitudinal data for a data set with thousands of data points?
For example, If I wanted to track the daily movement on a stock price over the last 10 years would having too large of a sample size be detrimental?
I was playing around with an arbitrary exchange traded fund, VTI, and found that when I use a linear regressions I get extremely tight confidence intervals due to the large sample size.
Here's the output from the data set I'm using. The regression line itself is accurate, but note how the confidence intervals are extremely tight and not actually representative of reality. I also tried a 99% confidence interval and the numbers didn't change much.
Is there a better stats tool out there for this kind of analysis?
This may not be the best forum to discuss this in great depth. Keep in mind we had thousands of posts here arguing over the answer to this question:
If you roll two dice until at least one of them is a two, then on that roll, what is the probability that both of them are a two?
Quote: WizardI think you should use exponential regression rather than linear. You should also make it configurable at what point to start. A topic I debate myself over is how far back should I go in analyzing NFL data. The further back you go, the more data you have, but the data gets stale. I'm sure there is the same problem with stocks. Let the user worry about it.
This may not be the best forum to discuss this in great depth. Keep in mind we had thousands of posts here arguing over the answer to this question:
If you roll two dice until at least one of them is a two, then on that roll, what is the probability that both of them are a two?
What's your reasoning behind switching to an exponential regression?
I was playing around with the data a bit and found a polynomial regression actually has the best fit, but it tends to be extremely inaccurate at the tails so its not really usable.
Isn't it just 11/36 chance to roll a single 2, 1/11 chance to roll the doubles?
Which conveniently comes out to 1/36 when multiplied together?
Quote: WizardThis may not be the best forum to discuss this in great depth. Keep in mind we had thousands of posts here arguing over the answer to this question:
If you roll two dice until at least one of them is a two, then on that roll, what is the probability that both of them are a two?
Mike, you bu663r !!!!! Why, Why, Why???
Months of therapy and just as I start to sleep nights, you rake it up again.
And now, in another thread, we have Mr M telling us how he would statistically prove that dice are, or are not biased. He'd no doubt set one of them as a Deuce.
I'm going to have to stay away from this insanity for a while. Let me know when its over :o)
Didn't they tell you? The internet is where you argue with, and laugh at, people to whom you would never wish to speak normally.Quote: OnceDearMonths of therapy and just as I start to sleep nights, you rake it up again.
Quote: QuasiIntellectuThe internet is where you argue with, and laugh at, people to whom you would never wish to speak normally.
http://www.gocomics.com/bc/2015/09/27
Quote: DonutsWhat's your reasoning behind switching to an exponential regression?
Because the value of any investment should grow exponentially.
Quote: OnceDearMike, you bu663r !!!!! Why, Why, Why???
Sorry about that. We have some very smart minds on this forum, for which I'm very proud. Unfortunately, everybody has an equal voice and those in camp 1/6 have the same right to speech as everybody else. To answer the why question, I was trying to warn the OP that not everybody here has the best of mathematical credentials, and those with the worst credentials tend to do a lot of talking.
Quote: WizardBecause the value of any investment should grow exponentially.Quote: DonutsWhat's your reasoning behind switching to an exponential regression?
I think that justification should only apply to stock prices if you add to the current price the value of all dividends paid since the beginning of the analysis period. It doesn't seem reasonable to expect an investment to grow exponentially if you keep siphoning off the earnings.
