I'm trying to survey people who come to my site. I would like to invite 3% of my visitors to participate in the survey. To do so, I currently have a survey popup based on the following logic:

Each time someone comes to my site, it generates a random whole number between 1 and 100. When the number is greater than 97 (=98, 99, or 100), then the visitor gets a survey popup.

My programmer told me that this gives 3% of chance for a given visitor to be prompted by the survey, so over time I'll have 3% of visitors being invited.

Well, I've had the survey for two days now, and my date says that only 0.6% of visitors are seeing the survey invitation. So I'm wondering whether the logic is sound. Would I really get AGGRIGATED 3% when each independent event had 3% of chance?

If someone can confirm or correct this logic, I'd appreciate it.

Thank you in advance,

shoribe

The logic appears to be sound, but remember that the more hits you get the closer you should get to the 3%. The more hits the closer the empirical probability will approach the theoretical probability. It does not necessarily mean that , say, you get 100 hits that there will actually be 3 that take the survey. It may be a little more or a little less.

Suppose a toss a coin 100 times. Theortical probability would dictate that I get 50 heads and 50 tails, but that will not necessarily happen. I may get 45 heads and 55 tails.

Thank you, galactus.

I understand the base size effect--the more you have, the closer it gets. But I had 16,138 visitors on Saturday, and 13,716 visitors on Sunday. 100 of 16,138 saw the popup on Saturday, and 95 of 13,716 saw the popup on Sunday. Do you think 16K is still too small to get close to 3%? And the fact that it was 0.6% on both Saturday and Sunday (well, it's 0.69%.. so I guess you could round it up to 0.7%.. but still) made me wonder if this is right. What do you think?

shoribe

The more the people, the closer to that percentage you'll approach. If even more people visit your site, you should see the increase in the percentage of popups appearing for the visitors. Just need more people and time, and you'll see by yourself

Let's look at this from a binomial perspective. As an example: The probability that someone

will take the survey is 3%. Suppose we choose 15,000 at random. What is the probability that

exactly 450 will take the survey? (450 is 3% of 15,000)



See there? Out of 15,000, there is about a 2% chance that 450 of them will take the survey.

But... using the same binomial, there is a 51% probability that less than 450 will take the survey.



Those are pretty good odds. That is where you come in with your .6%

OK, I understand that there's a tiny chance that I get exactly 3% over time even when the volume goes up to 15,000. And it makes sense that there's about 50% chance that I get less than 3%.

Is there any way to know what the chances are for me to get 0.6% and then 0.7%? I mean, what are the chances for me to be so far away from 3% twice in a row (and they are in the same range)?

I'm wondering if it's possible that the programming codes that generate and popup a screen aren't working properly and not actually prompting 3%. (My programmer says they are... I'm not savvy enough to read the codes.. )

shoribe

I calculated it through the binomial and it was very, very small. 0 for all intents and purposes.

I sure don't know, but perhaps someone has it set at .3% instead of 3%?

Just keep an eye on it and see if it doesn't come up in time.

Thank you so much for helping me out. I can now go back and tell my programmer that something has to be wrong. It was a great help!

shoribe