As Week 5 comes to an end, I am happily sitting on a small pile of data from our first run of the experiment!
After the excitement of hitting that Publish button and watching my experiment go live, I felt so mentally drained that when enough participants had completed the task, I turned off my laptop and went to bed. The next morning, I started digging into the results and found that, out of 10 people that we tested, we only had usable data from 7! One of the funny things about running experiments online is that you can apply filters and exclusion criteria all you want, but sometimes loophole-finders are inevitable. The silver lining is that I was able to use the data from those 3 random participants as practice for learning how to properly score the results and as predictors of what our actual participant population might look like. The more the merrier, apparently!
I am so happy to finally see the results of all our hard work and to experience the next steps in the experiment process!
What a great lesson! Approaching everything as a learning experience is the best thing we can do! Especially with so much uncertainty in our lives right now, learning through every experience is even more critical. The good news is, you also collected usable data that will contribute to the larger project! Congratulations!
It’s always a bummer to find out that some of your data isn’t usable, but I’m so happy you found a way to still use it for practice! That always makes it better and you don’t feel as crappy because you found a useful way to use the data you originally thought was unusable.