The value of experience

In a recent discussion about data informed product development I was asked about how we value professional experience. The thought had a very interesting angle in the fact that there are still a lot of things we have individual observations of and hold true even though they might lack data or scientific proof. I realized that I had already had a similar discussion earlier that day and that the pattern had occurred previously as well in different variations.

I think it's good start with the basic acknowledgement that although I'd always prefer detailed data it is very common to need action without data, often the only way to gather data is to do something for which you don't have data and previous experience. If possible you can do well without data by mapping a specific action or feature to first principles and see if they actually make sense, with a bit of work this can result in valuable insights. However investing effort to get the needed data is of course the obvious alternative unless...

When you look at all the design decisions needed to get something done it is easy to realize that even for trivial changes there are tons of decisions that are taken with very little specific knowledge of the particular quality of the method chosen. I guess this in some way is translated to the value of experience and many fields and professions rely heavily on them to get anything done. For any trivial part of a feature that is designed a number of professionals are making decisions on everything from font-size to code structure and algorithms to use way before you arrive at a reasonable prototype that you can actually expose to your customers in some way. It doesn't make sense to have experiments to verify everything to microscopic detail, eventually you would have an infinite amount of variations to test. From this perspective having experience is extremely valuable to get anything done in a reasonable amount of time.

However professional experience tend to be fairly general and frequently taught or gathered in a way that means it has a high risk of being outdated, this is a problem in a fast moving enterprise. What you learned from someone else five years ago, was perhaps based on a decade of professional experience which in a digital business is similar to trying to apply industrial practices from the 17th century. You needed this experience to do something but when you have any data on how this design actually worked that is a lot more valuable.

Most professionals will use new data to adjust their own personal experience accordingly. This is why professionals frequently love having the possibility to gather data and feedback on their efforts because it makes them even better professionals, which regardless of other future benefits has high value in itself. However it is not uncommon that discussions on methods and data are perceived as mistrust or questioning professional expertise when in fact it is just work needed to elevate the understanding of the problem and identify weakness in the existing data to ensure that future experience and data is of higher quality.

Scientific advancement moves at an amazing speed and this also works to establish experience and practices as proven methods. What was a few years ago considered as an unconventional method that can sometimes be applied might today be a proven method with well known boundaries for which applications it is suitable for. Keeping up to speed with these scientific advances is a critical aspect of being an expert, updating experience and learn new things are critical to any professional.

Occasionally experience and professional rules of thumb can be confused with having data, the human mind loves oversimplifying and drawing conclusions from where there are none. Things that are well known truths might be based assumptions that are similar to your problem at hand but "as we all know" the devil is in the details (pun intended). This is a good reason for also spending effort to test what is already known to ensure that results are consistent with existing theory and previous results. When in doubt (and sometimes even when not in doubt) verify that your experience isn't bad or broken.