2.9 Conclusion

As the career paths available in big data continue to grow so does the shortage of big data professionals needed to fill those positions. In the previous sections of this chapter the characteristics needed to be successful in the field of big data have been introduced and explained. The characteristics such as communication, knowledge of big data concepts, and agility are equally as important as the technical skill aspects of big data.

Big data professionals are the bridge between raw data and useable information. They should have the skills to manipulate data on the lowest levels, and they must know how to interpret its trends, patterns, and outliers in many different forms. The languages and methods used to achieve these goals are growing in strength and numbers, a pattern unlikely to change in the near future, especially as more languages and tools enter and gain popularity in the big data fray.

Regardless of language, method, or specialization, big data scientists face a unique technical challenge: working in a field where their exact role lacks a clear definition. Within an organization, they help to solve problems, but even these problems may be undefined. To further complicate matters, some data scientists work outside any specific organization and its direction, like in academic research. Future chapters will explore concrete applications of big data across multiple disciplines to demonstrate how diversely big data scientists can work.