Topic 1- Data Architecture within the organization
I was originally inspired to write this blog while reading the article “Data Hubs, Data Lakes and Data Warehouses: How They Are Different and Why They Are Better Together”; however, it was more of how organizations manage their data through Data hubs, lakes, or warehouses. As a ruling principle in the activation of an ongoing EA practice and in this case, it is Data architecture, the team needs strong sponsorship and ongoing support whether it is financial, operational to achieve the goals. In our case, this sponsorship was initiated through a royal decree to establish a national data management office and their key mandate was to put in place data management policies for all government entities to ensure alignment and unification. In addition, the royal decree included the enforcement for all government entities to establish a Data Management Office (DMO) function within the organization and abides by the regulations of the national DMO. This created a high level of sponsorship support initially and led the Board to establish the office as a strategy function under the strategy department. Until last year in early 2022, the board decided to move the function under the Technology deputy. It was interesting to hear during the class session one of my classmates made a comment regarding the structure of the data department and how it was also moved to the Technology department, and I thought it was interesting to see similarities and wonder if there are similarities in the operation as well.
Moving on, I wanted to provide a high level of the
- Data sources
- How the data is structured
- Key challenges
- Data Sources- As you can see in figure (1) below, the data sources are divided into 3 segments:
- Operations’ Data– Data provided to the organization by the 6 regulation bodies.
- Enterprise Data– Data generated from internal operations and developments.
- Market Data- Data generated from entities within the ecosystem of operations’ support.
Figure 1- Internal Document- sensitive information was removed.
- Data Structure- As you can see in Figure (2), the structure of the data within the organization:
Figure 2- Internal Document- sensitive information was removed.
- Key Challenges- here are a list of key enterprise challenges in the data management which I’m sure every organization:
- There are different procedures used for collecting data from different departments.
- Lack of defined enterprise data utilization
- Undefined roles and responsibilities for data related decisions.
- Lack of resources and expertise in the field of Data Management
- Lack of data infrastructure availability
- Lack of data management awareness within the organization
Topic 2- To cloud or not to cloud
As I was reading the different research documents by Gartner, one article titled “Understanding Cloud Data Management Architectures: Hybrid Cloud, Multi-cloud and Intercloud” triggered an interest of writing this blog because I came across a presentation prepared to be presented to committee regarding the strategy of moving to the cloud at the end of month. As done by many organizations who were established before cloud storage and services became a commodity, a high amount of investment was made to build the on-premises infrastructure with a specific standard to ensure availability and reliability was in place to build the infrastructure. There is even more pressure to ensure higher-than-average standards are in place due to the sensitivity of the services provided by the organization and having those services on high availability due to the nature of the business operations.
I interviewed a few relevant stakeholders regarding the current setup and what constraints are in place and what steps were taken toward moving to the cloud and referring to the below figure:
Throughout the interview and referring to the figure above, scenario number 2 was the closer scenario of how the setup was made. Due to the criticality of the services, services were hosted on the on-premises infrastructure while the backup for those services are in the cloud; however, there were two services (unable to disclose them) are indeed fully operational with a cloud service vendor within the country. One constraint discussed previously which had a lot of influence on the moving to the cloud strategy was the regulating government body enforcing all government entities to ensure their hosting operations are within the country. This created an obstacle for picking up the pace of moving to the cloud because there were limited-service providers available (more vendors are on their way to operate within the kingdom). Initial capital to be invested is not the feasible choice as current available vendors have marked up cost for service knowing that the supply was low in the market; thus, the organization will continue to operate its infrastructure as is and will conduct an annual feasibility assessment toward the cloud.
Your insight into the intricacies of data management, especially in the context of government entities, is fascinating. The emphasis on the importance of sponsorship and alignment, demonstrated through the royal decree, underscores the gravity and significance of data management in today’s digital age. I’m also intrigued by the challenges you’ve outlined, as they seem universally relevant and highlight areas of potential improvement for many organizations.