Data Engineering Process Fundamentals - Discovery

Introduction In this series of Data Engineering Process Fundamentals, we explore the Data Engineering Process (DEP) with key concepts, principles and relevant technologies, and explain how they are being used to help us deliver solutions. The first step, and important to never skip, in this process...

Data Engineering Process Fundamentals

Introduction Data Engineering is changing constantly. From cloud data platforms and pipeline automation to data streaming and visualizations tools, new innovations are impacting that way we build today’s data and analytical solutions. In this series of Data Engineering Process Fundamentals, we explore...

Use Remote Dev Container with GitHub Codespaces

As Software Engineers, we usually work with multiple projects in parallel. This forces us to configure our work stations with multiple software development tools, which eventually leaves our workstation performing poorly. To overcome this problem, we often use virtual machine (VM) instances that run...

Reduce code complexity by letting an API Gateway handle disparate services and document transformation

Modern Web applications use the microservice architecture for their API service integration. These APIs are often a combination of internal and external systems. When the system is internal, there is better control of the API endpoints and contract payload which the front-end components consume....

Understand Your Users with App Insights to Improve Adoption

After spending several months of software requirements, scrum and design meetings, hours of implementation and testing efforts, our application is finally deployed, but the work for a sustainable product is not complete. After the app is deployed, it is important to understand the users to improve...

Modernize SOAP Web Service with Azure API Management

Simple Object Access Protocol (SOAP) is an XML based protocol standard to build Web Services. It provides a communication channel for data exchange between applications that are built with different technologies. As a result, most companies have invested heavily in building SOAP Web Services. SOAP,...