Here is the repository of notes that I would like to remember, open and share. I hope you might find something useful to you too.

Feedbacks are welcome.

This site is built with fastpages, An easy to use blogging platform with extra features for Jupyter Notebooks.

Posts

• Sync-Async tasks pattern in MLOps pipeline

  This article discusses how ML training duration impacts MLOps pipeline and how to mitigate the impact using sync-async task patterns. It also includes code snippets when using Azure Pipeline and Azure Machine Learning Pipeline.

  Oct 10, 2022

• Security practices and design principles for implementing a data lakehouse solution in Azure Synapse

  notes about my security learning and design practices from using Synapse to build a cost-effective data lakehouse solution

  Mar 15, 2022

• Reflections on my 2020 Data Projects - Part II

  Last year, I participated in projects which process and analyze terabytes of data daily. We managed to have the system went production successfully and it is now processing data from different continents 24x7. Here are some learnings on the journey.

  Jan 1, 2021

• Reflections on my 2020 Data Projects - Part I

  Last year, I participated in projects which process and analyze terabytes of data daily. We managed to have the system went production successfully and it is now processing data from different continents 24x7. Here are some learnings on the journey.

  Jan 1, 2021

• Lessons learned from building a large scale historical data analysis system using Azure Data Explorer - Part 2

  notes about some lesson learned from building a large scale historical data analysis system that has hundreds of terabytes data using Microsoft Azure Data Explorer - Part II

  May 25, 2020

• Lessons learned from building a large scale historical data analysis system using Azure Data Explorer - Part 1

  notes about some lesson learned from building a large scale historical data analysis system that has hundreds of terabytes data using Microsoft Azure Data Explorer - Part I

  May 21, 2020

• Troubleshooting Azure Data Explorer (Kusto) cross-tenant access issues

  notes about some learning from fixing Azure Data Explorer (Kusto) cross-tenant access issues

  Mar 17, 2020

• Test

  Mar 3, 2019