Data Engineering Archives - ITFROMZERO

Processing Parquet Files in Python with PyArrow: Reading, Writing, and Production Optimization Tips

By IT From Zero October 27, 2026

Six months of production experience using PyArrow and Apache Parquet in Python: resolving VPS OOM errors, slashing log storage from 12GB to 1.4GB, and cutting processing time from 18 minutes to 42 seconds.

Artificial Intelligence tutorial - IT technology blog

Getting Started with ScrapeGraphAI: Automating Web Scraping with LLMs and Python

By IT From Zero October 9, 2026

A comprehensive guide on configuring ScrapeGraphAI with OpenAI and Ollama, optimizing token usage, and building self-healing web scraping pipelines for production.

Python

Stop ‘Dirty Data’ with Pandera: Insurance for Your Data Pipeline

By IT From Zero September 12, 2026

Don't let dirty data crash your system at midnight. Learn how to use Pandera for efficient and high-performance data validation in Pandas and Polars.

Git

Using DVC with Git: Don’t Let a 10GB Dataset “Break” Your Repo

By IT From Zero September 12, 2026

Don't let big data slow down your project. Learn how to combine DVC and Git to manage Datasets and Models professionally, keeping your Repo lightweight and versioning easy.

Python

Building ‘Immortal’ Data Pipelines with Prefect: Don’t Let Your Python Scripts Die in Silence

By IT From Zero September 10, 2026

Don't let your Python scripts die in silence. A guide to building professional Data Pipelines with Prefect: Auto-retry, Dashboard monitoring, and smart error handling.

Database

Mastering dbt Core with PostgreSQL: From Fragmented SQL to Standardized Data Workflows

By IT From Zero September 9, 2026

Eliminate the headache of manual SQL management. A detailed guide on using dbt Core with PostgreSQL to automate data transformations, data testing, and documentation.

Database

Mastering Airbyte: Automate Your ELT Data Pipelines Without Writing Code

By IT From Zero September 7, 2026

The era of manual ETL coding is over. Learn how to use Airbyte to automate data pipelines from MySQL and Postgres to BigQuery with just a few clicks.

Docker

Dockerizing Apache Airflow: Professional CeleryExecutor Deployment with Docker Compose

By IT From Zero September 6, 2026

How to Dockerize Apache Airflow with CeleryExecutor and Redis using Docker Compose. The optimal solution for deploying stable and scalable data workflows.

Python

Mastering PySpark on Linux: Processing Hundreds of GBs of Data with Python

By IT From Zero August 18, 2026

Struggling to process tens of gigabytes of data with Pandas? Discover how to install and optimize PySpark on Linux to handle Big Data efficiently through parallel computing.

Database

Mastering Apache Hop: Building Automated ETL Pipelines for Database Synchronization from A-Z

By IT From Zero August 16, 2026

Say goodbye to manual scripting. A detailed guide on using Apache Hop to build automated ETL pipelines for professional and efficient database synchronization.